In chess there's an idiom for this. "Long think, wrong think" because it's a quite common phenomenon that very good players will ruin positions by rather than playing with their good instinct, over-analyzing a position, there's a related situation of the hardest games to win being already won positions because there's so many ways to win that people will on occasion start doing something really stupid akin to the example of having too much choice in the article.
I think a nice collective analogue to this is Alfred Whitehead's observation that 'civilisation advances by extending the number of important operations which we can perform without thinking of them'. Progress is being made by holistically integrating knowledge in a way that makes it sort of ambient.
It also reminds me of a slightly snarky article why all the people in the rationalist cult never seem to actually be successful at anything other than rationalism. It's precisely because consciously thinking is easy, it's the integration of knowledge into the whole is what's difficult but actually necessary.
“In the words of the ancients, one should make his decisions within the space of seven breaths. Lord Takandobu said, “If discrimination is long, it will spoil.” Lord Naoshige said, “When matters are done leisurely, seven out of ten will turn out badly. A warrior is a person who does things quickly.
When your mind is going hither and thither, discrimination will never be brought to a conclusion. With an intense, fresh and undelaying spirit, one will make his judgments within the space of seven breaths. It is a matter of being determined and having the spirit to break right through to the other side.”
But taken from a site [0] I just found in the process that apparently has nothing to do with the OS and informs us that ;
"Ubuntu is a concept that we have in our Bantu languages at home. Ubuntu is the essence of being a person. It means that we are people through other people. We cannot be fully human alone. We are made for interdependence, we are made for family. When you have ubuntu, you embrace others. You are generous, compassionate. If the world had more ubuntu, we would not have war. We would not have this huge gap between the rich and the poor. You are rich so that you can make up what is lacking for others. You are powerful so that you can help the weak, just as a mother or father helps their children. This is God's dream.”
Ubuntu the operating system takes its name from the concept that you talk about here.
That is also why the logo of Ubuntu the operating system is three people holding hands, which kind of blew my mind when I first heard about it. Prior to that I thought the logo was just some lines and circles in a pretty pattern with no specific meaning.
What about when there is a huge length of time between the plan and the final execution? Do you just not let yourself think about it after the 7th breath? In the context of the hagakure, I suppose it would apply to battle strategy?
Keep in mind the expertise required to build that intuition, though. The average person who merely knows how to play Chess should probably think before they move, or else blunder away the whole game by putting their queen on a square threatened by a knight they didn't look at.
That said, most people are probably "experts"--in the sense that their intuitions can often outdo their conscious trains of thought--in more than a few things. I'll bet that drivers in their 40's make better decisions with instinct than conscious thought. The same probably goes for middle aged home cooks. But teenagers should go over their driving lessons in their heads, and stick to the recipes.
Learning something new requires a lot of concentration and deliberate thinking and practice. When I learned chess I remember the friend I was playing with becoming a bit bored waiting for my moves but with time it became more automatic and only certain moves would make me think more and when not being able to evaluate or choose between alternative I'd go to gut feeling, possibly distilled from thousands and thousands of chess games played. At some point the tipping point leaned too much towards gut feeling and after some bluffing I decided to take a break from chess and moved on to something else. I think I got bored and my brain would jump to shortcuts. Then I discovered musical instruments
>Alfred Whitehead's observation that 'civilisation advances by extending the number of important operations which we can perform without thinking of them'.
Reminds me a bit of a quote I heard recently: "Tradition is a set of solutions for which we have forgotten the problems. Throw away the solution and you get the problem back."
Of course, sometimes the underlying problem has gone away entirely or been dealt with by a different solution. Then you run into traditionalists defending ridiculous things with arguments that make no sense.
> civilisation advances by extending the number of important operations which we can perform without thinking of them
Interesting take. My first thought was that this relates to societal trust, a cornerstone of civilization. I can buy food from someone I've never met, and keep myself fed, 'without thinking'.
That’s an interesting point. Makes me think of the loops that are required to be jumped through to enable a trustless digital currency like Bitcoin. The entire network, with transactions of about 1kB in size, can do at most 7 transactions per second (3-4 orders of magnitude lower than Visanet, which itself is lower throughput than all the world’s physical cash transactions could theoretically do if everyone bought one or two things a day on average). That’s the same data throughput of a mid-1990s modem, but with power consumption of about 10GW to secure the transactions.
Pure trustlessness indeed has a massive computational and mental overhead. Some compromise (like the Lightning network, or even a central trusted banking institution) is essential to get anything done. (Even with Bitcoin, you’re trusting open source software, often an actual intermediary like Coinbase, etc... Try to do it without any crypto libraries whatsoever—writing it really from scratch—and it is an enormous project. How many people teach themselves Finite Field theory before doing Bitcoin because they take the trustlessness seriously?)
But most people are noticeably stronger with longer time than with shorter time. You need merely play a long time control and not give yourself the benefit of using that time and see whether your rating drops.
I’m working on a video game right now and I’m having this problem with the story and dialogue. In the past my game design was in the realm of tabletop rpgs, so you come up with the idea, add some details, and then you run it right away. You see the player responses in person, face to face, learn whether you did well or not, and then move on to do even better the next time. It works great. In a video game, there can be months between conception and a player experiencing it, and you won’t be there to see most of them react, so you can second guess every single detail hundreds of times until it’s hard to judge things clearly. I’ve never been into heavy up front design before, but it might actually be helpful for this specific problem - ie, to just write it and then trust it later on
I was thinking about this the other day. When I was a kid I'd think 1-12 moves ahead, I'd obviously learned to prune bad moves and it worked pretty well for me.
But then thinking about it, this is exactly the wrong way to train your brain. It's job isn't to evaluate every single position, but rather to train it in the form of a "neural network" to institutionally recognize good vs. bad positions, and to expand on that.
Looking at some high ranked players online, this seems to be the way they recognize positions and possibilities.
That's completely different, and not wholely beneficial either. The piper's due is merely deferred until the last of the original abstractor's guard dies, then everyome who relied on it has to rederive the fundamentals, or stagnate.
I don't think this proves anything other than there are often contradictory idioms in chess, here's one attributed to Larsen: "If you find a good move, look for a better one".
> very good players will ruin positions by rather than playing with their good instinct, over-analyzing a position
I wonder if this is due to errors happening and getting mixed into the search. Computers pretty monotonously improve with thinking time, and the algorithm is pretty simple once you have good intuition to guide it. However, a single memory error down a line will turn the whole thing to mush.
Even disregarding the errors, when "overthinking" is easy t go deeper in some branches than in others, without a very good probability estimation (which something the computer can do more easily: weight the branches).
The "intuitive" approach is not that deep, but the depth is more even.
This is true at every level of chess. The thinking optimum is somewhere between "playing without caution" and "over-analysing". Over-analysing is theorically the best but because we're not computers there are mistakes in our resulting assertions.
Being already known by my friends and family to be an "overthinker" and prone to "analysis paralysis", I sometimes wonder why I'm still a SWE. I've slowly started to realize over time that my profession is filled with overthinking and over-engineering, and that our interview process can even select for it. IMO its hard to be the person that aces the technical interview gauntlet then walks out of the building and turns the analytical skills off.
Many interview processes seem to favor how well a candidate can enumerate edge cases and problem spaces over effective risk assessment and cost management. They're both important to evaluate but often in practice the dumb solution is what my team ends up using because can be more maintainable, cheaper to build, easier to reason about, etc. Today my aim is to get my requirements, write as few lines of quality code in as short of a time as possible, test it, ship it and be done.
Narrow focus and the ability to scope things down to what exactly what matters helps a lot. I defeat over-analysis by meditation, intentional dumbness/willful ignorance, and flow state.
As someone who's been programming for years, one of my biggest complaints in software is over-engineering. It's so universally prevalent, I literally expect to get downvoted for even making this comment. I strongly advocate for simplicity in software implementation, allowing for maintainability and flexibility... the cleanest solution that reaches the goal and doesn't lock us too far into a narrow approach that may be difficult to deviate from. Sometimes that's not realistic, but it's actually feasible a lot more than people seem to believe.
It's a weird term, "over engineering". Think about over-engineering a bridge. What does it look like? Huge and bulky, way too expensive, able to hold much more weight than it should, lots of walkways and access points and design features that nobody really wants? But wait: "any fool can build a bridge that doesn't fall down. It takes an engineer to build a bridge that just barely doesn't fall down." The whole point of engineering is to figure out exactly what the bridge needs to do and make it do only that. So the "over-engineered" bridge is actually under-engineered -- they haven't thought enough about the actual problem. So too is most code that is called "over-engineered" -- it's far more common that people haven't though enough about the problem than that they've over-thought it.
I like to believe that experience will eventually drive people towards using scientific parsimony when creating solutions to engineering problems.
I don't think over-engineering is the only problem: shiny-new. Shiny-new syndrome wherein the typically junior programmer cannot help but jump from one new thing to the next, advocating that some new system must use the new tech that they recently discovered. While the enthusiasm is ok, working with people like this can be a drag.
Another problem: we are facebook too! In the sense that small companies heartily believe that they actually have problems at the scale of a company like facebook (or will ever get to that level). Again, hearing someone say, "We should do xyz because Netflix does it!" makes me want to give up on this profession.
"over-engineering" is a fuzzy term though. For someone, using a third-party service to handle users auth flow is a time saver, and building the whole flow themselves would be over-engineered. For another, using a third-party service for auth would be adding layers of complexity on top of a simple db request, and would thus be considered over-engineered.
Largely agree, but I've found distilling complex requirements down to a simple implementation takes effort, and sometimes longer than the more circuitous route.
A coarse rule of thumb from a few decades of my own experience in software development is that version 1 will be quick, dirty and (in retrospect) naive. Version 2 will improve it with what you learned the first time around. And version 3 tends to be the clean, solid, polished winner. I've learned to expect and encourage refactoring to arrive at this point.
"I would have written a shorter letter, but I did not have the time."
In my experience, you often start with a somewhat over engineered solution. Then you take the (long) process of really understanding it and slowly distill it down. This is true for a new feature and also the whole codebase.
Wholeheartedly agree. The best system is no system at all - problem is it doesn't really satisfy any requirements. Anything added beyond that is extra room where fragility creeps in & the tradeoff shouldn't ever be taken lightly.
In my opinion over-engineering is a symptom of not enough thinking. Complex, over engineered solutions don't happen because a problem is too well understood, they happen because a problem is not understood well enough.
This is based on the assumption that more thinking == better understanding.
Recently I've been interviewing and assessing technical tests regularly.
We have this one technical coding challenge. Most of the submissions are 1000+ LOC and quite heavily engineered. One submission though was 300 LOC, runs 3x as quick as everyone elses and is the only one to get 100% in our acceptance tests. The author was very self-deprecating about it - describing it as a quickly cobbled together submission.
I'm nearly 40 and find I over analyze the design of everything. Which is great when I'm architecting a high level software feature, but when I get to coding I'm almost at analysis paralysis over every, damn detail. I miss that sense of flow.
> the ability to scope things down to what exactly what matters
In the last few years, I started shifting my focus from what variety of cases to handle with an elaborate solution, to what possible cases I might be blocking with a simple solution. Gratuitous YAGNI was an important intermediate step in that process. It might not be very FAANG-friendly (or it might not scale to four-digits-engineers regardless of brand), but both in terms of real life performance and peer feedback, it's yielded very good results.
While the first is prone to becoming a bottomless pit of SWE-self-pleasuring; the latter is actually very useful in coming up with something that's cheap, easy and maintainable, but still flexible enough -- which doesn't necessarily mean extensible! Ease-of-refactoring is another flavor of flexibility, and that's where I found this kind of thinking beating YAGNI alone.
One actual drawback I see is that it makes DRY difficult to achieve, but the more people I work with, the more I'm becoming disillusioned with that anyway. It's such an overly adaptable dogma to generate work that is often pointless, and becomes harmful so easily. I think we found so much comfort in "don't type the same thing twice" (though I still try to keep it to 3) that we don't stop to think about "don't maintain the same information twice" anymore. But this is 3AM rambling, so I'll stop here.
It is not really "not thinking", but not using your conscious mind, which can only focus on a single thing.
You think with your subconscious mind too. But this can do multiple things at the same time, way faster than the logical mind.
The harder you focus on a single thing, the more you ignore the entire system and the slower you perform.
It is not just instinct as the article say, a tennis player does not play by instinct because nobody knows how to play tennis when he is born.
It is by training that you develop intuition. If you train well you can perform well without thinking consciously. Training well is hard work and takes a lot of time too.
If you follow your instincts you are predictable and an easy prey. I can hunt or fish animals because they follow their instincts too well.
There is a word that is missing from the language or at least common usage, and that is differentiating something done with conscious effort of every little detail, and something you just do without thinking about it. (there are probably lots of words which somewhat fit the bill)
The hardest way to do something is complete conscious control of every step, the easiest way is something you can do perfectly on autopilot.
"Flow" is about being able to do something very well with almost minimal conscious control. (I would expand flow to being more than this, but it is at least a defining characteristic)
Intuition versus instinct. Instinct is innate, intuition is developed. "The more you know", if you have a better breadth and depth of knowledge and experience to draw from you can "intuit" the (or a) proper course of action without the need for deliberate and thorough reasoning. Many people use the term "instinct" when "intuition" may be the better term.
"Tacit knowledge" I think captures what you are talking about. Its the knowledge embedded in our bodies and automatic responses. Most commonly this can refer to mechanical tasks, like when you go on autopilot while driving.
It can also be used to describe mental tasks and ways of thinking. An engineer has tacit knowledge that shapes their natural way of thinking and intuition. A novelist, a nurse, or a tailor, in contrast, will have very different mental structures.
In Dutch that word would be "automatisme": a habit so ingrained in your mechanical memory that it doesn't require conscious thought.
Thinking about it, wouldn't the word "habit" cover it in English? It makes me smile to realize that our habits are so ingrained, we don't even realize we have a word for it :)
Isn't that Type 1 vs Type 2 thinking, Daniel Kahneman?
Sure, it's great if we could use Type 1 thinking for everything, but the problem is when things are counterintuitive. So what you really need is a good rule to follow to know when to engage in Type 2 thinking.
My current programming team is doing way too much Type 1 thinking.
Thinking keeps me awake. It's getting worse as I get older.
Maybe it's anxiety or something related, and sometimes it's worried thoughts, but often it's just running through scenarios. Video games, media plots, what I need to do next week. My brain is far more active after 10pm, and I am personally more motivated. I get a burst of energy but it's not really the right time to clean the house so lying in bed has it all go to my head.
I've come to the conclusion I can't sleep with an active train of thought going on. It sounds like meditation should help but I haven't had much luck there. Maybe I just need to try more often until it becomes second nature.
I assume you have already considered and observed the effect of alcohol and/or caffeine intake.
One thing that definitely helps me is to have a routine for bedtime. A hot shower, bed preheated in winter (heated underblanket), keeping the room cold and reading a book while lying in bed. I typically pick technical material as it's less likely to keep me up all night with an engrossing story. At some point, I notice that the mind is tired and I am able to gather words from the page but not comprehend the material. I then set the book aside, turn off the bedlamp and try to sleep. Usually, this is sufficient and I fall asleep in no time. Some people just fall asleep while reading. For me, it's reading. For a friend of mine, it's watching a TV show. He has something playing on the computer or TV and falls asleep while watching it.
The idea is to get the body relaxed (hot shower in my case) and then do something (devoid of stress) that keeps ones mind off the hustle and bustle of one's day.
Also, going to bed and rising at (roughly) the same time everyday will also get the body and the mind trained to be sleepy at that time and it'll get easier to fall asleep.
EDIT: I naturally tend to drift to a nocturnal routine if left to my own inclinations. The hardest part of what I wrote above is just pulling myself away from the day to go to bed at what most would consider a reasonable hour. If you can not convince yourself that sleep is important and commit to maintaining discipline in your sleep/wake routine, no "technique", "advice" or "substance" will be effective on the long run.
The part about reading but no longer comprehending sounds very familiar. If it happens twice that I’m looking at the words, but my mind is going somewhere else, I judge myself ready to fall asleep.
Then after closing my eyes I go through the stuff I just read (to avoid getting stressed out over real-life stuff), and after a while will have one or more thoughts that make no sense.
That’s the sign that falling asleep will happen very soon. From there on I don’t know what happens next.
So what do you do about thoughts of others suffering while you live the good life? Do these kind of thoughts ever keep you up or torment you? Or how do you view a life of luxury while others die at a young age?
Physical exercise after work, cut out screen time a couple hours before you want to sleep (especially engaging screen time), consider cutting off caffeine earlier in the day (or totally for a while and ease back in), get treated for anxiety (not necessarily medication, therapy techniques like CBT can help in a wide variety of areas).
The biggest one for me was physical exercise. Working out hard for 1-3 hours after work (most people can't put in 3 hours, I know) left me sufficiently physically exhausted that whatever mentally engaging activities I got into later, I was crashing by midnight no matter what. Doing this after work was also critical as it helped create a clean break from work (where many stressful or technically engaging thoughts come from for me). Exercising in the morning did not have the same effect.
I've been dealing with a similar condition for a while. I've learned that it's impossible for me to "not think". Any attempts at emptying my head will only invite more anxieties and problems to be solved. Instead, I've learned to fill it with trivial, creative challenges that occupy keep my brain busy enough to ward off negative thoughts yet introduce no stakes.
Some examples include:
- If I found myself stranded on an island, what kinds of challenges would I have to overcome and how?
- If I could improve the magic system of Harry Potter, what changes would I introduce?
- If I were given a device that I could use to turn back the time to 6 am once everyday, how could I use it to my best advantage while avoiding any pitfalls?
- If I were given a small 12x12x12 room with an unlimited budget to create a living space for me to be confined in, how would I use that space?
These would be the kinds of things that I could think about for a while before finding myself fast asleep. Not sure if this would work for anyone else, but maybe you can give it a try?
Aside from (mostly) practicing normal sleep hygiene recommendations, I listen to a sitcom that I've already watched many times using wireless earbuds as I fall asleep. Sometimes I do both ears, somtimes one ear, depending on whether I feel like sleeping with my head turned. I'm normally a back sleeper.
Because I've already seen the episodes, they don't stimulate any thinking. They're comedy, so all happy and lighthearted. Well written comedy is funny on multiple-goarounds, but other than some chuckles the point is mostly to help me fall asleep happy. On a bad night I'll listen to two episodes, but mostly I'm asleep halfway through the first episode.
My go to shows fwiw are It's Always Sunny in Philadelphia and The Office. Parks and Recreation also works. It's OK to have plot, as long as you already know the plot / have seen the show enough, so you aren't trying to watch the episode.
Since some of these shows moved off Netflix I've had to buy them on Google Play store. FWIW Google Play's player is better than amazon video's for mobile (it's basically youtube's player).
The wireless earbuds I use are these: $35, https://www.amazon.com/Soundcore-Bluetooth-Headphones-Waterp... and they're connected by a cord so you don't lose one of the buds when you sleep (used to happen to me). I have two sets of earbuds now, because I really struggle to sleep without it now that it's part of my routine and I can't risk them not being charged.
Usually I'm awake in my bed for at least an hour. Sometimes two
Huh, lucky one. Some stay awake until morning.
I know that some people find it strange or even perverted, but try listening to ASMR. First find an artist and a specific 1-2h long video that calms your mind at day while you work or read, and then use it for going to sleep (use comfortable earbuds). They have special videos for sleep, but you may use any of them. You’ll skip a handful of artists and asmr types before you find that one, so be patient. Also try languages you don’t speak (french, chinese). Top, “general purpose” artists is a good start.
Also use browser youtube version and adblock that surely blocks youtube ads, or they will jumpscare you, and turn off autoplay.
I was in the exact same situation when I was a kid, I could stay awake in my bed for hours. It was extremely frustrating waking my mother at 3am to tell her that I can’t fall asleep. Nowadays, in my early 20s I can basically fall asleep in less than 3 minutes. I have no clue how this condition disappeared from my life. At least for me physical activity was not a factor in this condition, maybe I’m eating healthier than before (less sugar).
I often get symptoms like this if I am working on/thinking about highly technical things past about 8 PM. I feel like the pace and energy level of what I'm doing is a contributing factor, as I've found reading technical books rarely causes it.
I get that way sometimes. For me, it's usually because I'm not physically tired enough and have a lot of stress in my system. Rowing 7-10 10km pieces over two weeks fixes it.
Be engaged during the day. Use that mental energy. Whatever that means, but challenge yourself and exhaust yourself.
Exercise which is just generally really good for your mental and physical state. Do note that working out later at night can make the problem worse since exercises also causes stress chemicals in your brain and it takes a few hours for those to clear.
Melatonin[1] an hour before bed. It's safe, non-habit forming, dirt cheap, very mild and surprisingly effective in my experience. I take it as needed instead of every night, but I know a few folks take it every night.
I really do think that I personally made the mistake of allowing myself to think actively in bed and like a pebble down a mountain it is suddenly an avalanche that feels impossible to stop.
I have to be completely exhausted in order to get to sleep reasonably quickly after going to bed. I might try meditation at some point... but yes my mind as well suddenly turns on. I also seem to drift off lightly for some period of time and then I find myself unexpectedly awake again and my mind is obnoxiously alert and ready to grind through thoughts.
Yes me too! It definitely feels like something I trained myself into during my teens. I don't know why but I guess the quiet time lead to contemplation of all my teenage troubles. I do the same on bus rides since I used to catch the bus for school every day and had one of the longer rides. The problem is I actually enjoy it so it's hard to stop.
I also drift off and notice it and jerk wide awake. Sometimes with hypnagogia, shadows take the form of bad people or I see spiders hovering or running across my vision. It's a bit disturbing and definitely doesn't help when I get an adrenaline response. I seem far more prone to it when I'm very tired and parts of my brain are shutting down but this one stubborn conscious area keeps me up.
I'm a night owl living on a 9-5 schedule, and I have the same problem. Most of the day I'm a little tired, then at night I get a burst of energy. It's frustrating. I think the ideal solution would be to find a good job that lets me work my ideal hours, but barring that I've found a useful strategy. After my wife goes to bed, I'll go downstairs for an hour or two and read, write, and think. By letting my brain burn off some of its energy, I find myself naturally drifting into a healthy calm tired. I'm never exactly wiped to the point where I can just fall asleep, but I calm down enough that I can still get to sleep at a semi-decent hour. Combined with WFH I actually get close to 8.5 hours by sleeping in later, so it's not bad.
For a while last year, I did just accept it and stay up late until I felt more tired, usually around 1:30am. We were both working from home and I'd sleep in a little bit (which is also getting harder in my late 30s). This wasn't perfect but it was better. But now my wife is back at the office and is out of bed before 8am, which wakes me up and usually I can't get back to sleep easily so I've been struggling since.
I've tried exercise both during the day and the evening in case it's a physical thing, but it either doesn't help or makes it worse. Sometimes my active brain state amplifies physical aches and pains and itches and it just runs away from me and is very distracting.
I think what might really be missing is a sense of closure or achievement. A lot of things in life can give gratification without meaning, but the days I do something like finish fixing the shower or putting up shelves, it does seem a bit easier to switch off at night. Cooking is the closest I get most days.
What type of meditation did you try? Try following / counting breaths. Definitely counting in the beginning, breathe out, 1, breathe in, breathe out, 2, breathe in ... up to 10, then re-start to 1. Do it for 20 min before falling asleep. Then as you try to fall asleep do the same.
Don't be discouraged if it's not working right away. It has to work but you may need months of training before it starts to work well.
The hardest challenge is to be completely cold to the thoughts, no matter how important they may seem, ignore them and return to counting the breaths.
I've done a few meditation podcasts where a guy talks and I follow along, and as soon as it ends I'm relaxed but more alert. If my power trips overnight and the noise (fans or other electrical background noise) suddenly stops, I jerk awake. This is the same.
I've tried counting back from 5000 slowly on each breath. I got very far before stopping. Going backwards with large numbers is meant to take a bit more mental energy I think.
I've done the 1-10 then reset, we did that in Yoga at the gym. I've also done the box breathing where you breathe in and out slowly but at different rates.
I think mostly it will come down to practice like anything else. I get upset if it doesn't work but I enjoy ruminating so it's hard to let go. In many of these techniques I was very relaxed but still had an attentive core stopping the sleep from happening.
I'm not diagnosing you or anything but that thread put me on to the fact I might have that type of wiring too ... it would explain a lot ( for me ) and possibly some of that ^ ?
[ EDIT ] Anxiety is hands down my primary problem, but sometimes some issues can obfuscate others, and cause deductive fallacies by encouraging you to explain everything away under the umbrella of that first cause.
Yeah my mum and sister (around 65 / 35 years old) were diagnosed last year. I'm thinking I should go along soon since it runs in families. For some reason it's not that easy to book these things in when you have trouble planning and following through on stuff :)
My mum also had insomnia for years. She did get some sleeping pills as a twice a week "reset" and I think a lot of mental issues are exacerbated by poor sleep leading to weak willpower so that could help. But she said the ADHD meds helped a lot with her sleep too.
If you are not on any brain altering medication (or other drugs), try chamomile tea before bed time. Please read up on the chemical profile of chamomile tea as it will interact with certain brain medications (start here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2995283/).
Reducing artificial light after sundown and cutting out caffeine and listening to slow placed instrumental music (key being that there are no vocals/lyrics for the brain to attach to) has fixed my "awake brain" issue. Try to also avoid screen time before and in bed as each "new" thing you see (lets say, hackernews post titles) will activate your brain for a few minutes and keep it awake long after it has seen them. I use a single tea candle and switch off all other sources of light - it emits more than enough light! Basically you want to try to limit any inputs and have a consistent winding down ritual.
After a while the brain will accept this new ritual and will immediately get into a relaxed state if the lights go down and it hears a certain pace of music. For me, hang drum + flute music is the best, followed by calm asian instruments. When you listen to the same calm playlist over and over at bedtime, eventually you will know them. When you get to this stage, the brain will get bored with them at first - then you can double down and hum with the song (if you live alone of coarse). Try not to "talk" to yourself internally during your winding down ritual as it will keep the brain awake (don't try to convince the brain it is time to sleep, it will resist and fight back with more awareness).
Do this for 2 weeks and see if your night ritual improves. Obviously adjust it for your circumstances. Make it your special me-time. Choose the music to be unique and don't play those songs at other times. If you brain really wants to have a chat, ask it how it's been, tell it you are thankful for it and you will see it again in the morning! Make it your friend, don't fight it. I mean it. Be gentle and caring towards it, like you are it's father/mother, accept it for what it is.
The whole ritual last from 20 to 60 minutes, depending on how tired you are and how much conflict you accumulated during the day (conflicts being work frustrations, traffic frustrations, sexual frustration, doing things against your ideal solution, politics/news). These things all put pressure on the internal state and most people relieve those pressures either by sexual release, exercise or drugs/alcohol/food before bed. Having a self care ritual is another option. Sleep will do most of the work for us, but sleep is so much better if is not forced but rather gently simmered in. Fresh bed sheets & hot shower can also affect how comfortable the bed feels. A good mattress also helps. Natural smells also helps, so a drop of rose geranium, rubbed on your skin. Again, if you pick a specific flavour/smell to make part of your ritual, use it only during your winding time ritual and not during the day. You want to let the brain associate the music and smells and other environmental factors with "time to relax mode". Be consistent.
The thing that disturbs my peace the easiest is caffeine, even a single cup in the morning can effect my night time ritual.
Anyway. The above kind of what works for me, it's not medical/scientific based, hope it helps. Self care goes a long way!
Thanks, I'm glad it works for you. I might check if we still have some chamomile tea since I've never made a habit of it. And Kindle on my phone is probably risky because it's right next to the bad engagement-algorithm stuff, maybe I need to pick up some paper books.
For the rest - every time I read something like this I think about my wife, for two reasons.
One - any ritual will force her to do the same and play along but that might be asking a lot if its against her natural inclinations. She tends to want to stay up late (like me) and gets some of her best work done then. If I go to bed alone, then if she is still in the house (so I know she is coming along later) that anticipation alone is usually enough to keep me awake.
Two - she doesn't need any of that. Why can she fall asleep in 5 minutes after reading her phone in bed and breaking every single rule people give me? It always bothers me and I get very jealous :)
"The only reliable cure for overthinking seems to be enjoyment, something that both success and analysis can dull." This point at the end seems a bit forced without any support beyond the author's assertion. The feeling I get is someone who has mastered this unthinking is "enlightened" of sorts with an ability to get to satisfaction with a humility that can be mistaken for smugness.
From fiction, I might refer to the Wheel of Time series where Rand Al'Thor uses a trick to "find the void" in a way his father taught him. When you get overwhelmed from information and emotions, finding the void allows you to regain composure and think straight again.
What if you enjoy overthinking though? What if that is your relaxation, is to be incredibly consumed in thought and solving new and intricate problems versus the dull existence of just "being"?
Not exactly. Meditation is a solitary activity where one can train the mind to emptiness. What the article and I'm referring to is getting to that state while still being in the moment. Meditation might be the ability to sharpen the sword but this is about being able to use it in battle.
It reminds me of a graduate quantum class I had. The professor gave what he called "infinite time length exams". We would come in during the afternoon and we could stay as long as we wanted to work on the exam--though we had to slip it under his door by say 9:00am. You could bring as many books as you wanted (apparently at one point you could bring "anything" and someone brought a professor--but that could just be a story). But, if you didn't finish in a reasonable time, you probably weren't going to get the answer.
My modern physics professor did this for all our assignments. He didn’t want to rush students to fill out answers, so he made the questions very hard. We had to really understand the mechanics in order to answer the questions. I retained so much more by learning this way when compared to wrote memorization, and it was fun having discussions with other students to help each other understand the concepts.
i really like this sort of exams, where some artificial condition isn't imposed on the examinees for ranking purposes.
I had a computer science exam where you are allowed to bring any text book/reference book you wish, and any notes you've written yourself. The exam was long and hard yes, but it was also thorough. One of the more enjoyable exams i recall taking - vs all of the other types.
With online learning, basically all of my exams are like this. I’ve heard of some people having proctored online exams, but luckily my school’s Math and CS departments seem to have opted to just make the questions harder/unable to be looked up, and gave up on restricting us from using notes or the internet. It’s actually surprising how well it works, if the professor can design the exam cleverly enough. Really the only way you can cheat is by colluding with other students (which admittedly is a fairly big issue) or, for some types of programming questions, by just running the code on your own machine (which again can be avoided by clever test design).
Isn't this just an assignment where you're not sufficiently trusted to be in the location that suits you best (eg library, your desk?) A fully proctored assignment, you may not ask questions once you've read the problem.
I guess it has merit if cheating of that nature is a big problem.
Readit News
I think a nice collective analogue to this is Alfred Whitehead's observation that 'civilisation advances by extending the number of important operations which we can perform without thinking of them'. Progress is being made by holistically integrating knowledge in a way that makes it sort of ambient.
It also reminds me of a slightly snarky article why all the people in the rationalist cult never seem to actually be successful at anything other than rationalism. It's precisely because consciously thinking is easy, it's the integration of knowledge into the whole is what's difficult but actually necessary.
When your mind is going hither and thither, discrimination will never be brought to a conclusion. With an intense, fresh and undelaying spirit, one will make his judgments within the space of seven breaths. It is a matter of being determined and having the spirit to break right through to the other side.”
Initially from ; https://en.wikipedia.org/wiki/Hagakure
But taken from a site [0] I just found in the process that apparently has nothing to do with the OS and informs us that ;
"Ubuntu is a concept that we have in our Bantu languages at home. Ubuntu is the essence of being a person. It means that we are people through other people. We cannot be fully human alone. We are made for interdependence, we are made for family. When you have ubuntu, you embrace others. You are generous, compassionate. If the world had more ubuntu, we would not have war. We would not have this huge gap between the rich and the poor. You are rich so that you can make up what is lacking for others. You are powerful so that you can help the weak, just as a mother or father helps their children. This is God's dream.”
- Ubuntu, as explained by Bishop Desmond Tutu
[0] https://ubuntutheory.blogspot.com/2008/08/7-breaths.html
That is also why the logo of Ubuntu the operating system is three people holding hands, which kind of blew my mind when I first heard about it. Prior to that I thought the logo was just some lines and circles in a pretty pattern with no specific meaning.
That said, most people are probably "experts"--in the sense that their intuitions can often outdo their conscious trains of thought--in more than a few things. I'll bet that drivers in their 40's make better decisions with instinct than conscious thought. The same probably goes for middle aged home cooks. But teenagers should go over their driving lessons in their heads, and stick to the recipes.
Reminds me a bit of a quote I heard recently: "Tradition is a set of solutions for which we have forgotten the problems. Throw away the solution and you get the problem back."
"Tradition is a set of solutions for problems that don't exist anymore".
An example of the latter that immediately springs to mind is circumcision.
Of course, sometimes the underlying problem has gone away entirely or been dealt with by a different solution. Then you run into traditionalists defending ridiculous things with arguments that make no sense.
Interesting take. My first thought was that this relates to societal trust, a cornerstone of civilization. I can buy food from someone I've never met, and keep myself fed, 'without thinking'.
Related:
* https://en.wikipedia.org/wiki/Liars_and_Outliers
* https://www.schneier.com/books/liars-and-outliers/
Pure trustlessness indeed has a massive computational and mental overhead. Some compromise (like the Lightning network, or even a central trusted banking institution) is essential to get anything done. (Even with Bitcoin, you’re trusting open source software, often an actual intermediary like Coinbase, etc... Try to do it without any crypto libraries whatsoever—writing it really from scratch—and it is an enormous project. How many people teach themselves Finite Field theory before doing Bitcoin because they take the trustlessness seriously?)
But then thinking about it, this is exactly the wrong way to train your brain. It's job isn't to evaluate every single position, but rather to train it in the form of a "neural network" to institutionally recognize good vs. bad positions, and to expand on that.
Looking at some high ranked players online, this seems to be the way they recognize positions and possibilities.
Similar concept is applied to programming, we call that concept abstraction.
do you have a link? would love to read this, and it probably applies to me :)
I wonder if this is due to errors happening and getting mixed into the search. Computers pretty monotonously improve with thinking time, and the algorithm is pretty simple once you have good intuition to guide it. However, a single memory error down a line will turn the whole thing to mush.
Many interview processes seem to favor how well a candidate can enumerate edge cases and problem spaces over effective risk assessment and cost management. They're both important to evaluate but often in practice the dumb solution is what my team ends up using because can be more maintainable, cheaper to build, easier to reason about, etc. Today my aim is to get my requirements, write as few lines of quality code in as short of a time as possible, test it, ship it and be done.
Narrow focus and the ability to scope things down to what exactly what matters helps a lot. I defeat over-analysis by meditation, intentional dumbness/willful ignorance, and flow state.
I don't think over-engineering is the only problem: shiny-new. Shiny-new syndrome wherein the typically junior programmer cannot help but jump from one new thing to the next, advocating that some new system must use the new tech that they recently discovered. While the enthusiasm is ok, working with people like this can be a drag.
Another problem: we are facebook too! In the sense that small companies heartily believe that they actually have problems at the scale of a company like facebook (or will ever get to that level). Again, hearing someone say, "We should do xyz because Netflix does it!" makes me want to give up on this profession.
A coarse rule of thumb from a few decades of my own experience in software development is that version 1 will be quick, dirty and (in retrospect) naive. Version 2 will improve it with what you learned the first time around. And version 3 tends to be the clean, solid, polished winner. I've learned to expect and encourage refactoring to arrive at this point.
In my experience, you often start with a somewhat over engineered solution. Then you take the (long) process of really understanding it and slowly distill it down. This is true for a new feature and also the whole codebase.
There are other coping mechanisms you can use, although some of them are arguably worse than procrastination (eg overengineering).
This is based on the assumption that more thinking == better understanding.
We have this one technical coding challenge. Most of the submissions are 1000+ LOC and quite heavily engineered. One submission though was 300 LOC, runs 3x as quick as everyone elses and is the only one to get 100% in our acceptance tests. The author was very self-deprecating about it - describing it as a quickly cobbled together submission.
I'm nearly 40 and find I over analyze the design of everything. Which is great when I'm architecting a high level software feature, but when I get to coding I'm almost at analysis paralysis over every, damn detail. I miss that sense of flow.
Now you probably shouldn‘t code golf either, but KISS works as an operating principle for a reason.
In the last few years, I started shifting my focus from what variety of cases to handle with an elaborate solution, to what possible cases I might be blocking with a simple solution. Gratuitous YAGNI was an important intermediate step in that process. It might not be very FAANG-friendly (or it might not scale to four-digits-engineers regardless of brand), but both in terms of real life performance and peer feedback, it's yielded very good results.
While the first is prone to becoming a bottomless pit of SWE-self-pleasuring; the latter is actually very useful in coming up with something that's cheap, easy and maintainable, but still flexible enough -- which doesn't necessarily mean extensible! Ease-of-refactoring is another flavor of flexibility, and that's where I found this kind of thinking beating YAGNI alone.
One actual drawback I see is that it makes DRY difficult to achieve, but the more people I work with, the more I'm becoming disillusioned with that anyway. It's such an overly adaptable dogma to generate work that is often pointless, and becomes harmful so easily. I think we found so much comfort in "don't type the same thing twice" (though I still try to keep it to 3) that we don't stop to think about "don't maintain the same information twice" anymore. But this is 3AM rambling, so I'll stop here.
You've already seen how to be a better SWE than most.
Now you just have to execute.
You think with your subconscious mind too. But this can do multiple things at the same time, way faster than the logical mind.
The harder you focus on a single thing, the more you ignore the entire system and the slower you perform.
It is not just instinct as the article say, a tennis player does not play by instinct because nobody knows how to play tennis when he is born.
It is by training that you develop intuition. If you train well you can perform well without thinking consciously. Training well is hard work and takes a lot of time too.
If you follow your instincts you are predictable and an easy prey. I can hunt or fish animals because they follow their instincts too well.
The hardest way to do something is complete conscious control of every step, the easiest way is something you can do perfectly on autopilot.
"Flow" is about being able to do something very well with almost minimal conscious control. (I would expand flow to being more than this, but it is at least a defining characteristic)
It can also be used to describe mental tasks and ways of thinking. An engineer has tacit knowledge that shapes their natural way of thinking and intuition. A novelist, a nurse, or a tailor, in contrast, will have very different mental structures.
Thinking about it, wouldn't the word "habit" cover it in English? It makes me smile to realize that our habits are so ingrained, we don't even realize we have a word for it :)
Sure, it's great if we could use Type 1 thinking for everything, but the problem is when things are counterintuitive. So what you really need is a good rule to follow to know when to engage in Type 2 thinking.
My current programming team is doing way too much Type 1 thinking.
Maybe it's anxiety or something related, and sometimes it's worried thoughts, but often it's just running through scenarios. Video games, media plots, what I need to do next week. My brain is far more active after 10pm, and I am personally more motivated. I get a burst of energy but it's not really the right time to clean the house so lying in bed has it all go to my head.
I've come to the conclusion I can't sleep with an active train of thought going on. It sounds like meditation should help but I haven't had much luck there. Maybe I just need to try more often until it becomes second nature.
One thing that definitely helps me is to have a routine for bedtime. A hot shower, bed preheated in winter (heated underblanket), keeping the room cold and reading a book while lying in bed. I typically pick technical material as it's less likely to keep me up all night with an engrossing story. At some point, I notice that the mind is tired and I am able to gather words from the page but not comprehend the material. I then set the book aside, turn off the bedlamp and try to sleep. Usually, this is sufficient and I fall asleep in no time. Some people just fall asleep while reading. For me, it's reading. For a friend of mine, it's watching a TV show. He has something playing on the computer or TV and falls asleep while watching it.
The idea is to get the body relaxed (hot shower in my case) and then do something (devoid of stress) that keeps ones mind off the hustle and bustle of one's day.
Also, going to bed and rising at (roughly) the same time everyday will also get the body and the mind trained to be sleepy at that time and it'll get easier to fall asleep.
EDIT: I naturally tend to drift to a nocturnal routine if left to my own inclinations. The hardest part of what I wrote above is just pulling myself away from the day to go to bed at what most would consider a reasonable hour. If you can not convince yourself that sleep is important and commit to maintaining discipline in your sleep/wake routine, no "technique", "advice" or "substance" will be effective on the long run.
From time to time I also add a 5-minute meditation to the routine and it works quite well for me.
Then after closing my eyes I go through the stuff I just read (to avoid getting stressed out over real-life stuff), and after a while will have one or more thoughts that make no sense.
That’s the sign that falling asleep will happen very soon. From there on I don’t know what happens next.
Usually I'm awake in my bed for at least an hour. Sometimes two. I just really can't stop thinking. It's not anxiety in my case.
Any people here who were in a similar situation and were able to get rid of it, or make significant improvement?
The biggest one for me was physical exercise. Working out hard for 1-3 hours after work (most people can't put in 3 hours, I know) left me sufficiently physically exhausted that whatever mentally engaging activities I got into later, I was crashing by midnight no matter what. Doing this after work was also critical as it helped create a clean break from work (where many stressful or technically engaging thoughts come from for me). Exercising in the morning did not have the same effect.
Some examples include:
- If I found myself stranded on an island, what kinds of challenges would I have to overcome and how?
- If I could improve the magic system of Harry Potter, what changes would I introduce?
- If I were given a device that I could use to turn back the time to 6 am once everyday, how could I use it to my best advantage while avoiding any pitfalls?
- If I were given a small 12x12x12 room with an unlimited budget to create a living space for me to be confined in, how would I use that space?
These would be the kinds of things that I could think about for a while before finding myself fast asleep. Not sure if this would work for anyone else, but maybe you can give it a try?
Because I've already seen the episodes, they don't stimulate any thinking. They're comedy, so all happy and lighthearted. Well written comedy is funny on multiple-goarounds, but other than some chuckles the point is mostly to help me fall asleep happy. On a bad night I'll listen to two episodes, but mostly I'm asleep halfway through the first episode.
My go to shows fwiw are It's Always Sunny in Philadelphia and The Office. Parks and Recreation also works. It's OK to have plot, as long as you already know the plot / have seen the show enough, so you aren't trying to watch the episode.
Since some of these shows moved off Netflix I've had to buy them on Google Play store. FWIW Google Play's player is better than amazon video's for mobile (it's basically youtube's player).
The wireless earbuds I use are these: $35, https://www.amazon.com/Soundcore-Bluetooth-Headphones-Waterp... and they're connected by a cord so you don't lose one of the buds when you sleep (used to happen to me). I have two sets of earbuds now, because I really struggle to sleep without it now that it's part of my routine and I can't risk them not being charged.
Huh, lucky one. Some stay awake until morning.
I know that some people find it strange or even perverted, but try listening to ASMR. First find an artist and a specific 1-2h long video that calms your mind at day while you work or read, and then use it for going to sleep (use comfortable earbuds). They have special videos for sleep, but you may use any of them. You’ll skip a handful of artists and asmr types before you find that one, so be patient. Also try languages you don’t speak (french, chinese). Top, “general purpose” artists is a good start.
Also use browser youtube version and adblock that surely blocks youtube ads, or they will jumpscare you, and turn off autoplay.
Be engaged during the day. Use that mental energy. Whatever that means, but challenge yourself and exhaust yourself.
Exercise which is just generally really good for your mental and physical state. Do note that working out later at night can make the problem worse since exercises also causes stress chemicals in your brain and it takes a few hours for those to clear.
Melatonin[1] an hour before bed. It's safe, non-habit forming, dirt cheap, very mild and surprisingly effective in my experience. I take it as needed instead of every night, but I know a few folks take it every night.
[1] - https://slatestarcodex.com/2018/07/10/melatonin-much-more-th...
I have to be completely exhausted in order to get to sleep reasonably quickly after going to bed. I might try meditation at some point... but yes my mind as well suddenly turns on. I also seem to drift off lightly for some period of time and then I find myself unexpectedly awake again and my mind is obnoxiously alert and ready to grind through thoughts.
The next day is usually not enjoyable.
I also drift off and notice it and jerk wide awake. Sometimes with hypnagogia, shadows take the form of bad people or I see spiders hovering or running across my vision. It's a bit disturbing and definitely doesn't help when I get an adrenaline response. I seem far more prone to it when I'm very tired and parts of my brain are shutting down but this one stubborn conscious area keeps me up.
I've tried exercise both during the day and the evening in case it's a physical thing, but it either doesn't help or makes it worse. Sometimes my active brain state amplifies physical aches and pains and itches and it just runs away from me and is very distracting.
I think what might really be missing is a sense of closure or achievement. A lot of things in life can give gratification without meaning, but the days I do something like finish fixing the shower or putting up shelves, it does seem a bit easier to switch off at night. Cooking is the closest I get most days.
Don't be discouraged if it's not working right away. It has to work but you may need months of training before it starts to work well.
The hardest challenge is to be completely cold to the thoughts, no matter how important they may seem, ignore them and return to counting the breaths.
I've tried counting back from 5000 slowly on each breath. I got very far before stopping. Going backwards with large numbers is meant to take a bit more mental energy I think.
I've done the 1-10 then reset, we did that in Yoga at the gym. I've also done the box breathing where you breathe in and out slowly but at different rates.
I think mostly it will come down to practice like anything else. I get upset if it doesn't work but I enjoy ruminating so it's hard to let go. In many of these techniques I was very relaxed but still had an attentive core stopping the sleep from happening.
"Me and ADHD"
https://news.ycombinator.com/item?id=25992390
I'm not diagnosing you or anything but that thread put me on to the fact I might have that type of wiring too ... it would explain a lot ( for me ) and possibly some of that ^ ?
[ EDIT ] Anxiety is hands down my primary problem, but sometimes some issues can obfuscate others, and cause deductive fallacies by encouraging you to explain everything away under the umbrella of that first cause.
My mum also had insomnia for years. She did get some sleeping pills as a twice a week "reset" and I think a lot of mental issues are exacerbated by poor sleep leading to weak willpower so that could help. But she said the ADHD meds helped a lot with her sleep too.
This is like someone who doesn't have depression telling someone with depression to 'read a good book' ;)
Meditation is effectively what the article is talking about. Not following those thought avenue's.
Reducing artificial light after sundown and cutting out caffeine and listening to slow placed instrumental music (key being that there are no vocals/lyrics for the brain to attach to) has fixed my "awake brain" issue. Try to also avoid screen time before and in bed as each "new" thing you see (lets say, hackernews post titles) will activate your brain for a few minutes and keep it awake long after it has seen them. I use a single tea candle and switch off all other sources of light - it emits more than enough light! Basically you want to try to limit any inputs and have a consistent winding down ritual.
After a while the brain will accept this new ritual and will immediately get into a relaxed state if the lights go down and it hears a certain pace of music. For me, hang drum + flute music is the best, followed by calm asian instruments. When you listen to the same calm playlist over and over at bedtime, eventually you will know them. When you get to this stage, the brain will get bored with them at first - then you can double down and hum with the song (if you live alone of coarse). Try not to "talk" to yourself internally during your winding down ritual as it will keep the brain awake (don't try to convince the brain it is time to sleep, it will resist and fight back with more awareness).
Do this for 2 weeks and see if your night ritual improves. Obviously adjust it for your circumstances. Make it your special me-time. Choose the music to be unique and don't play those songs at other times. If you brain really wants to have a chat, ask it how it's been, tell it you are thankful for it and you will see it again in the morning! Make it your friend, don't fight it. I mean it. Be gentle and caring towards it, like you are it's father/mother, accept it for what it is.
The whole ritual last from 20 to 60 minutes, depending on how tired you are and how much conflict you accumulated during the day (conflicts being work frustrations, traffic frustrations, sexual frustration, doing things against your ideal solution, politics/news). These things all put pressure on the internal state and most people relieve those pressures either by sexual release, exercise or drugs/alcohol/food before bed. Having a self care ritual is another option. Sleep will do most of the work for us, but sleep is so much better if is not forced but rather gently simmered in. Fresh bed sheets & hot shower can also affect how comfortable the bed feels. A good mattress also helps. Natural smells also helps, so a drop of rose geranium, rubbed on your skin. Again, if you pick a specific flavour/smell to make part of your ritual, use it only during your winding time ritual and not during the day. You want to let the brain associate the music and smells and other environmental factors with "time to relax mode". Be consistent.
The thing that disturbs my peace the easiest is caffeine, even a single cup in the morning can effect my night time ritual.
Anyway. The above kind of what works for me, it's not medical/scientific based, hope it helps. Self care goes a long way!
For the rest - every time I read something like this I think about my wife, for two reasons.
One - any ritual will force her to do the same and play along but that might be asking a lot if its against her natural inclinations. She tends to want to stay up late (like me) and gets some of her best work done then. If I go to bed alone, then if she is still in the house (so I know she is coming along later) that anticipation alone is usually enough to keep me awake.
Two - she doesn't need any of that. Why can she fall asleep in 5 minutes after reading her phone in bed and breaking every single rule people give me? It always bothers me and I get very jealous :)
From fiction, I might refer to the Wheel of Time series where Rand Al'Thor uses a trick to "find the void" in a way his father taught him. When you get overwhelmed from information and emotions, finding the void allows you to regain composure and think straight again.
It's from (2012)
I had a computer science exam where you are allowed to bring any text book/reference book you wish, and any notes you've written yourself. The exam was long and hard yes, but it was also thorough. One of the more enjoyable exams i recall taking - vs all of the other types.
I guess it has merit if cheating of that nature is a big problem.