It seems very odd that the article seems to be measuring the information content of specific tasks that the brain is doing or specific objects that it is perceiving. But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer.
When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative to each other. You also pick up the fact that it's a Rubik's cube and not a bird or a series of binary digits or English text. If an orange cat lunged at your Rubik's cube while you were studying it, you wouldn't process it as "face 3 has 4 red squares on the first row, then an orange diagonal with sharp claws", you'd process it as "fast moving sharp clawed orange cat attacking cube". Which implies that every time you loom at the cube you also notice that it's still a cube and not any of the millions of other objects you can recognize, adding many more bits of information.
Similarly, when you're typing English text, you're not just encoding information from your brain into English text, you're also deciding that this is the most relevant activity to keep doing at the moment, instead of doing math or going out for a walk. Not to mention the precise mechanical control of your muscles to achieve the requisite movements, which we're having significant trouble programming into a robot.
Has anyone here even read more than the title? The article literally explains that you perceive at a rate of 10^9 bits per second. But after filtering and preprocessing in the outer brain you are only left with about 10 bits per second for conscious processing for things like motor functions. Yes, you can see a Rubic's cube and perceive all sorts of facts about it and the environment at the same time. But try solving it with your hands while someone shows you a bunch of objects and asks you visual comprehension questions at the same time. You might still perceive those other objects, but consciously classifying them verbally is gonna be really difficult. It's no surprise that the feature space that your deeper brain can actively work on is quite limited.
Send a query to your favorite database. The database preprocesses hundreds of gigabytes of data, map-reduces it, and finally returns a single 32-bit integer, taking exactly 3.2 seconds to do so.
Nobody would say that the database can only process 10 bits per second. The query just happened to ask for a very simplified answer.
On the other hand, the amount of actual entropic "information" that is processed when you identify a Rubik's cube as such may be nowhere near as much as you think it is, and most importantly, 10 bits may be nowhere near as little as you think it is.
If we use your example, which is that of identifying an object, we may simply ask the entropy of what the distribution of possible objects-to-be-identified is at t=0, prior to any analysis. Saying we can resolve 10 bits of this entropy per second is equivalent to saying that we can identify one object from a uniform distribution of 1024 per second. Let's suppose this is a low estimate by several orders of magnitude, and that it's really one from a billion objects instead that you can identify per second. Then this would still only be about 30 bits/sec.
None of this changes the main thesis of the paper, which is that this is much lower than the 10⁹ bits/sec our sensory systems transmit.
But you don't just perceive an object's category (like "cat"). We also perceive a high amount of detail about the object - colour, pattern, behaviour, we make comparisons to past behaviour, predictions of what's likely to happen next and so on.
Sure, some parts of the brain don't receive all that detail, but that's necessary for abstraction. If you pumped all the sensory data everywhere, the brain would get overwhelmed for no reason.
> deciding that this is the most relevant activity to keep doing at the moment, instead of doing math or going out for a walk.
How many bits of actual decision is going on here, as compared to the period of time that decision applies to?
For example, if a person decided once per second whether or not to go for a walk, that could be 1 bit per second. But if that person is constantly transitioning back and forth between walking and not-walking, we could consider their behavior pathological. Instead, for most people, the information density of these decisions is quite low, i.e. the per-second decision bits are very compressible.
Personally, I only decide whether to go for a walk (or not) _at most_ once every few minutes. Even if we add in bits for "where" and "precisely when" and "how long", I think we're still at just a small fraction of 1 bit per second.
Your conscious planner may be making 1 decision/second, but your senses, proprioception, balancing system, etc. are handing you about a gigabit/second of data, most of which never rises to your conscious attention.
When I'm reading, that's roughly 2000 bits/second, but I am engaging it with model-making systems that can completely enrapture me.
I/O is not the same as computation; conscious computation is not the same as all computation.
There is a whole "OS" in the background that is way more complex than all of our programmed systems. I may be hyped focusing on a complex program and really not think about anything else, but my body is still constantly senses, processes, and reacts to signals. E.g. I'm not ragdoll falling to the ground, which requires holding balance and tone, which is far from trivial. I also constantly listening to every possible danger, which would immediately override my current activity. I also react to internal signals like hunger or pain.
A bit dumb, but maybe relevant comparison might be asking why can an Apple Watch stay on for a single day only on a charge, while Garmin can do 2 weeks/a month? Because one is a general purpose computer, while the other is an embedded software that can only ever do that few things it is preprogrammed to do.
The vast majority of the brain's processing power, including conceptual processing, is not conscious. Conscious thought is the framebuffer of the mind, subconscious is the RAM.
> Personally, I only decide whether to go for a walk (or not) _at most_ once every few minutes.
And yet, if I walked into your office and shouted your name, you would have less than a second of reaction time, indicating that you are processing all sound around you and deciding whether to get up and talk to someone pretty much continuously.
Don’t forget typing in English is also outcome of processing all the context.
Also not so conscious context like „I am writing a reply on HN not on Reddit - making obvious silly pun about cats is not going to make upvotes - making quasi intellectual comment about subconscious processing should earn me some”.
Otoh, I just spent a minute to even comprehend your idea. We are living in the information era, the era of dodging an orange cat has ended. And we suck at this new thing, really. We’re like slow af emulators of logic programs and tend to fallback to animal mode when tired or clueless.
Do you really think that the fundamental human act of the current era is running logic programs? I don't dispute that digital logic and its uses deeply changed humanity, but this is like calling humans "weak af imitations of steam engines" during the late-1800s railroad boom.
Calling actions/thoughts that follow a concrete list of logical rules/algorithms "animal mode" is deeply anti-human.
Could the 10bits be some sort of instruction-set-address-width?
Meaning that while you are solving the rubik‘s cube, your brain has the capacity of switching into one of 2^10=1024 possible next states, which might fire nerve impulses for the body and also come with a new set of 1024 next states?
When focusing on the cube about 20 or 30 of the next states could be related to the cube, while the rest is concerned with bodily sensations, other thoughts etc.
What if learning something well meant that less states need to be reserved for the task/cube at hand because the decision tree going forward has become more efficiently packed, so more states are open for something else.
THe 10 bits is just very obviously made-up nonsense. The brain doesn’t operate on anything like a binary system, sensory data is not digitally encoded - it is all continuous data. Even the abstract kind of answers its own question. It says sensory data is something like the order of 10^9 bits per second but the “information throughput” is 10 bits per second. It asks “why do we need billions of neurons to process 10 bits per second?” Well we clearly don’t because that’s clearly not what we do. We process 10^9 bits per second, and we do a bunch of things which aren’t just processing sensory data as well. And on top of that we do things which aren’t measured in the 10 bits per second also.
> But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer.
Look, I like the sentence. I like it so much I might steal it. But, that said I think analogising what the brain does to "general purpose computer" is a bit awry because we can invent algorithms, and given a process we can implement algorithms but I still tend to think there is something ineffable about thought/cognition which is NOT general purpose computation. It's a heuristic. I dunno what I think.
The computer analogy is definitely off. For one thing, rationality is kind of bolted on to our linguistic ability. It's not deeply seated in our brain; it takes substantial energy to engage in logical reasoning and we're not that great at it; even very skilled mathematicians make trivial mistakes.
Deeper in our brain is reasoning about causality. But even that is more of an input to our reactions and emotions than it is as a consciously accessible function, and it too is often wrong.
As you said, it's a heuristic. We certainly can't solve the halting problem with brain magic.
Well and not only that, your brain is at the same time processing incoming sounds which it spatializes based on the minor phase/level differences and based onnreflections within the ears themselves. Your brain processes all feelings for all body parts, including a big chunk of skin, temperature perception, sense of gravity to balance yourself, etc.
If you received 10 bits once a second describing all of that, good luck, you probably won't survive the next 15 minutes.
The PhD student writing this could be excused for being young an inexperienced but their advisor, tagged second on the paper, should have cut this off at the pass.
> If the questions are properly designed, each will reveal 1 bit
of information about the mystery thing. If the guesser wins routinely, this suggests that the thinker can access about
2^20 ≈ 1 million possible items in the few seconds allotted. So the speed of thinking – with no constraints imposed –
corresponds to 20 bits of information over a few seconds: a rate of 10 bits per second or less.
This is an extrinsic definition of "information" which is task relative, and has little to do with any intrinsic processing rate (if such a thing can even be defined for the imagination).
The question of why does biological hardware capable of very high "intrinsic rates" deliver problem solving at "very low extrinsic rates" seems quite trivial. Its even a non-sequitur to compare them: properties of the parts are not properties of wholes. "Why does a gas move at 1 m/s, when its molecules move at 1000s m/s..."
All the 'intrinsic processing' of intelligence is concerned with deploying a very large array of cognitive skills (imagination, coordination, planning, etc.) that are fully general. Any given task has requires all of those top be in operation, and so we expect a much slower rate of 'extrinsic information processing'.
Consider how foolish the paper is to compare the intrinsic processing of a wifi network with the extrinsic task-specific processing of a human: it is likewise the case that if we set a computer the challenge of coordinating the solution of a task (eg., involving several LLMs) across a network, it's task-specific performance would drop off a cliff -- having a much slower 'solution rate' than 10bit/second.
These 'task-specific bits' represent a vast amount of processing work to solve a problem. And are at least as much to do with the problem, than the system solving it.
It seems to me all this paper does is define tasks in a highly abstract way that imposes a uniform cost to process '1 bit of task information'. Do the same for computers, and you'd likewise find tiny bitrates. The rate at which a problem is solved is 'one part of that problem per second' for a suitable definiton of 'part'
Another relevant point is the common anecdote about, eg, some master engineer who gets paid big bucks to come fix some problem that's been blocking up a factory for weeks. The engineer walks around, listens to a few of the machines, and then walks up to one of the machines and knocks it with his elbow Fonzi style and the factory starts running again. The factory boss is happy his factory is working, but annoyed that he paid so much for such an "easy" solution.
Ie, the amount of input and processing required to produce the "right" 10 bits might be far larger than 10 bits. Another obvious example is chess. The amount of bits conveyed by each move is small but, if you want to make the right move, you should probably put some deeper thought into it.
Humans are essentially organisms that collect and filter information, boil it down to a condensed soup of understanding, and emit a light sprinkle of carefully chosen bits intended to reshape the future towards their desires.
Exactly. English text is thought to have about 10 bits per word of information content, yet you can read much more quickly than 1 word per second. That includes not just ingesting the word, but also comprehending the meaning the author is conveying and your own reflections on those words.
Seems like this 10 number comes out of the kind of research where the objective isn’t to find the truth, but to come up with an answer that is headline grabbing. It’s the scientific equivalent of clickbait.
This type of comment is my least favorite on HN. "Seems quite trivial," "non-sequitur to compare them," "foolish." I am not able to read the paper as I do not have access, but the published perspective has 131 citations which seem to consider everything from task-specific human abilities, to cortical processing speeds, to perception and limb movements and eye movements, and so on.
I'm glad you thought about it too, but to assume that the authors are just silly and don't understand the problem space is really not a good contribution to conversation.
(Disclosure: I’m a former academic with more than a handful of papers to my name)
The parent comment is harshly criticizing (fairly, in my view) a paper, and not the authors. Smart people can write foolish things (ask me how I know). It’s good, actually, to call out foolishness, especially in a concrete way as the parent comment does. We do ourselves no favors by being unkind to each other. But we also do ourselves no favors by being unnecessarily kind to bad work. It’s important to keep perspective.
> Why can we only think about one thing at a time?
Maybe this is just a perception thing. Sure, you can only really keep up one stream of thought, visualization or inner dialogue (whatever you want to call it) at a time, but perhaps that's because we learn all our lives that direct communication is a one-channel, linear thing--speaking and listening focused on one topic at a time. Our brain does plenty of thinking in the background that leads to "a-ha!" moments even when the direct focus of our thoughts isn't on that topic. What if the mind could maintain multiple threads of thoughts at once, but our language coerces our thought patterns into being linear and non-concurrent?
As someone without an inner monologue, and someone that's spent a LOT of time meditating, it's not the language. It's the attention mechanisms themselves.
Buddhist scholars insist that while we can have multiple threads of attention in our awareness, like strings with pearls of experience/thoughts we can only actually hold one little pearl of information from that stream in our attention at a time, and that we flit between them quite rapidly.
Personally, I sort of agree, but I notice that there seems to be a time-compression thing happening where the pearl delivered to attention can contain a compressed summary of continuous perception. This seems to work for 2 things at once in awareness. When you start monitoring 3+ streams, there are gaps. And even maintaining the 2 streams continuously is exhausting so the mind tends to relax a little and leave gaps on a normal basis, but it seems like it can monitor dual feeds when its particularly important.
My understanding is that neuroscience largely seems to agree with the above.
(Actually, I'll note that the USUAL mode of being doesn't even monitor one stream continuously. A lot of the weird effects (and deeply interesting ones!) they talk about in meditative arts seem to pop up when you progress to being able to hold truly continuous attention.)
What you're describing here is software, not hardware—Cognitive Science is the relevant field, not Neuroscience.
That said, your understanding is largely supported by our current understanding of consciousness, attention, and perception. The attention mechanism doesn't handle parallel processing well—but can operate "multi-threaded", where it juggles several foci at once (with some obvious cost to switching between them). But I think its a mistake to assume that decision making has to be done within this attention context. While we may only be aware of a single thread at any given time, the brain is doing a lot of parallel processing. We can only focus our attention on a single cognitive task, but that doesn't mean we're not actively performing many others.
Sometimes I'll be deeply thinking about something while driving, and discover I'm at the last road to my house without remembering having driven the previous few blocks. It's quite disturbing. When I say deeply thinking I don't mean anything involving phones or external stimuli - really just thinking about a particular problem I'm working on. I also don't deliberately engage this deep mode of thought, I just sort of slide into it naturally.
Does anyone else have this happen? I don't think my driving is suffering, but it's hard to really honestly say?
I don't know what ever became of the line of research, but there was a very interesting book I read decades ago called Descartes' Error by Antonio Damasio that examined case studies of patients who had their corpus collosum severed, resulting in a split brain. You could show their left and right eyes different images and ask them what they saw and they would write and speak different answers, because speech and writing are controlled by different brain hemispheres.
This seems to suggest that any bottleneck in conscious attention is not an inherent limitation of an animal brain but rather a consensus mechanism we've developed to keep our chain of experience coherent. If we get rid of the constraint that all of our external communication channels need to present the same narrative, we can seemingly process more information even when it requires being a conscious center of attention.
It's like UIs being single-threaded, because otherwise you would have chaos if several background threads are trying to update the UI at the same time.
Also I do not agree with the premise that we can only think about one thing at a time.
We routinely communicate with multiple people at once and also communicate with the same persons in multiple threads of conversations.
Of cause this means that we switch between those tasks and do not really do them in parallel. At most we listen to one person, answer a second via speech, a third via text while thinking about what to respond to a fourth
This is the part that bothers me. I can definitely think of multiple things at a time. It really just depends on the complexity of the tasks. I can listen to and process and audiobook while driving to work every morning, for instance. I definitely can have multiple thoughts in parallel. I remember when I used to recite prayers, I would be reciting the memorized prayer while thinking about other things. Both things were happening at the same time. The memorized task takes less processing power, but it still requires some thought to execute.
We think about many things at a time. But for those with malfunctioning brains that have the internal monologue going constantly, they mistaken that monologue for their thoughts and so it must be "one thing at a time". The language they experience their monologue in is by its very nature, sequential, you can't speak or even hear/understand two parallel streams of speech.
>Our brain does plenty of thinking in the background that leads to "a-ha!" moments even
That's not "in the background". That's the real you, your real mind. That's the foreground. But, if your brain malfunctions as many do, then the monologue shows up and crowds out everything. Sometimes it is apparently loud enough that it even prevents those "a-ha!" moments.
>but our language coerces our thought patterns into being linear and non-concurrent?
The language should just be discarded. What you want is an internal silence.
I wouldn’t say it’s language so much as unnecessarily added language. Words and sentences can appear and be useful, but there is a lot of mental activity that is not essential but added on responses to things. I wouldn’t say a component that generates comments is a broken brain, it believing the comments or the beliefs embedded inside them can break your contentedness.
Heck, I can type way faster than 10 bits per second, even after gzipping the output.
And when I consider the amount of sensory information that I consciously process (not that comes in, but that I conceptually analyze), it's got to be way higher.
> Quick, think of a thing... Now I’ll guess that thing by asking you yes/no questions.” The game ‘Twenty Questions’ has been popular for centuries1as a thinking challenge. If the questions are properly designed, each will reveal 1 bit of information about the mystery thing. If the guesser wins routinely, this suggests that the thinker can access about 220≈ 1 million possible items in the few seconds allotted. So the speed of thinking – with no constraints imposed – corresponds to 20 bits of information over a few seconds: a rate of 10 bits per second or less.
As the answerer, if you have a wide vocabulary or if you're a technical person then it's not too difficult to routinely choose words the other person simply does not know so that no amount of yes/no questions will get them there.
Obscure medical terms (phlebotomy), names of uncommonly-known stars (Fomalhaut), obscure data structures (cache-oblivious lookahead arrays), mathematical constants (Feigenbaum's constants)... The list goes on and on!
The point I'm trying to make is that most people who play Twenty Questions aren't trying to maximize the number of bits per second in their answer. They're actually trying to play semi-cooperatively. The fun part of Twenty Questions is when the other person guesses your word with as few questions remaining as possible. Having them get all the way to 20 and then you tell them "no you were way off to guess toothache, it was actually temporomandibular joint dysfunction" makes you look rather unsporting!
Thus, since I think we can expect people who play Twenty Questions to actually try to choose a word they know the other person can guess within the space allowed, we can reasonably conclude that using the game as a way to establish some sort of rough constraint on the speed of thinking (in bits per second) is way off. In fact, I know from my own experience playing the game that I will think of and discard many words in a short time as I try to find one that will be in the sweet spot of challenge for the other person to guess.
So, in the context of random word lookup with filter for things, we have a latency of a few seconds and a total selection of 20 bits.
Meanwhile the machinery in understanding that it is a game, processing the audio input of the question, producing the output of the answer is all taken for granted.
If the questions were pre-determined, which they're usually not. Reminds me of Huffman coding and the reason that compression challenges measure submissions looking at artifacts required to run them in addition to compressed size. I tend to agree with OP that this doesn't pass the smell test
English is about one bit per letter. If you type at a very fast 120WPM then you’re right at 10bps. Computers just don’t represent English very efficiently, even with gzip.
Even very fast typists are unable to do stenography without a machine specialized to the task. Speech, in turn, can usually be understood at two or even three times the rate at which it is ordinarily produced. Meanwhile, I can read several times faster than I can understand speech, even at the highest speedup which I find coherent.
Ergo, 10 bits per second just doesn't hold up. It's an interesting coincidence that a reasonably fast typing speed hits that rate, but humans routinely operate on language at multiples of it.
These letters are jointly distributed, and the entropy of the joint distribution of a second of "plausible" English text is much lower than the naive sum of the marginal entropies of each letter. In fact, with LLMs that report the exact probability distribution of each token, it is now possible to get a pretty decent estimate of what the entropy of larger segments of English text actually is.
I was iterating over the different citations for bitrate, at least some of them, like Starcraft and the Rubik's cube, are literally a Guinness Book of Records that's a tiny blurb about APMs and a video of a guy solving the rubik's cube.
Going from APM and/or image wiggling to "bits per second" is .... hilariously reductive and I struggle to consider this response to be woefully incomplete at convincing this reader.
And yeah, my immediate response to reading the title was "where the hell are they getting that number", so I have gone and looked and am unsatisfied.
It seems weird to me. They say 10/bits/sec "behavioral throughput."
Have they not seen a football match? The brain controls 600 or so muscles in a rapid manner. That alone must be a lot of bits per second, certainly far better than computer controlled robots.
Re
>Why does the brain need billions of neurons to process 10
bits/s?
Tesla's FSD cars have a lot of processing power but still struggle not to drive into fire trucks. You probably need a lot.
I beg you, please read the paper before commenting. It's very interesting and it answers a lot of questions that might arise from just skimming the title.
Only 10 beatings per second? This is a just universe, Sir!
On a serious note, enlightenment only cures us of our selfish curiosity, i.e. any action which causes harm to others. The Way requires us to harmonize with universal compassion, so there is take and give (especially with regard to our required sustenance), but we definitely lose our propensity to experiment with our power at the expense of others. No, we are to increase our curiosity in how we can better help others, help being the cornerstone of compassion.
I don't need to read the paper. The problem is that mechanical systems have inertia and are limited in their ability to change direction and thereby their ability to signal discrete information.
At what speed is the brain processing when a pair of Chinese ping pong masters are playing? All of this requires seeing the ball coming from the opponent, subconsciously calculating its speed, angle, and rotation to decide how to move your body, legs, arms, hands, and even fingers to hit the ball at the desired location, with the desired angle, and at the desired speed. A standard ping pong table is 2.74 meters long, and in high-speed professional games, the ball can traverse this distance in as little as 90–140 milliseconds. This means players have less than a tenth of a second to react. It's not just the immense image-processing speed required but also the mental speed to transform that visual data into precise positioning and movement of hundreds of muscles in less than a second. Ten bits per second? No way.
The only time 'bits' will ever be an appropriate measure of human processing is when we are processing or producing diginal information artifacts, e.g. a typewritten document.
Our bodies' systems are biochemical wetware that will never be aptly described using a boolean basis. That is one of the primary problems of society's obsessions with classical notions of gender.
No one is male OR female. We are, every single one of us, a combination of male and female hormones. The more "male" a person is is the result of that balance favoring the male hormones; and vice versa. What humanity is now struggling with is that there are plenty of folks with lots of both or little of either and all kinds of combinations.
Of course, my not being a biochemist means my categorization of hormones into "male" and "female" is, itself, likely to be a poorly booleanized representation of their natures.
We are much more akin to Boltzmann's statistical mechanics description of reality, than to digital logic's boolean description.
Bits are a perfectly acceptable way to measure biological information processing. These are not the boolean logic digital bits like on a computer. They're the more abstract concept of a bit in information theory.
Take the number of distinct possible configurations a system can be in (accounting for statistical uncertainty/biases if needed), take the base 2 logarithm of that number, and you have the bits of information in the system. This can be applied to basically anything, biological or otherwise.
> Take the number of distinct possible configurations a system
Easy for an isolated system. Human body is 6000 billion cells, each of them has many possible configurations, most of them share and process informations. I respectfully doubt there’s much to do with bits outside of a tiny bit if flesh in a petri dish.
> That is one of the primary problems of society's obsessions with classical notions of gender.
What you go on to discuss is sex, and sexual dimorphism, which is a remarkably robust way of classification. The "classical" notions of gender (tbh, "classical" doesn't make much sense here) as sex based is fairly reasonable all things considered. Consider the arguments presented in this essay [0]. That, however, doesn't really mean much for how we should treaty people in public who desire to express their gender in different ways, which is, of course, respecting of their dignity and desires, in most cases.
Yeah, what I mean by classical would boil down to just genitalia, which doesn't really hold up in how we must respect the person and how they feel and choose to express themselves. Yes, so long as their expressions are not harming others, then we must respect their human right to choose who they are.
I've got to give a huge hat tip to Suzi (Eddie) Izzard, who -- beyond their being just a brilliant comic and generally good human being -- taught me and my fam about how the spectrum of human configuration is way more complex than just male and female.
The use of "bits" here doesn't mean we are working in binary.
It is more like the way it is used in information theory. The number of bits is log2 of the number of states that can be represented, and it doesn't have to be an integer. For example, with 10 bits of information, we can distinguish between 1024 different states, it can be 1024 colors for instance, or 1024 genders if you wish, it doesn't matter, the important part is that there are 1024 boxes to put things in, no matter what they are. Of course, it doesn't mean that only 1024 colors exist in the universe, there are an infinity of them, but with 10 bits, you can only distinguish between 1024 of them. If you want more, you need more bits, if you can do with less, you need less.
By the article results, it means your "inner brain" can process one color with 1024 nuances per second, or 2 independent colors with 16 nuances each per second. If the colors are not independent, it can process more, because, if, say, you know that the two color are highly contrasting, you don't have to allocate "boxes" for noncontrasting colors, may free some boxes for more nuances, so, you may, for instance, process two contrasting colors with 100 nuances each with these 10 bits.
A bit is the fundamental unit of information theory, and has nothing to do with digital logic in this context. No one is saying "ok use one bit to encode male or female". No one is saying "ok 101 means serotonin, and 110 is dopamine". What they are saying is that the information content produced by a human being can be compressed down to about 10 bits per second, but this is a statistical description.
I think you've mixed up a few mostly unrelated things together to make a point. You're correct in that the larger point to be made is that analog and digital computing are paradigmatically distinct and analogies are hard to draw across that divide.
However, "bits" is just a quantity of information in a certain base. We could discuss it in "nits" if you prefer. The point is that information per se remains real even if the specific representation is based on some assumption of digital computing.
The rest of your comment is unfortunately out of scope of this article although it deserves some discussion on its own merit.
Boolean logic extends just fine to handle complexity. Instead it’s the intuitive classification people come up with that are often a poor fit for reality.
Is someone’s DNA consistent throughout their body? Y/N
Does someone have any chromosomal anomalies? Y/N etc
Similarly it’s very possible for a girl to suffer from abnormally low testosterone levels which doesn’t fit with how the public thinks of it as a gendered hormone. During puberty it normally spikes in both girls and boys. From a range of (2.5 - 10) in prepubescents, the typical range in puberty for boys is much higher (100 - 970) vs (15 - 38) but that doesn’t make it a male hormone just a pathway used differently.
In biology, or really most sciences (math being an exception), the more closely you examine a delineated this or that categorization, the more you realize it's a scale, a range, or something fuzzy.
Like even things we talk about regularly like touch and space is vague in the details. Is it still touching if the repulsive force of electron to electron is keeping nucleus apart? Where is empty space begin and an atom end? Is it after the electron shell? Outside of it's repulsive force? Some hybrid value?
I can always be awestruckdumb by the understanding that we are all mostly space inhabited by fields, our littlest bits vibrating at mindblowing speeds.
What you're saying is interesting but I think the causality is backwards here and I can provide some examples to show why.
(By male hormone I'm assuming you mean testosterone, and by female hormone I assume you mean oestrogen.)
i in fact
If being "more male" came from having more testosterone (and vice versa), then logically when children go through puberty and develop into adults, they would become "more" male or "more" female.
As adults become elderly and naturally produce less sex-associated hormones, they would become "less" male or female.
(Fetuses do not all begin in the womb as female, that's a common misunderstanding. We start off physically undifferentiated, and develop along a genetically predetermined pathway as we grow. Some animals use temperature or other environmental triggers to pick, humans use genes.)
Would that mean a male bodybuilder who injects testosterone is more male than a man that doesn't? His phenotype may become visibly more masculine, but that doesn't change his sex at all. Same for a female bodybuilder that injects testosterone - she may develop stereotypically male physical characteristics like large muscles and a deeper voice, but her sex is unaffected.
The causality is the other way: being male - or - female results in a physiology (adult testicles/ovaries) that produces sex associated hormones in larger or lesser degrees depending on the person (and in some cases very low amounts or not at all).
This makes sense if sex is a binary (with rare differences of sex development - detailed here https://www.theparadoxinstitute.com/read/sex-development-cha... ) that results in different levels of sex hormones in the body and resulting phenotype. So yes, everyone is male or female.
(I'm not referring to gender here - I'm talking only about sex)
If there's a spectrum then some men could be biologically "more male" than others and vice versa for women. I've not seen any evidence of this myself, but I'm happy to be proven wrong!
it is a categorization, like all things in biology. One of the most robust and significant ones for all of life is sexual versus asexual reproduction. It is intentionally blurring understanding to say that it is not a binary. This is not a gaussian situation, and not fitting into this categorization is exceedingly rare due to defect/mutation which largely does not proliferate genetically.
Readit News
When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative to each other. You also pick up the fact that it's a Rubik's cube and not a bird or a series of binary digits or English text. If an orange cat lunged at your Rubik's cube while you were studying it, you wouldn't process it as "face 3 has 4 red squares on the first row, then an orange diagonal with sharp claws", you'd process it as "fast moving sharp clawed orange cat attacking cube". Which implies that every time you loom at the cube you also notice that it's still a cube and not any of the millions of other objects you can recognize, adding many more bits of information.
Similarly, when you're typing English text, you're not just encoding information from your brain into English text, you're also deciding that this is the most relevant activity to keep doing at the moment, instead of doing math or going out for a walk. Not to mention the precise mechanical control of your muscles to achieve the requisite movements, which we're having significant trouble programming into a robot.
Nobody would say that the database can only process 10 bits per second. The query just happened to ask for a very simplified answer.
If we use your example, which is that of identifying an object, we may simply ask the entropy of what the distribution of possible objects-to-be-identified is at t=0, prior to any analysis. Saying we can resolve 10 bits of this entropy per second is equivalent to saying that we can identify one object from a uniform distribution of 1024 per second. Let's suppose this is a low estimate by several orders of magnitude, and that it's really one from a billion objects instead that you can identify per second. Then this would still only be about 30 bits/sec.
None of this changes the main thesis of the paper, which is that this is much lower than the 10⁹ bits/sec our sensory systems transmit.
Sure, some parts of the brain don't receive all that detail, but that's necessary for abstraction. If you pumped all the sensory data everywhere, the brain would get overwhelmed for no reason.
How many bits of actual decision is going on here, as compared to the period of time that decision applies to?
For example, if a person decided once per second whether or not to go for a walk, that could be 1 bit per second. But if that person is constantly transitioning back and forth between walking and not-walking, we could consider their behavior pathological. Instead, for most people, the information density of these decisions is quite low, i.e. the per-second decision bits are very compressible.
Personally, I only decide whether to go for a walk (or not) _at most_ once every few minutes. Even if we add in bits for "where" and "precisely when" and "how long", I think we're still at just a small fraction of 1 bit per second.
When I'm reading, that's roughly 2000 bits/second, but I am engaging it with model-making systems that can completely enrapture me.
I/O is not the same as computation; conscious computation is not the same as all computation.
A bit dumb, but maybe relevant comparison might be asking why can an Apple Watch stay on for a single day only on a charge, while Garmin can do 2 weeks/a month? Because one is a general purpose computer, while the other is an embedded software that can only ever do that few things it is preprogrammed to do.
Consciously. Subconsciously much more is going on.
And yet, if I walked into your office and shouted your name, you would have less than a second of reaction time, indicating that you are processing all sound around you and deciding whether to get up and talk to someone pretty much continuously.
Also not so conscious context like „I am writing a reply on HN not on Reddit - making obvious silly pun about cats is not going to make upvotes - making quasi intellectual comment about subconscious processing should earn me some”.
Calling actions/thoughts that follow a concrete list of logical rules/algorithms "animal mode" is deeply anti-human.
Meaning that while you are solving the rubik‘s cube, your brain has the capacity of switching into one of 2^10=1024 possible next states, which might fire nerve impulses for the body and also come with a new set of 1024 next states?
When focusing on the cube about 20 or 30 of the next states could be related to the cube, while the rest is concerned with bodily sensations, other thoughts etc.
What if learning something well meant that less states need to be reserved for the task/cube at hand because the decision tree going forward has become more efficiently packed, so more states are open for something else.
Look, I like the sentence. I like it so much I might steal it. But, that said I think analogising what the brain does to "general purpose computer" is a bit awry because we can invent algorithms, and given a process we can implement algorithms but I still tend to think there is something ineffable about thought/cognition which is NOT general purpose computation. It's a heuristic. I dunno what I think.
Deeper in our brain is reasoning about causality. But even that is more of an input to our reactions and emotions than it is as a consciously accessible function, and it too is often wrong.
As you said, it's a heuristic. We certainly can't solve the halting problem with brain magic.
But stuff I am good at? I don’t see it at all. A terminal? I never have any truly tangible conscious recollection of serious coding.
It might be the same for people good at Rubik’s Cube.
If you received 10 bits once a second describing all of that, good luck, you probably won't survive the next 15 minutes.
Deleted Comment
This is an extrinsic definition of "information" which is task relative, and has little to do with any intrinsic processing rate (if such a thing can even be defined for the imagination).
The question of why does biological hardware capable of very high "intrinsic rates" deliver problem solving at "very low extrinsic rates" seems quite trivial. Its even a non-sequitur to compare them: properties of the parts are not properties of wholes. "Why does a gas move at 1 m/s, when its molecules move at 1000s m/s..."
All the 'intrinsic processing' of intelligence is concerned with deploying a very large array of cognitive skills (imagination, coordination, planning, etc.) that are fully general. Any given task has requires all of those top be in operation, and so we expect a much slower rate of 'extrinsic information processing'.
Consider how foolish the paper is to compare the intrinsic processing of a wifi network with the extrinsic task-specific processing of a human: it is likewise the case that if we set a computer the challenge of coordinating the solution of a task (eg., involving several LLMs) across a network, it's task-specific performance would drop off a cliff -- having a much slower 'solution rate' than 10bit/second.
These 'task-specific bits' represent a vast amount of processing work to solve a problem. And are at least as much to do with the problem, than the system solving it.
It seems to me all this paper does is define tasks in a highly abstract way that imposes a uniform cost to process '1 bit of task information'. Do the same for computers, and you'd likewise find tiny bitrates. The rate at which a problem is solved is 'one part of that problem per second' for a suitable definiton of 'part'
Ie, the amount of input and processing required to produce the "right" 10 bits might be far larger than 10 bits. Another obvious example is chess. The amount of bits conveyed by each move is small but, if you want to make the right move, you should probably put some deeper thought into it.
Humans are essentially organisms that collect and filter information, boil it down to a condensed soup of understanding, and emit a light sprinkle of carefully chosen bits intended to reshape the future towards their desires.
Or another way of saying it is, the answer was right there all along, the hard part was filtering all the non-answer out.
Seems like this 10 number comes out of the kind of research where the objective isn’t to find the truth, but to come up with an answer that is headline grabbing. It’s the scientific equivalent of clickbait.
Too bad people fall for it.
I'm glad you thought about it too, but to assume that the authors are just silly and don't understand the problem space is really not a good contribution to conversation.
The parent comment is harshly criticizing (fairly, in my view) a paper, and not the authors. Smart people can write foolish things (ask me how I know). It’s good, actually, to call out foolishness, especially in a concrete way as the parent comment does. We do ourselves no favors by being unkind to each other. But we also do ourselves no favors by being unnecessarily kind to bad work. It’s important to keep perspective.
Dead Comment
Maybe this is just a perception thing. Sure, you can only really keep up one stream of thought, visualization or inner dialogue (whatever you want to call it) at a time, but perhaps that's because we learn all our lives that direct communication is a one-channel, linear thing--speaking and listening focused on one topic at a time. Our brain does plenty of thinking in the background that leads to "a-ha!" moments even when the direct focus of our thoughts isn't on that topic. What if the mind could maintain multiple threads of thoughts at once, but our language coerces our thought patterns into being linear and non-concurrent?
Buddhist scholars insist that while we can have multiple threads of attention in our awareness, like strings with pearls of experience/thoughts we can only actually hold one little pearl of information from that stream in our attention at a time, and that we flit between them quite rapidly.
Personally, I sort of agree, but I notice that there seems to be a time-compression thing happening where the pearl delivered to attention can contain a compressed summary of continuous perception. This seems to work for 2 things at once in awareness. When you start monitoring 3+ streams, there are gaps. And even maintaining the 2 streams continuously is exhausting so the mind tends to relax a little and leave gaps on a normal basis, but it seems like it can monitor dual feeds when its particularly important.
My understanding is that neuroscience largely seems to agree with the above.
(Actually, I'll note that the USUAL mode of being doesn't even monitor one stream continuously. A lot of the weird effects (and deeply interesting ones!) they talk about in meditative arts seem to pop up when you progress to being able to hold truly continuous attention.)
That said, your understanding is largely supported by our current understanding of consciousness, attention, and perception. The attention mechanism doesn't handle parallel processing well—but can operate "multi-threaded", where it juggles several foci at once (with some obvious cost to switching between them). But I think its a mistake to assume that decision making has to be done within this attention context. While we may only be aware of a single thread at any given time, the brain is doing a lot of parallel processing. We can only focus our attention on a single cognitive task, but that doesn't mean we're not actively performing many others.
Does anyone else have this happen? I don't think my driving is suffering, but it's hard to really honestly say?
This seems to suggest that any bottleneck in conscious attention is not an inherent limitation of an animal brain but rather a consensus mechanism we've developed to keep our chain of experience coherent. If we get rid of the constraint that all of our external communication channels need to present the same narrative, we can seemingly process more information even when it requires being a conscious center of attention.
We routinely communicate with multiple people at once and also communicate with the same persons in multiple threads of conversations.
Of cause this means that we switch between those tasks and do not really do them in parallel. At most we listen to one person, answer a second via speech, a third via text while thinking about what to respond to a fourth
We just switch our focus of attention quite fast
>Our brain does plenty of thinking in the background that leads to "a-ha!" moments even
That's not "in the background". That's the real you, your real mind. That's the foreground. But, if your brain malfunctions as many do, then the monologue shows up and crowds out everything. Sometimes it is apparently loud enough that it even prevents those "a-ha!" moments.
>but our language coerces our thought patterns into being linear and non-concurrent?
The language should just be discarded. What you want is an internal silence.
Heck, I can type way faster than 10 bits per second, even after gzipping the output.
And when I consider the amount of sensory information that I consciously process (not that comes in, but that I conceptually analyze), it's got to be way higher.
10 bits/s doesn't pass the smell test.
> Quick, think of a thing... Now I’ll guess that thing by asking you yes/no questions.” The game ‘Twenty Questions’ has been popular for centuries1as a thinking challenge. If the questions are properly designed, each will reveal 1 bit of information about the mystery thing. If the guesser wins routinely, this suggests that the thinker can access about 220≈ 1 million possible items in the few seconds allotted. So the speed of thinking – with no constraints imposed – corresponds to 20 bits of information over a few seconds: a rate of 10 bits per second or less.
Obscure medical terms (phlebotomy), names of uncommonly-known stars (Fomalhaut), obscure data structures (cache-oblivious lookahead arrays), mathematical constants (Feigenbaum's constants)... The list goes on and on!
The point I'm trying to make is that most people who play Twenty Questions aren't trying to maximize the number of bits per second in their answer. They're actually trying to play semi-cooperatively. The fun part of Twenty Questions is when the other person guesses your word with as few questions remaining as possible. Having them get all the way to 20 and then you tell them "no you were way off to guess toothache, it was actually temporomandibular joint dysfunction" makes you look rather unsporting!
Thus, since I think we can expect people who play Twenty Questions to actually try to choose a word they know the other person can guess within the space allowed, we can reasonably conclude that using the game as a way to establish some sort of rough constraint on the speed of thinking (in bits per second) is way off. In fact, I know from my own experience playing the game that I will think of and discard many words in a short time as I try to find one that will be in the sweet spot of challenge for the other person to guess.
Meanwhile the machinery in understanding that it is a game, processing the audio input of the question, producing the output of the answer is all taken for granted.
No wonder they came up with such an obviously nonsensical answer in the end.
Every time I play this game, I can only think of one thing: https://t3.ftcdn.net/jpg/02/07/37/42/500_F_207374213_kNgoMel...
So I guess that means I can only think at 1 bit per second.
Ergo, 10 bits per second just doesn't hold up. It's an interesting coincidence that a reasonably fast typing speed hits that rate, but humans routinely operate on language at multiples of it.
Where did you get that number from? How would you represent a letter using 1 bit?
* when whole sentences or paragraphs are considered.
Going from APM and/or image wiggling to "bits per second" is .... hilariously reductive and I struggle to consider this response to be woefully incomplete at convincing this reader.
And yeah, my immediate response to reading the title was "where the hell are they getting that number", so I have gone and looked and am unsatisfied.
Have they not seen a football match? The brain controls 600 or so muscles in a rapid manner. That alone must be a lot of bits per second, certainly far better than computer controlled robots.
Re
>Why does the brain need billions of neurons to process 10 bits/s?
Tesla's FSD cars have a lot of processing power but still struggle not to drive into fire trucks. You probably need a lot.
A plea to reason, that is probably not outside the posting guidelines, but is certainly in a gray area :-)
This is... not a recipe for a successful discussion between people who have read the paper.
This is why enlightenment cures you of your curiosity.
On a serious note, enlightenment only cures us of our selfish curiosity, i.e. any action which causes harm to others. The Way requires us to harmonize with universal compassion, so there is take and give (especially with regard to our required sustenance), but we definitely lose our propensity to experiment with our power at the expense of others. No, we are to increase our curiosity in how we can better help others, help being the cornerstone of compassion.
Deleted Comment
Our bodies' systems are biochemical wetware that will never be aptly described using a boolean basis. That is one of the primary problems of society's obsessions with classical notions of gender.
No one is male OR female. We are, every single one of us, a combination of male and female hormones. The more "male" a person is is the result of that balance favoring the male hormones; and vice versa. What humanity is now struggling with is that there are plenty of folks with lots of both or little of either and all kinds of combinations.
Of course, my not being a biochemist means my categorization of hormones into "male" and "female" is, itself, likely to be a poorly booleanized representation of their natures.
We are much more akin to Boltzmann's statistical mechanics description of reality, than to digital logic's boolean description.
Take the number of distinct possible configurations a system can be in (accounting for statistical uncertainty/biases if needed), take the base 2 logarithm of that number, and you have the bits of information in the system. This can be applied to basically anything, biological or otherwise.
Sounds like the statistics in the papers from the social "sciences".
"There's lies, damned lies, and statistics." --Unknown
I don't think you're going to be able to count the "number of distinct possible configurations" of an even moderately complex living system.
Easy for an isolated system. Human body is 6000 billion cells, each of them has many possible configurations, most of them share and process informations. I respectfully doubt there’s much to do with bits outside of a tiny bit if flesh in a petri dish.
What you go on to discuss is sex, and sexual dimorphism, which is a remarkably robust way of classification. The "classical" notions of gender (tbh, "classical" doesn't make much sense here) as sex based is fairly reasonable all things considered. Consider the arguments presented in this essay [0]. That, however, doesn't really mean much for how we should treaty people in public who desire to express their gender in different ways, which is, of course, respecting of their dignity and desires, in most cases.
[0]: https://philosophersmag.com/unexceptional-sex/
Yeah, what I mean by classical would boil down to just genitalia, which doesn't really hold up in how we must respect the person and how they feel and choose to express themselves. Yes, so long as their expressions are not harming others, then we must respect their human right to choose who they are.
I've got to give a huge hat tip to Suzi (Eddie) Izzard, who -- beyond their being just a brilliant comic and generally good human being -- taught me and my fam about how the spectrum of human configuration is way more complex than just male and female.
Cheers, friend.
It is more like the way it is used in information theory. The number of bits is log2 of the number of states that can be represented, and it doesn't have to be an integer. For example, with 10 bits of information, we can distinguish between 1024 different states, it can be 1024 colors for instance, or 1024 genders if you wish, it doesn't matter, the important part is that there are 1024 boxes to put things in, no matter what they are. Of course, it doesn't mean that only 1024 colors exist in the universe, there are an infinity of them, but with 10 bits, you can only distinguish between 1024 of them. If you want more, you need more bits, if you can do with less, you need less.
By the article results, it means your "inner brain" can process one color with 1024 nuances per second, or 2 independent colors with 16 nuances each per second. If the colors are not independent, it can process more, because, if, say, you know that the two color are highly contrasting, you don't have to allocate "boxes" for noncontrasting colors, may free some boxes for more nuances, so, you may, for instance, process two contrasting colors with 100 nuances each with these 10 bits.
Are you using an information theory that is based upon something different from Shannon's?
However, "bits" is just a quantity of information in a certain base. We could discuss it in "nits" if you prefer. The point is that information per se remains real even if the specific representation is based on some assumption of digital computing.
The rest of your comment is unfortunately out of scope of this article although it deserves some discussion on its own merit.
Is someone’s DNA consistent throughout their body? Y/N Does someone have any chromosomal anomalies? Y/N etc
Similarly it’s very possible for a girl to suffer from abnormally low testosterone levels which doesn’t fit with how the public thinks of it as a gendered hormone. During puberty it normally spikes in both girls and boys. From a range of (2.5 - 10) in prepubescents, the typical range in puberty for boys is much higher (100 - 970) vs (15 - 38) but that doesn’t make it a male hormone just a pathway used differently.
Even that is a very smooth view of humanity as if was all going through more or less the same mindset.
Rest assured that most of humanity don’t conceive their life experience according to a scientific measure of information units.
Like even things we talk about regularly like touch and space is vague in the details. Is it still touching if the repulsive force of electron to electron is keeping nucleus apart? Where is empty space begin and an atom end? Is it after the electron shell? Outside of it's repulsive force? Some hybrid value?
Also remember that putting a topic under mathematical form or mere layman prose is also a spectral arbitrary categorization.
Yeah, those are great questions, for sure.
I can always be awestruckdumb by the understanding that we are all mostly space inhabited by fields, our littlest bits vibrating at mindblowing speeds.
(By male hormone I'm assuming you mean testosterone, and by female hormone I assume you mean oestrogen.) i in fact If being "more male" came from having more testosterone (and vice versa), then logically when children go through puberty and develop into adults, they would become "more" male or "more" female.
As adults become elderly and naturally produce less sex-associated hormones, they would become "less" male or female.
(Fetuses do not all begin in the womb as female, that's a common misunderstanding. We start off physically undifferentiated, and develop along a genetically predetermined pathway as we grow. Some animals use temperature or other environmental triggers to pick, humans use genes.)
Would that mean a male bodybuilder who injects testosterone is more male than a man that doesn't? His phenotype may become visibly more masculine, but that doesn't change his sex at all. Same for a female bodybuilder that injects testosterone - she may develop stereotypically male physical characteristics like large muscles and a deeper voice, but her sex is unaffected.
The causality is the other way: being male - or - female results in a physiology (adult testicles/ovaries) that produces sex associated hormones in larger or lesser degrees depending on the person (and in some cases very low amounts or not at all).
This makes sense if sex is a binary (with rare differences of sex development - detailed here https://www.theparadoxinstitute.com/read/sex-development-cha... ) that results in different levels of sex hormones in the body and resulting phenotype. So yes, everyone is male or female.
(I'm not referring to gender here - I'm talking only about sex)
If there's a spectrum then some men could be biologically "more male" than others and vice versa for women. I've not seen any evidence of this myself, but I'm happy to be proven wrong!
All physical systems are described on a base-2 basis using bits, or shannon entropy.
Deleted Comment