The prediction algorithm is actually very straightforward[1]. It's a fun exercise to write a sequence generator specifically to defeat it. Here is a sequence that can get the prediction rate down to 13%:
You can improve that by around 13% if you soften the inequality (your > should be >=, I believe)
table = {}
s = 'ddddd'
for i in range(100):
if i%5 == 0:
print(s)
state = table.setdefault(s, {'f':0, 'd':0})
s = s[1:] + ('f' if state['f'] <= state['d'] > 0 else 'd')
state[s[-1]] += 1
>The prediction algorithm is actually very straightforward[1]. It's a fun exercise to write a sequence generator specifically to defeat it.
if you are inspecting the prediction algorithm, and you want to specifically defeat it, doesn't that mean copy the prediction algorithm and throw a "not" on it? It's the old halting problem trick...
From the Github description, a quote: "In a class I taught at Berkeley, I did an experiment where I wrote a simple little program that would let people type either “f” or “d” and would predict which key they were going to push next. It’s actually very easy to write a program that will make the right prediction about 70% of the time. Most people don’t really know how to type randomly. They’ll have too many alternations and so on. There will be all sorts of patterns, so you just have to build some sort of probabilistic model. Even a very crude one will do well. I couldn’t even beat my own program, knowing exactly how it worked. I challenged people to try this and the program was getting between 70% and 80% prediction rates. Then, we found one student that the program predicted exactly 50% of the time. We asked him what his secret was and he responded that he “just used his free will.”"
I can get it to about 60%. Given the algorithm used, it seems like it should be trivial to "trick" it by just cycling through novel 5-grams over and over, but to be honest I can't be bothered to check. And that would be rather contrary to the point of the exercise, if somewhat undercutting of the premise that free will is equivalent or comparable to randomness.
> I can get it to about 60%. Given the algorithm used, it seems like it should be trivial to "trick" it by just cycling through novel 5-grams over and over, but to be honest I can't be bothered to check. And that would be rather contrary to the point of the exercise, if somewhat undercutting of the premise that free will is equivalent or comparable to randomness.
Right; if you guess completely randomly it will hover around 50%, but if you choose the least-likely 5-gram from your history, then it should end up less than 50%.
yes, I've done some of these before. Ive gotten up above 200 key presses at ~50% before getting bored.
my secret was decidedly less sexy than free will though. i just had XX years of working as a statistician and having a better intuition as to what random strings and numbers actually look like :/
Same here, though I don't have any years of experience working as a statistician.
I did have a job where I had to implement a bunch of cognition tests for scientific studies. Initially I worked purely off the research papers describing how the some of the tests work - apparently by presenting a series of random numbers.
After I had the first few up and running, the feedback from the researchers was that my random numbers weren't random enough. "Weird", I thought... I had seeds for their sessions so I could reproduce the sequences they'd seen and they looked pretty random to me.
In the end it turned out my sequences were "too random", they didn't like any number being repeated, having more than 3 instances of the same number within any 6-digit sequence, and a few other "tweaks". Turns out actual randomness is confusing even for some very experienced and well trained people!
It's easy to be random if you have memorized random (or normal) data that your adversary isn't focused on. For example, early digits of pi work in most cases. Or use a generator to create a sequence to memorize.
> Free will is equivalent to randomness with many (asymptotically infinite) degrees of freedom.
I don't think this is obviously true. I can make a random number generating device with as many degrees of freedom as you like by producing a bunch of vertically polarised photons and measuring their horizontal polarisation. I don't think anyone would describe this system as having "free will".
Conversely humans with "free will" don't usually behave very randomly. They often have reasoning (good or bad) and are able to explain why they took whatever actions they took.
Free will seems to be qualitatively different to randomness.
I just typed randomly and got 50%. Does that mean I've got more free will than you? Is my will random? I've got my doubts, but I don't have anything better either.
In what way is being unpredictable considered "free will"? Of course it's just being clickbaity but worth mentioning. I've seen this better used to show that humans are poor sources of randomness.
Is there a difference between non-determinism and a free will choice? The latter seems to be a contradiction. A choice implies a reason which would be a cause, thus determinism.
This test is asking to make random 'choices'. This does have value in something such as sports.
It's hard to come up with a scientifically valid test for something as wide and vague as 'free will'. So what happens is that the tests are designed to test for very specific things that are then used to reason better about vague things like 'free will'. What this test is actually focused on is decision making.
The participant in the test is asked to randomly press either button A or B. The cognitive/mental experience of the participant is the following sequence of operations that seem to happen sequentially in time:
1. The participants is asked which button to push.
2. the participant ponders his decision.
3. the participant pushes the chosen button.
But what this test shows is that while the participants internal monologue is still 'deciding' whether to press A or B in step 2. The software is already able to accurately predict the answer at step 1.
So this indicates that even though your internal monologue might still be going "hmm should I pick A or B?", the decision, where your internal monologue eventually will end up, has already been made.
This is what the clickbait headline refers to; even though it feels like we humans are able to dictate and direct our thoughts, it seems that it doesn't entirely works that way.
> even though your internal monologue might still be going "hmm should I pick A or B?", the decision, where your internal monologue eventually will end up, has already been made.
There is an older better experiment by Benjamin Libet of this. The participant is directed to choose whether to press a single button, and watching a fast clock display to report the instant the decision is taken. The measurement and comparison of the brain activity clearly shows deciding to press, many milliseconds before the participant experiences deciding. Therefore, the participant does not have free will, is it not so? Since the decision is done before it is known to be done by the subject.
The retort is simple. The leap to "no free will" ignores that consciousness itself is a processing: the brain component that reconciles the internal input and renders it for externality takes time to do its work. The decision is still made by the participant. The subjective experience is behind the objective activation simply just as the signal for a television screen must pass through the wire before it is translated for pixels.
> even though it feels like we humans are able to dictate and direct our thoughts, it seems that it doesn't entirely works that way.
Because it's not a binary thing, the thought process is more complex but we can take our body for example. We all control our body, right? But now you take an average office worker vs profession sportsman and see that one can do way more precise and measured movements.
Thoughts are similar, you take some buddhist monk who recites 5 hour long mantras by memory for dozen of years and their thought control would be on another level. It is just the question of training and developing the skill.
All the test does is show that humans are bad at bookkeeping historical choices and make ones that don't follow patterns. We are wired to make and follow patterns. Machines are much better, but they certainly don't have more/any free will.
I don't imagine this is how a top sports competitor outwits their opponent. I expect that they have a theory of mind anticipating what the opponent expects then does something different. If we evolved in this manner, does it help to pretend that the machine has a mind to theorize about?
> This is what the clickbait headline refers to; even though it feels like we humans are able to dictate and direct our thoughts, it seems that it doesn't entirely works that way.
The implicit assumption here is that we are not dictating and directing our thoughts just because the outcome of the process of dictating and directing our thoughts may be predictable. This just doesn't follow.
Consider this analogy: a computer program dictates and directs the state of computer memory, but if I'm able to semi-reliably predict the state of computer memory at any point, then that disproves that the computer program dictates and directs the state of computer memory.
Of course that argument is obviously nonsense, no such test can disprove the fact that the program literally dictates and directs the state of computer memory. I also claim by analogy that no such test can disprove that the process of deliberation that we view as free will actually happens, even if that deliberation is deterministic. Determinism and free will are compatible (see Compatibilism).
Hmm, I don't get it. The prediction was accurate 49% of the time, which is what I was expecting in a coin toss like this one. In which way does it predict the answer at step 1 if it's a binary choice with 50% accuracy?
The core of most confusion in the "free will" philosophical debates is that the "free will" concept is undefined, and people assume many different unspoken such definitions when debating it.
I assume they used "free will" in the title because the student who did the best in the original test, when asked how they did it, said they just used their free will. I assume nobody involved—either the author of the linked page, or Scott Aaronson—believe that this simple game disproves free will by itself (whatever they believe about it in general).
> Is there a difference between non-determinism and a free will choice? The latter seems to be a contradiction. A choice implies a reason which would be a cause, thus determinism.
So many technical (read: narrowly educated) people make philosophical arguments that just boil down to an incredulous stance.
Most arguments about free will are arguments about a definition. If you taboo "free will", and let the discussion continue, it turns into much more tractable discussions about determinism, neurology, and control or lack there of.
Examining the code, my first thought was to throw it a de Bruijn sequence[1]: a cyclic sequence of length 2^n that contains every string of length n as a subsequence. It should (and does) learn a length-32 de Bruijn sequence after only a couple of repeats, but performs miserably on a length-64 sequence.
With a little (computer-)elbow grease, one can find a sequence with perfectly free will. After a short time, it starts repeating every... that's right, 64 characters. So of course, we want to find the shortest possible prefix.
Note that the last two lines are a de Bruijn sequence. Repeat those until you get bored. That's how you exercise your free will.
There is a deterministic bias: when in doubt, the page guesses "f". If it flipped a coin instead, a few experiments show that the greedy algorithm I and omoikane use cannot beat 25% for long.
I can look at the digits of PI and press d and f depending it's odd or even. So what does it prove? Nothing. Being able to beat this algorithm doesn't mean that I have free will, and choose to fall into patterns (knowing the PI solution exists) doesn't mean I don't.
If your behaviour can be predicted then you don't have free will.
Obviously you can argue whether this really predicts human behaviour or even the above implication itself (for example I think compatibilism kinda disagrees with it) but I'd say it is related.
From my point of view, that someone can predict what I will do from past behavior does not imply I don't have a free will.
Only way that would prove I don't have a free will is someone who would predict my behavior without knowing anything about my past, like a fortune teller that would tell my friend (because if fortune teller tells it to me it might be self fulfilling prophecy :D) something I would do; without having any interaction with me.
There's admittedly a relationship, but it's very subtle and nuanced.
For example, you can predict that I'm going to pay my rent with > 90% accuracy, but that has nothing to do with whether I'm paying out of my free will or not.
A Marxist will argue that it's the oppression of the bourgeoisie that makes me pay, and a free market capitalist would counter that I made the choice "freely". But the truth is somewhere in between, and probably depends "subjectively" on how the person "feels" about doing it.
Of course if the prediction for all actions (not merely within an experimental setting) is close to 100% then there's probably something weird about it, but I don't think that's remotely possible. Not with current tech, and unlikely with future tech either.
It got less than half right, so worse than just guessing.
Observation- predicting learned habits and other cognitive-effort preserving strategies your brain uses are not a refutation of free will. The whole point of habits, as Marvin Minsky pointed out in Society of Mind, is to save us from having to think through every little decision, i.e. to make ourselves predictable and unburden ourselves from decisions.
So in this instance, performing "randomly" in a low-stakes event is likely to reflexively trigger off a strategy in the brain which avails itself of some emergent pattern which serves the purpose of reducing cognitive effort. Call this test-taker strategy 1, i.e. just stab at the keys "randomly".
Another approach for the test-taker, call it strategy 2, is to expend considerable cognitive effort trying to model the machine's own model of the test taker's emerging behavior by observing the machine's responses, and then out fox it. (HN is great b/c I know everyone followed that). That's what I did, but it took real effort. Pretty soon, I wanted to drop the heavy thing and just go back to strategy 1.
IMHO the underlying assumption, that this is a test of "free will", is just flawed. Most people will enact strategy 1, be predictable and "lose".
predicted: f, observed: d
predicted: f, observed: d
predicted: f, observed: d
predicted: f, observed: d
predicted: f, observed: f
predicted: f, observed: d
predicted: f, observed: d
predicted: d, observed: d
predicted: f, observed: d
predicted: f, observed: f
predicted: f, observed: f
predicted: f, observed: f
predicted: f, observed: d
predicted: f, observed: d
predicted: f, observed: f
predicted: f, observed: f
I think it's closer to an attempted refutation of a strawman.
As one who leans towards belief in free will and also towards compatibilism, and more importantly originates from a culture with a shared belief in free will (conservative Christian evangelicalism), I don't think anyone I've known who believed in it meant "I believe my actions are fundamentally unpredictable."
I think they just meant that they genuinely have an ability to choose between the options they perceive, and then to attempt to enact that choice.
It is actually about challenging the user to come up with a random sequence, which is pretty difficult without a coin to flip, an irrational number to copy, etc.
When I just "randomly" mash the keys, the prediction rate is around 70%. I have to intentionally focus to avoid patterns as much as I can and then I'm able to pull it down to 54%.
Dan Ariely[1] did a similar spiel when teaching according to his book. He asked half of class to flip a coin and write down the sequence. The other half was supposed to write down a random sequence without the coin flips. Then he collected the papers, read them out and he claims he was able to tell almost every student whether they did the real coin flips or just simulated ones.
[1] - yes, his papers were retracted because he manipulated data
I managed to keep it below 45% without any real thought for inputs greater than 100 keystrokes. The biggest problem humans have with randomness is that they underestimate the possibility of long runs (they are rare but not that rare).
"Hacker's Keyboard" was supposed to let you pull up the keyboard anywhere but some system update broke it. I got around it with Teamviewer to my desktop.
Readit News
[1] https://github.com/elsehow/aaronson-oracle/blob/master/src/i...
if you are inspecting the prediction algorithm, and you want to specifically defeat it, doesn't that mean copy the prediction algorithm and throw a "not" on it? It's the old halting problem trick...
ddddd dfddd dffdd dfdfd ddfff ddfdd fdffd
That's only 10%.
I can get it to about 60%. Given the algorithm used, it seems like it should be trivial to "trick" it by just cycling through novel 5-grams over and over, but to be honest I can't be bothered to check. And that would be rather contrary to the point of the exercise, if somewhat undercutting of the premise that free will is equivalent or comparable to randomness.
Right; if you guess completely randomly it will hover around 50%, but if you choose the least-likely 5-gram from your history, then it should end up less than 50%.
my secret was decidedly less sexy than free will though. i just had XX years of working as a statistician and having a better intuition as to what random strings and numbers actually look like :/
I did have a job where I had to implement a bunch of cognition tests for scientific studies. Initially I worked purely off the research papers describing how the some of the tests work - apparently by presenting a series of random numbers.
After I had the first few up and running, the feedback from the researchers was that my random numbers weren't random enough. "Weird", I thought... I had seeds for their sessions so I could reproduce the sequences they'd seen and they looked pretty random to me.
In the end it turned out my sequences were "too random", they didn't like any number being repeated, having more than 3 instances of the same number within any 6-digit sequence, and a few other "tweaks". Turns out actual randomness is confusing even for some very experienced and well trained people!
Do you have a memorized random string?
It's easy to be random if you have memorized random (or normal) data that your adversary isn't focused on. For example, early digits of pi work in most cases. Or use a generator to create a sequence to memorize.
Not sure if the experiment tests this. (Pressing a different key or closing the browser tab are also variables in this probability distribution.)
I don't think this is obviously true. I can make a random number generating device with as many degrees of freedom as you like by producing a bunch of vertically polarised photons and measuring their horizontal polarisation. I don't think anyone would describe this system as having "free will".
Conversely humans with "free will" don't usually behave very randomly. They often have reasoning (good or bad) and are able to explain why they took whatever actions they took.
Free will seems to be qualitatively different to randomness.
I have free will to eat the burger, or not.
The biggest impacts on that decision are whether I like burgers and if I'm hungry.
Whether I'm 'in the mood' is the only random component.
And the outcome isn't random. I can fairly accurately predict that after 20 times of being offered that choice, most people will say no.
Deleted Comment
Is there a difference between non-determinism and a free will choice? The latter seems to be a contradiction. A choice implies a reason which would be a cause, thus determinism.
This test is asking to make random 'choices'. This does have value in something such as sports.
The participant in the test is asked to randomly press either button A or B. The cognitive/mental experience of the participant is the following sequence of operations that seem to happen sequentially in time: 1. The participants is asked which button to push. 2. the participant ponders his decision. 3. the participant pushes the chosen button.
But what this test shows is that while the participants internal monologue is still 'deciding' whether to press A or B in step 2. The software is already able to accurately predict the answer at step 1.
So this indicates that even though your internal monologue might still be going "hmm should I pick A or B?", the decision, where your internal monologue eventually will end up, has already been made.
This is what the clickbait headline refers to; even though it feels like we humans are able to dictate and direct our thoughts, it seems that it doesn't entirely works that way.
There is an older better experiment by Benjamin Libet of this. The participant is directed to choose whether to press a single button, and watching a fast clock display to report the instant the decision is taken. The measurement and comparison of the brain activity clearly shows deciding to press, many milliseconds before the participant experiences deciding. Therefore, the participant does not have free will, is it not so? Since the decision is done before it is known to be done by the subject.
The retort is simple. The leap to "no free will" ignores that consciousness itself is a processing: the brain component that reconciles the internal input and renders it for externality takes time to do its work. The decision is still made by the participant. The subjective experience is behind the objective activation simply just as the signal for a television screen must pass through the wire before it is translated for pixels.
Because it's not a binary thing, the thought process is more complex but we can take our body for example. We all control our body, right? But now you take an average office worker vs profession sportsman and see that one can do way more precise and measured movements.
Thoughts are similar, you take some buddhist monk who recites 5 hour long mantras by memory for dozen of years and their thought control would be on another level. It is just the question of training and developing the skill.
All the test does is show that humans are bad at bookkeeping historical choices and make ones that don't follow patterns. We are wired to make and follow patterns. Machines are much better, but they certainly don't have more/any free will.
The implicit assumption here is that we are not dictating and directing our thoughts just because the outcome of the process of dictating and directing our thoughts may be predictable. This just doesn't follow.
Consider this analogy: a computer program dictates and directs the state of computer memory, but if I'm able to semi-reliably predict the state of computer memory at any point, then that disproves that the computer program dictates and directs the state of computer memory.
Of course that argument is obviously nonsense, no such test can disprove the fact that the program literally dictates and directs the state of computer memory. I also claim by analogy that no such test can disprove that the process of deliberation that we view as free will actually happens, even if that deliberation is deterministic. Determinism and free will are compatible (see Compatibilism).
So many technical (read: narrowly educated) people make philosophical arguments that just boil down to an incredulous stance.
https://en.wikipedia.org/wiki/Compatibilism
With a little (computer-)elbow grease, one can find a sequence with perfectly free will. After a short time, it starts repeating every... that's right, 64 characters. So of course, we want to find the shortest possible prefix.
Note that the last two lines are a de Bruijn sequence. Repeat those until you get bored. That's how you exercise your free will.There is a deterministic bias: when in doubt, the page guesses "f". If it flipped a coin instead, a few experiments show that the greedy algorithm I and omoikane use cannot beat 25% for long.
[1] https://en.wikipedia.org/wiki/De_Bruijn_sequence
I can look at the digits of PI and press d and f depending it's odd or even. So what does it prove? Nothing. Being able to beat this algorithm doesn't mean that I have free will, and choose to fall into patterns (knowing the PI solution exists) doesn't mean I don't.
Obviously you can argue whether this really predicts human behaviour or even the above implication itself (for example I think compatibilism kinda disagrees with it) but I'd say it is related.
From my point of view, that someone can predict what I will do from past behavior does not imply I don't have a free will.
Only way that would prove I don't have a free will is someone who would predict my behavior without knowing anything about my past, like a fortune teller that would tell my friend (because if fortune teller tells it to me it might be self fulfilling prophecy :D) something I would do; without having any interaction with me.
For example, you can predict that I'm going to pay my rent with > 90% accuracy, but that has nothing to do with whether I'm paying out of my free will or not.
A Marxist will argue that it's the oppression of the bourgeoisie that makes me pay, and a free market capitalist would counter that I made the choice "freely". But the truth is somewhere in between, and probably depends "subjectively" on how the person "feels" about doing it.
Of course if the prediction for all actions (not merely within an experimental setting) is close to 100% then there's probably something weird about it, but I don't think that's remotely possible. Not with current tech, and unlikely with future tech either.
Deleted Comment
It's easy to create a more complex example to defeat a more advanced adversary. (Many exist.)
Observation- predicting learned habits and other cognitive-effort preserving strategies your brain uses are not a refutation of free will. The whole point of habits, as Marvin Minsky pointed out in Society of Mind, is to save us from having to think through every little decision, i.e. to make ourselves predictable and unburden ourselves from decisions.
So in this instance, performing "randomly" in a low-stakes event is likely to reflexively trigger off a strategy in the brain which avails itself of some emergent pattern which serves the purpose of reducing cognitive effort. Call this test-taker strategy 1, i.e. just stab at the keys "randomly".
Another approach for the test-taker, call it strategy 2, is to expend considerable cognitive effort trying to model the machine's own model of the test taker's emerging behavior by observing the machine's responses, and then out fox it. (HN is great b/c I know everyone followed that). That's what I did, but it took real effort. Pretty soon, I wanted to drop the heavy thing and just go back to strategy 1.
IMHO the underlying assumption, that this is a test of "free will", is just flawed. Most people will enact strategy 1, be predictable and "lose".
predicted: f, observed: d predicted: f, observed: d predicted: f, observed: d predicted: f, observed: d predicted: f, observed: f predicted: f, observed: d predicted: f, observed: d predicted: d, observed: d predicted: f, observed: d predicted: f, observed: f predicted: f, observed: f predicted: f, observed: f predicted: f, observed: d predicted: f, observed: d predicted: f, observed: f predicted: f, observed: f
However, what the game does is refute a particular attempt to prove that will does exist.
I think it's closer to an attempted refutation of a strawman.
As one who leans towards belief in free will and also towards compatibilism, and more importantly originates from a culture with a shared belief in free will (conservative Christian evangelicalism), I don't think anyone I've known who believed in it meant "I believe my actions are fundamentally unpredictable."
I think they just meant that they genuinely have an ability to choose between the options they perceive, and then to attempt to enact that choice.
Dead Comment
It is actually about challenging the user to come up with a random sequence, which is pretty difficult without a coin to flip, an irrational number to copy, etc.
When I just "randomly" mash the keys, the prediction rate is around 70%. I have to intentionally focus to avoid patterns as much as I can and then I'm able to pull it down to 54%.
Dan Ariely[1] did a similar spiel when teaching according to his book. He asked half of class to flip a coin and write down the sequence. The other half was supposed to write down a random sequence without the coin flips. Then he collected the papers, read them out and he claims he was able to tell almost every student whether they did the real coin flips or just simulated ones.
[1] - yes, his papers were retracted because he manipulated data