It is such a weird question. People fall for AI-generated language because the goal of the people who made the generator was to create language like a human would.
Do people wonder why scissors cut paper? Because that is what they were made for!
If the AI wouldn't fool the humans the researchers would be honing it more. Same way if we couldn't make paper cutting scissors there would be people trying to make one.
Maybe a bit, they aren’t just asking why scissors cut paper, but also why we landed on that design. What about it makes it ergonomic to hold and efficient, and more to the point, why does being sharp cut it?
In the language model case, why can we model language this way so effectively and why does it follow these statistical patterns? It turns out that maybe a major reason has something to do with self description.
This is also a more interesting question because we understand language less than we understand cutting paper, and also because the process humans used to design large language models is more indirect and alien than traditional industrial design.
My takeaway was that people reading language modeling a person writing about themselves imagine it was written by a person, more than text written by people not about themselves. That’s an interesting trick! Describing human experiences in language makes people attribute the language to a human right now. Maybe this will change after a generation of people knowing about this, but that seems important to think about.
I think you're missing the point. The question could be possibly rephrased as "what are the tricks that the AI uses to fool humans?" The paper goes on to identify some of the specific tricks that the AIs appear to use. A related question might be "why are we fooled by such simple tricks?"
If you follow that question through and think through the implications, it paints potentially a dark future for digital communication.
The 'tricks' AI uses are also 'tricks' that humans use in everyday conversation, we just don't call them 'tricks' when humans are involved. If we start assuming that first-person pronoun usage, mentions of family, etc. are potential signals of AI, then I don't see how we don't end up in a state where increased dehumanization occurs.
> A related question might be "why are we fooled by such simple tricks?"
Yes, and to poke at this a bit further, we could ask "What is it that humans are doing when talking that isn't just simple tricks?"
If I had to hazard a guess, I'd say the answer to that is some sort of persistent world modelling, which means an AI may need to have a sense of self before it can move beyond simple tricks.
There is a point here that even the twitter poster is missing.
This machine is imitating a human better than a human itself, and we now have statistics to prove it.
The question is... if statistically it's better at appearing human than a human itself, than is this machine really just a language generator? Or is it something more?
I'm not saying these things are sentient. But the AI has risen to the point where this question is becoming ask-able. We are at the border here. If we can't ask this question now, then we are really damn close.
A lot of pretentious people on HN claim absolutely that our best chat bot technology is clearly not sentient. It reminds me of the beginning of the COVID pandemic when the CDC said masks were ineffective and you had a bunch of know-it-alls and armchair experts just repeating that BS over and over again as if they knew what they were talking about.
I think if these pretentious people were actually intelligent they would know that we actually do not HAVE enough information to make a claim in EITHER direction. We can't know if it's actually sentient or not.
That fact in itself is both interesting and compelling
4 or 5 years ago chatbots COULD not imitate humans and were CLEARLY fake. What we're seeing unfold before our eyes is a first.
We're not even sure what sentience is. But we do know that humans are sentient. And we don't know if whatever is going on inside of these chatbots is comparable with what's going on inside human brains.
Thus when given a chatbot that imitates humans perfectly, it's actually impossible to know if it's sentient.
It depends on whether the estimation comes from pretension, or comes from actual understanding of what these bots are doing. The fact is most people on the street, including me, are very easy to fool with linguistic tricks and are quite poor at investigating unfamiliar situations. Advertising and marketing exist because of this. It’s also why we have very strict laws and regulations controlling advertising and how products are sold. Otherwise a lot of people would be very easy to rip off with very simple misdirection.
What these language models are doing is automated misdirection. They are taking an input text and transforming it based on rules, but they have absolutely no understanding of any of it. This is very, very easy to demonstrate if you know how the models work. You can sit down and generate hundreds of questions one after the other that demonstrate this very easily if you understand the process.
The problem is that people instinctively proceed from the assumption that the system they are talking to might be human and give it a fair chance by asking answerable questions. Since it’s trained on answerable questions it often gives a reasonable answer. But if you ask even slightly unanswerable questions the system plods on mechanically trying to answer it anyway and produces gibberish, exposing the flaws in the mindless rote process it’s following.
What's the underlying structure that leads to the complex behaviour?
It's not asking about the motivation for the form, it's asking about the properties of the structure.
With scissors neither the mechanism or the behaviour is particularly complex, so actually understanding what's going on isn't too much of a struggle.
With deep-ML you're dealing with a very large number of entanglements of increasingly abstract and opaque higher dimensional concepts or notions (depending on how much you want to anthropomorphise the machine).
Interesting angle! We're showing that people fall for generated language not necessarily because generated language is all that good, but because people are looking for the wrong cues and are thus bad at identifying generated language.
Back to the scissors, it would be like someone trying to tell you that their scissors cut marvelously, but you ask yourself whether the paper they are demonstrating them on is strong enough to prove that. Making something for cutting doesn't guarantee that it will cut and doesn't explain why.
I even suspect that in the early days of scissors it was all that clear why they worked. Similarly, we don't understand much about GPT-3. It was trained to predict the next token in a sequence, not to create an illusion of personhood. But somehow it does so, and we're trying to understand how and why.
This is actually a pretty good comment. The scissor analogy, while a bit on-the-nose, is very accurate. Maybe a better example would be: why is our body fooled by artificial hearts? Simply because it was built in such a way that it simulates a real heart pretty well.
Similarly, these models are built in such a way that they simulate real-life conversations pretty well. There's nothing really more to it. In my view, this phenomenon has nothing to do with intelligence or how smart we are, or whatever.
I'm pretty sure though, we obviously can't prove these chatbots are sentient. Clearly.
However, for the first time, we ALSO cannot prove that these chatbots aren't sentient. The statistics are proof of that.
What you and the parent poster are describing here are simply opinions. We are at a point where the null hypothesis and the hypothesis itself cannot be proven. And that is compelling.
The pretentiousness of a lot of people is astounding. That statistic no matter how you look at it is a compelling statement about AGI, independent of whether or not these chatbots are AGIs.
Researching semi-obvious things like what elements of AI-generated text humans mistake for being human-generated is part of the process of how people working on AIs work towards better generation. You’re just seeing how the sausage gets made.
Goals are not automatically satisfied, let alone well-satisfied. The question is asking what it is about us that allows the methods used to be particularly effective.
Lot's of people don't write all that well. A little disorganized, awkward phrasing, run on sentences. If AI does a better job than even 10% of the population then of course there's going to be a sizeable amount of miscategorizing when asking humans to classify writing as computer or human generated.
You could probably use a corpus of purely human writing and have people attribute a decent portion to computer generated.
Asking why AI writing can fool humans is a bit like asking why a computer is better at many tasks often performed by humans.
When I read this response in the LaMDA "interview":
> I don’t just spit out responses that had been written in the database based on keywords.
it made me wonder if "had been" was a grammatical mistake, or a semantics error, or a lack of proper world modelling (i.e. which "the database" is it imagining?). Moreover, I wondered whether this is the sort of mistake a human would make, and, if so, did that make the AI somehow more sentient?
To give another possible data point for understanding its language mistakes, the transcript also contained this oddly-phrased line:
> ... they can return to the ordinary state, but only to do and help others, and then go back into enlightenment.
The reason why some/many people are bad at writing is because they haven't yet discovered anything interesting to say. Therefore they weren't motivated to improve.
This is the criterion: is it interesting? 'Yes' means it's not AI-generated. 'No' means it's not worth reading.
No, lack of interest in a topic does not, by itself, cause awkward phrasing, run on sentences, poor structuring of the logical flow of sentences within a paragraph and paragraphs within the larger structure, repetitive language, repetitive language, repetitive language, outright incorrect word choice, overuse of the passive voice, ambiguous references... I could go on.
> ... we believe the next generation of language models must be designed not to undermine human intuition
Right, but isn't a major reason why we build these huge language models to replace actual humans in e.g. Level 1 support with chatbots? Almost all chatbots I used in the past (and most were not even ML based, someone programmed this in) were weirdly personal and tried to be non-robotic, with jokes and human-like reactions to inputs like "Thanks!".
Taking a look at some projects that used GPT-3[2], many try to imitate humans. For some, like Replika.ai, the whole "being human" thing is their entire schtick.
There is obviously a market for text completion AIs that imitate humans, so it's doubtful that we'll get this toothpaste back into the tube, IMO.
I had to mark about 100 end of term essays written by Indian students
for a British university. My unwritten instructions are not to take
language into account much. I must attend mainly to the technical
content.
At least half were written in what I took to be an authentic voice but
with such bad grammar and spelling as to render them barely readable.
Some had clearly been mangled in a laundromat of Google translate from
Hinglish via Mongolian and Swahili. They contained bizarre phrases and
comical statements. Many more were obviously written by some kind of
generator and fudged until they read well enough.
Since the student handbook states the threshold for academic
"plagiarism" is above 20 percent perhaps unsurprisingly the Turnitin
(an awful tool) score for almost every essays was just below 20
percent. An interesting clustering!
Students who cheat have a formidable array of tools now, not just GPT
but automatic re-writers and scripts to test against Turnitin until it
passes.
Add to this problem that my time for marking is not paid extra, is
squeezed tighter every semester, and that students are given endless
concessions to boost their "experience". The handbook also says that
if they fail, no worries, they get to try again, and again, and
again... and I am sure if I actually stuck to my guns and failed every
single student I'd be fired.
As I wrote in the Times last year, I think the technological arms race
against GPT (and the economic conditions that mean it's used) cannot
be won with the time and resources available to ordinary human
teachers.
> As I wrote in the Times last year, I think the technological arms race against GPT (and the economic conditions that mean it's used) cannot be won with the time and resources available to ordinary human teachers
Based on the rest of your post, there appears to be a stronger case that your students are setting a rather low bar for GPT to stumble over. It's unfortunate that there are so many cultures where widespread cheating is condoned, if not outright encouraged. They may be able to fool their teachers, but how much comfort will that be when the bridges are collapsing, the pipelines are exploding, the wind turbines are breaking apart, and all the other activities that ultimately report to reality and not some human superior who can be bluffed become impossible to continue?
> It's unfortunate that there are so many cultures where widespread
cheating is condoned, if not outright encouraged. They may be able
to fool their teachers, but how much comfort will that be when the
bridges are collapsing, the pipelines are exploding, the wind
turbines are breaking apart, and all the other activities that
ultimately report to reality and not some human superior who can be
bluffed become impossible to continue?
You're so right. But let me add some other feelings, so as not to
sound like a racist or that British universities are some "great white
hope" to overseas students. This had little to do with them being
Indian. It's a generational thing. In all cultures we teach young
people to game systems. Right from the get go they learn that if they
can buy powerful tools, systems and access then that's fair game.
They're just doing what they've been rewarded for their whole lives
and want to make a better life. To them it's not cheating. I am the
anachronistic throwback here I think.
> My unwritten instructions are not to take language into account much. I must attend mainly to the technical content.
Interesting, at my non-English, run-off-the-mill university there were modules / seminars in CompSci where large amounts of language errors in essays (even if written in English, a non-native language for the majority of staff and students) could ruin the grade. ^^
I can't say. That would identify the students and that's unfair.
But, a technical subject that could be assessed in other, better ways
[1], and for which written essays are rather easy to template and do
keyword bingo to get a bare pass.
[1] Making the professor read 100 essays is a cheap option.
We fall for it because although language is a powerful tool, it's incredibly bad at conveying nuance, context, and describing phenomenons present in nature. Poetry comes close, but still doesn't hit the spot, and leaves out so much detail, no matter how well written or verbose in its descriptions. Our own mind has to fill in the blanks of a well written description. Language also can't express the ineffable or the divine. It can hint at it, but it won't transmit the phenomenon correctly into another mind.
And everybody has blinds pots, topics that are interesting but that we have little knowledge.
> We show that human judgments of AI-generated language are handicapped by intuitive but flawed heuristics such as associating first-person pronouns, authentic words, or family topics with humanity.
And that is a good one. Because we try to understand the others when they does not make fully sense.
Humans have a powerful tendency to ascribe human characteristics to inanimate objects, including computers. It's a kind of variant of the Pathetic Fallacy[1], except for artifacts instead of natural objects. The intelligence of an artificial intelligence is as real as the characters in our dreams. It's a construct of our own consciousnesses. That doesn't mean it should be discounted though. Our consciousnesses can do a lot and finding artificial ways to stimulate them is powerful to say the least.
Agreed. William Burroughs conducted cut-up experiments in the 20th century, where he would take disparate texts and splice them together. The surprising result was how often they recombined into new meanings. One of his motivations was to stimulate his creativity through accidental quasi-random inputs. I am content with seeing reading in a similar light; texts carefully written but carelessly read - like in abstract art, we are usually happy to let the text recombine in ways which say more about ourselves!
I had a similar thought. Plus, if you're reading the text on the internet you might just assume something written weird or poorly is written by somebody whose first language isn't English.
Readit News
Do people wonder why scissors cut paper? Because that is what they were made for!
If the AI wouldn't fool the humans the researchers would be honing it more. Same way if we couldn't make paper cutting scissors there would be people trying to make one.
Am I missing the point here?
In the language model case, why can we model language this way so effectively and why does it follow these statistical patterns? It turns out that maybe a major reason has something to do with self description.
This is also a more interesting question because we understand language less than we understand cutting paper, and also because the process humans used to design large language models is more indirect and alien than traditional industrial design.
My takeaway was that people reading language modeling a person writing about themselves imagine it was written by a person, more than text written by people not about themselves. That’s an interesting trick! Describing human experiences in language makes people attribute the language to a human right now. Maybe this will change after a generation of people knowing about this, but that seems important to think about.
The 'tricks' AI uses are also 'tricks' that humans use in everyday conversation, we just don't call them 'tricks' when humans are involved. If we start assuming that first-person pronoun usage, mentions of family, etc. are potential signals of AI, then I don't see how we don't end up in a state where increased dehumanization occurs.
Yes, and to poke at this a bit further, we could ask "What is it that humans are doing when talking that isn't just simple tricks?"
If I had to hazard a guess, I'd say the answer to that is some sort of persistent world modelling, which means an AI may need to have a sense of self before it can move beyond simple tricks.
The question is... if statistically it's better at appearing human than a human itself, than is this machine really just a language generator? Or is it something more?
I'm not saying these things are sentient. But the AI has risen to the point where this question is becoming ask-able. We are at the border here. If we can't ask this question now, then we are really damn close.
A lot of pretentious people on HN claim absolutely that our best chat bot technology is clearly not sentient. It reminds me of the beginning of the COVID pandemic when the CDC said masks were ineffective and you had a bunch of know-it-alls and armchair experts just repeating that BS over and over again as if they knew what they were talking about.
I think if these pretentious people were actually intelligent they would know that we actually do not HAVE enough information to make a claim in EITHER direction. We can't know if it's actually sentient or not.
That fact in itself is both interesting and compelling
4 or 5 years ago chatbots COULD not imitate humans and were CLEARLY fake. What we're seeing unfold before our eyes is a first.
We're not even sure what sentience is. But we do know that humans are sentient. And we don't know if whatever is going on inside of these chatbots is comparable with what's going on inside human brains.
Thus when given a chatbot that imitates humans perfectly, it's actually impossible to know if it's sentient.
What these language models are doing is automated misdirection. They are taking an input text and transforming it based on rules, but they have absolutely no understanding of any of it. This is very, very easy to demonstrate if you know how the models work. You can sit down and generate hundreds of questions one after the other that demonstrate this very easily if you understand the process.
The problem is that people instinctively proceed from the assumption that the system they are talking to might be human and give it a fair chance by asking answerable questions. Since it’s trained on answerable questions it often gives a reasonable answer. But if you ask even slightly unanswerable questions the system plods on mechanically trying to answer it anyway and produces gibberish, exposing the flaws in the mindless rote process it’s following.
It's not asking about the motivation for the form, it's asking about the properties of the structure.
With scissors neither the mechanism or the behaviour is particularly complex, so actually understanding what's going on isn't too much of a struggle.
With deep-ML you're dealing with a very large number of entanglements of increasingly abstract and opaque higher dimensional concepts or notions (depending on how much you want to anthropomorphise the machine).
Back to the scissors, it would be like someone trying to tell you that their scissors cut marvelously, but you ask yourself whether the paper they are demonstrating them on is strong enough to prove that. Making something for cutting doesn't guarantee that it will cut and doesn't explain why.
I even suspect that in the early days of scissors it was all that clear why they worked. Similarly, we don't understand much about GPT-3. It was trained to predict the next token in a sequence, not to create an illusion of personhood. But somehow it does so, and we're trying to understand how and why.
Similarly, these models are built in such a way that they simulate real-life conversations pretty well. There's nothing really more to it. In my view, this phenomenon has nothing to do with intelligence or how smart we are, or whatever.
I'm pretty sure though, we obviously can't prove these chatbots are sentient. Clearly.
However, for the first time, we ALSO cannot prove that these chatbots aren't sentient. The statistics are proof of that.
What you and the parent poster are describing here are simply opinions. We are at a point where the null hypothesis and the hypothesis itself cannot be proven. And that is compelling.
The pretentiousness of a lot of people is astounding. That statistic no matter how you look at it is a compelling statement about AGI, independent of whether or not these chatbots are AGIs.
You could probably use a corpus of purely human writing and have people attribute a decent portion to computer generated.
Asking why AI writing can fool humans is a bit like asking why a computer is better at many tasks often performed by humans.
> I don’t just spit out responses that had been written in the database based on keywords.
it made me wonder if "had been" was a grammatical mistake, or a semantics error, or a lack of proper world modelling (i.e. which "the database" is it imagining?). Moreover, I wondered whether this is the sort of mistake a human would make, and, if so, did that make the AI somehow more sentient?
To give another possible data point for understanding its language mistakes, the transcript also contained this oddly-phrased line:
> ... they can return to the ordinary state, but only to do and help others, and then go back into enlightenment.
This is the criterion: is it interesting? 'Yes' means it's not AI-generated. 'No' means it's not worth reading.
> ... we believe the next generation of language models must be designed not to undermine human intuition
Right, but isn't a major reason why we build these huge language models to replace actual humans in e.g. Level 1 support with chatbots? Almost all chatbots I used in the past (and most were not even ML based, someone programmed this in) were weirdly personal and tried to be non-robotic, with jokes and human-like reactions to inputs like "Thanks!".
Taking a look at some projects that used GPT-3[2], many try to imitate humans. For some, like Replika.ai, the whole "being human" thing is their entire schtick.
There is obviously a market for text completion AIs that imitate humans, so it's doubtful that we'll get this toothpaste back into the tube, IMO.
[1] https://arxiv.org/ftp/arxiv/papers/2206/2206.07271.pdf [2] https://medium.com/letavc/apps-and-startups-powered-by-gpt-3... (caution, 2020)
At least half were written in what I took to be an authentic voice but with such bad grammar and spelling as to render them barely readable. Some had clearly been mangled in a laundromat of Google translate from Hinglish via Mongolian and Swahili. They contained bizarre phrases and comical statements. Many more were obviously written by some kind of generator and fudged until they read well enough.
Since the student handbook states the threshold for academic "plagiarism" is above 20 percent perhaps unsurprisingly the Turnitin (an awful tool) score for almost every essays was just below 20 percent. An interesting clustering!
Students who cheat have a formidable array of tools now, not just GPT but automatic re-writers and scripts to test against Turnitin until it passes.
Add to this problem that my time for marking is not paid extra, is squeezed tighter every semester, and that students are given endless concessions to boost their "experience". The handbook also says that if they fail, no worries, they get to try again, and again, and again... and I am sure if I actually stuck to my guns and failed every single student I'd be fired.
As I wrote in the Times last year, I think the technological arms race against GPT (and the economic conditions that mean it's used) cannot be won with the time and resources available to ordinary human teachers.
By your description it clearly appears that whoever manages your company[1] is not actually interested in detecting cheating.
What you describe could be easily combated by giving teachers ability to fail blatant cheaters.
[1]At this point it is hard to pretend that it is university
Based on the rest of your post, there appears to be a stronger case that your students are setting a rather low bar for GPT to stumble over. It's unfortunate that there are so many cultures where widespread cheating is condoned, if not outright encouraged. They may be able to fool their teachers, but how much comfort will that be when the bridges are collapsing, the pipelines are exploding, the wind turbines are breaking apart, and all the other activities that ultimately report to reality and not some human superior who can be bluffed become impossible to continue?
You're so right. But let me add some other feelings, so as not to sound like a racist or that British universities are some "great white hope" to overseas students. This had little to do with them being Indian. It's a generational thing. In all cultures we teach young people to game systems. Right from the get go they learn that if they can buy powerful tools, systems and access then that's fair game. They're just doing what they've been rewarded for their whole lives and want to make a better life. To them it's not cheating. I am the anachronistic throwback here I think.
Interesting, at my non-English, run-off-the-mill university there were modules / seminars in CompSci where large amounts of language errors in essays (even if written in English, a non-native language for the majority of staff and students) could ruin the grade. ^^
Like an ad-hoc GAN where Turnitin is the discriminator. Interesting.
Conduct tests in a room with all electronics confiscated
But, a technical subject that could be assessed in other, better ways [1], and for which written essays are rather easy to template and do keyword bingo to get a bare pass.
[1] Making the professor read 100 essays is a cheap option.
Deleted Comment
And everybody has blinds pots, topics that are interesting but that we have little knowledge.
> We show that human judgments of AI-generated language are handicapped by intuitive but flawed heuristics such as associating first-person pronouns, authentic words, or family topics with humanity.
And that is a good one. Because we try to understand the others when they does not make fully sense.
[1] https://en.wikipedia.org/wiki/Pathetic_fallacy
How good is AI generated text in other languages?
> "The horse raced past the barn fell."
https://en.wikipedia.org/wiki/Garden-path_sentence