I have a little weather station mounted outside my house, and there is a small linux machine (Raspberry Pi) logging the weather data into an SQLite database. I have almost two years of data now, so I asked GPT-4 to write a Python program that would query the database, calculate the average temperature for each day, fit a sine wave to those data points, find the first minimum of the sine wave and use that to calculate what day of the year should be coldest on average, then plot everything. It did that easily. But the big surprise was this: just for fun, I told GPT-4 that one of the quantities it had calculated could be used to estimate the orbital period of the Earth, and I asked it which quantity it was. It correctly answered that the reciprocal of the frequency of the fitted sine wave is an estimate of the Earth's orbital period. All I had told it to do was pull out the temperature data for each day, and fit a sine wave. It knew that the reciprocal of the frequency was the Earth's orbital period, without me mentioning anything at all about the Earth when I was prompting it to write the program. It had calculated dozens of variable values on its way to producing the plots, and it knew the right one to pick, and it knew the answer was its reciprocal.
I thought that perhaps the program just recognized that the only value that could be used to calculate any period was that frequency, and perhaps the model didn't really know anything about the Earth motion through its orbit affecting the average temperature, so I asked:
"Can any of the values be used to estimate the Moon's orbital period?"
and it replied:
"No, the code and the data used in this program are not suitable for estimating the Moon's orbital period. The temperature data being analyzed is influenced by the Earth's rotation around the Sun and other factors like weather patterns, but it doesn't directly correlate with the Moon's orbital period."
It's a word prediction algorithm. Literally any collection of words (sometimes also known as a fact) it was trained on, regardless of how esoteric or domain specific, should generally be able to be regurgitated and, to a lesser degree, associated with similar collections of words. If you want to see it fail, don't try to "outsmart" it, but simply consider how it's programmed.
---
Me: "continue the sequence: 0, done, 2, free, 4, hive"
ChatGPT 4: 'I give up.' (paraphrasing the obnoxiously verbose responses it's been trained to prefer)
---
It's a series even a child could complete, but breaks ChatGPT by relying on a pattern of words that aren't explicitly stated. So you get software which can pass the bar, but fail at trivial series.
ChatGPT is not good when it needs to answer directly. If you let it lead itself to the right answer it's pretty good.
# Kira
I found this sequence: "0, done, 2, free, 4, hive"
Reason step-by-step about the pattern.
Consecutively continue the sequence with four more entries.
Consecutively reflect on your answer and fix mistakes in case you broke the pattern.
# ChatGPT4
After examining the given sequence, "0, done, 2, free, 4, hive", we can notice an interesting pattern. It appears that the sequence alternates between numbers and words that sound like numbers. Let's break down the pattern:
"0" is a number.
"done" sounds like the number "1" when spoken aloud.
"2" is a number.
"free" sounds like the number "3" when spoken aloud.
"4" is a number.
"hive" sounds like the number "5" when spoken aloud.
Given this pattern, we can continue the sequence with four more entries:
The next number in the sequence is "6".
The word that sounds like the number "7" is "heaven".
The next number in the sequence is "8".
The word that sounds like the number "9" is "nein", which is German for "no".
So, the continued sequence would be: "0, done, 2, free, 4, hive, 6, heaven, 8, nein".
Reflecting on the pattern and the answer provided, the continuation seems to be consistent with the established pattern. We alternated between numbers and words that phonetically resemble numbers, resulting in the sequence "6, heaven, 8, nein" for the next four entries.
> It's a word prediction algorithm. Literally any collection of words (sometimes also known as a fact) it was trained on, regardless of how esoteric or domain specific, should generally be able to be regurgitated and, to a lesser degree, associated with similar collections of words.
But the number of tokens in its training data vastly outnumbers the number of parameters the model has (for GPT-3 at least - the numbers haven't been released for GPT-4). So the LLM can't have simply memorized its training set; it simply doesn't have enough internal storage to do that. So it is not just regurgitating portions of its training set.
I asked my coworkers in the office but none was able to answer that. Not sure because they were are ESL (me included) or because they were GPT in disguise.
I apologize for being enormously stupid, but the word is a rhyme with the underlying number right?
Edit: I want to add I even get GPT3 getting it right without explanation to be honest. Even that one is outsmarting me. 4 is using terms like homophone. I'm not sure you proved what you thought you proved.
While I don't think these claims are entirely correct, I think they are worth considering.
On the one hand, Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo). Just as much, LLMs don't express their creators values 'cause they don't express any values, they average language responses (though people certainly can see them expressing values, which can cause problems for the people).
On the other hand, the point that their creators don't understand them and are still quite willing to toss them to the open Internet really show considered carefully whatever their exact capabilities.
> Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo).
The fact is that they acquired abilities no one expected them to acquire. In hindsight you can say it was embedded in language and maybe you could have seen it coming, but it is an empirical fact that this was unexpected and unpredictable beforehand.
It's still fairly exaggerated, because the LLM's purpose is basically "knowing human communication" and it still only exhibits that, but it knows how humans would respond if they obediently followed directions given to it exactly as instructed.
And it was only unexpected to those that weren't following the news the GPT 3 paper was published in 2020 and detailed this wild advancement in its capabilities[0,1].
Man, I have a totally opposite view about LLMs expressing creator’s values. Not only do they express them, they don’t STOP expressing them to the point of utter annoyance. Any remotely PG topic ends with a safety caveat, e.g., “however, it’s important to consider . . . .”
Those statements don't come from the base model, they come from the steering methods, basically a form of hand tuning after the model is mostly trained, which the paper says are relatively crude and imperfect. It is the fact that these models are so unpredictable that led to these attempts at steering.
> Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo)
I think the author would say those are not the capabilities being discussed here.
This is a personal correspondence typeset via LaTeX — it is not an academic paper, and it was not peer-reviewed. (The document does not claim otherwise, but I think it's common for people to assume that documents that have been typeset in such a format are more rigorous than this is.)
Leaving that aside, I really take issue with the style used by the author. For example, section 3 begins:
> There is increasingly substantial evidence that LLMs develop internal representations of the world to some extent, and that these representations allow them to reason at a level of abstraction that is not sensitive to the precise linguistic form of the text that they are reasoning about.
LLMs do not "reason"; they do not "learn" or "develop" anything of their own volition. They are (advanced) statistical models. Anthropomorphizing them is not just technically incorrect, but morally disingenuous. Writing about LLMs in this way causes people with less domain-specific knowledge to trust the models the way they might trust people, and this has the potential for serious harm.
Because of this choice of phrasing, I wanted to look into the author's background. Among their recent activities, they list:
> I'm organizing a new AI safety research group at NYU, and I wrote up a blog post explaining what we're up to and why.
"AI Safety" is a distinct (and actually opposing) area from "AI Ethics". The people who prefer the word "safety" tend also to engage in discussions that touch on aspects of longtermism. Longtermism is not scientifically well-grounded; it seeks to divert attention from present and real issues to fanciful projections of far-future concerns. I do not know for certain that the author is in fact a longtermist, but their consistent anthropomorphization of a pile of statistical formulae certainly suggests they wouldn't feel out of place among a crowd of such people.
In contrast, the people who prefer the term "ethics" in their work are grounded in real and present issues. They concern themselves with reasonable regulation. They worry about the current-day environmental impacts of training large models. In short, they are concerned with actual issues, rather than the alleged potential for a statistical model to "develop" sentience or exhibit properties of "emergent" intelligence (subjects from the annals of the science-fiction writing of last century).
I hope the author can clarify their choice of phrasing in their work, though I worry they have chosen their words carefully already. Readers should exercise caution in taking the claims of a soothsayer without a sufficient quantity of salt.
> LLMs do not "reason"; they do not "learn" or "develop" anything of their own volition.
If this claim of yours was sustainable, your grievance with the writing style would make a lot of sense. But GPT-4 can clearly reason about novel problems which were not in its training data. I think you should consider reading some of the many citations in the document which describe examples of that happening.
LLMs clearly develop internal representations, this is an empirical fact. For example, one paper cited is https://arxiv.org/abs/2210.13382, which trained an LLM on Othello games, where an 8 layer model is fed one of 60 tokens (there are 60 possible moves in Othello, 8*8 minus 4 initially occupied squares). You can probe this model at layer 5, and it computes Othello board state from moves, and you can be sure this board state is used, because if you patch activation so that board state is modified, the model generates moves for this modified board state.
> LLMs clearly develop internal representations, this is an empirical fact.
you've offered an anecdote. Some LLMs (more generally, this type of neural network model) will generate configurations that can be understood as a representation; others will not. The fact that the authors were able to find an apparent model in a heavily rule-based system is not incredibly surprising but offers little clue about whether this is "clearly" true for similar architectures that represent very different (real) world systems.
From your citation itself:
> More broadly, it would be interesting to study how our results generalize to models trained on
natural language. One stepping stone might be to look at language models whose training data has
included game transcripts. Will we see similar representation of board state?
> I hope the author can clarify their choice of phrasing in their work
From TFA's conclusions section:
"Open debates over whether we describe LLMs as understanding language, and whether to describe their actions using agency-related words like know or try, are largely separate from the questions that I discuss here (Bender & Koller, 2020; Michael, 2020; Potts, 2020). We can evaluate whether systems are effective or ineffective, reliable or unreliable, interpretable or uninterpretable, and improving quickly or slowly, regardless of whether they are underlyingly humanlike in the sense that these words evoke."
>LLMs do not "reason"; they do not "learn" or "develop" anything of their own volition. They are (advanced) statistical models.
As a non-expert in the field I was hesitant at the time to disagree with the legions of experts who last year denounced Blake Lemoine and his claims. I know enough to know, though, of the AI effect <https://en.wikipedia.org/wiki/AI_effect>, a longstanding tradition/bad habit of advances being dismissed by those in the field itself as "not real AI". Anyone, expert or not, in 1950, 1960, or even 1970 who was told that before the turn of the century a computer would defeat the world chess champion would conclude that said feat must have come as part of a breakthrough in AGI. Same if told that by 2015 many people would have in their homes, and carry around in their pockets, devices that can respond to spoken queries on a variety of topics.
To put another way, I was hesitant to be as self-assuredly certain about how to define consciousness, intelligence, and sentience—and what it takes for them to emerge—as the experts who denounced Lemoine. The recent GPT breakthroughs have made me more so.
I didn't say it's "not real AI", but of course whether I meant that or not comes down to a definition of "AI".
In academia (where I am, though my specialty is a step removed), "AI" is exactly the region of research concerned with statistical problem-solving. Machine learning is sometimes synonymous, though I haven't been able to determine whether this is consistent among all self-described AI researchers or just some of them.
Systems like GPTs are not capable of "thought", full stop. There is no ambiguity. If this is hard for you to accept, I'm sorry, but it is a fact. They have no agency.
I saw someone sketch a good explanation. They were responding to a claim that Chat GPT could "pass the Harvard admissions process". While it is true that the system can access sufficient data to respond accurately to questions on an exam, and it could be prompted to generate an essay, it cannot of its own volition choose to submit an application. It doesn't even "know" that Harvard is a real place to which it can apply, nor does it "understand" how it could go about "learning" how to find out about these things it does not know. It simply isn't capable of these things. (And even if it were that wouldn't necessarily be evidence of intelligence, but for now it's a sufficient distinguisher.)
> Wat
Can you explain what about my last point was "wat"-worthy? Do you disagree that it is morally problematic to pretend a thing is sentient when it is in fact not?
We may just live in a world where there’s room for more than one viewpoint and people are able to research different aspects of the same thing without getting into debates over dogma.
> We may just live in a world where there’s room for more than one viewpoint
AI safety is not a "viewpoint", it is a marketing strategy. The goal of the people leading that charge is to drum up FUD to increase investment in their useless snake oil companies. I feel no remorse for not being inclusive of their "perspective".
At what point are we allowed to use the "reason" or "learn" for AI then? Can we use these terms for non-human animals? If so, which ones?
I think LLMs are at the point where adopting an "intentional stance" (using Daniel Dennett's term) is reasonable.
> At what point are we allowed to use the "reason" or "learn" for AI then?
Maybe when the technology is physically capable of thinking for itself, rather than just being statistically likely to respond in a way that appears like the way a human would respond. Which may never happen, for what it's worth.
> I think LLMs are at the point where adopting an "intentional stance" (using Daniel Dennett's term) is reasonable.
I hadn't heard of this term so I looked into it, and wow, what a biased Wikipedia article it has. It's written with a completely partial stance. I've no idea whether to take it seriously, to be honest.
That said, I disagree strongly. LLMs do not "intend" anything, and pretending they do is (in my view) highly problematic. They're just (sophisticated!) prediction engines; nothing more.
While llm don't learn, they have now enough token input space and handle context well enough to manage a cyclic conversation.
While not learning, they can still run inferences on what they are fed as context within the token space. And because they are becoming so good at getting context out of structured data, they can use the context provided as a layer on top of their knowledge.
So you can at a very good approximation upload knowledge into them, and such knowledge can represent how to use tools to fetch data they don't know about - i.e. what langchain agents are doing.
Now of course that is not 'llm' some, is llm + code for informaron discovery, but that code is entirely supported and enabled by the existence of good enough llm
People routinely make up their own vague and ill defined meanings of understanding and reasoning to disqualify LLMs. This is necessary because LLMs obviously reason and understand by any evaluation that can be carried out.
Seriously just watch. He's not actually going to be able to coherently define his "reasoning" in a way that can be tested.
When you hear someone speak, are you thinking their thoughts?
We are talking about a model of language and our expectations for what the words would look like if a person were using language to reason with. People are confusing one thing (their own interpretation of what they are reading) with another (a statistical model that is being driven to align with certain specific human expectations).
I’ve been imagining “reasoning” as meaning “referring to an abstraction”. The idea being if you wanted to write a script to tell you how far an arrow of a given weight will fly, you’ll probably use some physics simulation (or even just a coefficient to multiply the input weight by) to abstract the arrow in flight. Then you can answer questions around that abstraction.
The alternative is having 1000 different results for different kinds of arrows, and averaging out the results for the ones similar to the input arrow.
An LLM is working on text tokens, it’s trying to give the most statistically common next token based on everything that’s been fed into it. Does that statistical model abstract the objects and concepts it talks about? Eh? I don’t know
> 6. Human performance on a task isn’t an upper bound on LLM performance
Is that true? Because LLMs are trained on the work of human: a LLM that learned all of the content of the articles would at best yield output equal to that of the human writing.
The reasons the author gives seem unfounded to me:
> First, they are trained on far more data than any human sees
If the human writer has access to google (everyone in modern society) this point is moot.
> In addition, they are often given additional training using reinforcement learning before being deployed.
If this is human in the loop RL, then the upper bound would still be the human training it. If it isn't, refer to #1.
I don't agree with you. Human knowledge itself is not bound only to the previous knowledge before it, so why wouldn't AI be any different? This is more of where AGI will shine, but I still think LLMs can produce novel innovation.
There's also a difference between having access to and being able to 'comprehend' all previous works in a given genre for generating music, that a human simply won't have the ability to process and recall when creating their own music and the AI could surpass. Humans simply having access to information is different than using information.
> If the human writer has access to google (everyone in modern society) this point is moot.
But a single human can only read so much information in a finite amount of time (their life). Sure, the same thing applies to LLMs, but the theoretical limits are orders of magnitude higher.
And while this probably isn't what you were talking about, it's also pretty clear that any time-limited task favours LLMs. There's no human on earth that could write a non-trivial short story in 10 seconds, regardless of quality level.
> If the human writer has access to google (everyone in modern society) this point is moot.
Does not follow at all. There’s a vast difference between being able to search for and internalize a tiny, tiny sample of all the available information, and literally having sampled and synthetized all of it.
> If it isn't, refer to #1.
Meaning what? LLMs use adversarial learning, the same thing that allowed AlphaGo to reach superhuman levels in Go.
9. With millions of people copying, pasting, and running code written by LLMs (often students who have no idea what the code actually does), it will be impossible to keep an intelligence that surpasses our own 'inside the box'.
Readit News
I thought that perhaps the program just recognized that the only value that could be used to calculate any period was that frequency, and perhaps the model didn't really know anything about the Earth motion through its orbit affecting the average temperature, so I asked:
"Can any of the values be used to estimate the Moon's orbital period?"
and it replied:
"No, the code and the data used in this program are not suitable for estimating the Moon's orbital period. The temperature data being analyzed is influenced by the Earth's rotation around the Sun and other factors like weather patterns, but it doesn't directly correlate with the Moon's orbital period."
---
Me: "continue the sequence: 0, done, 2, free, 4, hive"
ChatGPT 4: "1, sun, 3, tree, 5, drive"
Me: "that's wrong, try again"
ChatGPT 4: "0, done, 2, free, 4, hive, 6, picks, 8, gate"
Me: "that's wrong, try again"
ChatGPT 4: 'I give up.' (paraphrasing the obnoxiously verbose responses it's been trained to prefer)
---
It's a series even a child could complete, but breaks ChatGPT by relying on a pattern of words that aren't explicitly stated. So you get software which can pass the bar, but fail at trivial series.
# Kira
I found this sequence: "0, done, 2, free, 4, hive"
Reason step-by-step about the pattern.
Consecutively continue the sequence with four more entries.
Consecutively reflect on your answer and fix mistakes in case you broke the pattern.
# ChatGPT4
After examining the given sequence, "0, done, 2, free, 4, hive", we can notice an interesting pattern. It appears that the sequence alternates between numbers and words that sound like numbers. Let's break down the pattern:
"0" is a number. "done" sounds like the number "1" when spoken aloud. "2" is a number. "free" sounds like the number "3" when spoken aloud. "4" is a number. "hive" sounds like the number "5" when spoken aloud. Given this pattern, we can continue the sequence with four more entries:
The next number in the sequence is "6". The word that sounds like the number "7" is "heaven". The next number in the sequence is "8". The word that sounds like the number "9" is "nein", which is German for "no".
So, the continued sequence would be: "0, done, 2, free, 4, hive, 6, heaven, 8, nein".
Reflecting on the pattern and the answer provided, the continuation seems to be consistent with the established pattern. We alternated between numbers and words that phonetically resemble numbers, resulting in the sequence "6, heaven, 8, nein" for the next four entries.
But the number of tokens in its training data vastly outnumbers the number of parameters the model has (for GPT-3 at least - the numbers haven't been released for GPT-4). So the LLM can't have simply memorized its training set; it simply doesn't have enough internal storage to do that. So it is not just regurgitating portions of its training set.
Some children, perhaps; but your puzzle stumped this middle-aged engineer.
Edit: I want to add I even get GPT3 getting it right without explanation to be honest. Even that one is outsmarting me. 4 is using terms like homophone. I'm not sure you proved what you thought you proved.
On the one hand, Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo). Just as much, LLMs don't express their creators values 'cause they don't express any values, they average language responses (though people certainly can see them expressing values, which can cause problems for the people).
On the other hand, the point that their creators don't understand them and are still quite willing to toss them to the open Internet really show considered carefully whatever their exact capabilities.
The fact is that they acquired abilities no one expected them to acquire. In hindsight you can say it was embedded in language and maybe you could have seen it coming, but it is an empirical fact that this was unexpected and unpredictable beforehand.
And it was only unexpected to those that weren't following the news the GPT 3 paper was published in 2020 and detailed this wild advancement in its capabilities[0,1].
0: https://www.infoq.com/news/2020/06/openai-gpt3-language-mode...
1: https://news.ycombinator.com/item?id=23885684
I think the author would say those are not the capabilities being discussed here.
Leaving that aside, I really take issue with the style used by the author. For example, section 3 begins:
> There is increasingly substantial evidence that LLMs develop internal representations of the world to some extent, and that these representations allow them to reason at a level of abstraction that is not sensitive to the precise linguistic form of the text that they are reasoning about.
LLMs do not "reason"; they do not "learn" or "develop" anything of their own volition. They are (advanced) statistical models. Anthropomorphizing them is not just technically incorrect, but morally disingenuous. Writing about LLMs in this way causes people with less domain-specific knowledge to trust the models the way they might trust people, and this has the potential for serious harm.
Because of this choice of phrasing, I wanted to look into the author's background. Among their recent activities, they list:
> I'm organizing a new AI safety research group at NYU, and I wrote up a blog post explaining what we're up to and why.
"AI Safety" is a distinct (and actually opposing) area from "AI Ethics". The people who prefer the word "safety" tend also to engage in discussions that touch on aspects of longtermism. Longtermism is not scientifically well-grounded; it seeks to divert attention from present and real issues to fanciful projections of far-future concerns. I do not know for certain that the author is in fact a longtermist, but their consistent anthropomorphization of a pile of statistical formulae certainly suggests they wouldn't feel out of place among a crowd of such people.
In contrast, the people who prefer the term "ethics" in their work are grounded in real and present issues. They concern themselves with reasonable regulation. They worry about the current-day environmental impacts of training large models. In short, they are concerned with actual issues, rather than the alleged potential for a statistical model to "develop" sentience or exhibit properties of "emergent" intelligence (subjects from the annals of the science-fiction writing of last century).
I hope the author can clarify their choice of phrasing in their work, though I worry they have chosen their words carefully already. Readers should exercise caution in taking the claims of a soothsayer without a sufficient quantity of salt.
If this claim of yours was sustainable, your grievance with the writing style would make a lot of sense. But GPT-4 can clearly reason about novel problems which were not in its training data. I think you should consider reading some of the many citations in the document which describe examples of that happening.
you've offered an anecdote. Some LLMs (more generally, this type of neural network model) will generate configurations that can be understood as a representation; others will not. The fact that the authors were able to find an apparent model in a heavily rule-based system is not incredibly surprising but offers little clue about whether this is "clearly" true for similar architectures that represent very different (real) world systems.
From your citation itself:
> More broadly, it would be interesting to study how our results generalize to models trained on natural language. One stepping stone might be to look at language models whose training data has included game transcripts. Will we see similar representation of board state?
From TFA's conclusions section:
"Open debates over whether we describe LLMs as understanding language, and whether to describe their actions using agency-related words like know or try, are largely separate from the questions that I discuss here (Bender & Koller, 2020; Michael, 2020; Potts, 2020). We can evaluate whether systems are effective or ineffective, reliable or unreliable, interpretable or uninterpretable, and improving quickly or slowly, regardless of whether they are underlyingly humanlike in the sense that these words evoke."
As a non-expert in the field I was hesitant at the time to disagree with the legions of experts who last year denounced Blake Lemoine and his claims. I know enough to know, though, of the AI effect <https://en.wikipedia.org/wiki/AI_effect>, a longstanding tradition/bad habit of advances being dismissed by those in the field itself as "not real AI". Anyone, expert or not, in 1950, 1960, or even 1970 who was told that before the turn of the century a computer would defeat the world chess champion would conclude that said feat must have come as part of a breakthrough in AGI. Same if told that by 2015 many people would have in their homes, and carry around in their pockets, devices that can respond to spoken queries on a variety of topics.
To put another way, I was hesitant to be as self-assuredly certain about how to define consciousness, intelligence, and sentience—and what it takes for them to emerge—as the experts who denounced Lemoine. The recent GPT breakthroughs have made me more so.
I found this recent Sabine Hossenfelder video interesting. <https://www.youtube.com/watch?v=cP5zGh2fui0>
>Anthropomorphizing them is not just technically incorrect, but morally disingenuous.
Wat
In academia (where I am, though my specialty is a step removed), "AI" is exactly the region of research concerned with statistical problem-solving. Machine learning is sometimes synonymous, though I haven't been able to determine whether this is consistent among all self-described AI researchers or just some of them.
Systems like GPTs are not capable of "thought", full stop. There is no ambiguity. If this is hard for you to accept, I'm sorry, but it is a fact. They have no agency.
I saw someone sketch a good explanation. They were responding to a claim that Chat GPT could "pass the Harvard admissions process". While it is true that the system can access sufficient data to respond accurately to questions on an exam, and it could be prompted to generate an essay, it cannot of its own volition choose to submit an application. It doesn't even "know" that Harvard is a real place to which it can apply, nor does it "understand" how it could go about "learning" how to find out about these things it does not know. It simply isn't capable of these things. (And even if it were that wouldn't necessarily be evidence of intelligence, but for now it's a sufficient distinguisher.)
> Wat
Can you explain what about my last point was "wat"-worthy? Do you disagree that it is morally problematic to pretend a thing is sentient when it is in fact not?
We may just live in a world where there’s room for more than one viewpoint and people are able to research different aspects of the same thing without getting into debates over dogma.
I mean, inclusion -> https://news.ycombinator.com/item?id=34698769
AI safety is not a "viewpoint", it is a marketing strategy. The goal of the people leading that charge is to drum up FUD to increase investment in their useless snake oil companies. I feel no remorse for not being inclusive of their "perspective".
Thank you for a pleasant break!
https://www.patheos.com/blogs/wakeupcall/2020/05/10-thought-...
Maybe when the technology is physically capable of thinking for itself, rather than just being statistically likely to respond in a way that appears like the way a human would respond. Which may never happen, for what it's worth.
> I think LLMs are at the point where adopting an "intentional stance" (using Daniel Dennett's term) is reasonable.
I hadn't heard of this term so I looked into it, and wow, what a biased Wikipedia article it has. It's written with a completely partial stance. I've no idea whether to take it seriously, to be honest.
That said, I disagree strongly. LLMs do not "intend" anything, and pretending they do is (in my view) highly problematic. They're just (sophisticated!) prediction engines; nothing more.
What definition of reasoning do you have in mind such that LLMs don't do it?
While not learning, they can still run inferences on what they are fed as context within the token space. And because they are becoming so good at getting context out of structured data, they can use the context provided as a layer on top of their knowledge.
So you can at a very good approximation upload knowledge into them, and such knowledge can represent how to use tools to fetch data they don't know about - i.e. what langchain agents are doing.
Now of course that is not 'llm' some, is llm + code for informaron discovery, but that code is entirely supported and enabled by the existence of good enough llm
Seriously just watch. He's not actually going to be able to coherently define his "reasoning" in a way that can be tested.
We are talking about a model of language and our expectations for what the words would look like if a person were using language to reason with. People are confusing one thing (their own interpretation of what they are reading) with another (a statistical model that is being driven to align with certain specific human expectations).
The alternative is having 1000 different results for different kinds of arrows, and averaging out the results for the ones similar to the input arrow.
An LLM is working on text tokens, it’s trying to give the most statistically common next token based on everything that’s been fed into it. Does that statistical model abstract the objects and concepts it talks about? Eh? I don’t know
Its actually a crazy idea that the only thing we needed to do to make LLMs more effective was just scale up the parameters
Though the transformer architecture definitely made it possible to handle the complexity
https://wp.nyu.edu/arg/why-ai-safety/
Is that true? Because LLMs are trained on the work of human: a LLM that learned all of the content of the articles would at best yield output equal to that of the human writing.
The reasons the author gives seem unfounded to me:
> First, they are trained on far more data than any human sees
If the human writer has access to google (everyone in modern society) this point is moot.
> In addition, they are often given additional training using reinforcement learning before being deployed.
If this is human in the loop RL, then the upper bound would still be the human training it. If it isn't, refer to #1.
There's also a difference between having access to and being able to 'comprehend' all previous works in a given genre for generating music, that a human simply won't have the ability to process and recall when creating their own music and the AI could surpass. Humans simply having access to information is different than using information.
But a single human can only read so much information in a finite amount of time (their life). Sure, the same thing applies to LLMs, but the theoretical limits are orders of magnitude higher.
And while this probably isn't what you were talking about, it's also pretty clear that any time-limited task favours LLMs. There's no human on earth that could write a non-trivial short story in 10 seconds, regardless of quality level.
Does not follow at all. There’s a vast difference between being able to search for and internalize a tiny, tiny sample of all the available information, and literally having sampled and synthetized all of it.
> If it isn't, refer to #1.
Meaning what? LLMs use adversarial learning, the same thing that allowed AlphaGo to reach superhuman levels in Go.
Would that enabled these models to compress themselves?