Really interesting analogy in the video is the discussion about the system thinking from book Thinking, Fast and Slow by Daniel Kahneman
System 1 thinking: Fast automatic thinking and rapid decisions. For example is when someone ask you 2 + 2, you don't think. You just reply quickly instantly. LLMs currently only have system 1 thinking.
System 2 thinking: Rational slow thinking to make complex decisions. For example when someone ask you 17 x 24 you think slowly and rationally to multiply. This kind of thinking is a major component we need for AGI. Current rumor from OpenAI about so called "Q*" algorithm could be something related to system 2 thinking (Just speculation at this point)
System 1 thinking: Does x sentence sound like a correct English sentence.
System 2 thinking: Verify x sentence is a correct English sentence by using grammar rules.
Someone fluent in English can form correct English sentences using only system 1 thinking, while someone that has just started learning English must think about grammar rules (using system 2 thinking) to do it.
I wonder why OpenAI doesn't try to get more feedback and training data from its users, though I do notice that sometimes it'll give me two answers and ask me to pick the better one.
For example I've noticed that a lot of the time when I ask ChatGPT a coding question it might get 90% of the answer. When I tell it what to fix and/or add, it usually gets the answer. I wonder if they're using these refined answers to fine-tune those original prompts.
I wonder how the LLM interacts with other software like the calculator or Python interpreter. It would be great if this were modular so that the LLM OS could be more like Unix than Windows which is what OpenAI seems to be trying to emulate.
Ultimately though it seems to me like AGI is fairly straightforward from here. Just train on more quality data - in particular enabling the machine to generate this training data, increase parameter size, and the LLM just gets better and better. Seems like we don't even need any new major breakthroughs to create something resembling AGI.
They should be capturing the changes that people make to the ChatGPT outputs. Many people will be copying the outputs to some other application and then make changes. If open AI would make it easier to modify the outputs right within ChatGPT, they could use that as feedback. Basically, fuse the end-user UI with the UI of the annotates.
I have zero faith that the average ChatGPT user will make quality edits. If anything, this invites trolling and active dataset poisoning/manipulation the moment people figure out that's what they're doing.
Karpathy has an excellent zero-to-hero series on the topic in which he explains the very core of the neural networks, LLMs and the related concepts. With no background on the topic, I was able to get an idea what's all this about and even become dangerous: https://karpathy.ai/zero-to-hero.html
There's something enlightening in hands-on learning without using metaphors. He even opens the code of production grade tools to show you how exactly the concepts he explained and build together are actually implemented IRL.
This is a style of teaching that clicks with me. I don't learn well with metaphors and high abstractions and find it magical to remove the magic of amazing things and bring it down to easy to reason pieces which can create a complex structure with composition so you can just disregard the complexity as a separate thing of the core.
This was an incredibly informative talk, especially the ideal of giving LLMs the System 2 thinking capability. I think if LLMs can do system 2 thinking we are one more step closer to AGI.
I’ve summarised the talk here - https://gist.github.com/anupj/f3a778dcb26972ba72c774634a80d7... - if you anyone wants to RAG the text for their custom GPT :)
This video was very informative and clear for a beginner like me who is curious about AI and ML. I'd like to learn more about how to finetune llama for different tasks and domains. Does anyone have any recommendations for resources that explain this concept in a simple way and gradually introduce the technical details and tools required?
Readit News
I was thinking of this other video he had published early this year: "Let's build GPT: from scratch, in code, spelled out" https://www.youtube.com/watch?v=kCc8FmEb1nY
Karpathy is generally well-reputed as a good tutor, especially for complex topics in AI / ML.
System 1 thinking: Fast automatic thinking and rapid decisions. For example is when someone ask you 2 + 2, you don't think. You just reply quickly instantly. LLMs currently only have system 1 thinking.
System 2 thinking: Rational slow thinking to make complex decisions. For example when someone ask you 17 x 24 you think slowly and rationally to multiply. This kind of thinking is a major component we need for AGI. Current rumor from OpenAI about so called "Q*" algorithm could be something related to system 2 thinking (Just speculation at this point)
System 1 thinking: patterns.
System 2 thinking: logic.
For example:
System 1 thinking: Does x sentence sound like a correct English sentence.
System 2 thinking: Verify x sentence is a correct English sentence by using grammar rules.
Someone fluent in English can form correct English sentences using only system 1 thinking, while someone that has just started learning English must think about grammar rules (using system 2 thinking) to do it.
For example I've noticed that a lot of the time when I ask ChatGPT a coding question it might get 90% of the answer. When I tell it what to fix and/or add, it usually gets the answer. I wonder if they're using these refined answers to fine-tune those original prompts.
I wonder how the LLM interacts with other software like the calculator or Python interpreter. It would be great if this were modular so that the LLM OS could be more like Unix than Windows which is what OpenAI seems to be trying to emulate.
Ultimately though it seems to me like AGI is fairly straightforward from here. Just train on more quality data - in particular enabling the machine to generate this training data, increase parameter size, and the LLM just gets better and better. Seems like we don't even need any new major breakthroughs to create something resembling AGI.
There's something enlightening in hands-on learning without using metaphors. He even opens the code of production grade tools to show you how exactly the concepts he explained and build together are actually implemented IRL.
This is a style of teaching that clicks with me. I don't learn well with metaphors and high abstractions and find it magical to remove the magic of amazing things and bring it down to easy to reason pieces which can create a complex structure with composition so you can just disregard the complexity as a separate thing of the core.
An aside... incredibly, it looks like he recorded in one cut from his hotel room.
Deleted Comment