The day an AI answers "Drive." without all the fuss. That's when we are near AGI ig
Deleted Comment
What I'm slightly worried about is that eventually they are going to want to monetize LLMs more and more, and it's not going to be good, because they have the ability to steer the conversation towards trying to get you to buy stuff.
That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do build separate mental models of the llm and are able to work around it).
This is a neatly isolated toy-case, which is interesting, because we can assume similar issues arise in more complex cases, only then it's much harder to reason about why something fails when it does.