It really looks like ChatGPT suffered from some form of "initial over hype syndrome". I'm not underestimating it, actually I think the application of transformers to do what ChatGPT is capable of doing is something really impressive, novel and caught the world by surprise. I'm sure we'll find many equally impressively good applications for it.
But people eventually found its limitations. And did it quite fast. People learned that it is not as trustworthy as initially thought and it is also very convincing when it is wrong. It maybe very interesting to generate texts, to startup small code for functions, to query information it has "cataloged", find some trivial mistake, make suggestions and... well, not much more than that. It may save time to boot projects, but it is not capable of managing anything larger than its "memory".
I think people are now actually more impressed by things it can't do easily. It can't play hangman, chess, tic-tac-toe... It got the phase of the Moon wrong when I asked it "What was the phase of the Moon when John Lennon was killed".
So, once people get hit by one of its mistakes or limitations, it sticks more than the "impressive part". That means, people will certainly ask themselves "Should I trust a thing that can't even play tic-tac-toe?"
I had a brief period of thinking it wasn’t as impressive as I thought, but came to realize that it’s still an incredibly useful tool for research and discovery, and occasionally debugging or loosely architecting code. I’m sure for others it has genuinely powerful and useful capabilities.
If you want it to do your job for you though, you’re going to have a disappointing time.
I almost think of it as a helpful method of finding what work I should do. It doesn’t really do any of the labour for me. Like if I want to find research papers about x, it’ll find things really well and make interesting connections between papers I otherwise might not. But I won’t rely on it to condense the papers or surface important details; you really still need to do that legwork. Same with code. It can give broad, loose, useful suggestions, but you really need to write it yourself. That’s fine with me.
This is my experience. I was initially sceptical, then wowed, then stopped using it entirely when I found out how utterly wrong it was about something it really should have been able to do well.
I gave it a programming problem I was having a hard time with, and it came up with the same incorrect answer I had. I explained which part of the answer was wrong, and why. It's response was to apologise, acknowledge that part as being incorrect ... then give me the exact same answer again. I explained that it had given me the same answer, and it apologised, acknowledged it's mistake, and gave the same answer.
If any tool I use behaved like that, I would not use that tool, as the dangers far outweigh any benefits.
> I was initially sceptical, then wowed, then stopped using it entirely...
Similar as well in my case.
I think that my problem is that the core of its answers are extremely confident, and it's personally hard for me to switch into the "doubt"-mode when I read such a confident reply.
I still use it, but very very rarely, and only when I'm desperate to get at least some hints (and I'm therefore psychologically ready to doublecheck all infos contained in the answer).
People were being told it would replace massive numbers of workers—upper-middle-class knowledge workers, like us—almost instantly. That implies a very high level of cognition and understanding that anyone who interacted with it without a strong preconceived bias in favor of seeing such things would quickly see does not exist.
No, it's not Eliza, but it's also not Skynet, and it was basically being hyped as Skynet. Not by everyone, and not to everyone, but easily enough that when people see that it's way, way less than the strong AI they were promised, they're going to go away disappointed and decide it's not worth their time at all.
I know this is sarcasm, but is this not similar to what ChatGPT with bing integration was doing?
I had limited exposure before it got removed, but it would actually spell out the searches it would do (often more than one) to answer your question - when it worked it was actually pretty magical.
I never did look into the mechanism for when it decided to search the web, versus just provide you an answer however.
This is what i thought, until i stumbled across ReAct prompts. The tldr is you prompt the llm to give its thought, provide it with guides for actions to take,(you do this through code) then give the answer back to the llm, itll continue in this loop. for the lennon questions
> Me: What was the moon phase when john lennon was killed?
> GPT: Thought: I need to find out the moon phase on the date when John Lennon was killed.
Action: google: Moon phase on December 8, 1980
PAUSE
I didnt code this I just did the search my self following is the google result
> ME: The Moon phase for December 8th, 1980 is a Waxing Crescent phase.
> GPT: Observation: The Moon phase for December 8th, 1980 was a Waxing Crescent phase.
Answer: The Moon phase when John Lennon was killed on December 8th, 1980 was a Waxing Crescent phase.
these iterative thoughts and actions really sold me on it. theres papers that go more indepth
It's amazing that the article's author talked at length about the hype decreasing yet didn't realize that this is just the Gartner hype cycle in action [0]. We are currently on the downward slope towards the trough of disillusionment after the huge peak, but over time we'll equalize towards the new status quo of using AI everywhere and not even thinking about it, same as we do with smartphones now.
This article paints an overly negative picture. It's fairly obvious that AI is only going to improve from where it's at today, and the hyperbolic language about no one caring makes me less inclined to believe that the author is attempting to paint an honest picture. I derive immense value out of Gpt4 as an aid for coding, a search engine replacement that accepts open ended questions, a writing aid, etc. I know someone who has poor writing skills and used Gpt4 to improve their resume / cover letters.
He links to an article stating that ChatGpt use is down 10% in June as the source for the doom and gloom, but this is orthogonal to whether usage is low. The drop likely was due to school ending for the summer.
We're talking about an app that was the fastest growing app in human history prior to Threads launching. Decreased usage numbers implies that not everyone is deriving value from it, but there are a lot of people who are, it really depends on what your job is and your personality type probably plays a role too. There's also probably a backlash because people overstated what it could do, but what it can do is still extremely useful. I think stating AI is a fad ignores all of this, and again, ignores the massive increase in AI capabilities that we're going to see over the coming years.
It’s suffering from extreme overhype pushed by OpenAI. They wanted people to think the human-led world was over and they were first to market on creating the new overlords. This soured them on a lot of tech people and ensured whatever they built could never live up. Still a great product and I use a chatbot everyday for work, but it’s just that a chatbot.
Absolutely, combine this with things like AI that describe videos? We haven't even begun to milk what these LLM can offer. From mass surveillance to easing every day burdens anything that handles messy information has potential for improvement and its a long time before all the possible improvements have been implemented.
It's weird to me that some people just don't see the oncoming train. AI is here to stay. Use cases are still being worked out, but it's here to stay.
Reminds me of the Astound application around 1995-1996. It was a pretty major PowerPoint competitor and I used it a lot. For whatever reason, they only had a Windows 3.1 version. So I called them asking when the Windows 95 version was coming out. They told me they didn't think Windows 95 will be anything more than a fad and therefore they are sticking with what they have.
I remember us laughing in the office about how an entire company could be so clueless.
Funny, because AI has seen plenty of applications before ChatGPT, yet "the oncoming train" in most peoples heads is "this is going to completely replace artists, programmers, etc"
This is what is laughable, and has been from the very beginning. Will we _continue_ to use AI to try and find novel drug treatments? Yes. Will chatbots still be the first thing a customer support flow uses (and will it still suck)? Yes.
I gave a presentation to our CTO this morning on utilizing LLMs to do a myriad of enterprise DX tasks. The amount of excitement that came out of that meeting was thick enough to cut through.
This ain't going no where and will only be integrated into everything we have around us.
Heck, soon, i'll start building my own dumb version of Jarvis i can talk to at home, which i'm truly excited for.
Even with the graphic in the article, it has tablets in around 2010, there were definitely tablets well before that. They took a while to grow and take off, seems similar here.
AI has been "the next big thing" since the 30's. The problem with AI is they still haven't found out how to actually deliver on that second word - intelligence. Until then, it's nothing more than an interesting toy.
> With every passing day, OpenAI looks more like Napster or the many defunct piracy platforms—it relies on the creativity of others to make a buck. And there are plenty of laws against that.
This seems like an interesting comparison.
Napster didn’t take over the record industry of course. But, people don’t buy physical albums anymore for the most part, either. The founder of Spotify called Napster part of his inspiration.
Napster was the half formed idea that resulted from the technical aspects getting ahead of the legal and business ones. Modern music streaming was a business-friendly response to it, a necessary response, something almost as convenient as file sharing.
This OpenAI stuff is weird because, imagine if Sony had invested heavily in Napster and tried to build products around it before people worked out the legal issues. What would streaming look like now, if Napster had been able to borrow some legitimacy and lawyers from incumbents? (And also if Napster had been able to buy a bunch of hardware that enabled business models that, like, Kazaa or Limewire weren’t able to replicate).
IMO the final version wouldn’t look much like p2p file sharing, but it also might not look like the, basically, record industry storefront streaming we have now.
I think we’re in weird times. I don’t think the LLMs will look much like what OpenAI has now in the end, but I wonder if they’ll end up less pushed aside than Napster, given the support they’ve already got.
I feel this might be more of a mild 'Trough of disillusionment' in the hype cycle than a real shrinking of demand. The other thing with ChatGPT specifically is that during really high usage times, its usability was impacted, so may have driven off some users. Also, ChatGPT has been RLHFed to death. Outside of regular corporate or homework like use cases, the output often feels bland and overspiced with adjectives. I hope other competitors can come up with models that are not so Politically Correct and more creative even if at the risk of offending a user or two.
This meme that models are (mainly) hindered by the constraints on political correctness is really tiring. The alignment tax to get that is really not that huge, and what are the use cases enabled by not having it? Kinky roleplay dialogues? The real thing holding back these models IMO is more fundamental stuff like truthfulness and reasoning capabilities.
The vast majority of people don't care about truthfulness which ChatGPT doesn't care about anyway. Entertainment is a way bigger driver of usage than utility.
I think I'm a relatively slow adopter of AI, but this feels a bit premature to call it a fad at this point. Everyone is still in a bit of a "wait and see" period, but we already see some of the disputes over rights to use AI recreations of actors indicate there is demand for companies who would want to use it if the legal situation was more defined and favourable.
I think at this point it's hard to tell if ChatGPT is like the early demos of self-driving cars (neat, but just never able to get the trust levels for mass adoption), or like the LG Prada, a sign of what to come, just needing someone to wrap it in iPhone levels of execution to succeed. Or maybe it's even the Newton to the iPhone.
There's a special place in hell for Substack writers that regurgitate other people's journalism under flashy headlines without contributing anything new.
Are you referring to the a36z sponsored conference for thought leaders? IIRC, "Flat Blocks of World Coin Eats Softest Chain Wares Con, 2024 And Beyond."
Something about searching and parsing through content myself just feels so satisfying. Being able to dive deep and find parts that are meaningful for the context I'm driving towards just makes the whole experience more enjoyable then having an AI tell me what meaning is based on training.
I prefer searching when I know what to search for, but I really like GPT for figuring out terminology and concepts when researching unfamiliar matters. But I agree, doing the actual leg work of researching and collecting data is ultimately still best done manually. I tend to use a combination quite often though.
Readit News
But people eventually found its limitations. And did it quite fast. People learned that it is not as trustworthy as initially thought and it is also very convincing when it is wrong. It maybe very interesting to generate texts, to startup small code for functions, to query information it has "cataloged", find some trivial mistake, make suggestions and... well, not much more than that. It may save time to boot projects, but it is not capable of managing anything larger than its "memory".
I think people are now actually more impressed by things it can't do easily. It can't play hangman, chess, tic-tac-toe... It got the phase of the Moon wrong when I asked it "What was the phase of the Moon when John Lennon was killed".
So, once people get hit by one of its mistakes or limitations, it sticks more than the "impressive part". That means, people will certainly ask themselves "Should I trust a thing that can't even play tic-tac-toe?"
If you want it to do your job for you though, you’re going to have a disappointing time.
I almost think of it as a helpful method of finding what work I should do. It doesn’t really do any of the labour for me. Like if I want to find research papers about x, it’ll find things really well and make interesting connections between papers I otherwise might not. But I won’t rely on it to condense the papers or surface important details; you really still need to do that legwork. Same with code. It can give broad, loose, useful suggestions, but you really need to write it yourself. That’s fine with me.
I gave it a programming problem I was having a hard time with, and it came up with the same incorrect answer I had. I explained which part of the answer was wrong, and why. It's response was to apologise, acknowledge that part as being incorrect ... then give me the exact same answer again. I explained that it had given me the same answer, and it apologised, acknowledged it's mistake, and gave the same answer.
If any tool I use behaved like that, I would not use that tool, as the dangers far outweigh any benefits.
Similar as well in my case.
I think that my problem is that the core of its answers are extremely confident, and it's personally hard for me to switch into the "doubt"-mode when I read such a confident reply.
I still use it, but very very rarely, and only when I'm desperate to get at least some hints (and I'm therefore psychologically ready to doublecheck all infos contained in the answer).
https://en.wikipedia.org/wiki/Gartner_hype_cycle
No, it's not Eliza, but it's also not Skynet, and it was basically being hyped as Skynet. Not by everyone, and not to everyone, but easily enough that when people see that it's way, way less than the strong AI they were promised, they're going to go away disappointed and decide it's not worth their time at all.
What we got was a better MegaHAL.
What I don't understand is this: when the AI doesn't have >95% confidence it it's answer, why doesn't it just Google like the rest of us? /s
I had limited exposure before it got removed, but it would actually spell out the searches it would do (often more than one) to answer your question - when it worked it was actually pretty magical.
I never did look into the mechanism for when it decided to search the web, versus just provide you an answer however.
> Me: What was the moon phase when john lennon was killed?
> GPT: Thought: I need to find out the moon phase on the date when John Lennon was killed. Action: google: Moon phase on December 8, 1980 PAUSE
I didnt code this I just did the search my self following is the google result
> ME: The Moon phase for December 8th, 1980 is a Waxing Crescent phase.
> GPT: Observation: The Moon phase for December 8th, 1980 was a Waxing Crescent phase. Answer: The Moon phase when John Lennon was killed on December 8th, 1980 was a Waxing Crescent phase.
these iterative thoughts and actions really sold me on it. theres papers that go more indepth
[0] https://en.wikipedia.org/wiki/Gartner_hype_cycle
You picked an interesting example, were smartphones ever even part of the gartner hype cycle.
I think it was just a straight steady upwards line that rounded off.
He links to an article stating that ChatGpt use is down 10% in June as the source for the doom and gloom, but this is orthogonal to whether usage is low. The drop likely was due to school ending for the summer.
We're talking about an app that was the fastest growing app in human history prior to Threads launching. Decreased usage numbers implies that not everyone is deriving value from it, but there are a lot of people who are, it really depends on what your job is and your personality type probably plays a role too. There's also probably a backlash because people overstated what it could do, but what it can do is still extremely useful. I think stating AI is a fad ignores all of this, and again, ignores the massive increase in AI capabilities that we're going to see over the coming years.
It's hard to gauge hype except retrospectively. We will know in 4-6 years if it was over hyped.
Reminds me of the Astound application around 1995-1996. It was a pretty major PowerPoint competitor and I used it a lot. For whatever reason, they only had a Windows 3.1 version. So I called them asking when the Windows 95 version was coming out. They told me they didn't think Windows 95 will be anything more than a fad and therefore they are sticking with what they have.
I remember us laughing in the office about how an entire company could be so clueless.
This is what is laughable, and has been from the very beginning. Will we _continue_ to use AI to try and find novel drug treatments? Yes. Will chatbots still be the first thing a customer support flow uses (and will it still suck)? Yes.
Instead of having to collect a bunch of data, clean it, and train on it, you can simply write plain, human prompts. That’s extremely powerful.
This ain't going no where and will only be integrated into everything we have around us.
Heck, soon, i'll start building my own dumb version of Jarvis i can talk to at home, which i'm truly excited for.
This seems like an interesting comparison.
Napster didn’t take over the record industry of course. But, people don’t buy physical albums anymore for the most part, either. The founder of Spotify called Napster part of his inspiration.
Napster was the half formed idea that resulted from the technical aspects getting ahead of the legal and business ones. Modern music streaming was a business-friendly response to it, a necessary response, something almost as convenient as file sharing.
This OpenAI stuff is weird because, imagine if Sony had invested heavily in Napster and tried to build products around it before people worked out the legal issues. What would streaming look like now, if Napster had been able to borrow some legitimacy and lawyers from incumbents? (And also if Napster had been able to buy a bunch of hardware that enabled business models that, like, Kazaa or Limewire weren’t able to replicate).
IMO the final version wouldn’t look much like p2p file sharing, but it also might not look like the, basically, record industry storefront streaming we have now.
I think we’re in weird times. I don’t think the LLMs will look much like what OpenAI has now in the end, but I wonder if they’ll end up less pushed aside than Napster, given the support they’ve already got.
I think at this point it's hard to tell if ChatGPT is like the early demos of self-driving cars (neat, but just never able to get the trust levels for mass adoption), or like the LG Prada, a sign of what to come, just needing someone to wrap it in iPhone levels of execution to succeed. Or maybe it's even the Newton to the iPhone.
I’m back to search being more useful.