$6 billion dollars raised in 2 months for their series C is blowing my mind. What does Anthropic have that OpenAI or other LLM startups don't? What other companies have raised that much in a single round?
It's such a small playing field with too few players.
I have a slide in a presentation on the topic that's an inverted pyramid, as pretty much the entire LLM field rests on only a few companies, who aren't necessarily even doing everything correctly.
The fact that they even have a seat at such a small table with so much in the pot already and much more forecasted means they command a large buy in regardless of their tech. They don't ever need to be in the lead, they just need to maintain their seat at the table and they'll still have money being thrown their way.
The threat of missing the next wave or letting a competitor gain exclusive access is too high at this point.
Of course, FOMO driving investments is also the very well known pattern of a bubble, and we may have a bit of a generative AI bubble among the top firms where large sums of money are going to go down the drain on investing into overvalued promises because the cost of missing a promise that will actually come to fruition is considered too high.
Ironically the real payoff is probably in focusing on integration layers at this point, particularly given the gains in performance over the past year in research by developing improved interfacing with SotA models.
LLMs at a foundational pretrained layer are in a race towards parity. Having access to model A isn't going to be much more interesting than having access to model B. But if you have a plug and play intermediate product that can hook into either model A or B and deliver improved results to direct access to either - that's where the money is going to be for cloud providers in the next 18 months.
The "big context window" talk reminds me of high wattage speakers in the 80s. Yes, it's loud. "Does it sound good?", is the question.
Having a large context window is pointless unless the model is able to attend to attention on the document submitted. As RAG is basically search, this helps set attention regardless of the model's context window size.
Stuffing random thing into a prompt doesn't improve things, so RAG is always required, even if it's just loading the history of the conversation into the window.
This is just a prompting hack you can use with any LLM, not exclusive to Claude. But I do like the fact that they include these tricks in their documentation.
The large(100k tokens) context window together with the fact that it can actually use the information in that context window. From personal experience other models including open ai fail to properly answer when provided large(more than 5k tokens) inputs as context even when the model officially accepts much larger contexts. But Claude 2 models are uncannily good at taking all that context into consideration.
I don't see how that could even register. We use OpenAI at home (both me and my wife) but everything ChatGPT spouts must, invariably, be verified.
It's not even "trust but verify" but "Oh yup, I didn't thought of that. However it may be a lie because ChatGPT is a pathological liar, and as I cannot possibly trust a lying tool, I'll now verify" (I know, I know, there's a discussion regarding nomenclature: "lying" / "hallucinating" or whatever. But anyone who's actually using ChatGPT knows what I mean).
Basically the output of ChatGPT, for me, goes directly into Google / Wikipedia / etc.
The one case where I can use the output of ChatGPT directly is when I translate from, say, english to french or vice-versa and I know both languages well enough to be able to tell if the translation is okay or not.
Those believing they can use the output of ChatGPT without verifying it are basically these lawyers who referenced hallucinated cases to a judge.
As another person as commented: it didn't even make a dent in Google's search requests and that is no surprise.
It starts with the power users. GPT4 has certainly had a big impact on my Google searches.
For example, all my tech support searches are now GPT4. Those are painful on Google. There's no need to verify with a Google search, since you can just try out what GPT4 says.
Concrete example: I use it all the time to help me with Excel. What it suggests is nearly always correct. It has turned me into an Excel power user within a few weeks.
You need to develop a sense for what it's likely to be correct on, but once you do it's insanely useful. Simple rule of thumb: if you think you'd find a direct response to your question by wading through pages of ad infested Google results, it'll definitely work great on GPT4.
The way I recall it, it took multiple years for Google to go from a secret power user thing to displacing Yahoo and Altavista for the broad user base. And that was at a time where being online in itself was sort of an early adopter thing.
Anyway, I guess my point is, I would be worried if I was Google, and ignore this tech at your own risk...
ChatGPT remixes info from beyond the first page of Google search. That’s the value. If you ask it for a list of nice nature spaces in Tokyo, like I just did, it returns 12 spots that all seem appealing. Already, that’s more information density than a Google Search. But now I have to go look up if these actually exist (this isn’t the sort of mistake it usually makes though), where it is, the hours, admission prices etc. So that’s going to be a few Google searches for this question. Of course, I’ll have to actually go to the sites for these gardens, if they exist, because you can’t quite trust Google Maps’s accuracy either - hours and opening days can be off, especially around the holidays, when I’d like to go to Tokyo. One ChatGPT query -> several Google searches.
If for this text output, ChatGPT also linked me directly to the gardens’ sites, scraped the info from the live site, and summarized this - that would actually save a ton of time. Google could have a leg up bc it has a knowledge graph, but so does Microsoft. This requires a lot more than training an LLM - this requires it to be an actual product, not a tech demo like it is. A chat with an agent that occasionally lies is a terrible UI.
I think there’s great scope for UI innovation here. But such an experience might be pretty expensive in terms of compute - lots of LLM queries and extra lookup systems. Someone who does this hard integration work and is willing to spend a lot of resources per query will deliver a delightful, time-saving user experience, and can probably charge for it. And that may be a great value-prop for local AI - you can give it tons of resources to solve your particular problem. As I see it, mass market LLMs that are provided for free will never do this extra work for you. ChatGPT might be in a good position bc it already has a ton of paying customers that it can continue to draw a wall around. Their early Nov announcement might be something along these lines.
I don't know, I make plenty of queries that don't need verification. I would say the majority.
Write a polite email to X saying Y -> doesn't need verification.
Rewrite this in formal English for a grant application -> doesn't need verification.
How to tar gz a folder in Linux? -> doesn't need verification. When I get the answer, I will probably think "oh, sure, it was czvf". And even if I didn't know what the arguments were, I would know enough to know that a tar command isn't going to delete my files or anything like that, so I would just try and see if it worked.
Write Python code to show a box plot with such and such data -> doesn't need verification, I'm too lazy to write the code myself (or look into how seaborn worked) but once I see the code, I can quickly understand it and check that it does what it should. Or actually run it and see if it worked.
Brainstorming (give me a bunch of titles for a paper about X/some ideas about how I could do Y) -> doesn't need verification, I can see the ideas and decide which I like and which I don't.
I get that if you're a journalist, a lawyer or something like that, probably the majority of your ChatGPT queries will need verification, but that's definitely not my experience... probably because I don't often ask ChatGPT for things that I don't know, most of my use is either for things that I could do myself but require a time investment and I'd rather ChatGPT does them in a few seconds, or for brainstorming. Neither of those require a Google search at all.
I don’t use chatgpt or LLMs for anything factual at all given the workflow you’ve described of asking a question and then needing to separately search all results. But they are very useful for framing documents, summarizing things, etc.
I use it with Browse with Bing and it works quite well for keeping up with research on a diverse set of topics - CS, AI, economics, neuroscience etc. The citations help you verify quickly.
And this is besides the other stuff that search ones can’t do directly but it does quite well. It has definitely cut web searches for me.
The types of queries where gpt is useful (or I hope will soon be useful) to me are those where google themselves have "destroyed the internet.^"
If you want to basic information about a physical exercise, a recipe, a travel destination or such... All the content in Google seems to be content farmed. Written by people who don't know, quickly, by copying similar articles. It's just a buggy, manual version of the proverbial gpt anyway. At this point, I may as well just go direct.
If I want to know about doing weighted crunches instead of sit ups, Google results already represent all the downsides of using gpt. The got UI is nicer. You can also narrow in on your questions, push back and generally get the same level of content in a better package.
Maybe the old chestnut of "computerized recipe book" will finally be solved.
Casual users don't care about ChatGPT or any other smart chat app because they are used to Google Search or classical search bar type web search for decades now.
If chat apps take over, I think they will first taker over desktop computers because on mobile it is easier to type a few keywords in the web search bar and get 5 or 6 relevant web results than it is to chat with a chat bot for like 5 minutes.
Informational quires are most likely to be more useful in chat apps than in the classical web search but navigational and transactional quires are here to stay on Google or whatever search engine comes about.
I was going to say that mobile can just chat on the mic to the gbt but then I remembered speech to text is an apparently hard problem that no one has made progress on in 15 years
Google reportedly has $118 billion cash on hand. I think they would have happily invested more than $2 billion in an OpenAI competitor if they thought more money than that would matter.
I'm struggling in finding the source, but I read some in depth analysis that reports that bing only got 0.53% market share since the launch of chatGPT integration. Apparently they are cannibalising small players but not google.
the supermajority of humanity doesn't know what chatGPT is useful for, so google's total amount of searches are almost certainly the same. you could only detect the difference for, say, searches that would return stackoverflow results
In my experience, claude.ai outperforms OpenAI's ChatGPT in several key areas. Notably, claude.ai excels in long-tail document reading and recall, demonstrating superior comprehension and information retrieval. Additionally, claude.ai offers enhanced contextual understanding, allowing it to provide more relevant and precise responses.
However, while Claude.ai certainly showcases its strengths in specific areas, it doesn't quite measure up to OpenAI's ChatGPT in terms of adaptability and precision. ChatGPT stands out for its capability to understand intricate queries and produce nuanced, tailored responses.
Both platforms have distinct strengths; I firmly believe they'll evolve to dominate different niches in the AI ecosystem.
> In my experience, claude.ai outperforms OpenAI's ChatGPT in several key areas.
ChatGPT on GPT-3.5? I can buy this. GPT-4? No fucking way. If Claude got anywhere close, this would be front page news in every tech and tech-adjacent outlet.
This has to be said again and again: "ChatGPT" is meaningless without specifying whether you mean GPT-3.5 or GPT-4; statements about "ChatGPT" (or LLMs in general) capability limits are invalid unless tested on GPT-4.
> ChatGPT on GPT-3.5? I can buy this. GPT-4? No fucking way. If Claude got anywhere close, this would be front page news in every tech and tech-adjacent outlet.
Have you compared both? For my use case of generating & modifying simple code & Q&A based off of docs claude is in a similar range to GPT-4 & you can pay as you go instead of 20 bucks a month.
I’ve had the opposite experience. Soon after claude 2 came out, i pasted in a long document that started with the document’s title and it couldn’t answer a “what is the title?” question.
Amazon recently annouced their investment in Anthropic as well. Two, of the largest ad providers on the planet. Invest in the competition so you can have "input" into various aspects of the company, all the while profitting on any sucess.
There may be some interest in having access to Google’s machine learning hardware as well. What we don’t know publicly is the extent of any deals these companies have to share technology and infrastructure. Google is one of the only hyperscalers to have succeeded in building its own ML hardware. That’s a key asset.
Anthropic will be interesting company if founders lost voting power in the board, with FTX, Amazon and Google representatives arguing to each other on key decisions protecting their interests.
"""The Trust is an independent body of five financially disinterested members with an authority to select and remove a portion of our Board that will grow over time (ultimately, a majority of our Board). Paired with our Public Benefit Corporation status, the LTBT helps to align our corporate governance with our mission of developing and maintaining advanced AI for the long-term benefit of humanity."""
I'm confused. Ok, Bard isn't cutting edge but it's pretty damned good for every day stuff ... and I actually detest Google ... but I've been using it because I'm sick of Edge on my PC and I think the UI is just 'friendlier' and quicker.
They've already announced a replacement / better model coming but considering how long Google has been doing the AI thing, it's amazing they dropped the ball this badly.
This investment is Claude sounds like their backup plan.
Bard is so stupid that I tried the take a picture feature, asked a question about a picture, then when I asked a followup question about the picture it said "Sorry I can't look at pictures."
I do expect it to get much smarter by the end of next year.
I asked bard to make a comparison chart of OpenAI, Claude, and PaLM's specs (size and context window) and it repeatedly gave me a nice looking table with obviously wrong values.
If you've used the competition (ChatGPT, Claude, even Perplexity free version), you will find that Bard is not good in terms of quality of answers. (I'm forced to use Bard at work. The UI is responsive I'll give you that, but the answers are often more wrong than ChatGPT and Claude)
this happened to him several times, and each time he ended up betting it all again. if anthropic would make him whole, he'd bet it all on more bullshit crypto
Readit News
The investments are absurdly large but if you believe LLMs will be fundamentally transformative then I can understand the logic.
I have a slide in a presentation on the topic that's an inverted pyramid, as pretty much the entire LLM field rests on only a few companies, who aren't necessarily even doing everything correctly.
The fact that they even have a seat at such a small table with so much in the pot already and much more forecasted means they command a large buy in regardless of their tech. They don't ever need to be in the lead, they just need to maintain their seat at the table and they'll still have money being thrown their way.
The threat of missing the next wave or letting a competitor gain exclusive access is too high at this point.
Of course, FOMO driving investments is also the very well known pattern of a bubble, and we may have a bit of a generative AI bubble among the top firms where large sums of money are going to go down the drain on investing into overvalued promises because the cost of missing a promise that will actually come to fruition is considered too high.
Ironically the real payoff is probably in focusing on integration layers at this point, particularly given the gains in performance over the past year in research by developing improved interfacing with SotA models.
LLMs at a foundational pretrained layer are in a race towards parity. Having access to model A isn't going to be much more interesting than having access to model B. But if you have a plug and play intermediate product that can hook into either model A or B and deliver improved results to direct access to either - that's where the money is going to be for cloud providers in the next 18 months.
The big context window is pretty magical for some use cases. There are lots of things RAG with limited context can't do.
Having a large context window is pointless unless the model is able to attend to attention on the document submitted. As RAG is basically search, this helps set attention regardless of the model's context window size.
Stuffing random thing into a prompt doesn't improve things, so RAG is always required, even if it's just loading the history of the conversation into the window.
I dunno I kind of get the perception Microsoft's locked up OpenAI's funding so Google couldn't throw money at them even if they wanted to?
https://docs.anthropic.com/claude/docs/give-claude-room-to-t...
You can see how the AI arrived at the response.
Dead Comment
Anecdotally, I now use ChatGPT for at least 25-50% of the queries that I previously would have had no other channel for other than a search engine.
If I was in charge of Alphabet I'd be starting to worry. This move makes them look a bit desperate.
It's not even "trust but verify" but "Oh yup, I didn't thought of that. However it may be a lie because ChatGPT is a pathological liar, and as I cannot possibly trust a lying tool, I'll now verify" (I know, I know, there's a discussion regarding nomenclature: "lying" / "hallucinating" or whatever. But anyone who's actually using ChatGPT knows what I mean).
Basically the output of ChatGPT, for me, goes directly into Google / Wikipedia / etc.
The one case where I can use the output of ChatGPT directly is when I translate from, say, english to french or vice-versa and I know both languages well enough to be able to tell if the translation is okay or not.
Those believing they can use the output of ChatGPT without verifying it are basically these lawyers who referenced hallucinated cases to a judge.
As another person as commented: it didn't even make a dent in Google's search requests and that is no surprise.
For example, all my tech support searches are now GPT4. Those are painful on Google. There's no need to verify with a Google search, since you can just try out what GPT4 says.
Concrete example: I use it all the time to help me with Excel. What it suggests is nearly always correct. It has turned me into an Excel power user within a few weeks.
You need to develop a sense for what it's likely to be correct on, but once you do it's insanely useful. Simple rule of thumb: if you think you'd find a direct response to your question by wading through pages of ad infested Google results, it'll definitely work great on GPT4.
The way I recall it, it took multiple years for Google to go from a secret power user thing to displacing Yahoo and Altavista for the broad user base. And that was at a time where being online in itself was sort of an early adopter thing.
Anyway, I guess my point is, I would be worried if I was Google, and ignore this tech at your own risk...
If for this text output, ChatGPT also linked me directly to the gardens’ sites, scraped the info from the live site, and summarized this - that would actually save a ton of time. Google could have a leg up bc it has a knowledge graph, but so does Microsoft. This requires a lot more than training an LLM - this requires it to be an actual product, not a tech demo like it is. A chat with an agent that occasionally lies is a terrible UI.
I think there’s great scope for UI innovation here. But such an experience might be pretty expensive in terms of compute - lots of LLM queries and extra lookup systems. Someone who does this hard integration work and is willing to spend a lot of resources per query will deliver a delightful, time-saving user experience, and can probably charge for it. And that may be a great value-prop for local AI - you can give it tons of resources to solve your particular problem. As I see it, mass market LLMs that are provided for free will never do this extra work for you. ChatGPT might be in a good position bc it already has a ton of paying customers that it can continue to draw a wall around. Their early Nov announcement might be something along these lines.
Write a polite email to X saying Y -> doesn't need verification.
Rewrite this in formal English for a grant application -> doesn't need verification.
How to tar gz a folder in Linux? -> doesn't need verification. When I get the answer, I will probably think "oh, sure, it was czvf". And even if I didn't know what the arguments were, I would know enough to know that a tar command isn't going to delete my files or anything like that, so I would just try and see if it worked.
Write Python code to show a box plot with such and such data -> doesn't need verification, I'm too lazy to write the code myself (or look into how seaborn worked) but once I see the code, I can quickly understand it and check that it does what it should. Or actually run it and see if it worked.
Brainstorming (give me a bunch of titles for a paper about X/some ideas about how I could do Y) -> doesn't need verification, I can see the ideas and decide which I like and which I don't.
I get that if you're a journalist, a lawyer or something like that, probably the majority of your ChatGPT queries will need verification, but that's definitely not my experience... probably because I don't often ask ChatGPT for things that I don't know, most of my use is either for things that I could do myself but require a time investment and I'd rather ChatGPT does them in a few seconds, or for brainstorming. Neither of those require a Google search at all.
And this is besides the other stuff that search ones can’t do directly but it does quite well. It has definitely cut web searches for me.
If you want to basic information about a physical exercise, a recipe, a travel destination or such... All the content in Google seems to be content farmed. Written by people who don't know, quickly, by copying similar articles. It's just a buggy, manual version of the proverbial gpt anyway. At this point, I may as well just go direct.
If I want to know about doing weighted crunches instead of sit ups, Google results already represent all the downsides of using gpt. The got UI is nicer. You can also narrow in on your questions, push back and generally get the same level of content in a better package.
Maybe the old chestnut of "computerized recipe book" will finally be solved.
^Ra Ra!
you are just forgetting to add word "reddit" at the end of your query..
If chat apps take over, I think they will first taker over desktop computers because on mobile it is easier to type a few keywords in the web search bar and get 5 or 6 relevant web results than it is to chat with a chat bot for like 5 minutes.
Informational quires are most likely to be more useful in chat apps than in the classical web search but navigational and transactional quires are here to stay on Google or whatever search engine comes about.
Deleted Comment
However, while Claude.ai certainly showcases its strengths in specific areas, it doesn't quite measure up to OpenAI's ChatGPT in terms of adaptability and precision. ChatGPT stands out for its capability to understand intricate queries and produce nuanced, tailored responses.
Both platforms have distinct strengths; I firmly believe they'll evolve to dominate different niches in the AI ecosystem.
ChatGPT on GPT-3.5? I can buy this. GPT-4? No fucking way. If Claude got anywhere close, this would be front page news in every tech and tech-adjacent outlet.
This has to be said again and again: "ChatGPT" is meaningless without specifying whether you mean GPT-3.5 or GPT-4; statements about "ChatGPT" (or LLMs in general) capability limits are invalid unless tested on GPT-4.
You're correct, I should have specified. I rarely if ever use GPT-3.5. I have 7 paid / premium accounts, and I primarily use GPT-4.
Have you compared both? For my use case of generating & modifying simple code & Q&A based off of docs claude is in a similar range to GPT-4 & you can pay as you go instead of 20 bucks a month.
Deleted Comment
Quoting:
"""The Trust is an independent body of five financially disinterested members with an authority to select and remove a portion of our Board that will grow over time (ultimately, a majority of our Board). Paired with our Public Benefit Corporation status, the LTBT helps to align our corporate governance with our mission of developing and maintaining advanced AI for the long-term benefit of humanity."""
and how large is that portion?
N=1
They've already announced a replacement / better model coming but considering how long Google has been doing the AI thing, it's amazing they dropped the ball this badly.
This investment is Claude sounds like their backup plan.
I do expect it to get much smarter by the end of next year.
Deleted Comment
The fact that 1 or 2 years later the people will get their money back doesn’t mean there was any hope of surviving this a year ago.