I'd love to see this analysis done for ChatGPT, which has a much bigger 'consumer' marketshare.
I'm also very wary of their analysis method, given classifiers-gonna-classify. We already see it in their example of someone asking why their game is crashing and it buckets them into Computer & Mathematical occupation. I'm guessing the original question was not that of a game developer but rather a game player, so can you really call this an occupational task? Sure it's in that domain, I guess, but in a completely different context. If I'm asking a question about how to clean my dish washer, that's hardly in repairman or industrial occupations.
What I see is it going from 58 to 40 (the scale is ???), and it’s only continued to rise over time. So that maybe a common use (~30%), but it’s not the only use.
Most of those kids will continue to use it as they graduate, having embedded it in their workflow (unfortunately many will probably fully outsource all thinking to it, having learned a lot less since it did it all for them).
Yeah. reminds me of the ancient okcupid data analysis blogs and not the creepy one by sleep8. The group I'm surprised not to see represented in their analysis is "personal", where people I know use ChatGPT as a therapist/life coach/sms analysis&editor. and of course they crucially but understandably left off the denominator. 35% of a million requests is different than 35% of a billion. and also how many of the conversations had 1 message, indicating "just testing" vs 10 or 100 messages.
This more or less confirms what I imagine most of us thought, AI is mostly used by engineers, or for engineering tasks. Makes sense, I wonder how much traffic comes from automated tasks (co-pilot, etc). Every time I read a report like this I do wonder if we'll ever see an ROI on LLMs. HUNDREDS of billions of dollars of spend, and 3 years in its still primarily the same crowd using it, and has yet to create a "killer" app beyond ChatGPT and Co-Pilot style IDE's. And its not like people aren't trying! Look at the recent YC batches, its all AI-for-industry. Idk man, I fear the economic reality on the backside of this kind of spend.
Software engineering is a weird niche that is both a high paying job and something you can almost self-teach from widely available free online content. If not self-teach, you can rely on free online content for troubleshooting, examples, etc.
A lot of other industries/jobs are more of an apprenticeship model, with little data and even less freely available on open internet.
> something you can almost self-teach from widely available free online content.
I think you massively underestimate just how much data is online for everything, especially once you include books which are freely available on every possible subject (illegally, perhaps, but if Meta can download them for free then so can everyone else).
There's less noise for many other subjects than for software engineering, there's often just a couple rather than 100s of competing ways to do everything. There might just be one coursebook rather than 1000s of tutorials. But the data for self teaching is absolutely there.
> Software engineering is a weird niche that is both a high paying job and something you can almost self-teach
If you meant programming, I agree it could be self-taught, but not SE. SE is the set of techniques, practices, tools that we have collected over decades for producing multi-versioned software that meets a certain reliability rating. Not all of these is freely available online.
I would agree that the products coming out so far lack imagination, but hard disagree on the impact. LLMs have completely transformed multiple industries already. In SWE, I would estimate that junior positions shrank by 70-80%, but even that is less extreme than what is going on in other industries.
In marketing, the entire low-end to mid-tier market is gone. Instead of having teams working on projects for small to mid-sized companies, there's now a single Senior managing projects with the help of LLMs. I know multiple agencies who cut staff by 80-90% without dropping revenue.
Translation (of books, articles, subtitles) was never well paid, even for very complex and demanding work. My partner did it a bit on the side, mostly justifying the low pay with some moral bla about spreading knowledge across cultures... With LLMs you can completely cut out the grunt part. You define the hard parts (terms that don't translate well), round out the edges and edge out the fluff, and every good translator becomes two to ten times more productive. Since work is usually paid by the page, people in the industry got a very decent (at least temporary) pay jump, I would imagine around 100%.
Support is probably the biggest one though. It is important to remember that outsourcing ot India only works for English speaking countries. And even that isn't super cheap.
Here in Germany, if you don't have back-up wealth, it is your constitutional right to get some support from the state (~1400 euro), but you are obligated to find a job as soon as possible, and they will try to help you find a role. Support was always one of the biggest industries to funnel people towards. I talked to a friend working there, and according to them the complete industry basically stopped advertising new positions, the only ones that are left are financial services. The rest went all in on LLMs and just employ a fraction of the support stuff to deal with things escalating enough.
And that's not even touching on all the small things. How much energy is spent on creating pitch decks, communicating proposals, writing documentation etc? It probably goes up as far as 50% of work in large Orgs, and even if you can just save 5% of your time by using LLMs to phrase or organize, there is a decent ROI for companies to pay for them.
I think a lot of this is because the economic pressure is weak right now both on the side of labor and consumers, due to decades of severe upward wealth transfer. A lot of these companies are not improving or even maintaining their productivity or quality of service, and while there are probably some productivity gains for engineers, I suspect based on what I'm seeing that this is going to burn a lot of people out, as there is significant social pressure both from peers and employers to exaggerate this somewhat. People can have too heavy a workload for a decent amount of time before breaking.
There's just no countervailing force to make these decisions that immediately painful for them. Sectors are monopolized, people are tired and desperate, tech workers are in a basically unprecedented bout of instability.
The situation is super dark from a lot of angles, but I don't think it's really "the overwhelming usefulness of AI" that's to blame here. As far as I can tell, the biggest thing these technologies are doing is providing a cover story for private-equity-style guttings of various knowledge work verticals for short-term profit, which was kind of inevitable given that's been happening across the board in the larger economy, it's just another pretense that works for different verticals.
There are cases where LLMs seem really genuinely useful (Mostly ones that are for and by SWEs, like generating documentation or smoothing some ramp processes in learning new libraries or languages) and those don't seem to be "transformative" at scale yet, unless we count "transforming" many products into buggier products that are more brittle and frustrating to interact with
>I know multiple agencies who cut staff by 80-90% without dropping revenue.
I'm finding it hard to reconcile this with my own experiences. My whole team ( 5 people ) left last year ( for better pay I guess ) and the marketing agency in germany Im working for had to substitute them with freelancers. To offset the cost they fired the one guy who was hired to push the whole LLM AI topic.
We managed to fill one junior position by offering 10k+ more then in their last job. The firm would love to hire people to replace the freelancers.
We had to cut stuff lately. But mostly they closed the kitchen which wasn't used due to work from home policy.
Definitely don't see any stuff reduction due to automation / LLM use. They still pay (external) people 60€ per written text/article. Because clients don't like LLM written stuff.
Actually I have interacted with multiple translators in multiple industries and I haven't seen any disruption (although I agree with your statement that it was never well paid)
- Synchronous translation at political/economic events still needs a personm as it ever did
- LLMs are nowhere near the level to be able to translate fine literature at a high enough quality to be publishable
- Translating software is still very hard, as the translator usually needs a ton of context/reference for commonly used terminology - we partnered with a machine translation company, and what they produced sucked balls.
I have friends who work as translators, and we make use of translation services as a company, and I haven't seen the work going away.
We’re trying to use it for industrial apps. Been over a year of R&D. Some good but often mixed results. Adherence to prompts is a big issue for us. It’s most useful not as a chatbot but to give explained descriptions of what the user is seeing, so they don’t need to dig down into 20 graphs and past history. That necessitates being able to refer to things with URIs which works 95% of the time but the 5% is killer since it’s difficult to detect issues and leads to dead links.
I tried to build a BIG E2E automation pipeline, along the lines of like, replace a team of 5 with this one simple tasks. And as I was doing it, all I could think was, just use chatgpt. Sure it can't actually automate what you're doing, but it'll get you there 80% as fast as fully-automated, with 90% less risk of error/nonsense at the end. Ironically, this company blocks all LLM websites, they even block GH on their employees' computers.
I'm curious about your approach and the nature of those industrial apps. Is it more of recommender agents accessing available sources (through URIs) - or more like explainers. Would be great to connect https://shorturl.at/xdOee
Can I ask a dumb question as an LLM newbie? What is it about Claude that makes it so good at basic software engineering tasks? Do you think it was finely tuned to be good at these tasks? No joke/trolling: A bunch of people have posted on HN in the last 6 months about creating MVPs (Minimum Viable Products) -- usually web apps -- using Claude. As a non-web-app programmer, I think this is amazing progress!
Fair, but do you think OpenAI and Gemini are going to be like directionally similar? How much of OpenAI's traffic is from Co-Pilot and other related tools, for example. My local IDE probably generates more queries a day than (pick a profession, idk, nurse? insurance sales? construction worker?) does in a month!
But "AI" tools have more or less seeped into every mainstream product... this is a strong "defensive move" for companies in anticipation of more to come.
We aren't leaving MS Office or Adobe because they already pushed out some minimal innovation. But what about the products you don't even know about? For lawyers, doctors, logistics, sales, marketing, wood workers, handymen? In Europe or Asia?
New product by bringing true innovation could easily push out legacy business by "shiny new thing"(AI) and better UX alone. A lot of software in these areas simply hasn't improved for 10 years - with a great idea and a dedicated team it's a landslide waiting to happen.
Claude is indeed far more familiar amongst software engineers.
Google Gemini integration into their docs/sheets/slides and Gmail perhaps will show different demographics in a few months, and that is yet before we heard from OpenAI.
Maybe spend and better models will help this (I’ve not used the deep research models so maybe we are there already). But even day to day coding, the LLMs are great helpers but giving them anything more than a slightly complicated prompts and it seems like these models become completely helpless. You just constantly need a human in the loop because these models are too dumb or lack the context to understand the big picture.
Maybe these models will get better as they’re given more context and can understand the full stack but for now they cannot.
And this is just with code where it already has billions of examples. Nevermind any less data-rich fields. The models still need to get smarter.
I don't think it's necessarily because of lack of generalizability. We (SWEs) built it, so we naturally have the most intimate knowledge of how to dogfood/use it. And so the cycle intensifies (use, provide feedback, improve). There's many positive examples of LLMs being useful in document based workflows in other domains as well!
Maybe! But you could say the inverse of lots of things that SWE's built. SWE's built the bloomberg terminal! SWE's built CRMs! I think its at least possible that LLMs are VERY useful for SWEs and a small number of other professions, but is unlikely to massively scale beyond that
If you were on the early internet talking to someone about music or woodworking or whatever, you could reasonably assume they were a tech person because it was not simple to get online. It took a minute for it to spread.
Daniel Rock has done some interesting work on the ROI of AI in general (also, I believe two of his papers are referenced in this study). Note that this doesn't explicitly restrict itself to covering LLMs, but... still a very interesting body of work.
My term for this is “Whitey’s goin’ to the data center”. We are looking at an arms race, where there really is genuine new technology and it will make a difference - but at the 1-2% per annum of an economy level - compounded over fifty years that is geo political dominance yes, but it’s not “machines of loving grace” level growth.
We already have thousands of geniuses working across our economies and teaching our youth. The best of our minds have every year or so been given a global stage in Nobel speeches. We still ignore their arses and will ignore it when AI tells us to stop fighting or whatever.
The real issue here is that wafer scale chips give 900,000 cores, and nothing but embarrassingly parallel code can use it - and frankly no coder I know writes code like that - we have to rethink our whole approach now Moores law is over. Only AI has anything like ability to use the processing ability being built today - the rest of us can stick to cores from 2016 and nothing would change.
Throwing hundreds of billions at having a bad way to program 1 million cores because we have not rethought software and businesses to cope seems wrong - both because “Whitey” can spend it on better things but also because it is an opportunity - imagine being 900,000 times faster than your competitors - what’s does that even mean?
Edit: Trying to put it another way - there are two ways AI can help us - it can improve cancer treatments at every stage of medical care, through careful design and creation of medical AI models that can slowly ratchet up diagnosis, treatment and even research and analysis. This is human organisations harnessing and adapting around a new technology
Or AI can become so smart it just invents a cure for cancer.
I absolutely think the first is going to happen and will benefit the denizens of the first world first. The second one requires two paradigm shifting leaps in the same sentence. Ten years ago I would have laughed in Anthropics face. Today I just give it a low probability multipled by another low probability- and that is an incredible shift.
As someone who works in finance, I would disagree. I asked ChatGPT:
Is the noun spend rare?
ChatGPT said:
The noun "spend" is relatively rare compared to its more common form as a verb. While "spend" is widely used as a verb (meaning to give money or time for something), as a noun, it refers to an expenditure or the act of spending, and it’s not as commonly encountered.
In most contexts, people would use alternatives like "expenditure," "spending," or "outlay" instead of "spend" as a noun. That said, it is still used occasionally in certain contexts, especially in financial or informal language.
Seems like Anthropic has too much money on their hands and are looking for ways to spend it. It’s surprising to see lean AI startups accumulate fat so quickly. Usually this sort of wheel spinning is reserved for large corporations.
And it’s not just them. To me this trend screams “valuations are too high”, and maybe hints at “progress might start to stagnate soon”.
Anthropic is a Public Benefit Corporation whose governance is very different from a typical company in that it doesn’t put shareholder ROI above all else. A majority of its board seats are reserved for people who hold no equity whatsoever and whose explicit mandate is to look out for humanity.
This is why I cancelled my chatgpt subscription and moved to claude. Its kinda silly, but I feel like the products are about equivalent for my use case so I'd rather do business with a company that is acting in good (better?) faith.
I'm not sure if you are being sarcastic or not, but the practical upshot of this new "Public Benefit Corporation" thing, with or without a trust or non-profit attached, is that you can tell both the public and your investors to fuck off. The reason why all the big AI startups suddenly want to use it is because they can. Normally no sane investor would actually invest in such a structure, but right now the fear that you might be left out of the race for humanity's "last invention" is so acute that they do it anyway. But if Dario Amodei actually cared about humanity any more than Sam Altman, that would be the surprise of the year to me.
The exact opposite. Relative to ChatGPT Anthropic has an enormous "brand problem." What they should be doing is exclusive deals like this, but with deals with large publishers on a recurring basis and figure out how to teach consumers who they are and how to use them best. For like 99% of the use cases all these products are parody and the real business gains are finding a way into consumers lives.
Semi-relevant sidenote: ChatGPT, spent $8m on a super bowl commercial yesterday just to show cool visualizations instead of any emotional product use case to an ultra majority audience has never had a direct experience with the product.
These companies would be best served building a marketing arm away from the main campus in a place like LA or NY to separate the gen pop story from that of the technology.
I disagree. I think Anthropic, like the other big players, is trying to get some of that government money. Releasing policy-adjacent papers seems like a way to alert government officials that Anthropic ought to be in the room when the money starts changing hands.
I am inclined to agree. If you’re at the precipice of automating or transforming knowledge work and the value for being the first is nearly infinite (due to “flywheel effects”), why would you dedicate any energy to studying the impact of AI on jobs directly? The thesis is everything changes.
I think AI in its current iteration is going to settle into being like a slightly worse version of Wikipedia morphed with a slightly better version of stackoverflow.
A lot of assumptions there. Why isn't Ford the only motor company?
And if the flywheel is that AI begets AI exponentially in an infinite loop then those share certificates you own probably won't be worth much. The AI won.
I don't see it. This is just an analysis of how Anthropic customers are using the product and what investment areas seem most promising in the future - why wouldn't they want that?
It's clearly more than an interesting tech blog post written by one of the data guys in their spare time. It's an "initiative".
That said, this doesn't seem like completely superfluous "fat" like what Mozilla does. It seems very much targeted at generating interesting bits of content marketing and headlines, which should contribute to increasing Anthropic's household brand name recognition vs. other players OpenAI, as well as making them seem like a serious, trustworthy institution, rather than a rapacious startup that has no interest in playing nice with the rest of society. That is: it's a good marketing tool.
My guess is that they developed it internally for market research, and realized that the results would make them look good if published. Expect it to be "sunset" if another AI winter approaches.
Even on the contrary, this is very important information to have, in order to understand your customer base and how sticky you are with them, what features you need to focus on, etc etc
We live in a world where there's a lot of talk about how AI might impact societies and economies - but little actual data. To me it seems very worthwhile to try to add 'any' data to that discussion and track how things change over time. Are reports of economic or labour trends pointless? Should companies not track how people use their products? I don't think it costs Anthropic much to do this - it's work for a couple of people to analyze their database.
idk, the models themselves are quickly becoming a commodity. it makes sense to spend money figuring out go to market rather than just improve the models themselves.
And yet they don't have the resources to let job applicants know when their application was unsuccessful. You just get an email after you applied saying: "We may not reach out unless we think you are a strong fit for the role you applied to. In the meantime, we truly appreciate your patience throughout our hiring process." They also tell you not to use AI in the application.
Everyone in the comments seem to not like the article or see it as a waste of time. I just don't think we are the audience they wanted for this, I think they want to show the average business owner the realistic potential and public (journalists that will distill this later) they are aware of the impacts and what to expect.
I don't read it as fear AI, I read change is happening because of AI.
Tools empower those with knowledge further than those without knowledge. The fact that people were concerned layman were simply going to be able to take on experienced programmers at their day jobs was farcical.
I honestly don’t know who the audience could be, other than “people who like to tell others they’re in the know because they read AI companies’ press releases.”
At no point do I see an actual elevator pitch/tl;dr/summary of what the frak this index actually is, except that it’s part of some effort to track AI adoption. It just rains down figures about which industries are using how much AI without first grounding the new concept they’re introducing.
When you say you have a new economic index, you need to give me a number, how I should interpret that number, and where it comes from. I don’t see that.
GDP: measure if a country’s total economic output by adding up end product purchases.
CPI: general price level by taking a weighted average of prices throughout the economy
Big Mac index: how expensive goods are in a country relative to the US by reference to the local cost of a Big Mac, converted through the exchange rate.
Here I expect something like “the economic output-weighted fraction of production taken over by AI”, but instead it’s just a list of AI adoption by industry.
Why introduce an index and not headline with a definition of an index? Which audience prefers that?
Awesome that there are releasing this paper and the associated data. I hope they'll do this regularly, so that changes can be tracked.
One thing I hope they'll correct going forward is inclusion of API usage. Anecdotally, I only use Anthropic models via Cursor. So none of that usage shows up in here. I'd expect that specialized tools/interfaces like Cursor will grow and thus more usage will shift to API. It would be a shame to miss out on that in the data set.
Feels like a page of graphs where Anthropic team discovers that Claude is the best coding model and is used mostly by devs. And they have no penetration in general population compared to OpenAI
I laughed out loud at their chart showing that Sonnet had a higher share of coding questions, whereas Opus had more writing.
If they would just look at their product, they'd see that it literally says it in the model description that Opus is better for writing. If you advertise one of your models as geared for task X, the insight that people use it more for task X isn't really an insight.
Read it as that as well, I think this mixes the "there is no penetration in that market" with "we have not been able to get these people to use our tools".
I used to use Claude.ai as my go-to LLM for everything. But then my conversations around taxes and finance got very frequently patronized by the LLM and even flagged. All legal stuff! It is just that my personal tax situation is a bit more complex than other people's because of businesses I run and geographic complications (living in more than one country, etc).
It got to the point where I was forced to go to ChatGPT if I wanted to just be left alone and get my answers. Then o1, o1 pro, o3-mini and Deep Research dropped and I have almost no reason to go back to Claude anymore. These days my main use case is using it as part of Cursor for code generation / co-piloting. But that's it.
If Anthropic wants to get me back, they should treat me as an adult again.
At day job - finance/office stuff - essentially zero traction despite everyone having enterprise AI subs & brainstorming sessions about use cases etc.
Then go home & do some hobby coding and suddenly it's next level useful.
It's not that the one is harder than the other, but rather that many jobs don't have an equivalent to a code base. The AI could I think grok parts of the job but typing up relevant content & what is required would take longer much than doing the task. There is nothing there to copy & paste for a quick win in the same way as code.
Readit News
I'm also very wary of their analysis method, given classifiers-gonna-classify. We already see it in their example of someone asking why their game is crashing and it buckets them into Computer & Mathematical occupation. I'm guessing the original question was not that of a game developer but rather a game player, so can you really call this an occupational task? Sure it's in that domain, I guess, but in a completely different context. If I'm asking a question about how to clean my dish washer, that's hardly in repairman or industrial occupations.
Still, it's cool they're doing this.
https://trends.google.com/trends/explore?date=today%205-y&ge...
Which suggests that the most common use is as a tutor / cheating on homework.
Most of those kids will continue to use it as they graduate, having embedded it in their workflow (unfortunately many will probably fully outsource all thinking to it, having learned a lot less since it did it all for them).
Dead Comment
What are you referring to?
Not statistically.
Software engineering is a weird niche that is both a high paying job and something you can almost self-teach from widely available free online content. If not self-teach, you can rely on free online content for troubleshooting, examples, etc.
A lot of other industries/jobs are more of an apprenticeship model, with little data and even less freely available on open internet.
I think you massively underestimate just how much data is online for everything, especially once you include books which are freely available on every possible subject (illegally, perhaps, but if Meta can download them for free then so can everyone else).
There's less noise for many other subjects than for software engineering, there's often just a couple rather than 100s of competing ways to do everything. There might just be one coursebook rather than 1000s of tutorials. But the data for self teaching is absolutely there.
If you meant programming, I agree it could be self-taught, but not SE. SE is the set of techniques, practices, tools that we have collected over decades for producing multi-versioned software that meets a certain reliability rating. Not all of these is freely available online.
In marketing, the entire low-end to mid-tier market is gone. Instead of having teams working on projects for small to mid-sized companies, there's now a single Senior managing projects with the help of LLMs. I know multiple agencies who cut staff by 80-90% without dropping revenue.
Translation (of books, articles, subtitles) was never well paid, even for very complex and demanding work. My partner did it a bit on the side, mostly justifying the low pay with some moral bla about spreading knowledge across cultures... With LLMs you can completely cut out the grunt part. You define the hard parts (terms that don't translate well), round out the edges and edge out the fluff, and every good translator becomes two to ten times more productive. Since work is usually paid by the page, people in the industry got a very decent (at least temporary) pay jump, I would imagine around 100%.
Support is probably the biggest one though. It is important to remember that outsourcing ot India only works for English speaking countries. And even that isn't super cheap. Here in Germany, if you don't have back-up wealth, it is your constitutional right to get some support from the state (~1400 euro), but you are obligated to find a job as soon as possible, and they will try to help you find a role. Support was always one of the biggest industries to funnel people towards. I talked to a friend working there, and according to them the complete industry basically stopped advertising new positions, the only ones that are left are financial services. The rest went all in on LLMs and just employ a fraction of the support stuff to deal with things escalating enough.
And that's not even touching on all the small things. How much energy is spent on creating pitch decks, communicating proposals, writing documentation etc? It probably goes up as far as 50% of work in large Orgs, and even if you can just save 5% of your time by using LLMs to phrase or organize, there is a decent ROI for companies to pay for them.
There's just no countervailing force to make these decisions that immediately painful for them. Sectors are monopolized, people are tired and desperate, tech workers are in a basically unprecedented bout of instability.
The situation is super dark from a lot of angles, but I don't think it's really "the overwhelming usefulness of AI" that's to blame here. As far as I can tell, the biggest thing these technologies are doing is providing a cover story for private-equity-style guttings of various knowledge work verticals for short-term profit, which was kind of inevitable given that's been happening across the board in the larger economy, it's just another pretense that works for different verticals.
There are cases where LLMs seem really genuinely useful (Mostly ones that are for and by SWEs, like generating documentation or smoothing some ramp processes in learning new libraries or languages) and those don't seem to be "transformative" at scale yet, unless we count "transforming" many products into buggier products that are more brittle and frustrating to interact with
I'm finding it hard to reconcile this with my own experiences. My whole team ( 5 people ) left last year ( for better pay I guess ) and the marketing agency in germany Im working for had to substitute them with freelancers. To offset the cost they fired the one guy who was hired to push the whole LLM AI topic. We managed to fill one junior position by offering 10k+ more then in their last job. The firm would love to hire people to replace the freelancers. We had to cut stuff lately. But mostly they closed the kitchen which wasn't used due to work from home policy. Definitely don't see any stuff reduction due to automation / LLM use. They still pay (external) people 60€ per written text/article. Because clients don't like LLM written stuff.
- Synchronous translation at political/economic events still needs a personm as it ever did - LLMs are nowhere near the level to be able to translate fine literature at a high enough quality to be publishable - Translating software is still very hard, as the translator usually needs a ton of context/reference for commonly used terminology - we partnered with a machine translation company, and what they produced sucked balls.
I have friends who work as translators, and we make use of translation services as a company, and I haven't seen the work going away.
This just isn't true, it's nowhere close.
If this was true we would see the results in productivity and unemployment stats. We don't though, so far the effect hasn't registered.
I love Claude, but let's not ignore that in the LLM race, they're not exactly the leading player.
We aren't leaving MS Office or Adobe because they already pushed out some minimal innovation. But what about the products you don't even know about? For lawyers, doctors, logistics, sales, marketing, wood workers, handymen? In Europe or Asia?
New product by bringing true innovation could easily push out legacy business by "shiny new thing"(AI) and better UX alone. A lot of software in these areas simply hasn't improved for 10 years - with a great idea and a dedicated team it's a landslide waiting to happen.
Google Gemini integration into their docs/sheets/slides and Gmail perhaps will show different demographics in a few months, and that is yet before we heard from OpenAI.
Maybe these models will get better as they’re given more context and can understand the full stack but for now they cannot.
And this is just with code where it already has billions of examples. Nevermind any less data-rich fields. The models still need to get smarter.
https://www.danielianrock.com/research
We already have thousands of geniuses working across our economies and teaching our youth. The best of our minds have every year or so been given a global stage in Nobel speeches. We still ignore their arses and will ignore it when AI tells us to stop fighting or whatever.
The real issue here is that wafer scale chips give 900,000 cores, and nothing but embarrassingly parallel code can use it - and frankly no coder I know writes code like that - we have to rethink our whole approach now Moores law is over. Only AI has anything like ability to use the processing ability being built today - the rest of us can stick to cores from 2016 and nothing would change.
Throwing hundreds of billions at having a bad way to program 1 million cores because we have not rethought software and businesses to cope seems wrong - both because “Whitey” can spend it on better things but also because it is an opportunity - imagine being 900,000 times faster than your competitors - what’s does that even mean?
Edit: Trying to put it another way - there are two ways AI can help us - it can improve cancer treatments at every stage of medical care, through careful design and creation of medical AI models that can slowly ratchet up diagnosis, treatment and even research and analysis. This is human organisations harnessing and adapting around a new technology
Or AI can become so smart it just invents a cure for cancer.
I absolutely think the first is going to happen and will benefit the denizens of the first world first. The second one requires two paradigm shifting leaps in the same sentence. Ten years ago I would have laughed in Anthropics face. Today I just give it a low probability multipled by another low probability- and that is an incredible shift.
I feel like this has less to do with what LLMs are best at and more to do with which folks are mostly likely to spend time using a chat bot.
Minor nitpick. Use of the word 'spend' as a noun is not widespread and not well known.
And it’s not just them. To me this trend screams “valuations are too high”, and maybe hints at “progress might start to stagnate soon”.
https://www.anthropic.com/news/the-long-term-benefit-trust
https://time.com/6983420/anthropic-structure-openai-incentiv...
Semi-relevant sidenote: ChatGPT, spent $8m on a super bowl commercial yesterday just to show cool visualizations instead of any emotional product use case to an ultra majority audience has never had a direct experience with the product.
These companies would be best served building a marketing arm away from the main campus in a place like LA or NY to separate the gen pop story from that of the technology.
I think AI in its current iteration is going to settle into being like a slightly worse version of Wikipedia morphed with a slightly better version of stackoverflow.
And if the flywheel is that AI begets AI exponentially in an infinite loop then those share certificates you own probably won't be worth much. The AI won.
Coincidentally, Anthropic's mission is AI safety.
That said, this doesn't seem like completely superfluous "fat" like what Mozilla does. It seems very much targeted at generating interesting bits of content marketing and headlines, which should contribute to increasing Anthropic's household brand name recognition vs. other players OpenAI, as well as making them seem like a serious, trustworthy institution, rather than a rapacious startup that has no interest in playing nice with the rest of society. That is: it's a good marketing tool.
My guess is that they developed it internally for market research, and realized that the results would make them look good if published. Expect it to be "sunset" if another AI winter approaches.
I don't read it as fear AI, I read change is happening because of AI.
At no point do I see an actual elevator pitch/tl;dr/summary of what the frak this index actually is, except that it’s part of some effort to track AI adoption. It just rains down figures about which industries are using how much AI without first grounding the new concept they’re introducing.
When you say you have a new economic index, you need to give me a number, how I should interpret that number, and where it comes from. I don’t see that.
GDP: measure if a country’s total economic output by adding up end product purchases.
CPI: general price level by taking a weighted average of prices throughout the economy
Big Mac index: how expensive goods are in a country relative to the US by reference to the local cost of a Big Mac, converted through the exchange rate.
Here I expect something like “the economic output-weighted fraction of production taken over by AI”, but instead it’s just a list of AI adoption by industry.
Why introduce an index and not headline with a definition of an index? Which audience prefers that?
One thing I hope they'll correct going forward is inclusion of API usage. Anecdotally, I only use Anthropic models via Cursor. So none of that usage shows up in here. I'd expect that specialized tools/interfaces like Cursor will grow and thus more usage will shift to API. It would be a shame to miss out on that in the data set.
Even if they don’t train on the data they could break it down by user agent / API client ID and infer something about cursor traffic.
If they would just look at their product, they'd see that it literally says it in the model description that Opus is better for writing. If you advertise one of your models as geared for task X, the insight that people use it more for task X isn't really an insight.
It got to the point where I was forced to go to ChatGPT if I wanted to just be left alone and get my answers. Then o1, o1 pro, o3-mini and Deep Research dropped and I have almost no reason to go back to Claude anymore. These days my main use case is using it as part of Cursor for code generation / co-piloting. But that's it.
If Anthropic wants to get me back, they should treat me as an adult again.
At day job - finance/office stuff - essentially zero traction despite everyone having enterprise AI subs & brainstorming sessions about use cases etc.
Then go home & do some hobby coding and suddenly it's next level useful.
It's not that the one is harder than the other, but rather that many jobs don't have an equivalent to a code base. The AI could I think grok parts of the job but typing up relevant content & what is required would take longer much than doing the task. There is nothing there to copy & paste for a quick win in the same way as code.