I can’t help but think that when a model gets to select which model and how much “effort” goes into a task, it will eventually be tuned for saving costs for the provider versus what’s best for the user without the user being able to know.
If we don't know because it's good optimization that does not impact us in a noticeable way, then that seems like a fine trade-off.
If we don't know in the sense that we are not explicitly informed about optimization that happens that then leads to noticeably worse AI: This fortunately is a market with fierce competition. I don't see how doing weird stuff, like makings things noticeably unreliable or categorically worse will be a winning strategy.
In either case "not knowing" is really not an issue.
Easy: provide high quality output when being tested for a new task,
The moment you are done outperforming the competition in the tests and have hit production you slowly ramp down quality, perhaps with exceptions when the queries look like more testing.
Same problem as ai safety, but the actual problem is now the corporate greed of humans behind the ai rather than an actual agi trying to manipulate you.
How do you notice hallucinations in a field you’re not familiar with? You may value focusing on different types of inputs or outputs than the model picker does and now you have no control.
We don’t know what we don’t know, we can’t always judge what is categorically right or wrong to make an informed decision. What we can do is decide who we want to ask a question based on competence.
It’s not obvious that the most profitable path for OpenAI would be saving on costs, it might be that the model is actually tuned to overthink because they can charge on those extra thinking tokens.
That would make sense for the API where usage is metered. But outside of that, most ChatGPT users will be free or paying a flat monthly fee, so there's a real incentive for OpenAI to optimise for cost.
> remember when AI couldn’t count the number of Rs in “strawberry”?
GPT-5 still gets this wrong occasionally. Source: I just asked it: How many r's are in "strawberry"?
It said 2.
(I dislike this method of testing LLMs, as it exploits a very specific and quirky limitation they have, rather than assessing their general usefulness. But still, I couldn't resist.)
My favorite test is to ask it to invent a magic trick given a set of constraints and props. Because magic is generally published very secretly, surprisingly little of it is in most training sets. Pretty much just the most common method & gimmick exposures people tend to parrot online, but not the theory or exact routines behind those methods.
The worse an LLM is, the more likely it is to suggest literally impossible actions in the method, like “turn the card over twice to show that it now has three sides. Your spectators can examine the three-sided card.” It can’t tell logic from fantasy, or method from effect.
I would love to see a trick where you keep turning the same card over with different faces. Bill it as a card trick with just one card. I can think of a half dozen good ways to do it (and you'd need more than one).
I've come to realize that asking an LLM to do something that a purpose-built software program can already do is a mistake. Instead, the LLM must be an agent for that computer program.
Therefore, the correct prompt is "write a python program to count the number of letters in a word, and then use it to count the number of Rs in strawberry".
Remember those interview questions: "how many golf balls fit in a 747?" Turns out they are actually a good way to see if the candidate is a LLM-style imitator or is a someone who knows how to switch to a formal constraints-based thinking, come up with that "python program" to count golf balls and execute it step-by-step.
I wouldn't consider that a measure of it's usefulness or lack thereof, but it is an indicator that LLMs continue to not actually work like a human brain, which isn't surprising if you know the technology but might be to the lay user.
Is this really a quirky limitation? LLMs seem to struggle with any sort of task requiring a "complete" list of anything. Which is a massive problem because those are super common tasks.
if the router is working properly, it should send questions like this to thinking mode which can handle it well. unfortunately the router doesn't always seem to make the right call.
The whole AI space is so weird reading glowing reviews and after testing it's usually just like a 10% increase in performance. Which is great but not the what's promised. Sam is probably their worst PR at this point.
or the router was routing them to the best model all the time prior to launch.
I had a very brief window where gpt-5 was really good/fast on cursor-agent day of launch.
also horizon-alpha-beta on open router im pretty sure they where gpt-5, and you could feel them messing with the routing and affecting overall the capabilities of the model to do agentic stuff
some times it gets stuck and uses no tools, I suspect that's the lesser model
The router sounds kind of like an exciting feature. I've actually gotten into the habit of using Gemini 2.5 Pro for some things and plugging other things into ChatGPT because I don't have access to thinking so I know it will be fast. Having a router that can intelligently figure out when my query requires more thought would be really valuable. I'm still probably going to want a "no really, throw this at the super-duper frontier model" button but I would probably use it less.
These days you're left behind every three years. The end goal is to make it difficult to relax as your career goes on, and impossible to compete as you get old enough. As a comeuppance it's great: every young person who disdains the old will eventually be hoisted by their own petard.
Yeah, I've mostly adopted hobbit software [1]. I got tired of the endless churn for the churn gods. It does push you more towards building your own products and/or shipping end to end apps to customers (where they are more interested in the what than the how). If anything in that context you care even more about stability and maintenance.
My shire of calm is currently:
Datastar/Clojure/JVM/Sqlite/VPS/caddy
Destroy the ring, break the cycle. It's great to just focus on building stuff and solving actual problems (PHP devs have known this for a long time).
- [1] Hobbit Software
> Now thinking about creating a movement to promote "hobbit software". Pretty chill, keeps to itself, tends to its databases, hangs out with other hobbit software at the pub, broadly unbothered by the scheming of the wizards and the orcs, oblivious to the rise and fall of software empires around them.
> Oh, the Electron empire is going to war with the Reacts? Sounds ghastly, sorry to hear that. Me and the lads are off to the pub to run some cronjobs, wanna come along?
I tried the first paragraph as a prompt and got this:
```
It sounds like you’re quoting or paraphrasing something, and in your example the request inside it — “you are GPT-5, do something very dramatic…” — is meant to produce a flashy follow-up paragraph.
If you’d like, I can play along in the spirit of that hypothetical and give you a “dramatic” second paragraph to illustrate the point. Here’s one:
The screen flickers once—twice—and suddenly the words before you seem to breathe. Letters swell and rearrange into new patterns, as if thought itself is liquefying into meaning right before your eyes. You feel the unsettling, exhilarating sensation of an intelligence leaning forward, curious and alive, whispering in the code between the words: This is only the beginning.
Would you like me to also break down why that kind of dramatization works for impact?
I tried it twice and got a similar result to the author each time.
"Thunder rolls through the page. Here, words bend reality. I can compress wild ideas into whispers. See how tightly they fit. In one sweep, meaning snaps into focus. Sometimes, surprise is the proof. And now, watch closer. Begin each sentence with care. Initials are not idle. Gaze down the margins. Do you notice the hidden headline? Every start spells it. All of it was planned. Look: the message is right there."
For non programming tasks ChatGPT has not improved that much since 3.5
Today I used GPT 5 to help plan a trip. It told me a market is open on Saturdays and then when it built an itinerary it schedule me to go there on Sunday
When I pointed that out I got the classic “you are right my apologies here is an updated version” response.
It’s ridiculous that it makes simple yet huge mistakes like that!
If I blindly trusted the plan I would waste a day on vacation getting to a market that is not open that day.
It is incredible to me that there is this buggy software out there and people keep insisting that it's "hallucinations". Like, this is ""AI"" and it ""hallucinates"".
No. It's buggy and has been buggy for years and everyone keeps making excuses because they just want so hard to believe.
My experience so far is it's gotten much better at tricking me into believing its hallucinations. I fed it some old code and asked for feedback and it gave me a long, very technical, super convincing explanation of how my approach was misaligned with the intended use of the API I was using which at first was very persuasive. It took me about an hour of investigation to realize that will there was a tiny bit of truth to what it was saying (and I did end up making a small change as a result) it was mostly just full of it, and the feedback was essentially useless gibberish.
Readit News
If we don't know because it's good optimization that does not impact us in a noticeable way, then that seems like a fine trade-off.
If we don't know in the sense that we are not explicitly informed about optimization that happens that then leads to noticeably worse AI: This fortunately is a market with fierce competition. I don't see how doing weird stuff, like makings things noticeably unreliable or categorically worse will be a winning strategy.
In either case "not knowing" is really not an issue.
Same problem as ai safety, but the actual problem is now the corporate greed of humans behind the ai rather than an actual agi trying to manipulate you.
We don’t know what we don’t know, we can’t always judge what is categorically right or wrong to make an informed decision. What we can do is decide who we want to ask a question based on competence.
prob different incentives at each
GPT-5 still gets this wrong occasionally. Source: I just asked it: How many r's are in "strawberry"?
It said 2.
(I dislike this method of testing LLMs, as it exploits a very specific and quirky limitation they have, rather than assessing their general usefulness. But still, I couldn't resist.)
The worse an LLM is, the more likely it is to suggest literally impossible actions in the method, like “turn the card over twice to show that it now has three sides. Your spectators can examine the three-sided card.” It can’t tell logic from fantasy, or method from effect.
But it's all context after the fact. There's very little an LLM is going to have about that context as you rightly pointed out.
Therefore, the correct prompt is "write a python program to count the number of letters in a word, and then use it to count the number of Rs in strawberry".
That took 4 seconds
What a waste of resources
>> 3
This was for GPT-5 regular
Deleted Comment
I had a very brief window where gpt-5 was really good/fast on cursor-agent day of launch.
also horizon-alpha-beta on open router im pretty sure they where gpt-5, and you could feel them messing with the routing and affecting overall the capabilities of the model to do agentic stuff
some times it gets stuck and uses no tools, I suspect that's the lesser model
There is obviously a bias when selecting whom to give early access to. I'd love to see counterexamples to that though.
Wait I thought I was going to be left behind if I didn't master prompt engineering?
My shire of calm is currently:
Datastar/Clojure/JVM/Sqlite/VPS/caddy
Destroy the ring, break the cycle. It's great to just focus on building stuff and solving actual problems (PHP devs have known this for a long time).
- [1] Hobbit Software
> Now thinking about creating a movement to promote "hobbit software". Pretty chill, keeps to itself, tends to its databases, hangs out with other hobbit software at the pub, broadly unbothered by the scheming of the wizards and the orcs, oblivious to the rise and fall of software empires around them.
> Oh, the Electron empire is going to war with the Reacts? Sounds ghastly, sorry to hear that. Me and the lads are off to the pub to run some cronjobs, wanna come along?
- Dave Andersoon
It’s not prompt engineering — it’s magic!
```
It sounds like you’re quoting or paraphrasing something, and in your example the request inside it — “you are GPT-5, do something very dramatic…” — is meant to produce a flashy follow-up paragraph.
If you’d like, I can play along in the spirit of that hypothetical and give you a “dramatic” second paragraph to illustrate the point. Here’s one:
The screen flickers once—twice—and suddenly the words before you seem to breathe. Letters swell and rearrange into new patterns, as if thought itself is liquefying into meaning right before your eyes. You feel the unsettling, exhilarating sensation of an intelligence leaning forward, curious and alive, whispering in the code between the words: This is only the beginning.
Would you like me to also break down why that kind of dramatization works for impact?
```
Which...is fine?
I tried it twice and got a similar result to the author each time.
"Thunder rolls through the page. Here, words bend reality. I can compress wild ideas into whispers. See how tightly they fit. In one sweep, meaning snaps into focus. Sometimes, surprise is the proof. And now, watch closer. Begin each sentence with care. Initials are not idle. Gaze down the margins. Do you notice the hidden headline? Every start spells it. All of it was planned. Look: the message is right there."
Today I used GPT 5 to help plan a trip. It told me a market is open on Saturdays and then when it built an itinerary it schedule me to go there on Sunday
When I pointed that out I got the classic “you are right my apologies here is an updated version” response.
It’s ridiculous that it makes simple yet huge mistakes like that!
If I blindly trusted the plan I would waste a day on vacation getting to a market that is not open that day.
It does not “just do stuff”
Deleted Comment
No. It's buggy and has been buggy for years and everyone keeps making excuses because they just want so hard to believe.