Specifically, I was curious about how Harvard's endowment has grown from its initial £780 in 1638, so I asked Google to calculate compound interest for me. A variety of searches all yield a reasonable formula which is then calculated to be quite wrong. For example: {calculate the present value of $100 compounded annually for 386 years at 3% interest} yields $0.736. {how much would a 100 dollar investment in 1638 be worth in 2025 if invested} yields $3,903.46. {100 dollars compounded annually for 386 years at 3 percent} yields "The future value of the investment after 386 years is approximately $70,389." And my favorite: {100 dollars compounded since 1638} tells me a variety of outcomes for different interest rates: "A = 100 * (1 + 0.06)^387 A ≈ 8,090,950.14 A = 100 * (1 + 0.05)^387 A ≈ 10,822,768.28 A = 100 * (1 + 0.04)^387 A ≈ 14,422,758.11"
How can we be so reasonable and yet so bad!?
Aside from the general limitations of this technology, Google needs this to be quite cheap if it runs for every request.
There is not a lot of revenue for a single search, and right now the AI results are actually pushing the links people are paying Google to display further down the page.
"slopsquatting" is the term coined for this.
Essentially, bad actors are registering these packages and uploading malware. If you happen to just blindly follow the AI, there's a chance your system gets infected.
The AI overview is worse than useless. It either hallucinates things or it treats shitposts as equally valid information-wise as anything else.
Though also as a sidenote Harvard's endowment probably wasn't put in a bank account with a flat 3% interest rate for a few hundred years...
Like people asked "does Lululemon use <name of some chinese company> to make its products" and Google says "yes", with no source except one tiktok video that falsely claims it to boost sales in face of tariffs. (Ignoring that it's not in the actual supplier list published by lululemon on their site)
Which means basically people would see that tiktok, go to fact check on google if it's true, and google overview will say "yes" (+ paragraphs of text that no one reads) citing that tiktok.
Vicious circle of LLM factchecking. Google used to be immune to it until it started to shove chatbot output to people's faces.
His example was "A swan won't prevent a hurricane meaning"
https://simonwillison.net/2025/Apr/23/meaning-slop/