Today, I noticed that my behavior has shifted over the past few months. Right now, I exclusively use ChatGPT for any kind of search or question.
Using Google now feels completely lackluster in comparison.
I've noticed the same thing happening in my circle of friends as well—and they don’t even have a technical background.
How about you?
You hear about this new programming language called "Frob", and you assume it must have a website. So you google "Frob language". You hear that there was a plane crash in DC, and assume (CNN/AP/your_favorite_news_site) has almost certainly written an article about it. You google "DC plane crash."
LLMs aren't ever going to replace search for that use case, simply because they're never going to be as convenient.
Where LLMs will take over from search is when it comes to open-ended research - where you don't know in advance where you're going or what you're going to find. I don't really have frequent use cases of this sort, but depending on your occupation it might revolutionize your daily work.
Just yesterday I was trying to remember the name of a vague concept I’d forgotten, with my overall question being:
“Is there a technical term in biology for the equilibrium that occurs between plant species producing defensive toxins, and toxin resistance in the insect species that feed on those plants, whereby the plant species never has enough evolutionary pressure to increase it’s toxin load enough to kill off the insect that is adapting to it”
After fruitless searching around because I didn’t have the right things to look for, putting the above in ChatGPT gave an instant reply of exactly what I was looking for:
“Yes, the phenomenon you're describing is often referred to as evolutionary arms race or coevolutionary arms race.”
Evolutionary arms race is somewhat tautological; an arms race is the description of the selective pressure applied by other species on evolution of the species in question. (There are other, abiotic sources of selective pressures, e.g. climate change on evolutionary timescales, so while 'evolution' at least carries a broader meaning, 'arms race' adds nothing that wasn't already there.)
That said, using your exact query on deepseek r1 and claude sonnet 3.7 both did include red queen in their answers, along with other related concepts like tit for tat escalation.
Or updated for the LLM age, "the best way to get the right answer from an LLM is not to ask it a question and use its answer; it's to post its response on a site of well-educated and easily nerdsniped people"
They’re very helpful for helping me ask more refined questions by getting the terminology correct.
I think of AI as an intelligent search engine / assistant and, outside of simple questions with one very specific answer, it just crushes search engines.
Google 55% as GPT is not a local search engine
GPT 45% but use it for more intelligent learning/conversations/knowledgebase.
If I had a GPT phone ... sorta like H.E.R. the movie I would rarely leave my phone's lockscreen. My AI device / super AI human friend would do everything for me including get me to the best lighting to take the best selfies...
For example: Take the ingredient list of a cosmetic or other product that could be 30-40 different molecules and ask ChatGPT to list out what each of them is and if any have potential issues.
You can then verify what it returns via search.
Dead Comment
The reason is pretty simple. If the result you want is in the first few search hits, it's always better. Your query is shorter so there is less typing, the search engine is always faster, the results are far better because you side step the LLM hallucinating as it regurgitates the results it remembers on the page your would have read if you searched.
If you aren't confident of the search times, it can take 1/2 an hour of dicking around with different terms, clicking though a couple of pages of search results for each set of term, until you finally figure out the lingo to use. Figuring out what you are really after from that wordy description is the inner magic of LLM's.
Most often not true in the kind of searches I do. Say, I search for how to do something in the Linux terminal (not just the command, but the specific options to achieve a certain thing). Google will often take me to pages that do have the answer, but are full of ads and fluff, and I have to browse through several options until I find the ones I want. ChatGPT just gives me the answer.
And with any halfway decent model, hallucination only seems to be a problem in difficult or very specialized questions. Which I agree shouldn't be asked to LLMs (or not without verifying sources, at least). But over 90% of what I search aren't difficult or specialized questions, they're just things I have forgotten, or things that are easy but I don't know just because they're not in my area of expertise. For example as a learner of Chinese, I often ask it to explain sentences to me (translate the sentence, the individual words, and explain what a given word is doing in the sentence) and for that kind of thing it's basically flawless, there's no reason why it would hallucinate as such questions are trivial for a model having tons of Chinese text in its training set.
I asked Claude to give me a recipe that uses mushrooms and freezes well and it give me a decent looking soup recipe. It might not be the best soup ever, but it's soup, kinda hard to mess up. The alternative would be to get a recipe from the web with a couple dozen paragraphs about how this is the bestest soup ever and it comes from their grandma and reminds them of summer and whatnot.
Interesting, I just random words. LLM not care sentence.
But what I'm talking about is when I want to read the page for myself. Waste of time to have to wait for an LLM to chew on it.
Really, for many “page searches”, a good search engine should just be able to take you immediately to the page. When I search “Tom Hanks IMDB”, there’s no need to see a list of links - there’s obviously one specific page I want to visit.
https://notes.npilk.com/custom-search
Are you feeling lucky?
Grok is great for finding details and background info about recent news, and of course it's great for deep-diving on general knowledge topics.
I also use Grok for quick coding help. I prefer to use AI for help with isolated coding aspects such as functions and methods, as a conversational reference manual. I'm not ready to sit there pretending to be the "pilot" while AI takes over my code!
For the record, I do not like Google's AI generated results it spams at me when I search for things. I want AI when I choose to use AI, not when I don't choose it. Google needs a way to switch that off on the web (without being logged in).
I know what I'm looking for. I just need exact URL.
Perplexity miserably fails at this.
Traditional search is dead, semantic search through AI is alive and well.
I can't yet count once AI misunderstood the meaning of my search while Google loves to make assumptions, rewrite my search query, and deliver the results that pay it the best which have the best ads (in my opinion as a lifetime user).
Lets not even mention how they willingly accept misleading ads atop the results which trick the majority of common users into downloading malware and adware on the regular.
The reason Google is still seeing growth (in revenue etc.) is that for a lot 'commercial' search still ends with this kind of action.
Take purchasing a power drill for example, you might use an LLM for some research on what drills are best, but when you're actually looking to purchase you probably just want to find the product on Home Depot/Lowe's etc.
Deleted Comment
Dead Comment
What? On Planet Earth, this is already a thing.
Kind of like a manual, with an index.
RTFM people.
Sounds trivial to integrate an LLM front end with a search engine backend (probably already done), and be able to type "frob language" and it gives you a curated clickable list of the top resources (language website, official tutorial, reference guide, etc) discarding spam and irrelevant search engine results in the process.
Deleted Comment
Or any other LLM that’s continuously trained on trending news?
> In responding to user queries, Grok has a unique feature that allows it to decide whether or not to search X public posts and conduct a real-time web search on the Internet. Grok’s access to real-time public X posts allows Grok to respond to user queries with up-to-date information and insights on a wide range of topics.
Other considerations:
- Visiting the actual website, you’ll see the programming languages logo. That may be a useful memory aide when learning.
- The real website may have diagrams and other things that may not be available in your LLM tool of choice (grok).
- The ACT of browsing to a different web page may help some learners better “compartmentalize” their new knowledge. The human brain works in funny ways.
- i have 0 concerns of a hallucination when readings docs directly from the author/source. Unless they also jumped on the LLM bandwagon lol.
Just because you have a hammer in your hand doesn’t mean you should start trying to hammer everything around you friend. Every tool has its place.
For some cases I absolutely prefer an LLM, like discoverability of certain language features or toolkits. But for the details, I'll just google the documentation site (for the new terms that the LLM just taught me about) and then read the actual docs.
I recently configured Chrome to only use google if I prefix my search with a "g ".
I don't like LLMs for two reasons:
* I can't really get a feel for the veracity of the information without double checking it. A lot of context I get from just reading results from a traditional search engine is lost when I get an answer from a LLM. I find it somewhat uncomfortable to just accept the answer, and if I have to double check it anyways, the LLM's answer is kind of meaningless and I might as well use a traditional search engine.
* I'm missing out on learning opertunities that I would usually get otherwise by reading or skimming through a larger document trying to find the answer. I appreciate that I skim through a lot of documentation on a regular basis and can recall things that I just happened to read when looking for a solution for another problem. I would hate it if an LLM would drop random tidbits of information when I was looking for concrete answers, but since its a side effect of my information gathering process, I like it.
If I were to use an AI assistant that could help me search and curate the results, instead of trying to answer my question directly. Hopefully in a more sleek way than Perplexity does with its sources feature.
At least that has been my experience. I admit I don't use LLMs very much.
This is my main reason for not using LLMs as a replacement for search. I want an accurate answer. I quote often search for legal or regulatory issues, health, scientific issues, specific facts about lots of things. i want authoritative sources.
You check the information you decide should be verified.
An LLM response without explicit mention of its provenance... There's no way to even guess whether it is authoritative.
What do you even use for double-check? Some random low-quality content farm? A glitchy LLM? An dodgy mirror of official docs full of ads? Or do you actually dig the source code for this?
And do you keep double-checking with all other information on the page... "A TOMLDecodeError will be raised on an invalid TOML document." - are you going to start an interactive session and check which error will be raised?
Just because you can find multiple independent sources saying the same thing doesn't mean it's correct.
"What I tell you three times is true"
Part of why I prefer to use a search engine is that I can see who is saying it, in what context. It might be Wikipedia, but also CIA world fact book. Or some blog but also python.org.
Or (lately) it might be AI SEO slop, reworded across 10 sites but nothing definitive. Which means I need to change my search strategy.
I find it easier (and quicker) to get to a believable result via a search engine than going via ChatGPT and then having to check what it claims.
And this is how LLMs perform when LLM-rot hasn't even become widely pervasive yet. As time goes on and LLMs regurgitate into themselves, they will become even less trustworthy. I really can't trust what an LLM says, especially when it matters, and the more it lies, the more I can't trust them.
Really, these days, either I know some resource exists and I want to find it, in which case a search engine makes much more sense than an LLM which might hallucinate, or I want to know if something is possible / how to do it, and the LLM will again hallucinate an incorrect way to do it.
I've only found LLMs useful for translation, transcription, natural language interface, etc.
LLMs have mostly been useful for three things: single line code completion (in GoLand), quickly translating JSON, and generating/optimizing marketing texts.
I use LLMs as a sounding board. Often if I'm trying to tease out the shape of a concept in my head, it's best to write it out. I now do this in the form of a question or request for information and dump it into the LLM.
"Search" can mean a lot of things. Sometimes I just want a website but can't remember the URL (traditional); other times I want an answer (LLMs); and other times, I want a bunch of resources to learn more (search+LLMs).
Bad: summarizing scientific research or technical data
Great: finding travel ideas or clarifying aspects of a franchise's fictional universe.
Instead I use a search engine and do my own reading and filtering. This way I learn what I'm researching, too, so I don't fall into the vicious cycle of drug abu ^H^H^H^H^H laziness. Otherwise I'll inevitably rely more on more on that thing, and be a prisoner of my own doing by increasingly offloading my tasks to a black box and be dependent on it.
Google recently (unrequested) provided me with very detailed AI generated instructions for server config - instructions that would have completely blown away the server. There will be someone out there who just follows the bouncing ball, I hope they've got good friends, understanding colleagues, and good backups!
Deleted Comment
What a weird sentence. What accuracy guarantees does Kagi have? Or, if you're not "offloading your brain to it", can't you do the same with an LLM?
Moreover, Kagi is a paid service. It has no ads, no hidden ranking, nothing to earn money by manipulating you. On the contrary you, the user, can add filters and ranking modifiers to promote the sites you find to be useful/truthful and demote others which push slop and SEO optimized content to your eyeballs. This is per user, and is not meddled with.
This makes Kagi very deterministic (unlike LLMs), very controllable (unlike LLMs), and very personalized (unlike LLMs). Moreover, Kagi gives you ~20 results or so per search, and no fillers (again, unlike LLMs).
I don't use Kagi's AI assistance features, and I don't pay for the "assistant" part of it, either.
I don't offload my brain to Kagi, because I don't prompt it until it gives me something I like. Instead, I get the results, read them, learn what I'm looking for, and possibly document what I got out from that research. This usage pattern, is again very different than prompting an LLM until it gives you something somewhat works or sounds plausible.
I do the hard work of synthesizing and understanding the answer. Not reading some slop and accepting it at face value.
As for AI search, I do find it extremely useful when I don't know the right words to search for. The LLM will instantly figure out what I'm trying to say.
And the ratio between using search engine and Kagi’s LLM agent with search is still 70% search. Sometimes, searching is faster, sometimes asking AI is faster.
I use LLM-s for what they are good at, generative stuff. I know some task take me a long time and I can shortcut with LLM-s easily.
So here's a ChatGPT example query* which is completely off:
https://chatgpt.com/share/67f5a071-53bc-8013-9c32-25cc2857e5...
* It's intentionally bad be able to compare with Google.
And here's the web result, which is spot on:
https://imgur.com/a/6ELOeS1
LLM's are great when you want AN answer, and not get side tracked.
Search is great when you want to know what answers are out there. The best example is Recipes... From what spices go into chai to the spice mix in any given version of chili (let's not start on beans).
The former is filling in missing knowledge the latter is learning.
https://imgur.com/a/boNS2YZ
https://chatgpt.com/share/67f5a9f9-f0a8-800d-9101-aafb88e455...
which I think is way better than google.
Google offered me a few hits with existing businesses, with ChatGPT I need to do another query.
Out of curiosity I tried it and it did take me to a wholesale company (single result), but the Google results are better with cheaper options (multiple good results), I can also parse the list faster with my eye.
Sure, I can just write a better prompt:
https://chatgpt.com/share/67f5b09b-c154-8013-840f-934af8302f...
This is my third attempt to get it right, but it found me one which I haven't seen before. However I would still do a Google search to be thorough and get the best deal.
So yeah, I do still use search engines, specifically Kagi and (as a fallback) DuckDuckGo. From either of them I might tack on a !g if I'm dissatisfied with the results, but it's pretty rare for Google's results to be any better.
When I do use an LLM, it's specifically for churning through some unstructured text for specific answers about it, with the understanding that I'll want to verify those answers myself. An LLM's great for taking queries like "What parts of this document talk about $FOO?" and spitting out a list of excerpts that discuss $FOO that I can then go back and spot-check myself for accuracy.