Surely people can relate to the situation where you end up on an article based on some technical query you have. The article repeats your question 7 times, has endless casually-related filler text that still does not answer the question and then ends with: try to unplug it.
It is so freaking obvious that it's a malicious content farm, but Google with all of its technical might seem unable or unwilling to detect it. If tech can't do it, organize some type of curation or feedback?
Same for image search. You search for "red flower Thailand" and flowers of various other colors from various locations appear. The idea that Google is spectacularly good at subject detection from imagery does not seem to actually work out in practice.
Most people's search queries consist of just 2-3 words. Nowadays Google consistently just drops the last word as if it knows better than I do what I need.
High value elaborate articles on various topics do not rank. Instead, dated articles do. You have to manually bookmark high quality content as you see it, because you'll never find it back via search.
Is everybody asleep at Google? This is not a small thing, this is your bread and butter. Teens are using Tiktok for search, you're in real trouble and better start cleaning up your act.
> Same for image search. You search for "red flower Thailand" and flowers of various other colors from various locations appear.
Now, if any of these flowers are next to a red dress, tapping the dress will reveal links to places you can buy it.
Google is not asleep. It has just got its priorities wrong. (Or rather, incentives in this organization seem to reward not what users like me appreciate.)
The thing is, all the stuff you’re listing as things they suck at properly detecting, filtering, categorizing, they used to be extremely good at. The google search that exists today is markedly worse than the Google search of 5 years ago. Wtf happened to cause search to just rot away into a useless mess of results that used to be very high quality? It’s such a night and day regression that I’ve legit wondered if this is a sign of the Mandela effect.
What you describe here in some order could be pointed at the fact they let someone who used to focus on Ads run all of organic search AND ads. This used to never happen, they had proper separation of church and state. The other part to blame imo is turning down the link graph, over reliance on NLP, and attempting to continue to prop up traffic to old media sites.
I wonder if they've misapplied the Youtube algorithm to searching. Other people who liked the pictures of red flowers also liked green flowers and purple flowers, so let's include that in the results, since that will probably generate more engagement and more engagement is obviously always good.
It always starts at the top. Sundar sure seems like Google's Balmer. He's good at keeping the lights on but from an outsider's perspective he doesn't seem to have any vision.
As an example of how bad the searchability is nowadays, I’ve been creating and expanding my own knowledge base (something like a personal wiki with links to interesting content I find) for about a year. It seems to work very well despite the effort it takes to keep it organized.
I've maintained my own hosted wiki since 2004, and yeah I'm glad I didn't completely outsource information management. It's definitely getting hard to find certain things I know I've seen.
Now I just need some kind of open source search engine to run on it ... (a bunch of text files that render to HTML, and ideally following the links 1 or 2 levels deep)
~20 years ago Google desktop search was a fantastic piece of software ... very fast and accurate on your local files. I don't think something like that exists now, and maybe never existed for Linux
Search engines are extremely modular and Unix-y. You have a bunch of indexed corpora and you intermingle them at ranking time, with respect to a query. But unfortunately there is no real incentive to provide something that has measurably good results and is also open to your own data and modifications
The incentive is to make a walled garden out of it
Same here, but in my notes app. Been doing it for years.
And if I may expand a little, not just for crappy Google, I also create alternative local knowledge bases at work.
I can't find anything at work. Everything is spread out across chat, Wikis, SharePoint, email. All having different owners, content may at any time disappear or move, there's constant authorization headaches.
Whenever I come across something useful that I expect to be of some future use, I make a local copy. File, web page, wiki, anything. Because our information systems are a massive failure.
I was about to ask you to share your list, but that got me wondering: is there any tooling for curating, sharing, and most importantly, consolidating curated lists of sites (based on tags rather than categories), such that the consolidated list is then searchable?
This is basically what I use my Wikipedia User Page for. Keeping links to all the news websites I happen to have found reasonably interesting content at. Stories I like that Wikipedia will probably not accept. Articles that I want to keep, yet might not be accepted for an article, or I have little faith they'll remain in the article. Probably just need to dump my bookmarks in occasionally.
Honest question, how much of this is simply due to Google slowing showing more & more ads on Page 1 of search results?
There use to be a time when paid placement was only 1-2 results.
It’s frequent now that the top 5-6 results are paid placement.
(And when I’m doing a search for a specific product I know I want, competitors are bidding up those search terms which is annoying because I’m being shown not what I’m explicitly searching for)
I have a hypothesis that a lot of this is the result of hyper focusing on short term reward. Just think about how we measure a top exec's performance. It is often dependent on cutting costs and increasing revenue. Cycle through that a bit and if the previous person did their job well then they cut a lot of fat. In fact, they probably cut as much as they thought they could get away with. So next person comes in and they gotta start cutting more than fat. Of course you could go other avenues to increase revenue but cost cutting measures are the easiest and quickest.
Kagi just renders those toxic sites into a grouping called "Listicles" which I then ignore. It's far from perfect, but it's clear that a company with far less money and access than Google doesn't find this an impossible problem to address.
So I would suggest that Google knows what it's doing, it just makes them money.
Google has commercialized a huge amount of search terms. I'll use biology as an example. You search for particular species and search results prioritize products that kill the species. You search for a particular plant and you'll have a hard time learning about the species as it only shows cultivated versions and products related to how to care for them.
Pure information/knowledge for the sake of learning and curiosity is de-prioritized.
Back to image search, it's unable to figure out original sources or doesn't care. Pinterest is the well known manifestation of that.
Google shopping results: completely broken. Click through on the products and half the time the price, availability, discounts and stock do not match.
Everything is so goddamn broken, and nobody at Google seems to care. I can't explain it, but it's been going on for a good 6-7 years or so.
It's because Google has never been focussed on search quality imo. No search engine produces high quality results anymore.
Especially since you get the clear spam sites that somehow reference your query in the page content (where they've just spammed loads of keywords, but also pretty sure some spam sites are doing something dynamic with it).
Google has maximized advertisement $ and that's all.
We're all technically minded here but very few people really understand how technical choices add up to greater detriments.
and that's today's Google. they minimized the index and maximized the searches that yield profit through Google ads. those websites you hate? they monetize Google ad words.
the one that pisses me off most is installing/configuring a software package... Top articles always end up being "apt-get install foo" and never address the configuration at all.
> We find that only a small portion of product reviews on the web uses affiliate marketing, but the majority of all search results do. [...] We further observe an
inverse relationship between affiliate marketing use and content complexity, and that all search engines fall victim to large-scale affiliate link spam
campaigns.
I think this is an excellent methodology for testing the quality of search results. I would love to see a standard search engine test and scoring system based on this, maybe similar to some of the LLM scoring systems.
This doesn't apply to the content complexity finding, but the finding that "product reviews which are in top search results are more likely to contain affiliate links than product reviews which are not" can also be explained by the fact that if and only I am getting a bunch of hits on my product reviews, I'm incentivized to monetize that with affiliate links.
We should also ask ourselves if affiliate links are really that bad. Someone could be making honest complete reviews and monetizing those with affiliate links, does that inherently mean that the search results are lower quality?
That approach also misses all the copied-a-github-issue low-effort content that seem to crop up on Google.
Forgive my naivety, but wouldn't a simple way for a search engine (like Kagi) to avoid falling victim here to detect affiliate link programs? There's got to be a small handful of patterns for affiliate link tracking:
1. Domain Interception & HTTP redirects
2. Tracking codes embedded in the URL directly
> Kagi surfaces shopping results featuring unbiased reviews and no affiliate links to help you identify the best product across categories. Top results include discussions focused on helping you find the best item to purchase - you are not bombarded with affiliate links and ads. Continue to scroll and you will see product comparisons across multiple vendors so you can pick what best suites you. Kagi's shopping search will always return a detailed discussion of which product to buy not a competition amongst advertisers promoting where you should buy. Kagi is focused on providing you the best results to make an informed decision not polluted by affiliate links and advertisements.
Currently, Kagi has (if you hover/click the shield icon to the right of a result) an indication of the information it knows about a website (as well as a way for you to rank it higher or lower for yourself).
One of these is "ads/trackers". I imagine that it would be feasible for this to include some of the more common affiliate URL types, or third party lead/affiliate tracking bounce hops like awin.
Clearly there will always be some amount of ability to "defeat" this kind of measure by obfuscating links, but eventually the user needs forwarded to a link that has a referral parameter or a site that sets an affiliate cookie or similar.
The "tracker category" also can give a bit of extra information - things like "invasive fingerprinting, advertising"
I am working on a search engine that checks a site for affiliate links from known providers and demotes sites that have a large amount of them.
In addition:
* it demotes sites with popups (think newsletter sign ups)
* it demotes sites that block (or complain about) ad blockers
* it demotes sites with a high number of ads and favors sites with no ads
* it demotes sites using certain sketchy ad companies.
* It demotes sites that have paywalls
* It detects possible link networks and flags them for human review/removal.
* sites with RSS feeds get promoted.
* There is a toggle to hide all sites with ads or external trackers, but it is still WIP (The whole project is).
There are many other features. No idea if I am going to make it public, I created it to update my skillset. I actually thought about setting up a nonprofit and making it open source, but I haven’t decided.
I don't know if this would be a very long-term solution if the big ones (ok, google) did it. Advertisers would catch on very quickly, and some legitimate review sites which might get funding through affiliate links unrelated to the product being reviewed would lose out to straight-up paid for "reviews" that are funded wholly by the manufacturer and just don't use affiliate links.
What would be the difference between an affiliate link program and a web ring?
I don't know if discovery is actually a bottleneck to be automated away. It might be the fun part. I'm thinking back to the Napster approach where you could browse other people's libraries for music ideas.
this is one way to do it, but I wouldn't say its sufficient. If I search for 'things to do in Seattle', you get many 'blogs' and such that a writer gets paid by sources to insert their place into the things to do list. I didn't word that well, so for example: I own a coffee shop, I pay them moneys, and the '25 things to do in Seattle' writer puts my coffee shop in the list.
If I do an image search for the word 'strawberry', how many of those results are not stock images, images from a store, etc. of a strawberry? can you find an actual picture of a strawberry sitting in the wild? or just some picture of a strawberry a person uploaded without trying to sell you something?
I don't know, feels like a paper titled "Is Google Getting Worse" could have benefited from actually looking at Google results rather than only results of other search engines.
Edit: This got downvoted to hell, so let me be more explicit. This study did not look at Google results, the title is pure clickbait. They used Startpage results as a proxy for Google results. I don't think that's a valid assumption, even if Startpage is using Google's index.
Using a reasonable proxy is not "pure clickbait", though it may be misleading. It adds an additional assumption about the Google results not being tampered with by Startpage, but it seems like a reasonable one, compared to the alternative of Google doctoring the search results when detecting a scraper and/or doing personalization.
If the authors have done their due diligence and confirmed the results from Startpage are actually Google results, then I don't see why they couldn't claim so in their title.
The authors did not do any work to establish it as a reasonable proxy. We know that's the case, because they don't describe any of that work in the paper. they just state that they're using this proxy measure, and then seem to be using Startpage and Google interchangably for the rest of the paper. Sure, if they'd actually done the searches on Google, the title would not be outright dishonest. But they didn't, so this defense of yours is just absurd.
It's pretty obvious why the results won't be the same: the feature sets of the search engines are different despite being based on the same index. Startpage's own documentation even has a page on how and why the results are different!
(Personalization is a part of the Google feature set, and has to be taken into account when considering the results. It's also a part of the Bing feature set. This didn't stop them from reporting the results of their testing on Bing in detail.)
Yes. Google used to be amazing, then it turned into an advertisement company. Slowly at first, then about a decade ago the pace increased.
But the worst part is, Google SEO has infected the entire web and made it into complete garbage.
Hopefully, this last decade or so will just be a blip before we return to baseline, where it can be wild and free again.
Are there stats* of Google Search across the years? I felt I don't use Google as much as I used to. And it isn't because "I know more stuff", but mainly because the way we use internet has changed. I wonder if kids or teens (who most of them don't know how to use an email inbox) would use Google... (I guess yeah?)
* Of course, the stats should include the total amount of internet users globally, or normalize the amount of searches based on that...
I've been in web development and SEO for almost 20 years now.
When I first started out all the veterans of SEO kept telling me not to do this, don't do that with things that could get your site buried in the SERP's. At the time Google's algorithm was really good at ferreting out affiliate links, link farms and other nefarious black hat techniques SEO's used to game Google.
Now? Complete opposite. I have several freelance clients and I've used every dirty SEO trick in the book and all of them have worked like magic to get my clients sites ranked on page 1 or 2 of the SERP's.
I have no idea what changed, but Google is super easy to manipulate now to get your site or specific pages ranking really high. I haven't heard or seen any of the horror stories I read and people blogged about constantly when I first started out for years - which tells me they're all probably doing the same thing I am and not seeing any repercussions.
Maybe Google doesn't care because users have become so savvy, they can filter through a ton of garbage in minutes to find what they really want?
Definitely I think the way we use the internet has changed profoundly. There are a lot of apps that provide useful information but they may not be made indexable by search engines. Much less useful information is simply out there on the open web, and much of it are locked behind logins. There were previous deals like Twitter sending a completely copy of all new tweets to Google, but these are basically dying.
It's especially interesting since you mentioned normalizing searches by the number of internet users. The country with the largest number of internet users is China, with more than 1 billion of them. And they don't have access to Google. And their local copycat, Baidu, is years behind Google in terms of technological sophistication and simultaneously years ahead of Google in terms of user hostility. So what do Internet users in China do in a post-search world? They simply open various apps and use the full text search feature of different apps. For general knowledge they might open ZhiHu and search there; for something resembling the old-time personal blogs by individual users they might open XiaoHongShu and search there; for short videos they might open Douyin and for long ones Bilibili. For reaching an organization be it a store or a museum or a hospital or a government department they might open WeChat and search there for an official account or mini program (a mini program is a website that uses WeChat APIs and can only be opened in WeChat).
I made these observations on a recent trip to China and it's clear to me what a post-search world looks like because China is already there.
> And their local copycat, Baidu, is years behind Google in terms of technological sophistication and simultaneously years ahead of Google in terms of user hostility.
Baidu search was fine during early days, issue as you hinted was PRC internet went mobile first and content got locked behind various platforms and made deliberately hard to scrape - crawling/indexing got locked much earlier than west. Hence now as more gets locked in west behind logins, western behaviours also shifting towards that model, how much of default search is query + wiki/reddit/youtube or straight into short video services like looking up recipes on XiaoHongShu. Reddit especially, simply because reddit app has horrible search experience. Also technically, Baidu rankdex predated Google PageRank which Larry Page referenced for Pagerank patents. Eitherway, depending on how ChatGPT copyright drama plays out, imo more people will just go the lazy route and have AI generate good enough summaries for most queries.
>I made these observations on a recent trip to China and it's clear to me what a post-search world looks like because China is already there.
You are talking like open web is dead but it's not. There are millions of blogs and personal sites out there. Walled gardens are user hostile and hungry for money, that's why enshittification[0] happens.
Long-term stats are tricky because of how much of the landscape has changed. There's the desktop/mobile split, developing countries increasing their internet use, heavier use of apps, growth and decline of results getting indexed, and change in what we search for.
It's not worse when you append site:reddit.com to every single search but this is only a function of the fact that reddit can't figure out how to build their own search. Outside of maybe programming stuff where I'll still click on links, I don't think google has driven me organically to a new site in years.
> It's not worse when you append site:reddit.com to every single search
Could you give some examples of search queries that would benefit from filtering by reddit?
(My own example: I've been looking for recommendations for a solid Linux laptop. A good result would be a list of reviews written from personal experience of owning such laptops. Reddit was useless for that.)
Fellow kagi user here. I have been very happy with the service so far. Switched over to it when they announced their new pricing model a little while ago. It is more than worth the cost. Even without tweaking, the quality of results is far higher than what I had gotten used to getting from DDG or DDG with !g. The ability to rank domains is amazing though.
Kagi has improved immensely and I no longer ever feel the need to check Google against it. It's a great product that I am more than happy to pay $10/mo
I'm so happy that their new pricing scheme has done away with search quotas. Their claim that the average user does 100 searches a month seems absurdly low. I would blow through my 300 search quota in a few days.
Every time I see someone talk about this feature on HN (almost every day) I get jealous, it's such a good idea. When I'm searching (on google) I'm always thinking; when I get Kagi I'm gonna downrank this site, uprank that one etc.
That is an interesting list; I'm not at all surprised to see pinterest holding the top ~10 spots, but things like facebook, instagram and twitter being so high does surprise me; perhaps it's indicative of the type of user they have (I too block those domains).
I could take that list, as-is and use it as a block list on my entire network.
Also a kagi user since they changed their pricing. The results have been decent for me for the most part, except for shopping which I still use google for. I do wish they’d step up their css game though. Maybe it’s just safari on iOS, but I get style problems constantly. The address bar doesn’t behave properly with the cursor if you type in a decently long query, and so often the search results will overflow the container horizontally. It’s a small price to pay for good search results, but stuff like that should be table stakes for a tool like this.
I get the reason why Google does not do domain level filtering due to antitrust concerns, but at least they should be able to adopt this kind of user preference approach. Why aren't they doing this years ago?
Quite likely underreporting affiliate links due to obfuscation like cloaking, hiding redirects behind javascript (they mention in the paper not rendering the page), using JS and a POST, other URL minifiers etc.
One interesting solution to the problem is to have more than one dominant search engine and its algorithmic choices, having half a dozen web-scale engines with some variation at least gives the user a choice into other avenues of information discovery. (There isn't really much point in using Startpage and DDG here since they're effectively meta search engines of Google and Bing). For SEOs in English speaking countries there is not much point in thinking beyond pleasing Google.
Clearly AI and whack-a-mole spam sites have been a problem for a while due to the prevalance of people tacking on 'reddit' to their query to find other humans talking about stuff.
Readit News
Surely people can relate to the situation where you end up on an article based on some technical query you have. The article repeats your question 7 times, has endless casually-related filler text that still does not answer the question and then ends with: try to unplug it.
It is so freaking obvious that it's a malicious content farm, but Google with all of its technical might seem unable or unwilling to detect it. If tech can't do it, organize some type of curation or feedback?
Same for image search. You search for "red flower Thailand" and flowers of various other colors from various locations appear. The idea that Google is spectacularly good at subject detection from imagery does not seem to actually work out in practice.
Most people's search queries consist of just 2-3 words. Nowadays Google consistently just drops the last word as if it knows better than I do what I need.
High value elaborate articles on various topics do not rank. Instead, dated articles do. You have to manually bookmark high quality content as you see it, because you'll never find it back via search.
Is everybody asleep at Google? This is not a small thing, this is your bread and butter. Teens are using Tiktok for search, you're in real trouble and better start cleaning up your act.
Now, if any of these flowers are next to a red dress, tapping the dress will reveal links to places you can buy it.
Google is not asleep. It has just got its priorities wrong. (Or rather, incentives in this organization seem to reward not what users like me appreciate.)
https://www.wired.com/story/prabhakar-raghavan-isnt-ceo-of-g...
And it's not just Google. Amazon is a mess. Social media is a mess. It all used to kind-of work but it's rapidly falling apart.
it's really not a deeper technical problem.
Now I just need some kind of open source search engine to run on it ... (a bunch of text files that render to HTML, and ideally following the links 1 or 2 levels deep)
~20 years ago Google desktop search was a fantastic piece of software ... very fast and accurate on your local files. I don't think something like that exists now, and maybe never existed for Linux
Search engines are extremely modular and Unix-y. You have a bunch of indexed corpora and you intermingle them at ranking time, with respect to a query. But unfortunately there is no real incentive to provide something that has measurably good results and is also open to your own data and modifications
The incentive is to make a walled garden out of it
And if I may expand a little, not just for crappy Google, I also create alternative local knowledge bases at work.
I can't find anything at work. Everything is spread out across chat, Wikis, SharePoint, email. All having different owners, content may at any time disappear or move, there's constant authorization headaches.
Whenever I come across something useful that I expect to be of some future use, I make a local copy. File, web page, wiki, anything. Because our information systems are a massive failure.
There use to be a time when paid placement was only 1-2 results.
It’s frequent now that the top 5-6 results are paid placement.
(And when I’m doing a search for a specific product I know I want, competitors are bidding up those search terms which is annoying because I’m being shown not what I’m explicitly searching for)
So I would suggest that Google knows what it's doing, it just makes them money.
Google has commercialized a huge amount of search terms. I'll use biology as an example. You search for particular species and search results prioritize products that kill the species. You search for a particular plant and you'll have a hard time learning about the species as it only shows cultivated versions and products related to how to care for them.
Pure information/knowledge for the sake of learning and curiosity is de-prioritized.
Back to image search, it's unable to figure out original sources or doesn't care. Pinterest is the well known manifestation of that.
Google shopping results: completely broken. Click through on the products and half the time the price, availability, discounts and stock do not match.
Everything is so goddamn broken, and nobody at Google seems to care. I can't explain it, but it's been going on for a good 6-7 years or so.
Especially since you get the clear spam sites that somehow reference your query in the page content (where they've just spammed loads of keywords, but also pretty sure some spam sites are doing something dynamic with it).
We're all technically minded here but very few people really understand how technical choices add up to greater detriments.
and that's today's Google. they minimized the index and maximized the searches that yield profit through Google ads. those websites you hate? they monetize Google ad words.
https://imgur.com/a/sJCECzQ
Looks mostly red to me. A little pink too I guess.
Dead Comment
I think this is an excellent methodology for testing the quality of search results. I would love to see a standard search engine test and scoring system based on this, maybe similar to some of the LLM scoring systems.
That approach also misses all the copied-a-github-issue low-effort content that seem to crop up on Google.
1. Domain Interception & HTTP redirects 2. Tracking codes embedded in the URL directly
> Kagi surfaces shopping results featuring unbiased reviews and no affiliate links to help you identify the best product across categories. Top results include discussions focused on helping you find the best item to purchase - you are not bombarded with affiliate links and ads. Continue to scroll and you will see product comparisons across multiple vendors so you can pick what best suites you. Kagi's shopping search will always return a detailed discussion of which product to buy not a competition amongst advertisers promoting where you should buy. Kagi is focused on providing you the best results to make an informed decision not polluted by affiliate links and advertisements.
1. https://help.kagi.com/kagi/features/shopping.html
One of these is "ads/trackers". I imagine that it would be feasible for this to include some of the more common affiliate URL types, or third party lead/affiliate tracking bounce hops like awin.
Clearly there will always be some amount of ability to "defeat" this kind of measure by obfuscating links, but eventually the user needs forwarded to a link that has a referral parameter or a site that sets an affiliate cookie or similar.
The "tracker category" also can give a bit of extra information - things like "invasive fingerprinting, advertising"
In addition:
* it demotes sites with popups (think newsletter sign ups)
* it demotes sites that block (or complain about) ad blockers
* it demotes sites with a high number of ads and favors sites with no ads
* it demotes sites using certain sketchy ad companies.
* It demotes sites that have paywalls
* It detects possible link networks and flags them for human review/removal.
* sites with RSS feeds get promoted.
* There is a toggle to hide all sites with ads or external trackers, but it is still WIP (The whole project is).
There are many other features. No idea if I am going to make it public, I created it to update my skillset. I actually thought about setting up a nonprofit and making it open source, but I haven’t decided.
I don't know if discovery is actually a bottleneck to be automated away. It might be the fun part. I'm thinking back to the Napster approach where you could browse other people's libraries for music ideas.
If I do an image search for the word 'strawberry', how many of those results are not stock images, images from a store, etc. of a strawberry? can you find an actual picture of a strawberry sitting in the wild? or just some picture of a strawberry a person uploaded without trying to sell you something?
Edit: This got downvoted to hell, so let me be more explicit. This study did not look at Google results, the title is pure clickbait. They used Startpage results as a proxy for Google results. I don't think that's a valid assumption, even if Startpage is using Google's index.
If the authors have done their due diligence and confirmed the results from Startpage are actually Google results, then I don't see why they couldn't claim so in their title.
It's pretty obvious why the results won't be the same: the feature sets of the search engines are different despite being based on the same index. Startpage's own documentation even has a page on how and why the results are different!
(Personalization is a part of the Google feature set, and has to be taken into account when considering the results. It's also a part of the Bing feature set. This didn't stop them from reporting the results of their testing on Bing in detail.)
Deleted Comment
But the worst part is, Google SEO has infected the entire web and made it into complete garbage. Hopefully, this last decade or so will just be a blip before we return to baseline, where it can be wild and free again.
* Of course, the stats should include the total amount of internet users globally, or normalize the amount of searches based on that...
When I first started out all the veterans of SEO kept telling me not to do this, don't do that with things that could get your site buried in the SERP's. At the time Google's algorithm was really good at ferreting out affiliate links, link farms and other nefarious black hat techniques SEO's used to game Google.
Now? Complete opposite. I have several freelance clients and I've used every dirty SEO trick in the book and all of them have worked like magic to get my clients sites ranked on page 1 or 2 of the SERP's.
I have no idea what changed, but Google is super easy to manipulate now to get your site or specific pages ranking really high. I haven't heard or seen any of the horror stories I read and people blogged about constantly when I first started out for years - which tells me they're all probably doing the same thing I am and not seeing any repercussions.
Maybe Google doesn't care because users have become so savvy, they can filter through a ton of garbage in minutes to find what they really want?
It's especially interesting since you mentioned normalizing searches by the number of internet users. The country with the largest number of internet users is China, with more than 1 billion of them. And they don't have access to Google. And their local copycat, Baidu, is years behind Google in terms of technological sophistication and simultaneously years ahead of Google in terms of user hostility. So what do Internet users in China do in a post-search world? They simply open various apps and use the full text search feature of different apps. For general knowledge they might open ZhiHu and search there; for something resembling the old-time personal blogs by individual users they might open XiaoHongShu and search there; for short videos they might open Douyin and for long ones Bilibili. For reaching an organization be it a store or a museum or a hospital or a government department they might open WeChat and search there for an official account or mini program (a mini program is a website that uses WeChat APIs and can only be opened in WeChat).
I made these observations on a recent trip to China and it's clear to me what a post-search world looks like because China is already there.
Baidu search was fine during early days, issue as you hinted was PRC internet went mobile first and content got locked behind various platforms and made deliberately hard to scrape - crawling/indexing got locked much earlier than west. Hence now as more gets locked in west behind logins, western behaviours also shifting towards that model, how much of default search is query + wiki/reddit/youtube or straight into short video services like looking up recipes on XiaoHongShu. Reddit especially, simply because reddit app has horrible search experience. Also technically, Baidu rankdex predated Google PageRank which Larry Page referenced for Pagerank patents. Eitherway, depending on how ChatGPT copyright drama plays out, imo more people will just go the lazy route and have AI generate good enough summaries for most queries.
You are talking like open web is dead but it's not. There are millions of blogs and personal sites out there. Walled gardens are user hostile and hungry for money, that's why enshittification[0] happens.
[0] https://en.wikipedia.org/wiki/Enshittification
Quick: If you want to search on what is the best toaster to buy, what URL do you type?
How do you find out the weather?
How do you find a local brewery in your area?
For me the first answer is Amazon. And the second is ask Siri. The third is Apple Maps.
Google is now below 50% of my search terms. Heck, I’m more likely to search Reddit for some queries.
Could you give some examples of search queries that would benefit from filtering by reddit?
(My own example: I've been looking for recommendations for a solid Linux laptop. A good result would be a list of reviews written from personal experience of owning such laptops. Reddit was useless for that.)
People actually discussing how they did this is the only real answer, generic websites with pictures and descriptions of his weapons never help.
[0] https://blog.kagi.com/kagi-features
I'm so happy that their new pricing scheme has done away with search quotas. Their claim that the average user does 100 searches a month seems absurdly low. I would blow through my 300 search quota in a few days.
I could take that list, as-is and use it as a block list on my entire network.
One interesting solution to the problem is to have more than one dominant search engine and its algorithmic choices, having half a dozen web-scale engines with some variation at least gives the user a choice into other avenues of information discovery. (There isn't really much point in using Startpage and DDG here since they're effectively meta search engines of Google and Bing). For SEOs in English speaking countries there is not much point in thinking beyond pleasing Google.
Clearly AI and whack-a-mole spam sites have been a problem for a while due to the prevalance of people tacking on 'reddit' to their query to find other humans talking about stuff.