- Applebot was originally written in Go (and uncovered a user agent bug on redirects, revealing it's Go origins to the world, which Russ Cox fixed the next day).
- Up until the release of iOS 9, Applebot ran entirely on four Mac Pro's in an office. Those four Mac Pro's could crawl close to 1B web pages a day.
- In it's first week of existence, it nearly took Apple's internal DNS servers offline. It was then modified to do it's own DNS resolution and caching, fond memories...
This is true but a little misleading. On Windows Go uses GetAddrInfo and DNSQuery which does the right thing. But on Linux there are two options: netgo and netcgo -- a pure Go implementation that doesn't know about NSS, and a C wrapper that uses NSS.
Since netgo is faster, by default Go will try its best to determine if it must use netcgo by parsing /etc/nsswitch.conf, looking at the tld, reading env variables, etc..
If you're building the code you can force it to use netcgo by adding the netcgo build tag.
If you're an administrator the least intrusive method I think would be setting LOCALDOMAIN to something or '' if you can't think of anything which will force it to use NSS.
>Up until the release of iOS 9, Applebot ran entirely on four Mac Pro's in an office. Those four Mac Pro's could crawl close to 1B web pages a day.
The scale of web stuff sometimes surprises me. 1B web pages sounds like just about the daily web output of humanity? How can you handle this with 4 (fast) computers?
Doesn't it depend on a lot of things? For example you can only do head requests to see if a page changed since a given timestamp. If not then there is no need to process it.
Does it use a traditional relational database or another existing database-like product? Or is built from scratch just sitting on top of a file system.
Nope, you don't really need a database. What you need for fast, scalable web crawling is more like key-value storage: a really fast layer (something like RocksDB on SSD) for metadata about URL's, and another layer that can be very slow for storing crawled pages (like Hadoop or Cassandra). In reality, writing directly to Hadoop/Cassandra was too slow (because it was in a remote data center) so it was easier to just write to RAID arrays over Thunderbolt, and sync the data periodically as a separate step.
Interesting stuff. I've used libcurl to crawl at that kind of pace, is the parsing/indexing separate from that count per day? Also interested in how you dealt with DNS and/or rate limiting
I've done similar at smaller scale. Instead of messing with underlying DNS or other caching in our code we just dropped a tuned dnsmasq as the resolver in front. The crawler had a separate worker to fill hosts so it was mostly hot when the crawler was asking.
No idea, it's been years since I last worked on it. It was also not the only Go service written at Apple (90% of cloud services at Apple were written in Java), though it may have been the first one used in production.
You should check out Manning's "Introduction to Information Retrieval", it has far more detail about web crawler architecture than I can write in a post, and served as a blueprint for much of Applebot's early design decisions.
With 1b pages per day I guess you needed 1gbit/s connections on each of those machines? Especially if they also wrote back to centralized storage.
I guess there are not many places where you can easily get 4GB/s sustained throughput from a single office (especially with proxy servers and firewalls in front of it). Is that standard at Apple or did the infrastructure team get involved to provide that kind of bandwidth?
I know Apple's walled garden approach isn't for everybody, but I'd personally love to see them make a search engine that isn't built around Ads. The strategy also is consistent with their pursuit of vertical integration.
I use DuckDuckGo for search, and do trust them with privacy. But if Apple did a half-decent job at nailing the search experience, I'd switch.
I think it's too soon to assume that Apple's search engine would be accessible directly from the web, rather than just being a backend for search functionality from within iOS. After all, they don't host any Apple Maps equivalent to Google Maps on the web.
A search engine that optimizes for good content instead of SEO-spam and ads would be a very welcome addition. Google seems to have given up on fighting malicious/spammy SEO (I'd expect them to identify and downrank spammy tricks like what the recipe websites do).
Google search has gone down the toilet. Most Google search results are ads or SEO optimized junk. It was funny once when I was searching for a medical term, all articles on first page were clearly SEO optimized and all articles were very similar from word usage, sentence construction, verbiage, like written by same person. I am surprised that Google doesn't filter out the results using some sort of similarity analysis to offer variety in results. I don't want to see all links giving exactly the same info.
And the same rumours has been going on for years, and I seriously doubt that.
Reason is that Apple already has a Search Engine. It is called Siri. It is what you get when you ask Siri Questions. What I think Apple hopes to achieve is that get 80% of what you want, From may be Recipes, Simple Answers like Exchange Rate, Nutrients, Sport Scores... etc away from Google.
Letting you to Google the absurd, hard, questions that has less Data value.
All while collecting roughly $10 per user for Default Search Engine Placement in Safari.
And remember the Privacy stand Apple has giving less Data to Google.
Basically Apple is squeezing Google in every single direction. And pitching them against Azure and AWS for Cloud Services for discount.
I can't believe this hasn't happened yet honestly. Apple Maps has been out for 7 years now. Also a bit troubling if it gains traction, considering Apple has been a lot more heavy handed in keeping content they don't approve of out from their ecosystem.
In all seriousness, Apple may do certain things well, but cloud-anything just doesn't appear to be in its DNA. Maps, mail, iCloud, Timemachine, etc. Pretty much every service I can think of is laden with bugs, quirks, actual data loss risk, or is slow enough to be unusable. I'm not even remotely surprised that there is no "Apple search" yet.
Also of note, Apple Maps in the iOS 14 beta just started accepting ratings and photos at map locations, rather than dumping you out to Yelp reviews. I'm happy with this change.
Google's results are already pretty clean. Search results are heavily filtered (e.g. copyright, adult content) so I don't think Apple would filter more. If anything, filtering more than Google could deteriorate results.
But the SEO market would certainly change if webmasters suddenly have to appeal to more than just Google to be successful.
Apple built their own search engine over 5 years ago, under the Siri / Spotlight umbrella. When people talk about Apple building their own search engine, they generally seem to expect a website dedicated primarily to web page results, but under the covers what powers Apple's Spotlight results is basically a search engine.
The big question would be what Apple would gain from a dedicated website for search results. Would people really switch to it from Google? Why would it be a better delivery mechanism for search results than Spotlight? Not sure the answers to these questions has changed much, from 5 years ago to today.
Let's see if Apple is the first one to nail it. I love using DDG but their results are not close to the same quality as Google for me, especially with localized results and unspecific requests. And DDG is backed by Microsoft's Bing which invested billions into this. Though having run a few websites, Google's crawler easily crawls 10x the amount of all other crawlers combined if they don't detect page slowdowns. I never understood why Bing was so conservative when it comes to crawling.
I believe if anyone can build a search engine that can truly deliver the same quality as Google, the whole market could change. Google is dependent on search ad revenue like nothing else, if they lose that monopoly it could really change the company.
my first reaction , was oh god, please dont! considering how worst siri is, i dont want that.. but after reading other comments like native ad tracking etc etc it could be just fine.
But google pays Apple in billions to keep its search as default, what could be the key motivation for Apple to build its own engine? its a highly saturated market and we have Duck Duck Go for privacy based , bing for alternative and heck, even wolfram alpha for computational! what Apple can improve?
Based on usage of DDG, bing and wolfram alpha, I would think Apple can improve a great deal. I'm not saying it can necessarily achieve that, but I do think there is a lot of space to improve (indexing, ranking, UX)
I remember when I rolled my eyes at Apple making a Maps product. I thought it was a fruitless, dumb idea.
I prefer Apple Maps to Google Maps now... don't underestimate Apple's ability to plant a flag and move it inch by inch each year.
In my personal experience Google Maps has better navigation data, better place information and overall much better maps. The user contributed content makes the platform such a pleasure to use even if you are in remote parts of the world.
On my iPhone I think Apple Maps has a vastly better navigation UI. Part of that is its first-party ability to turn the phone screen on when issuing instructions for upcoming maneuvers, which I much prefer to having the screen on constantly for a multi-hour trip (not to mention working well with Siri). Part of it is the overall aesthetic of the map features. I really like how it highlights traffic lights along your route to let you easily count how many lights are remaining until you need to turn.
That said, even here in the Bay Area I tend to do a sanity check of the route on Google Maps first if I’m taking an unfamiliar route. Just a few months ago a section of I-80 was closed just south of SF and Apple Maps had no clue! I believe it was a fairly last-minute schedule change for some planned construction due to vastly reduced traffic during the lockdowns.
Judging from the responses here, some of the reasons seem to be because Apple Maps is much more vertically integrated with iOS, and that doesn't seem like a good thing to me. Sure, you may get a better experience, but that's more likely because Apple is keeping Google Maps from getting as good than because it's a better solution, period. I have no love for Google but that's not a good model, and I feel like this kind of thing is making Spotify's case for Apple being a monopoly much stronger.
I also find that Apple Maps respects local mapping conventions. For example in New Zealand Google treats all state highways as equivalent and uses the same weight and colour line for all of them, but Apple Maps follows the NZ convention and assigns appropriate weights to the minor roads.
For those not familiar "state highway" simply means the central government agency pays for upkeep, rather than indicating any particular road quality or status.
Not OP, but I'm in the same boat. Agreed Google Maps' data is better. Their routing is sometimes better. But I still prefer Apple Maps in most cases. For me, it comes down to a few UX things:
* iPhone + Apple Watch combo is amazing to use w/ Apple Maps. Audible & haptic alerts on your wrist change how one interacts with the turn-by-turn directions, and it's an improvement.
* The fact that iPhone nav is available on the lock screen is a huge help. Doesn't require full unlock to get to the map.
* Apple Maps just seems to handle the accelerometer in the iPhone better. I found that Google Maps never knows what direction I'm facing/walking, and it jumps all over the place all the time. Apple Maps is just smoother and better to use.
The only thing that's missing in Apple Maps is bicycle directions, but apparently that's coming in iOS 14, which is great.
The main time I find myself using Apple Maps is when visiting China. Google Maps is blocked so the place data isn't as great, and the local apps are in Chinese, which I'm illiterate in.
I similarly find Apple Pay great for traveling, and easier to use than a credit card.
But, on my no-longer-bleeding-edge iPhone, the turn-by-turn directions suck. It's constantly telling me to turn onto a street I just passed, or rerouting to find a way to get me back to the street that I'm already on. The worst part is that I can't trust it even when it's right. If it says "in 300 feet turn left," I have to desperately look at the street names because it's quite possible that I actually have to turn in 30 feet.
It was such a relief when I realized that it's not my GPS hardware, Apple Maps gives directions perfectly. And the map data is way better than it used to be. Not as good as Google Maps, but good enough.
I hate Google maps when it shuts itself off with a 'journey complete' status while I'm still looking for the corner or the building that I was supposed to arrive at.
Its really small, but for me its traffic lights. It's SOOOO much easier to be navigating and say, turn in two lights, instead of "turn in the next three streets". Especially on busy roads when you need to be in a certain lane to make that left or right hand turn.
Google maps added this to their desktop app [1] and maybe their iOS app, but apple has had this for years.
Also Apple maps offers saved places in an offline, non-synced way. Google requires access to all my location data to save places (like web bookmarks) and that's just too privacy invasive for me.
Even a few years ago I preferred the way a few-block area was represented on Apple Maps vs Google Maps. Apple Maps seemed to be much better at showing relevant landmarks and business names, where Google Maps would randomly select some businesses that happened to have an address there, but often no signage, etc.
Both applications would select the same number of businesses to show (obviously not showing ALL names due to density restrictions on the screen and clutter), but the Google ones were just so painfully arbitrary, and Apple's building skeletons seemed a lot more relevant/recognizable.
In my experience, Apple Maps has been more up to date with recent changes to roads here in Norway. There was a new tunnel here recently, and it was updated in Apple Maps when it opened, while with Google Maps it took a while.
I'm guessing that it depends a lot on the area. Google Maps is still probably better in most places, but Apple Maps is catching up and may be better in some populated places. It's impressive considering the head-start Google has.
Apple should add an option of OpenStreetMaps overlay or something. OSM has been better than Google Maps in mapping paths in the forest and such here.
Not the parent commenter, but while I still use Google Maps over Apple Maps, I constantly have issues with the former.
I've put in business addresses or searched the business themselves, and then had it route me to a residential block behind the business on a completely different street, sometimes multiple blocks away from it. This is even after confirming the exact address is being shown and everything.
I only really use Google Maps because it's what I've been using, and I kind of like that its UI is a bit simpler with less animations and things than Apple Maps.
I don't have unlimited data, and I have found Apple maps is much lighter on data use than Google. I have also found that even if I don't have a solid connection Apple maps seems to still be capable of giving me directions quickly, whereas on the other hand with Google I may never get directions.
1 - It actually runs smoothly on my iPhone 11. Google Maps stutters and yanks. Yes, Google cannot build a performing maps app for the fastest phone in the world.
2 - It is a map. Not a yellow pages of coffee places, not an Instagram with amateur photos of locations.
Have you tried it lately? It gets better with each iteration, and the street photos are way better where they exist. The satellite and aerial photos are also better in most places that I travel too.
Also, the Google Maps app has gotten much more cluttered and complex than in past versions. They are still the gold standard map, but they regress over time.
My house has not been in Apple Map's index for approaching three years now. It's caused _all kinds_ of problems. Apple seems to drop the ball on the details that are needed to really get a product to be quality.
EDIT: I looked back in my emails and Twitter DMs and it's been 25 months at this point. So "approaching three years" is not accurate.
Have you tried reporting it? My parents recently moved into a new house built one year prior and I reported it a few days after they moved in. It took about 9 months, but they finally added the road in and populated ALL the addresses on the street, not just the one I reported.
Google maps still does not have the roads or street numbers listed. Just wanted to throw that out there.
Edit: Also want to add that it has been two years since my parents moved in. Went in to Waze Map Editor and added the road myself, so their turnaround time was instant. Apple Maps took 9 months, like I said, but Google Maps still has no info on the street despite having all the houses in clear satellite view.
Google Maps is making it easier on them by seemingly becoming worse and worse, in my opinion. The newest act of telling you it's changing your route if you don't click something is so idiotic it borders on malice. So going 70mph down the expressway, I need to take my eyes off the road to read the message and click a button asking it not to take action.
Further, it repeatedly picks a hard route. There's one park I go to that's easier to drive to using one few mile stretch of highway. Google insists it's faster to take these back, poorly lit, windy 2 lane roads. Maybe it is. But it turns a safe easy 3 turn trip into a nerve-wracking 10 turn trip. And it will, as noted above, keep trying to reroute me if I just ignore it and go the easy way.
Another thing google maps always does to me is telling me what the next turn is when its too late and I can't make it anymore. Using google maps with audio only is impossible.
Noticed that recently too - it insisted on taking stupid backroads and what not, instead of straight highway and no traffic. It seems to be getting dumber, or something.
I tried AppleMaps for the first time like 4-5 years ago. Feature-wise it wasn't where it is now but what annoyed me the most is that the GPS on map lagged FAR (1-2 seconds) behind my real position. More than once I missed a turn because of that, especially when you are driving in an urban area with a lot of smaller streets.
Fast-forward to 2020 - in the meantime I used Google Maps with no issues at all. A friend of mine suggested that I should try the latest version and maybe I would change my mind. So I used it for a day - same urban area - and it's still lagging behind - mind you: at this time I had a new iPhone and a new carrier. For me this makes the App unusable. I don't want to plan ahead 500 - 1000m (Germany) while using navigation.
I'm really curious, does someone have the same problem?
Absolutely this. Not sure if it's 1-2 seconds but the delay is enough to make it unusable. I've missed turns and exits because of it. Google Maps seems to have some predictive ability that allows it to appear to be "real-time".
I used to say that but lately apple maps seems to be wrong a lot - back to how it was when it first started, probably a temporary regression (I hope).
Yesterday for example it told me to do a U turn in a (sort of) one way street in the middle of the city, and not the first time its been dumb in the last few weeks.
Apple Maps is way better than Google for the wandering pedestrian. Google hides so many restaurants and bars that don’t pay them for prominent display. Apple Maps shows every place near my location. Obviously hasn’t mattered recently, but I always go to Apple naps for nearby browsing.
Google needs the competition. Search it's getting more and more laden with ads, and I have found the kind of specific technical searches it used to excel at are increasingly on page 2 after articles and videos and product listings.
Not to mention the dozen poorly-formatted Stack Overflow and GitHub issue aggregators that show up before the actual Stack Overflow and GitHub results.
hell, it ain't just ads, it's outright malware at this point. i had to look up some shit about my printer, and hoo boy, searching for that churns up even more of a cesspool than i get when trying to search specialized windows topics. those don't turn up anything useful most of the time, but it's at least benign--a wall of posts on official MS forums where the top answer is some "top community contributor" yeeting out a "my friend, have you tried rebooting and reinstalling drivers?" copy-pasta response to any conceivable question
the printer stuff is something else. i put some fault on canon here for pushing me to unofficial sites because their official shit requires you to download a fucking exe to deliver a PDF manual (cmon, wtf, how can your docs delivery pipeline be THAT bad), but the unofficial stuff is some ninth circle of hell
the first was some bog-standard infini-redirect ad fraud crap, but the second was uh... well, i was impressed to the degree that the indian tech chop shop "YOU ARE INFECT DIAL 1-800-LEGIT-MICROSOFT TO CLEAN COMPUTER" site was able to shut down a fully up-to-date copy of firefox, somehow eating all my other tabs, grabbing hold of my mouse pointer in strange ways when the window was in focus AND preventing the windows "close window" button from killing it.
task manager was still able to execute it but damn, that was some ARTISTRY put into making the fraud site as clingy as possible--puts 90s-era malware sites to shame, and this shit kicks around on right on the first page of google search results for a printer manual query, not some shady-ass porn site you'd drunkenly click through to via 3 levels of ever-sketchier ads
people ranted and raved about goods of questionable quality gaming amazon reviews, but eh--honestly, most of those i've bought were fine, if not artisanal high-end shit--they just engaged in a lot of dubious "get free shit for a good review" growth hacking tactics that are A-OKAY THATS JUST HOW YA DO BUSINESS when it's some SF startup pulling the same crap in a different form. it's weird how seemingly nobody gives a fuck that a lot of google searches will boost things that are far worse--offering purestrain fraud on a platter is fine, selling mediocre flash drives and kitchen gadgets with inflated reviews is THE HIGHEST OF CRIMES
Diversity would definitely be positive, the web is too important to have it indexed and sorted from one perspective only. Would prefer half a dozen engines and perhaps all enabled to provide results to metas who could further offer more diversity.
Part of that, I think, is just that the SEO industry has matured so much any random person can quickly climb the rankings by short-circuiting Google with basic SEO, taking over space which should've gone to more quality content instead.
I’m hearing a lot of theories in this thread about Apple building a search product.
Yes, that’s possible, but Applebot is used in a lot of ways today that are wholly unrelated to search. (If search consists of crawling, indexing, ranking, retrieval, frontend, etc., Applebot only does the crawling part.)
Applebot is used for generating attachment previews in iMessage (eg. Send someone a URL — the preview is from Applebot crawling it). From the docs, it sounds like it’s also used for similar previews in Siri.
Applebot was built for crawling web pages, to be used for search results in Spotlight and Siri. That user agent might also be used for attachment previews, but the original intent of Applebot was for search indexing.
> Applebot is used for generating attachment previews in iMessage (eg. Send someone a URL — the preview is from Applebot crawling it)
Is this true? I would have assumed that would be done locally on device. The preview image, for example, is what's designated by the Open Graph[0] og:image tag.
Crawling and caching is pretty common for any service that offers unfurled previews to avoid sending extra traffic to sites that may not be able to handle it, and in many cases I'd imagine retrieving the pre-extracted cached version to be faster.
But do you remember when Apple launched their maps? A decade later and it's still not nearly as good as Google maps. But it's "good enough". I imagine that's what they're going for with this new search service
The data that Apple previously paid TomTom and others to use was certainly subpar compared to Google's data.
However, Apple started collecting their own data in 2015, and rolling out their own maps in 2018.
>Apple is filling its map with so many [details] that Google now looks empty in comparison and all of these details create the impression that Apple hasn’t just closed the gap with Google—but has, in many ways, exceeded it.
Think the summary from what I've read the past few days
- It's established applebot is already used for Siri, but there is no general web search
- Much commentary diverts to how poor Apple maps (is?) was.
- Google pay apple a handsome amount of money to be the default search engine on Apple devices
- If Apple were no longer to accept that payment, about 80% of people would still use Google anyway, given a choice [0]
- Googles high earnings per search means they can offer more than any other search engine, like Bing, DDG et al
- People would generally like more competition in the search space and Apple has the means to do it
- Anecdotally, there isn't a drop in replacement for a search default other than Google that would meet the quality requirements for end users. In this case, the only other alternative would be an Apple search engine.
- As per recent articles, there does seem to be more activity from Apple wrt applebot crawling and hiring
If they get a choice with no silent default yes, but if the default is set for them and they have to go into preferences to change it the results are almost completely reversed.
This is why Apple Maps is so heavily used on iOS. Most people don't know or don't care. In the first year after the launch of Apple Maps it hit over 65% of maps usage on iPhones in the UK, even though back then the UK maps were awful.
Remember: ARPANet was originally just a DOD research project to connect some universities and military sites. Back when it was opened to the public you could get a .com domain name for free just by asking. Same deal with IP addresses: in the very early days you just had to ask and you could have your own /8 (assuming you were a university, medium-to-large company, or government entity).
There has never been any point in clawing back the large class A assignments. If that were done it wouldn't even give us an extra year of IPv4 address assignments before we exhaust the pool again... not that there is any legal mechanism to take them back anyway.
The solution is IPv6. Despite the slow rollout it is finally happening now that even the regional RIRs have started to run out of addresses.
Readit News
- Applebot was originally written in Go (and uncovered a user agent bug on redirects, revealing it's Go origins to the world, which Russ Cox fixed the next day).
- Up until the release of iOS 9, Applebot ran entirely on four Mac Pro's in an office. Those four Mac Pro's could crawl close to 1B web pages a day.
- In it's first week of existence, it nearly took Apple's internal DNS servers offline. It was then modified to do it's own DNS resolution and caching, fond memories...
Source: I worked on the original version.
Unlike other languages, Go bypasses system's DNS cache, and goes directly to the DNS server, which is a root cause of many problems.
Since netgo is faster, by default Go will try its best to determine if it must use netcgo by parsing /etc/nsswitch.conf, looking at the tld, reading env variables, etc..
If you're building the code you can force it to use netcgo by adding the netcgo build tag.
If you're an administrator the least intrusive method I think would be setting LOCALDOMAIN to something or '' if you can't think of anything which will force it to use NSS.
If you're on a system with cgo available, you can use `GODEBUG=netdns=cgo` to avoid making direct DNS requests.
This is the default on MacOS, so if it was running on four Mac Pro's I wouldn't expect it to be the root cause.
As I understand it, Go and Java are both trying to avoid FFI and calling out to system libs for name resolution.
I tend to always offer a local caching resolver available over a socket.
Considering the timeline, are those Trash Can Mac Pro? Or was it the old Cheese Grater ?
The scale of web stuff sometimes surprises me. 1B web pages sounds like just about the daily web output of humanity? How can you handle this with 4 (fast) computers?
Does it use a traditional relational database or another existing database-like product? Or is built from scratch just sitting on top of a file system.
Deleted Comment
I guess there are not many places where you can easily get 4GB/s sustained throughput from a single office (especially with proxy servers and firewalls in front of it). Is that standard at Apple or did the infrastructure team get involved to provide that kind of bandwidth?
All three uses of "it's" should be "its".
And I would just write "Mac Pros" instead of Mac Pro's".
https://appleinsider.com/articles/20/08/27/apple-may-launch-...
I use DuckDuckGo for search, and do trust them with privacy. But if Apple did a half-decent job at nailing the search experience, I'd switch.
The ads that Apple sells on the app store contribute $2 billion in revenue -- I don't think they'd build a search engine that isn't built around ads.
Google search has gone down the toilet. Most Google search results are ads or SEO optimized junk. It was funny once when I was searching for a medical term, all articles on first page were clearly SEO optimized and all articles were very similar from word usage, sentence construction, verbiage, like written by same person. I am surprised that Google doesn't filter out the results using some sort of similarity analysis to offer variety in results. I don't want to see all links giving exactly the same info.
Reason is that Apple already has a Search Engine. It is called Siri. It is what you get when you ask Siri Questions. What I think Apple hopes to achieve is that get 80% of what you want, From may be Recipes, Simple Answers like Exchange Rate, Nutrients, Sport Scores... etc away from Google.
Letting you to Google the absurd, hard, questions that has less Data value.
All while collecting roughly $10 per user for Default Search Engine Placement in Safari.
And remember the Privacy stand Apple has giving less Data to Google.
Basically Apple is squeezing Google in every single direction. And pitching them against Azure and AWS for Cloud Services for discount.
https://9to5mac.com/2020/08/26/apple-maps-reviews-inhouse/
But the SEO market would certainly change if webmasters suddenly have to appeal to more than just Google to be successful.
How does this benefit Apple? They’re a hardware company.
The big question would be what Apple would gain from a dedicated website for search results. Would people really switch to it from Google? Why would it be a better delivery mechanism for search results than Spotlight? Not sure the answers to these questions has changed much, from 5 years ago to today.
I believe if anyone can build a search engine that can truly deliver the same quality as Google, the whole market could change. Google is dependent on search ad revenue like nothing else, if they lose that monopoly it could really change the company.
But google pays Apple in billions to keep its search as default, what could be the key motivation for Apple to build its own engine? its a highly saturated market and we have Duck Duck Go for privacy based , bing for alternative and heck, even wolfram alpha for computational! what Apple can improve?
In my personal experience Google Maps has better navigation data, better place information and overall much better maps. The user contributed content makes the platform such a pleasure to use even if you are in remote parts of the world.
That said, even here in the Bay Area I tend to do a sanity check of the route on Google Maps first if I’m taking an unfamiliar route. Just a few months ago a section of I-80 was closed just south of SF and Apple Maps had no clue! I believe it was a fairly last-minute schedule change for some planned construction due to vastly reduced traffic during the lockdowns.
For those not familiar "state highway" simply means the central government agency pays for upkeep, rather than indicating any particular road quality or status.
* iPhone + Apple Watch combo is amazing to use w/ Apple Maps. Audible & haptic alerts on your wrist change how one interacts with the turn-by-turn directions, and it's an improvement.
* The fact that iPhone nav is available on the lock screen is a huge help. Doesn't require full unlock to get to the map.
* Apple Maps just seems to handle the accelerometer in the iPhone better. I found that Google Maps never knows what direction I'm facing/walking, and it jumps all over the place all the time. Apple Maps is just smoother and better to use.
The only thing that's missing in Apple Maps is bicycle directions, but apparently that's coming in iOS 14, which is great.
I similarly find Apple Pay great for traveling, and easier to use than a credit card.
But, on my no-longer-bleeding-edge iPhone, the turn-by-turn directions suck. It's constantly telling me to turn onto a street I just passed, or rerouting to find a way to get me back to the street that I'm already on. The worst part is that I can't trust it even when it's right. If it says "in 300 feet turn left," I have to desperately look at the street names because it's quite possible that I actually have to turn in 30 feet.
It was such a relief when I realized that it's not my GPS hardware, Apple Maps gives directions perfectly. And the map data is way better than it used to be. Not as good as Google Maps, but good enough.
Probably 75% of my Google Maps usage is to check business hours or grab a phone number; they've really got that down. Apple is catching up, though!
Google maps added this to their desktop app [1] and maybe their iOS app, but apple has had this for years.
Also Apple maps offers saved places in an offline, non-synced way. Google requires access to all my location data to save places (like web bookmarks) and that's just too privacy invasive for me.
[1]: https://arstechnica.com/gadgets/2020/08/google-maps-starts-w...
Both applications would select the same number of businesses to show (obviously not showing ALL names due to density restrictions on the screen and clutter), but the Google ones were just so painfully arbitrary, and Apple's building skeletons seemed a lot more relevant/recognizable.
I'm guessing that it depends a lot on the area. Google Maps is still probably better in most places, but Apple Maps is catching up and may be better in some populated places. It's impressive considering the head-start Google has.
Apple should add an option of OpenStreetMaps overlay or something. OSM has been better than Google Maps in mapping paths in the forest and such here.
I speak one of the languages, not the other. Guess which language Google decides is the correct one?
I've put in business addresses or searched the business themselves, and then had it route me to a residential block behind the business on a completely different street, sometimes multiple blocks away from it. This is even after confirming the exact address is being shown and everything.
I only really use Google Maps because it's what I've been using, and I kind of like that its UI is a bit simpler with less animations and things than Apple Maps.
1 - It actually runs smoothly on my iPhone 11. Google Maps stutters and yanks. Yes, Google cannot build a performing maps app for the fastest phone in the world.
2 - It is a map. Not a yellow pages of coffee places, not an Instagram with amateur photos of locations.
Also, the Google Maps app has gotten much more cluttered and complex than in past versions. They are still the gold standard map, but they regress over time.
EDIT: I looked back in my emails and Twitter DMs and it's been 25 months at this point. So "approaching three years" is not accurate.
Edit: Also want to add that it has been two years since my parents moved in. Went in to Waze Map Editor and added the road myself, so their turnaround time was instant. Apple Maps took 9 months, like I said, but Google Maps still has no info on the street despite having all the houses in clear satellite view.
Further, it repeatedly picks a hard route. There's one park I go to that's easier to drive to using one few mile stretch of highway. Google insists it's faster to take these back, poorly lit, windy 2 lane roads. Maybe it is. But it turns a safe easy 3 turn trip into a nerve-wracking 10 turn trip. And it will, as noted above, keep trying to reroute me if I just ignore it and go the easy way.
Fast-forward to 2020 - in the meantime I used Google Maps with no issues at all. A friend of mine suggested that I should try the latest version and maybe I would change my mind. So I used it for a day - same urban area - and it's still lagging behind - mind you: at this time I had a new iPhone and a new carrier. For me this makes the App unusable. I don't want to plan ahead 500 - 1000m (Germany) while using navigation.
I'm really curious, does someone have the same problem?
Yesterday for example it told me to do a U turn in a (sort of) one way street in the middle of the city, and not the first time its been dumb in the last few weeks.
For instance Apple Maps doesn't have U-Bahn support in Munich, which makes it unusable for most people.
Meanwhile Bing maps manages to be the most frustrating experience of any current online maps.
[1] https://appleinsider.com/articles/15/05/06/apple-details-new...
[2] https://appleinsider.com/articles/15/05/06/apple-challenges-...
https://web.archive.org/web/20190429220555/https://support.a...
What seems to be new:
- "About search rankings" section listing its purpose
- Expanded bot identification and verification information
the printer stuff is something else. i put some fault on canon here for pushing me to unofficial sites because their official shit requires you to download a fucking exe to deliver a PDF manual (cmon, wtf, how can your docs delivery pipeline be THAT bad), but the unofficial stuff is some ninth circle of hell
the first was some bog-standard infini-redirect ad fraud crap, but the second was uh... well, i was impressed to the degree that the indian tech chop shop "YOU ARE INFECT DIAL 1-800-LEGIT-MICROSOFT TO CLEAN COMPUTER" site was able to shut down a fully up-to-date copy of firefox, somehow eating all my other tabs, grabbing hold of my mouse pointer in strange ways when the window was in focus AND preventing the windows "close window" button from killing it.
task manager was still able to execute it but damn, that was some ARTISTRY put into making the fraud site as clingy as possible--puts 90s-era malware sites to shame, and this shit kicks around on right on the first page of google search results for a printer manual query, not some shady-ass porn site you'd drunkenly click through to via 3 levels of ever-sketchier ads
people ranted and raved about goods of questionable quality gaming amazon reviews, but eh--honestly, most of those i've bought were fine, if not artisanal high-end shit--they just engaged in a lot of dubious "get free shit for a good review" growth hacking tactics that are A-OKAY THATS JUST HOW YA DO BUSINESS when it's some SF startup pulling the same crap in a different form. it's weird how seemingly nobody gives a fuck that a lot of google searches will boost things that are far worse--offering purestrain fraud on a platter is fine, selling mediocre flash drives and kitchen gadgets with inflated reviews is THE HIGHEST OF CRIMES
Yes, that’s possible, but Applebot is used in a lot of ways today that are wholly unrelated to search. (If search consists of crawling, indexing, ranking, retrieval, frontend, etc., Applebot only does the crawling part.)
Applebot is used for generating attachment previews in iMessage (eg. Send someone a URL — the preview is from Applebot crawling it). From the docs, it sounds like it’s also used for similar previews in Siri.
Is this true? I would have assumed that would be done locally on device. The preview image, for example, is what's designated by the Open Graph[0] og:image tag.
[0]: https://ogp.me
But do you remember when Apple launched their maps? A decade later and it's still not nearly as good as Google maps. But it's "good enough". I imagine that's what they're going for with this new search service
However, Apple started collecting their own data in 2015, and rolling out their own maps in 2018.
>Apple is filling its map with so many [details] that Google now looks empty in comparison and all of these details create the impression that Apple hasn’t just closed the gap with Google—but has, in many ways, exceeded it.
https://www.justinobeirne.com/new-apple-maps
Apple's map data now covers the US and its territories and data for the UK and Ireland are currently in testing.
https://www.justinobeirne.com/new-apple-maps-expansion-9
- It's established applebot is already used for Siri, but there is no general web search
- Much commentary diverts to how poor Apple maps (is?) was.
- Google pay apple a handsome amount of money to be the default search engine on Apple devices
- If Apple were no longer to accept that payment, about 80% of people would still use Google anyway, given a choice [0]
- Googles high earnings per search means they can offer more than any other search engine, like Bing, DDG et al
- People would generally like more competition in the search space and Apple has the means to do it
- Anecdotally, there isn't a drop in replacement for a search default other than Google that would meet the quality requirements for end users. In this case, the only other alternative would be an Apple search engine.
- As per recent articles, there does seem to be more activity from Apple wrt applebot crawling and hiring
[0] https://spreadprivacy.com/search-preference-menu-research/
This is why Apple Maps is so heavily used on iOS. Most people don't know or don't care. In the first year after the launch of Apple Maps it hit over 65% of maps usage on iPhones in the UK, even though back then the UK maps were awful.
There has never been any point in clawing back the large class A assignments. If that were done it wouldn't even give us an extra year of IPv4 address assignments before we exhaust the pool again... not that there is any legal mechanism to take them back anyway.
The solution is IPv6. Despite the slow rollout it is finally happening now that even the regional RIRs have started to run out of addresses.