Readit News logoReadit News
dreamfactory commented on We Need Viable Search Engine Competition   peebs.org/2014/01/04/we-n... · Posted by u/nemesisj
antics · 12 years ago
Hi OP. Your points, as I understand them, are the following. Correct me if I'm wrong.

* MS is not a good example of a strong engineering org. Further, it is not a good example because Windows sucks.

* Google doesn't employ every smart person ever.

* Google can't actually compete. (ed note: in any field, or just against iOS and Fb?)

* My argument is like the argument that no one would supplant Windows.

* Given enough time, everything dies.

Points 2 and 3 are about Google. Let's start there. You're right that Google doesn't employ every smart person ever, but then, who said they did? :) But there are a limited number of search relevance engineers, and finding enough to keep up with Google is a monumental, and maybe insurmountable feat. If you want to compete with Google, you will need to have a serious advantage that supercedes this. That is just a fact.

Point 3 is about competition. I'll grant you that G+ is no Facebook, but Android is the most widely adopted mobile OS on the planet, and by a huge margin -- iOS is basically not even competitive, except for the top 5% of the market. Further, a billion smartphones will be bought this year, the most of which will be Internet enabled Android devices, and most of which will be bought in developing countries by people coming on the Internet for the first time. You tell me who's forward thinking there, because when that market comes online, it will be huge. The fact that you mention this as being non-competitive indicates to me that you might not know what you're talking about. :(

Point 1 is about the MS org. What can I say, OP, I work here, so maybe I'm not the best person to have this discussion with. But FWIW I chose this place over some much sexier jobs because the team I work with is arguably the best of its type in the world. There are bad neighborhoods, but the disparity between a good team in MS and a good team at Google is basically negligible. Also I think Windows is one of the great engineering feats of CS, so ... :| (Note that I still use UNIX at home.)

Point 4 is probably the result of confusion. I don't think no one can compete with Google. Bing has 20% market share! Clearly we can. But I do think it will be hard to compete with Google on search quality. I don't see how you can argue that.

And point 5 is obviously true but not relevant.

EDIT: Actually I now see your point 1 as saying "MS couldn't pull this off because they're not a good engineering org, but someone else could". Maybe someone else can build a better search engine, but I think what MS has pulled off with Bing is a monumental feat.

For starters, we built the entire Bing stack from scratch. No OSS. No common platforms like the JVM. Nothing like that. We started from nothing, and invented the server infrastructure, the data pipeline, the runtime that would support the site, the ML tools, everything. The fact that the site runs at all is a small miracle, but the site does not "just" run: the most remarkable thing by far is that the quality of our tooling is quite incredible, generally an order of magnitude better than the OSS equivalents. For example, the largest deployment of an OSS NoSQL datastore seems to be a few thousand nodes. The small NoSQL cluster backing our MapReduce implementation is stably deployed on a cluster an order of magnitude larger than this. This is something you only really see at companies like Amazon, Google, or MS.

I understand that the consumer market is not something MS is strong at, but I am hoping this gives you a taste of the scale and quality of what's happening behind the scenes. Happy to talk more about this if you drop me a line or skype me at `mrclemmer` :)

dreamfactory · 12 years ago
> For starters, we built the entire Bing stack from scratch. No OSS. No common platforms like the JVM. Nothing like that. We started from nothing, and invented the server infrastructure, the data pipeline, the runtime that would support the site, the ML tools, everything. The fact that the site runs at all is a small miracle, but the site does not "just" run: the most remarkable thing by far is that the quality of our tooling is quite incredible, generally an order of magnitude better than the OSS equivalents. For example, the largest deployment of an OSS NoSQL datastore seems to be a few thousand nodes. The small NoSQL cluster backing our MapReduce implementation is stably deployed on a cluster an order of magnitude larger than this. This is something you only really see at companies like Amazon, Google, or MS.

You lost me here. Not building on OSS seems like setting yourself up or failure from the start, particularly when you are fighting a manpower war, which is where OSS is beating every proprietary entity. OSS already powers Google and Amazon and OSS db's will scale to billions of nodes, not a few thousand.

dreamfactory commented on Rap Genius is Back on Google   news.rapgenius.com/Rap-ge... · Posted by u/tomlemon
nknighthb · 12 years ago
Government bureaucrat. I don't care if Google, Microsoft, or whoever else has humans who directly modify search results. I'm free to choose among the competing search engines based on the value they provide me.

Any external notion of correctness imposed industry-wide will destroy that choice.

dreamfactory · 12 years ago
I wasn't talking about anyone outside google, just critiquing the game being rigged internally, but it's maybe interesting you brought that in.

On the one hand that could be criticised as an illusion of choice in the market (behind the curtain there is often ownership across entire industry sectors, regardless of 'competitors' within a sector). On the other, governments quite correctly get a say in any case (what do we representatively govern otherwise - the alternative is to cede governance to global corporations). Google is particularly sensitive to the latter as it has a huge de facto monopoly and has become part of our infrastructure - it is absolutely a huge target for regulation wherever it has traction.

Talking of competition, is there anything viable in the shape of an open source effort where the algorithms and indexes could be crowd-managed - perhaps indexing via browser plugins?

dreamfactory commented on Rap Genius is Back on Google   news.rapgenius.com/Rap-ge... · Posted by u/tomlemon
nknighthb · 12 years ago
Whether a search engine presents "the most relevant results" is a subjective determination that can only be made with certainty on a per-user basis.

I'm confident in this assertion because people have, at various times, claimed that other search engines provide them more relevant results, yet for me, those search engines almost always failed to even match Google, and never exceeded it.

This was also the case pre-Google -- e.g. lots of people swore by AltaVista, but I almost never used it, because it never seemed to work well for me. I actually remained primarily a WebCrawler user for quite a while, right up until settling into Google in the 1999~2000 timeframe.

I don't want to be limited to search engines that provide what some bureaucrat has decided are the most relevant and/or neutral results. I want to be able to choose the search engine that provides the most relevant results for me, and I have seen no evidence that what's relevant for one person is necessarily relevant for everyone else.

dreamfactory · 12 years ago
> what some bureaucrat has decided

That is precisely my point.

(And if you are looking for personalisation, an algorithm isn't necessarily counter to that but a global smack-ban on behaviours most certainly is.)

dreamfactory commented on Rap Genius is Back on Google   news.rapgenius.com/Rap-ge... · Posted by u/tomlemon
robbrown451 · 12 years ago
That's great in the short term. In the long term, having consistent and harsh punishments (whether manual or algorithmic) for SEO cheaters is very much in the interest of Google's users, because it preserves the integrity of the search results.
dreamfactory · 12 years ago
Disagree. The sanctions that Google applies indicate that it is unable to effectively index a free web and is instead relying on editorial control and favouritism.
dreamfactory commented on Rap Genius is Back on Google   news.rapgenius.com/Rap-ge... · Posted by u/tomlemon
skrebbel · 12 years ago
You're forgetting something.

As a search engine user, I don't care whether the search engine firm is "fair", whether their rules apply equally to everyone or not. It's not a country, it's a search engine.

You can say what you want about Rap Genius, but they're just about the only lyrics site that doesn't suck. As a searcher, I want to get to sites that don't suck. In the case of lyrics, that's Rap Genius.

Google has an interest to show me the best results. How they determine those results, and how much of that process is manual, is up to them. It's nice that they decided that penalties and virtual spankings are a good thing in the long run, and it's lovely that you think that they're too inconsistent and unfair about it, but I'm a user. I don't care. I just want the best results.

dreamfactory · 12 years ago
As a user I care whether Google finds me the most relevant results from the biggest haystack or whether it editorialises those results - it's a big distinction. By that measure I'd say they definitely _shouldn't_ be penalising sites for finding loopholes in their algorithms but should simply close those holes up.
dreamfactory commented on Evernote, the bug-ridden elephant   jasonkincaid.net/2014/01/... · Posted by u/ssclafani
jonursenbach · 12 years ago
I've been meaning to move off of Evernote because their OSX client is just slow as all hell. Are there any alternatives, aside from something like Dropbox? Evernote's OCR implementation was/is really useful.
dreamfactory · 12 years ago
About a year ago I tried it for receipts due to the OCR but at least half the scanned images got lost (some in transit, some after they had been viewed in evernote). Shame as I'd have have gone for at least one paid account if it had worked.
dreamfactory commented on Evernote, the bug-ridden elephant   jasonkincaid.net/2014/01/... · Posted by u/ssclafani
protomyth · 12 years ago
I'm hoping someone does a cloud service for note syncing using a code control back-end. I would really love to be able to look at the changes and rollback a change or delete.
dreamfactory · 12 years ago
google docs?
dreamfactory commented on Does Snapchat's CEO Need to Go?   finance.fortune.cnn.com/2... · Posted by u/goronbjorn
dreamfactory · 12 years ago
Seems the journalist and a lot of commenters here are saying that a service like this needs to be regulated so that there are consequences for negligence or carelessness. There's probably something in that, but in the absence of a regulatory framework for social media systems, snapchat surely has no liability or requirement around data security (beyond existing regulations).
dreamfactory commented on Does Snapchat's CEO Need to Go?   finance.fortune.cnn.com/2... · Posted by u/goronbjorn
foobarqux · 12 years ago
What better measure is there for selling ads than the number of ads you sell?
dreamfactory · 12 years ago
demographics and dwell time
dreamfactory commented on Fired? Speak No Evil   nytimes.com/2014/01/03/op... · Posted by u/uptown
gamerdonkey · 12 years ago
Non-compete agreements may not be the best example for you. They can have large impacts on employees (http://web.mit.edu/newsoffice/2011/non-compete-agreements-10...) and far-reaching effects on entire regions (http://www.techdirt.com/articles/20071204/005038.shtml).
dreamfactory · 12 years ago
That's precisely why they are usually legally worthless, where they act as a restraint on trade for the individual concerned.

u/dreamfactory

KarmaCake day338February 11, 2013View Original