Like many have said, this contains a ton of insightful valuable criticism.
That said, I take immense issue with the characterization of the "heroism bad" sentiment at Google:
> There are documents that explicitly and proudly deride “heroism” and assert that not only should product teams not encourage “heroes”, they should actively dissuade them. If someone chooses to work twice as hard as is expected of them, they usually will be prevented from doing so because they have to work with others and doing so would force the others to work harder too.
As it's been explained to me, this is actually a principle primarily around avoiding inefficient hard work to keep a system healthy, more of a "work smart not hard" thing. Being the only person on the team who can keep prod healthy is not valuable when you could get hit by a bus tomorrow, and this behavior doesn't scale well. That's the gist of the argument and I think it is a good one. His criticism of people not taking enough risk is apt but that's not what this particular piece of culture is about.
Disclaimer: I have been employed by Google and my views are my own.
I can also see the value for a large organization in culturally enforcing interchangeability and replaceability of workers.
I always considered this bit of culture (and felt the message this way from coworkers) to be even more focused on on well-being of the would-be hero. It always comes alongside messages about how to deal with burnout or achieve a work-life balance that works for you. That's always been my preferred perspective on it, anyway.
Still, I'm with you in pushing back against the interpretation you quoted, which is at best a weak (and at worst toxically macho) understanding of real motivation behind this anti-"hero" sentiment. It's mildly absurd to say there's a culture of holding back hard workers in order to lighten the expectations for everyone else. Anyone who works "twice as hard as expected of them", broadly speaking, just gets recognized/rewarded/promoted until expectations and output match.
The reason I'm skeptical of heroism is that it hides inefficiencies from view (which often exacerbates them), by papering over them with hard work. I have written about it in the past: https://two-wrongs.com/hidden-cost-of-heroics.html
Amen to this. Years ago I did a contract for a company that was very pro-heroism. Every single promotion email I saw had some anecdote about how the person had, say, stayed at the office multiple nights in a row to save an in-trouble project.
You know what? Every project needed heroics just to get done on time because the place was profoundly fucked up. And each round of heroics left the code in poorer and poorer shape, demanding more heroics next time. Instead of doing a truly brave thing, like telling executives that a schedule was absurd and the team needed more time to do it right, people did the cowardly thing and said, "three bags full sir" to every absurd request.
And that was the right choice for promotion, because everybody higher up had succeeded by heroics.
Google doesn't disincentivize any of that, they just don't encourage it. Google has mechanisms for rewarding people for doing extraordinary work: peer bonuses, spot bonuses. People who make outsized contributions are compensated, and people who consistently contribute more than others are promoted. But systems that require heroic efforts to continue functioning are fundamentally broken and unsustainable, and need to be made easier to maintain.
I don’t think Big Tech needs to overtax its engineers to deliver innovation. They can afford to hire enough folks to keep the innovation engine humming without requiring heroics from ICs.
I understand there are some exceptions: sometimes during outages/emergencies, you do want folks to fix issues ASAP regardless of how long they are up.
Being a genius isn't being a hero. We need geniuses to create, not heroes to paper over our flaws. If you are busy being a hero, you don't have time to be a genius. And what happens when Superman takes a sick day?
as the old saying goes: "fix a production stopping bug after 30 seconds, you're an arsehole, fix it after 10 hours, a hero"
The problem with heroes is that they tend to be single people with a specialised and bottle-neck like ability. They step up because of a systematic failure.
If those heroe-likes are needed more often than in the odd Sev 1-2 bug fixing, I always consider it as a red flag. As you said, they are needed for a reason, and that reason, if not some odd special cause, is more often than not systematic.
This got a lot more comments than I expected. I highly recommend reading https://sre.google/sre-book/eliminating-toil/ (also referenced in a nested reply below) to understand a bit more about the toil-specific aspect of this I was describing.
I think your clarification is important, but also within the context that he painted, it doesn't seem like this is the type of "heroism" that is "staying up all night fixing the bugs", it's heroism to take risk with your product:
> There are documents that explicitly and proudly deride “heroism” and assert that not only should product teams not encourage “heroes”, they should actively dissuade them. If someone chooses to work twice as hard as is expected of them, they usually will be prevented from doing so because they have to work with others and doing so would force the others to work harder too. If someone says they can finish a project in a month, their manager will tell them to be realistic, pad it to four months, and tell the VP it is six months.
To me, that doesn't sound like avoiding all-nighters by lone wolfs so that everyone can support the system when it fails, it seems to me like it's preventing people from giving their all to create valuable products.
^this. I think it belies some of the things I don't agree with in this take. Overall I thought it was good, but I think his is a very old sentiment that leads to more worker burnout which is bad.
I read hero as someone who works hard(er), not the one person keeping things up. I mean, the one person is a hero, but heroes can just be the one doing most of the work. I have a couple of heroes at my current job and they're the ones working 12+ hours a day sometimes weekend to deliver on a NEW project. So it isn't much about bus factor, but trying to deliver even if it means sacrificing personal life.
The "heroism bad" sentiment OP's article is describing within Google's culture is specifically the "one person keeping things up" variety, and the characterization in the article is inaccurate.
I don't think a hero is someone who works harder. A hero is someone who resolves a critical issue that nobody else could resolve. How many hours a day they work doesn't enter into it.
>> on the other hand, any individual customer you dissatisfy creates zero risk unless it is a mega-customer, so customer satisfaction is just a concept on a dashboard to be trotted out at an all-hands meeting, tut-tutted about, and then forgotten about
From my exposure to GCP from the enterprise side, this rings true. The folks we worked with were polite, but seemed to be asking us about our needs as a one-off / outside the normal process.
With credit to the teams we interacted with, they were helpful and acted on our feedback (things like "As a customer, I need a dashboard that tracks my GCP spending"), but it was apparent this wasn't something they frequently did with customers.
"The maze is in the rat" is a great metaphor.
Execs want certain things ("Realtime visibility into all projects!" or "Stability!") but underappreciate the organizational opportunity cost in requiring them.
Is it more important to have perfectly tracked projects or projects which ship 20% faster?
And if the former, we all understand you're training your people to be like forever, right?
The problem with the article is that every company wants to manage risk; turning risk into money is what a company, any company, does.
Now bigger companies have processes, hierarchies, etc. to manage risk in a distributed way, because well they are bigger. But guess what these companies also have? profits! revenue! multiple products/services that produce that revenue that can be then assigned down at the appropriate level of hierarchy that took/mitigated/managed the original risk.
Google doesn't. It's a single-product company with a single Profit Line and thousands of Loss lines and it's pretending to be something else. That's why they are so busy managing risk all the time; most of them don't have revenues to manage, aim for, use to be rewarded for.
Google is a monopoly, a rentier on the internet. They need to be broken up and repurposed to multiple, actually value-creating companies. If not for the health of the internet, at least for the mental health of its employees it seems.
(I would argue that Google's "free" products, like Android, are the worst thing that has happened to the internet; they cannot be broken up soon enough).
> Google is a monopoly, a rentier on the internet. They need to be broken up and repurposed to multiple, actually value-creating companies. If not for the health of the internet, at least for the mental health of its employees it seems.
> (I would argue that Google's "free" products, like Android, are the worst thing that has happened to the internet; they cannot be broken up soon enough).
Yes! I agree with this entirely. Google is the behemoth of mediocrity, the McDonald’s of tech companies. All their products are good enough to deter competition and to keep feeding search, and not an ounce better. Humans and human attention are just grist for their mill.
> turning risk into money is what a company, any company, does.
I must say that's one of the most unorthodox description of what a company does I've seen.
How is a bread factory or a plumber "turning risk into money"? It sounds like you're describing an insurance company or a bank, not, you know a company like Google who produces free software & services to billions of people in exchange for monetizing their attention. Focusing on risk mitigation in a company is explicitly optimizing for protecting existing income stream at the expense of innovation, customers, and new income streams. It's not what "any company does": it's what uninspired companies who have lost their ways focus on.
It sounds like what a consultant used to "financialize" everything would say to solve Google's problems and slowly turn it into an intangible conglomerate like GE who is just the shadow of its previous self. But maybe this time has arrived for Google?
For example, it could be fascinating to see YouTube as a stand-alone company. Currently its revenues are masked under the "Google" part of Alphabet. It sounds like it's extremely profitable. It could be interesting to see YouTube perform as an independent entity on public markets alongside other media entities, rather than bundled into a "tech" company.
Google is famous for basically just ignoring customers. Or more accurately, ignoring users, since users are not customers.
Customers are those spending money posting ads on search. Frankly even they get crap customer support, but at least they are customers.
I'm no googler but it's clear "user support" (and "customer support") are not priorities at google, and that means other people _are_ priorities. So it's not exactly a shock to discover employees optimize to make other employees happy (measured) versus keeping users happy (not measured.)
I factor that in when I decide if I want to be more than a user, if I want to be a customer. (So far the answer is basically "no".) And, as a user, I understand I'm not a customer and Google doesn't care about me, or my gmail account... So I treat my gmail account with the importance it deserves.
So what though? This makes them happy, and they're all paid, so what does it matter what the outside world think or care?
I have been a enterprise customer of google cloud.
There is a chasm between google cloud and microsoft azure. Azure sucks as a product, but they really, really, really want you as costumer. Damn, I get a weekly one-to-one with the guy which was part of the team that created their kubernetes offering, just because I needed someone with expertise in the matter to help us maximize the benefit or ours. NO EXTRA COST.
With Google I got a guy that knows Kubernetes less than me. I sometimes get embarrassed asking them questions because they don't know how to answer but must come up with something to answer because the account manager is in the call. It feels like their poor account managers take a uphill battle just to get someone to talk to us. I feel sorry for them.
Guess what happens? must deployments are in azure, even if the azure CLI sucks, the azure portal is a mess and azure products are inconsistent.
But Google's GKE ingress takes 20 minutes to update (there is a years old bug opened, nobody cares), vs Azure application gateway taking 30 seconds. Both being very bad for us, but I get to talk to the Microsoft the guy who worked on that integration and discussing with him the pros and cons of using ingress-nginx instead.
Google don't want us as costumers, they just want to brag about having a better cloud (they don't).
(as a PS, please microsoft your employees don't like to answer questions on stack overflow. But great to be able to get stackoverflow answers from the product managers).
My companies have been google customers for years, have spent hundreds of thousands of dollars with them, they’ve never cared about us. Bad service and sometimes impossible to reach a person.
They don't seem to care about their corporate customers either. eg. corporate gmail spam filtering is fundamentally broken and no one seems to be able to do anything about it.
Is it more important to have perfectly tracked projects or projects which
ship 20% faster?
It's far more important to have perfectly tracked projects. A perfectly tracked project which is guaranteed to deliver on a particular day is gold. It makes planning for the rest of the organization much easier.
To use a programming analogy, it's like the difference between latency and jitter for real-time systems. Many real-time systems will happily sacrifice considerable amounts of average latency, in order to minimize jitter. It's far better to have a process that completes in 200 ms every single time than it is to have a process that completes in 2ms most of the time, but, occasionally takes 2000ms.
Similarly, from a managerial perspective, it's far better to have a team that gives you good visibility, allowing you to plan for a completion date (even if that date is farther into the future than you'd like) than it is to have a team that mostly finishes projects quickly, but occasionally bogs down and takes a year to finish a project that was initially estimated at two months.
The problem is that far too many organizations have neither. They don't have visibility, and they finish projects late. For these organizations, your dichotomy is a false one — better tracking is how they will ship faster.
Tracking does not guarantee anything. Tracking only tells you have missed the release after all of your plan is already destroyed.
You seem to want estimation. And you will keep doing exactly that, "wanting", because estimation isn't something that scales to big projects or multiple ones.
There's a reason PMs are reviled to mildly tolerated by engineers: they believe the map is the end product, not the territory.
The correct answer to which is more important is "It depends."
There are many businesses where shipping 20% later leads to ceding first mover advantage and losing the game. Largely what's being discussed in the article.
More detailed and accurate project tracking and projection doesn't deliver quicker shipping. Effectively prioritizing outstanding work does.
> A perfectly tracked project which is guaranteed to deliver on a particular day is gold. It makes planning for the rest of the organization much easier.
This is such a good example of prioritizing internal desires over customer needs.
> better tracking is how they will ship faster
My long experience is that heavy tracking is strongly correlated with slower shipping. Because tracking is treated as a substitute for actual domain competence.
So are most account managers, their paired SAs and escalations into product teams.
Hell I got a sit down with the engineers in Amazon Go (the walk out of the shops and pay automatically thing) because I was curious. As a consulting partner!
One things alluded to but not mentioned explicitly: many Googlers (especially in middle management roles) have work at Google all or almost all of their career. If you left university and went straight to Google, you might have a warped perception of the tech world and what's "normal". I think this contributes to the arrogant culture at Google because if you firmly believe that this company is the best of the best and you've never seen actual industry best practices, you might not even be aware of how weird and bad some of the internal processes and tools really are.
Had the exact same experience in Goldman Sachs. People with 20 year tenures in the bank still advocating for CVS and Ant build system in 2017. They're just stuck in their narrow mindsets and don't know any better. That and the superiority complex made me leave quite quickly.
pshh thats everywhere. and yes I know Im shadow banned but I thought it was important to comment on this thread BECAUSE I am shadow banned due to the fact that I bring facts to the discussion but the current "lets disrupt google" forced zeitgeist moment. This "lets disrupt google" marketing push happening on ycombinator right now reeks of bad ideas from people who dont work in search.
It's even worse that these Googlers (managers in particular) are picked up by other companies as a shining example, e.g.: "They'll know how to run a team / organisation."
And usually it's just them doing things like they did before which apparently isn't necessarily a good thing (sometimes is but often isn't). I cannot fault them, given the hiring expectation. But that's the problem. There's this cult that if Google does something ot must be the right way. It's filling books.
>> Does anyone at Google come into work actually thinking about “organizing the world’s information”?
This is the poignant question IMO.
The texture of the internet has changed drastically since the golden age of "Googling" for things. I feel like the current search vs. SEO paradigm has become a losing battle. The bad actors have adapted to Google's algorithms and now resemble the holes in Search's strategy like some over evolved contagion. The main issues I see as a lay person are:
- Bounce time and other metrics actively incentivizing content to be obfuscated and waste user's time.
- Content theft being viable and disincentivizing high quality content that can be copied easily.
- Walled garden sites that don't want to surrender their content for ad impressions and aren't easily or impossible to index.
I feel like solutions to the above problems would involve Google killing its own golden advertisement goose.
Maybe there are high influence Googlers that do come into work and think about “organizing the world’s information . . .” but a "in way that makes Google the most money" is inevitably tacked on.
I don't think it's the bad actors winning over the algorithm. It's the algorithm forgetting what the internet was actually about.
There was a major search update a few years ago that emphasized "authority". If your website was considered more reputable or an established brand, you ranked higher.
This is completely against the ethos of the internet. The internet was always about new ways to find and organize information. On the internet, CNN has no greater reputation than some random blog. Yet, if you were a mainstream brand, Google would deem you to be more authoritative than some internet-only website.
You can see this most clearly in medical queries. WebM and MayoClinic top the results, even though they're filled with generic fluff. Internet-only websites and forums dedicated to a specific illness rarely get on the front page, even though they have superior information.
Funnily enough, many people on here have the opposite perception of Google: they argue that search results have been hijacked by no-name blogs and that reputable authorities are further down the page.
> This is completely against the ethos of the internet. The internet was always about new ways to find and organize information
I disagree this being applicable to Search. Granted, SEO spam is ranking higher, but a lot of the "internet" today is littered with low quality content. It's important to rank higher quality content.
The second problem is misinformation. It's hard to differentiate information from misinformation. Sometimes misinformation can become information with new data.
The question is, how can an algorithm determine which content is higher quality and not misinformation. "Authority" can be one proxy signal for it.
I feel in this stage of the internet, we don't need an index over everything, but just curated content. This is hard to do for Google or Microsoft, because they'd get sued hard. Look at Section 230 case in the supreme court [1].
The web as a whole never came up with an answer to the question "As someone in possession of valuable information, why would I want to allow that information to be indexed, which would allow someone to trivially copy it?"
"Information wants to be free" was fine for ~1990-2010, until Google et al. took advantage of it to build walled gardens they could profit from.
In retrospect, I feel like if the in-web-standards micromonetization efforts had been adopted, we'd be in a better place today, because there would have been better revenue channels than "Whoever controls ads."
And/or differentiate and regulate search as a privileged common carrier-style business class, prevented from reusing their web scraping for other products.
I think a big reason why ads became the dominant way of revenue generation on the web (as opposed to micromonetization) was systemic issues that just took time to solve.
a) early internet adopters were mainly young people who didn’t have a lot of purchasing power yet
b) a complete lack of trusted means for online payment
More recently things have started to trend away from this because these two issues are now solved. You can see this in social media like Discord or Telegram who have a freemium model, movie/tv streaming like netflix or disney plus, new entrants like kagi or the general proliferation of SaaS offerings which can now sustain a premium userbase where previously you would’ve opted for an ads based model (think of the n different todolist providers etc.)
It certainly feels like there are some regulatory loopholes/oversights being abused.
Maybe just make it illegal (or actually just enforce existing laws) to link to websites that violate copyright laws (from pages with monetization) and force the market to sort things out?
It would be the end of an era for the internet in many different ways, but maybe the wild west needs to just end?
Sure Google is a lot more than search at this point, so it's not the right mission. Though, I would say "organizing the world's information" is at least one of the right missions for the company.
This is a good article and it does a nice job of explaining some of the problems at Google. Of course, much of them are overstated and others overfit to the particular group this person was hired into.
As someone who has first-hand knowledge of this person and the project he led, I can say (and I'm sure everyone would expect this): he does a great job of seeing all the things wrong with the org and the company; but he fails to recognize his own mistakes and shortcomings.
I would love to see one of these kinds of blog posts where the author owns up to their own mistakes.
I worked with the author for a couple of years, pre- and post- acquisition, and I have to admit that he drove me somewhat crazy sometimes too. Leaving that aside, I also had an immense amount of personal respect for him as I could see how much he very genuinely cares about what he is doing. And, that's actively doing his best to do right by his customers. I think the author is 110% spot-on with his critique of Google here.
Indeed, and additionally the proposed fixes aren’t all that inspiring. Mandating every manager do a meaningless, symbolic one hour stint in customer support isn’t going to change anything.
It’s funny that he cites the Waze blog, which similarly gives the impression of a blameless author
> Mandating every manager do a meaningless, symbolic one hour stint in customer support isn’t going to change anything.
I am of the opposite sentiment. I think it would do wonders. It's very easy in a mega-corp to completely lose track of what the product is, who the customers are. And my intuition is that most VPs at mega-corps think about products in the abstract and focus more on politics & empire building/strategy than actually caring about the customers - and it definitely shows for companies like Google.
Just last week I tried to spawn a GPU instance on GCP, and despite being very familiar with the space (I work on cloud infra), after 20 minutes of fighting the overly complex UI and ridiculously broken flows, I still hadn't my instance up. In the next 3 minutes, I signed up for Lambda Labs, entered a CC, clicked a single button and ssh'ed into my VM.
That's what any start-up focuses on: the product. And that's why they always win over bloated mega-corps focused on internal processes and infighting, whose employees self-select for being complacent about mediocre products as long as the monopoly keeps printing fat paychecks.
ex-googler here, who has had similar criticisms but was able to recognize their own flaws and mistakes, chipping in.
Doesn't the culture, as described, exacerbate certain flaws in certain people while hiding flaws of a different kind in others?
At google it's easy to see how a flawed engineer might suffer from impatience, lack of tact, inability to read the room, poor social skills, not being attentive to details or to everyone's needs. It's easy to ascribe flaws to someone who makes mistakes and, instead of writing a lengthy postmortem and installing a process, goes ahead and fixes them while pleading for everyone to move on and keep shipping.
Yet at the same time, the sins of complacency, cowardice, inability to make decisions or take calculated risks, rigidity, and aversion to hard, uncomfortable work remain well-hidden. Sometimes, they are even elevated to praise! At Google, it's better to waste everyone's time or do nothing, than take a risk and ruffle some feathers (and I'm not even talking about taking externally visible risks here--a decision to build a certain piece of infra or shut down a project).
And the cycle repeats. A brilliant but "flawed" engineer realizes they can't work in that culture. They blow up and move on. Everyone says "eh, they make mistakes, can't read the room, and are difficult to work with. Poor culture fit". Well, for them, a bunch of complacent do-nothings is also extremely difficult to work with. But the do-nothings are in charge, and they are not. So they leave. Rinse and repeat.
In every failed relationship there are two sides. Maybe one is entirely at fault, maybe the other, maybe both share blame of maybe there's just a basic incompatibility. This is true for work just like it is for marriage.
In each case, it is easy for one to point out flaws in the other. And in some cases, "the other" is the only one that has flaws. But in most cases it is probably a combination.
Just because a company has culture issues doesn't mean that every employee is blameless or perfect. When someone doesn't work out for a job it is hard to disentangle how much of the blame belongs on each party.
In this case, I'm fairly familiar with a lot of the facts on the ground. As I said above the article makes a good critique of the real problems Google has. But it also blames some of those problems for mistakes/failures the author himself made (in my opinion).
I don't feel articles like this are "wrong". I feel like they fail to go deep enough and if they went one layer deeper then it would be clear why the problems they describe are at best actually features and at worst the "least bad" option.
Specifically, all the issues being discussed are actually just features of Google (and the rest of FAANG really) being a mature company in a mature industry.
Google cannot (it seems) launch and maintain new services. Why? Because it already has a single, ultra successful, profitable service: advertising (primarily via search, gmail, youtube etc). When you have no existing business (say you are a start up) you have an urgent need to get one. But when you have a multi billion dollar core business asset (search), you have no urgency to create another. Instead you "urgently" need to not break your existing asset.
This is the same reason GE and AT&T etc did NOT beat Google to the online space.
This is the natural result of success.
This is the healthy, efficient, economically correct position for a company like google to be in.
People have a weird expectation that Google (etc) will be a start up forever. Or an incubator for start ups. Only thats basically functionally impossible.
So the problem here is people's expectations not being realistic and refusal to accept the reality that google (with 175000 employees) is no longer a plucky 10 person startup...
Yep, the primary distorting effect at a place like Google is the seemingly infinite source of revenue from something which is in large-part fully automated. The ad revenues are a crazy firehose.
Yes it takes a large crew of SREs and SWEs to keep that firehose fully primed, but nothing close to the labour force that Google employs.
When I joined Google at the end of 2011 there was something like 20,000 full time employees. At that point it was already clearly a company that had already transitioned from a disruptor of the industry to a maintainer of the status quo. e.g. they clearly bought the ad-tech company I was an employee at just to make us shut up and go away.
The conservative review and permissions structures in place there that the writer is complaining about are in place to avoid fucking up the firehose. And they're legit important. One of the most intimidating times I had working in my career was on the Ad Exchange release rotation, where I had production access for the deployment of binaries to thousands of machines in many datacentres that produced millions out transactions per second and were responsible for buckets and buckets of revenue. Please Don't Fuck Up was like an airplane banner circling around in my skull the whole time.
Any innovations that happen at Google happen because there are very smart keen people there, and Google is constantly seeking out alternate $$-firehoses. But it's never happened and I doubt it ever will. It's not a culture accustomed to being hungry enough to work aggressively to hunt $$.
This reminds me a lot of AT&T and Xerox in their prime: they both had a firehose of seemingly-infinite revenue (phone calls and photocopies), and used some of that money to hire a bunch of really smart people and have them work on research projects, which the company then completely mismanaged and wasted.
Thank you for phrasing the comment I came here to make better than I could.
This article (much like a lot of views expressed on this site) mostly just reads as someone jumping from a startup to large-business environment for the first time, and not grokking the very legitimate and unshakable reasons such an org has for being more conservative and process-heavy. Yes, that kind of thing has to be monitored and constantly worked on to ensure it doesn't grow out of control, but some level of bureaucracy is a feature, not a bug, for a large, successful organization. At Google's scale, that "some level" is quite high.
The author had worked at Microsoft for 12 years before starting the company that Google acquired. So, not only does he have a fair amount of experience from both sides, he's also worked at MS during its "lost decade" i.e. the Ballmer years.
The similarities between what MS was going through then and what Google is now facing is to me the most interesting aspect of the story. Whether you're printing money via OS or search monopoly, it seems like both the direction and timing of what's going to eventually happen to your business are almost inevitable.
MS today is quite different from what it was 10 years ago when the writer left. By getting acquired by Google in 2020, it's almost as he travelled back in time, into another Big Tech company at that same stage of the enterprise lifecycle.
That's quite true; but then again, if the goal is to change nothing, not make waves, and let the ad billions roll in, do they need 175,000 people to do that?
Well now you will be labelled an "activist investor"! :)
I think this is an excellent question.
I think Google has tried to be an incubator etc. And it's management have had a go at new products. That means hiring
I think they have also been pretty big acquirers. That also pushes the headcount up. Cynically people also say Google acquires companies to stop those companies becoming anything. When you do that, you need to keep those employees on the books too (or they will just go re-found the thing you just paid to kill). Not sure how true that is, I leave it to the reads discretion.
When both these sources of new services have failed, they have NOT fired people. Closing Stadia did not lead to firing 90% of the "Stadia Team". Ditto 101 other initiatives and take overs. It was actually a joke on "Silicon Valley" (the TV show) that no one ever get's let go by tech giants. No matter if they are useless or incompetent.
So the head count is probably pretty bloated (I have no idea how many people are actually needed to run Google).
And that brings us to actual Activist Investors and last months lay-offs.
I think investors are finally calling for efficiency not growth. I think management are finally listening. I say finally not because I think I would have done it sooner, only because I think it makes sense in hind sight.
I'm going to call out one specific point I disagree with, which I'm curious whether others will or won't agree with me on: google3 and the associated set of internal tools actually are better than what exists "in the real world".
I went from startup world (mostly rails and frontend frameworks of the early teens) early in my career to google3 for about half a decade to the whole suite of public cloud stuff starting about a year ago and I really miss google's internal tooling.
It's not at all that any given component is much or at all better than any given public competitor. Indeed I think the article is right that on an individual component basis there are strong competitors to everything google has internally. But taken as a whole, I found it much easier to figure out what tool to use for a given job and get it working well for my task within google than it is with all the public tools. But it's an integration and analysis paralysis / paradox of choice and sales cycle problem, rather than a technical problem.
Do I go with a fully managed service or self-host? How well does the self-hosting work with my infrastructure? What about cost comparisons? Do these three tools work together at all? Do they support my particular sso setup? It's just so much crap before even getting to the point where the technical capabilities are important. And it is either prohibitively time consuming or just actually impossible to do any sort of objective analysis of different options, so everyone just guesses and cargo cults. "This seems to have the most mindshare in the community."
I experienced variants of some of these same problems with google internal tools, but there were easy and good answers to a large enough set of the questions that it always seemed a lot easier to get an answer and move on.
this is something i've really felt moving from aws to a company that isn't a cloud provider. even when most aws services are third best at best in their category, you still get the following benefits
1. everything works reasonably well together and share the same cross cutting primitives (IAM, VPC).
2. you have service teams and support all under the same roof, so you get deep access to them and can get them to work together on solving your problem
3. you do not get a choice in the matter, so you do not have to waste time deliberating the choice. you also get rock bottom prices guaranteed.
this all made things much simpler to work with, much simpler to debug/get support for, and removed a lot of the churn we would've had if we had to re-evaluate services every N years.
Do you think this is more a reflection on the fact that the more mature company has already made these decisions for you vs the other company having to still figure out this stuff? Your point isn't really saying much about que quality of the internal tools vs the external ones. I see your comparison more along the lines of a company that's already decided how they'll tackle the problem and answered all your questions vs one that's starting to.
> Do you think this is more a reflection on the fact that the more mature company has already made these decisions for you
Not really. There are many competing tools internally too, and you have to choose the appropriate one. One thing that's common though is they all integrate with everything else - monitoring, building, launching, codebase, documentation, etc. This makes it much easier to just make it work.
Absolutely yes. I actually meant to make the point that I think all mature companies have this "a small set of things that work together" to some extent or another! So yeah, maybe the point still stands that this is not a point of unique exceptionalism for google. Though I haven't worked at any other large mature companies so I can't compare their tooling to google's.
Wow. When I read this I was so struck by just how accurately it captured Google culture. Things that really resonated;
The way I see it, Google has four core cultural problems. They are all the natural consequences of having a money-printing machine called “Ads” that has kept growing relentlessly every year, hiding all other sins.
(1) no mission, (2) no urgency, (3) delusions of exceptionalism, (4) mismanagement.
For history, my first big contribution inside Google was criticized for being "not technical enough." (and that was fair because the technology was straight forward), however it had pushed through the organizational inertia of "don't change anything" across several groups to get into product. My contribution was to skillfully push things through the org without activating the organizational antibodies that resist change at all cost. But this was not a skill that was appreciated at the time by the people that mattered.
The second thing was this;
Any disagreement with the management chain is career risk, so always say yes to the VP, and the VP says yes to the senior VP, all the way up.
People who were there when I was often didn't know me personally, but did know "of" me by my calling out management on their bogosity. I was not so enamored with "working at Google" to bury my irritation with bad faith communications and actions. To the point where I got direct feedback from my management chain that they would have an easier time in "calibration" if I was not causing so much friction. :-)
I suggested they might get a lot less friction if they stopped doing the kinds of things they were being called out on. I have never been one of those "go along to get along" people in the presence of basically evil[1] people.
When people ask me if they should work for Google this is the litmus test I give them, "Can you suppress all of your feelings of justice or ethics in the face of senior leaders making unethical choices chasing promotion, market monopoly, or additional profits for the company?" If the answer is "no" then I suggest they look to work elsewhere.
[1] I know that reads hypercritically, I don't have a good neutral way of describing people who make decisions that help advantage them at the cost of disadvantaging others. Especially when that decision is unnecessary or frivolous.
Readit News
That said, I take immense issue with the characterization of the "heroism bad" sentiment at Google:
> There are documents that explicitly and proudly deride “heroism” and assert that not only should product teams not encourage “heroes”, they should actively dissuade them. If someone chooses to work twice as hard as is expected of them, they usually will be prevented from doing so because they have to work with others and doing so would force the others to work harder too.
As it's been explained to me, this is actually a principle primarily around avoiding inefficient hard work to keep a system healthy, more of a "work smart not hard" thing. Being the only person on the team who can keep prod healthy is not valuable when you could get hit by a bus tomorrow, and this behavior doesn't scale well. That's the gist of the argument and I think it is a good one. His criticism of people not taking enough risk is apt but that's not what this particular piece of culture is about.
I can also see the value for a large organization in culturally enforcing interchangeability and replaceability of workers.
I always considered this bit of culture (and felt the message this way from coworkers) to be even more focused on on well-being of the would-be hero. It always comes alongside messages about how to deal with burnout or achieve a work-life balance that works for you. That's always been my preferred perspective on it, anyway.
Still, I'm with you in pushing back against the interpretation you quoted, which is at best a weak (and at worst toxically macho) understanding of real motivation behind this anti-"hero" sentiment. It's mildly absurd to say there's a culture of holding back hard workers in order to lighten the expectations for everyone else. Anyone who works "twice as hard as expected of them", broadly speaking, just gets recognized/rewarded/promoted until expectations and output match.
Edit: reorganized order of thoughts
Deleted Comment
You know what? Every project needed heroics just to get done on time because the place was profoundly fucked up. And each round of heroics left the code in poorer and poorer shape, demanding more heroics next time. Instead of doing a truly brave thing, like telling executives that a schedule was absurd and the team needed more time to do it right, people did the cowardly thing and said, "three bags full sir" to every absurd request.
And that was the right choice for promotion, because everybody higher up had succeeded by heroics.
What you’re saying is a virtue in peacetime and a sin in wartime. Precisely the argument the author is trying to make
I understand there are some exceptions: sometimes during outages/emergencies, you do want folks to fix issues ASAP regardless of how long they are up.
interesting! it's precisely the point made at the end of The Caine Mutiny.
https://www.youtube.com/watch?v=Jw6gwGawbXA
as the old saying goes: "fix a production stopping bug after 30 seconds, you're an arsehole, fix it after 10 hours, a hero"
The problem with heroes is that they tend to be single people with a specialised and bottle-neck like ability. They step up because of a systematic failure.
The saying I know is "fix an outage => hero; prevent an outage => what are we even paying you for?
Heroes are a bottleneck in KTLO, but they are a necessity for innovation.
It's not a coincidence Google no longer innovates. They threw the baby out with the bathwater.
> There are documents that explicitly and proudly deride “heroism” and assert that not only should product teams not encourage “heroes”, they should actively dissuade them. If someone chooses to work twice as hard as is expected of them, they usually will be prevented from doing so because they have to work with others and doing so would force the others to work harder too. If someone says they can finish a project in a month, their manager will tell them to be realistic, pad it to four months, and tell the VP it is six months.
To me, that doesn't sound like avoiding all-nighters by lone wolfs so that everyone can support the system when it fails, it seems to me like it's preventing people from giving their all to create valuable products.
https://sre.google/sre-book/eliminating-toil/ is a good public example of this sentiment.
From my exposure to GCP from the enterprise side, this rings true. The folks we worked with were polite, but seemed to be asking us about our needs as a one-off / outside the normal process.
With credit to the teams we interacted with, they were helpful and acted on our feedback (things like "As a customer, I need a dashboard that tracks my GCP spending"), but it was apparent this wasn't something they frequently did with customers.
"The maze is in the rat" is a great metaphor.
Execs want certain things ("Realtime visibility into all projects!" or "Stability!") but underappreciate the organizational opportunity cost in requiring them.
Is it more important to have perfectly tracked projects or projects which ship 20% faster?
And if the former, we all understand you're training your people to be like forever, right?
Now bigger companies have processes, hierarchies, etc. to manage risk in a distributed way, because well they are bigger. But guess what these companies also have? profits! revenue! multiple products/services that produce that revenue that can be then assigned down at the appropriate level of hierarchy that took/mitigated/managed the original risk.
Google doesn't. It's a single-product company with a single Profit Line and thousands of Loss lines and it's pretending to be something else. That's why they are so busy managing risk all the time; most of them don't have revenues to manage, aim for, use to be rewarded for.
Google is a monopoly, a rentier on the internet. They need to be broken up and repurposed to multiple, actually value-creating companies. If not for the health of the internet, at least for the mental health of its employees it seems.
(I would argue that Google's "free" products, like Android, are the worst thing that has happened to the internet; they cannot be broken up soon enough).
Yes! I agree with this entirely. Google is the behemoth of mediocrity, the McDonald’s of tech companies. All their products are good enough to deter competition and to keep feeding search, and not an ounce better. Humans and human attention are just grist for their mill.
> They need to be broken up and repurposed to multiple, actually value-creating companies.
These obviously can't both be true. You need to take your pick of what you critizice.
I must say that's one of the most unorthodox description of what a company does I've seen.
How is a bread factory or a plumber "turning risk into money"? It sounds like you're describing an insurance company or a bank, not, you know a company like Google who produces free software & services to billions of people in exchange for monetizing their attention. Focusing on risk mitigation in a company is explicitly optimizing for protecting existing income stream at the expense of innovation, customers, and new income streams. It's not what "any company does": it's what uninspired companies who have lost their ways focus on.
It sounds like what a consultant used to "financialize" everything would say to solve Google's problems and slowly turn it into an intangible conglomerate like GE who is just the shadow of its previous self. But maybe this time has arrived for Google?
Search is 58% of revenue but I'd still call it at least a 4 product company with the "other" products contributing a decent 11%.
Customers are those spending money posting ads on search. Frankly even they get crap customer support, but at least they are customers.
I'm no googler but it's clear "user support" (and "customer support") are not priorities at google, and that means other people _are_ priorities. So it's not exactly a shock to discover employees optimize to make other employees happy (measured) versus keeping users happy (not measured.)
I factor that in when I decide if I want to be more than a user, if I want to be a customer. (So far the answer is basically "no".) And, as a user, I understand I'm not a customer and Google doesn't care about me, or my gmail account... So I treat my gmail account with the importance it deserves.
So what though? This makes them happy, and they're all paid, so what does it matter what the outside world think or care?
There is a chasm between google cloud and microsoft azure. Azure sucks as a product, but they really, really, really want you as costumer. Damn, I get a weekly one-to-one with the guy which was part of the team that created their kubernetes offering, just because I needed someone with expertise in the matter to help us maximize the benefit or ours. NO EXTRA COST.
With Google I got a guy that knows Kubernetes less than me. I sometimes get embarrassed asking them questions because they don't know how to answer but must come up with something to answer because the account manager is in the call. It feels like their poor account managers take a uphill battle just to get someone to talk to us. I feel sorry for them.
Guess what happens? must deployments are in azure, even if the azure CLI sucks, the azure portal is a mess and azure products are inconsistent.
But Google's GKE ingress takes 20 minutes to update (there is a years old bug opened, nobody cares), vs Azure application gateway taking 30 seconds. Both being very bad for us, but I get to talk to the Microsoft the guy who worked on that integration and discussing with him the pros and cons of using ingress-nginx instead.
Google don't want us as costumers, they just want to brag about having a better cloud (they don't).
(as a PS, please microsoft your employees don't like to answer questions on stack overflow. But great to be able to get stackoverflow answers from the product managers).
To use a programming analogy, it's like the difference between latency and jitter for real-time systems. Many real-time systems will happily sacrifice considerable amounts of average latency, in order to minimize jitter. It's far better to have a process that completes in 200 ms every single time than it is to have a process that completes in 2ms most of the time, but, occasionally takes 2000ms.
Similarly, from a managerial perspective, it's far better to have a team that gives you good visibility, allowing you to plan for a completion date (even if that date is farther into the future than you'd like) than it is to have a team that mostly finishes projects quickly, but occasionally bogs down and takes a year to finish a project that was initially estimated at two months.
The problem is that far too many organizations have neither. They don't have visibility, and they finish projects late. For these organizations, your dichotomy is a false one — better tracking is how they will ship faster.
You seem to want estimation. And you will keep doing exactly that, "wanting", because estimation isn't something that scales to big projects or multiple ones.
There's a reason PMs are reviled to mildly tolerated by engineers: they believe the map is the end product, not the territory.
The correct answer to which is more important is "It depends."
There are many businesses where shipping 20% later leads to ceding first mover advantage and losing the game. Largely what's being discussed in the article.
More detailed and accurate project tracking and projection doesn't deliver quicker shipping. Effectively prioritizing outstanding work does.
They're similar but definitely not the same.
This is such a good example of prioritizing internal desires over customer needs.
> better tracking is how they will ship faster
My long experience is that heavy tracking is strongly correlated with slower shipping. Because tracking is treated as a substitute for actual domain competence.
Their paid technical support (the 100-200$ one) is great and responsive, with an easy way to escalate if a response isn't helpful.
Hell I got a sit down with the engineers in Amazon Go (the walk out of the shops and pay automatically thing) because I was curious. As a consulting partner!
And usually it's just them doing things like they did before which apparently isn't necessarily a good thing (sometimes is but often isn't). I cannot fault them, given the hiring expectation. But that's the problem. There's this cult that if Google does something ot must be the right way. It's filling books.
It's similar for Amazon hire in my experience.
This is the poignant question IMO.
The texture of the internet has changed drastically since the golden age of "Googling" for things. I feel like the current search vs. SEO paradigm has become a losing battle. The bad actors have adapted to Google's algorithms and now resemble the holes in Search's strategy like some over evolved contagion. The main issues I see as a lay person are:
- Bounce time and other metrics actively incentivizing content to be obfuscated and waste user's time.
- Content theft being viable and disincentivizing high quality content that can be copied easily.
- Walled garden sites that don't want to surrender their content for ad impressions and aren't easily or impossible to index.
I feel like solutions to the above problems would involve Google killing its own golden advertisement goose.
Maybe there are high influence Googlers that do come into work and think about “organizing the world’s information . . .” but a "in way that makes Google the most money" is inevitably tacked on.
There was a major search update a few years ago that emphasized "authority". If your website was considered more reputable or an established brand, you ranked higher.
This is completely against the ethos of the internet. The internet was always about new ways to find and organize information. On the internet, CNN has no greater reputation than some random blog. Yet, if you were a mainstream brand, Google would deem you to be more authoritative than some internet-only website.
You can see this most clearly in medical queries. WebM and MayoClinic top the results, even though they're filled with generic fluff. Internet-only websites and forums dedicated to a specific illness rarely get on the front page, even though they have superior information.
I disagree this being applicable to Search. Granted, SEO spam is ranking higher, but a lot of the "internet" today is littered with low quality content. It's important to rank higher quality content.
The second problem is misinformation. It's hard to differentiate information from misinformation. Sometimes misinformation can become information with new data.
The question is, how can an algorithm determine which content is higher quality and not misinformation. "Authority" can be one proxy signal for it.
I feel in this stage of the internet, we don't need an index over everything, but just curated content. This is hard to do for Google or Microsoft, because they'd get sued hard. Look at Section 230 case in the supreme court [1].
[1] https://news.bloomberglaw.com/us-law-week/google-case-at-sup...
"Information wants to be free" was fine for ~1990-2010, until Google et al. took advantage of it to build walled gardens they could profit from.
In retrospect, I feel like if the in-web-standards micromonetization efforts had been adopted, we'd be in a better place today, because there would have been better revenue channels than "Whoever controls ads."
And/or differentiate and regulate search as a privileged common carrier-style business class, prevented from reusing their web scraping for other products.
a) early internet adopters were mainly young people who didn’t have a lot of purchasing power yet
b) a complete lack of trusted means for online payment
More recently things have started to trend away from this because these two issues are now solved. You can see this in social media like Discord or Telegram who have a freemium model, movie/tv streaming like netflix or disney plus, new entrants like kagi or the general proliferation of SaaS offerings which can now sustain a premium userbase where previously you would’ve opted for an ads based model (think of the n different todolist providers etc.)
Maybe just make it illegal (or actually just enforce existing laws) to link to websites that violate copyright laws (from pages with monetization) and force the market to sort things out?
It would be the end of an era for the internet in many different ways, but maybe the wild west needs to just end?
Larry Page said almost 10 years ago (!!!) that Google's mission probably needed to be updated. That's a long time to be lost in the wilderness.
https://www.theguardian.com/technology/2014/nov/03/larry-pag...
As someone who has first-hand knowledge of this person and the project he led, I can say (and I'm sure everyone would expect this): he does a great job of seeing all the things wrong with the org and the company; but he fails to recognize his own mistakes and shortcomings.
I would love to see one of these kinds of blog posts where the author owns up to their own mistakes.
It’s funny that he cites the Waze blog, which similarly gives the impression of a blameless author
I am of the opposite sentiment. I think it would do wonders. It's very easy in a mega-corp to completely lose track of what the product is, who the customers are. And my intuition is that most VPs at mega-corps think about products in the abstract and focus more on politics & empire building/strategy than actually caring about the customers - and it definitely shows for companies like Google.
Just last week I tried to spawn a GPU instance on GCP, and despite being very familiar with the space (I work on cloud infra), after 20 minutes of fighting the overly complex UI and ridiculously broken flows, I still hadn't my instance up. In the next 3 minutes, I signed up for Lambda Labs, entered a CC, clicked a single button and ssh'ed into my VM.
That's what any start-up focuses on: the product. And that's why they always win over bloated mega-corps focused on internal processes and infighting, whose employees self-select for being complacent about mediocre products as long as the monopoly keeps printing fat paychecks.
If you think talking to customers for an hour is pointless, then you are part of the problem...no matter who you work for.
Doesn't the culture, as described, exacerbate certain flaws in certain people while hiding flaws of a different kind in others?
At google it's easy to see how a flawed engineer might suffer from impatience, lack of tact, inability to read the room, poor social skills, not being attentive to details or to everyone's needs. It's easy to ascribe flaws to someone who makes mistakes and, instead of writing a lengthy postmortem and installing a process, goes ahead and fixes them while pleading for everyone to move on and keep shipping.
Yet at the same time, the sins of complacency, cowardice, inability to make decisions or take calculated risks, rigidity, and aversion to hard, uncomfortable work remain well-hidden. Sometimes, they are even elevated to praise! At Google, it's better to waste everyone's time or do nothing, than take a risk and ruffle some feathers (and I'm not even talking about taking externally visible risks here--a decision to build a certain piece of infra or shut down a project).
And the cycle repeats. A brilliant but "flawed" engineer realizes they can't work in that culture. They blow up and move on. Everyone says "eh, they make mistakes, can't read the room, and are difficult to work with. Poor culture fit". Well, for them, a bunch of complacent do-nothings is also extremely difficult to work with. But the do-nothings are in charge, and they are not. So they leave. Rinse and repeat.
In each case, it is easy for one to point out flaws in the other. And in some cases, "the other" is the only one that has flaws. But in most cases it is probably a combination.
Just because a company has culture issues doesn't mean that every employee is blameless or perfect. When someone doesn't work out for a job it is hard to disentangle how much of the blame belongs on each party.
In this case, I'm fairly familiar with a lot of the facts on the ground. As I said above the article makes a good critique of the real problems Google has. But it also blames some of those problems for mistakes/failures the author himself made (in my opinion).
Which in no way impacts how right or wrong his criticisms of Google are, and is unrelated to the topic he was speaking about.
Specifically, all the issues being discussed are actually just features of Google (and the rest of FAANG really) being a mature company in a mature industry.
Google cannot (it seems) launch and maintain new services. Why? Because it already has a single, ultra successful, profitable service: advertising (primarily via search, gmail, youtube etc). When you have no existing business (say you are a start up) you have an urgent need to get one. But when you have a multi billion dollar core business asset (search), you have no urgency to create another. Instead you "urgently" need to not break your existing asset.
This is the same reason GE and AT&T etc did NOT beat Google to the online space.
This is the natural result of success.
This is the healthy, efficient, economically correct position for a company like google to be in.
People have a weird expectation that Google (etc) will be a start up forever. Or an incubator for start ups. Only thats basically functionally impossible.
So the problem here is people's expectations not being realistic and refusal to accept the reality that google (with 175000 employees) is no longer a plucky 10 person startup...
Yes it takes a large crew of SREs and SWEs to keep that firehose fully primed, but nothing close to the labour force that Google employs.
When I joined Google at the end of 2011 there was something like 20,000 full time employees. At that point it was already clearly a company that had already transitioned from a disruptor of the industry to a maintainer of the status quo. e.g. they clearly bought the ad-tech company I was an employee at just to make us shut up and go away.
The conservative review and permissions structures in place there that the writer is complaining about are in place to avoid fucking up the firehose. And they're legit important. One of the most intimidating times I had working in my career was on the Ad Exchange release rotation, where I had production access for the deployment of binaries to thousands of machines in many datacentres that produced millions out transactions per second and were responsible for buckets and buckets of revenue. Please Don't Fuck Up was like an airplane banner circling around in my skull the whole time.
Any innovations that happen at Google happen because there are very smart keen people there, and Google is constantly seeking out alternate $$-firehoses. But it's never happened and I doubt it ever will. It's not a culture accustomed to being hungry enough to work aggressively to hunt $$.
Politics is eating everything there.
This article (much like a lot of views expressed on this site) mostly just reads as someone jumping from a startup to large-business environment for the first time, and not grokking the very legitimate and unshakable reasons such an org has for being more conservative and process-heavy. Yes, that kind of thing has to be monitored and constantly worked on to ensure it doesn't grow out of control, but some level of bureaucracy is a feature, not a bug, for a large, successful organization. At Google's scale, that "some level" is quite high.
The similarities between what MS was going through then and what Google is now facing is to me the most interesting aspect of the story. Whether you're printing money via OS or search monopoly, it seems like both the direction and timing of what's going to eventually happen to your business are almost inevitable.
MS today is quite different from what it was 10 years ago when the writer left. By getting acquired by Google in 2020, it's almost as he travelled back in time, into another Big Tech company at that same stage of the enterprise lifecycle.
I think this is an excellent question.
I think Google has tried to be an incubator etc. And it's management have had a go at new products. That means hiring
I think they have also been pretty big acquirers. That also pushes the headcount up. Cynically people also say Google acquires companies to stop those companies becoming anything. When you do that, you need to keep those employees on the books too (or they will just go re-found the thing you just paid to kill). Not sure how true that is, I leave it to the reads discretion.
When both these sources of new services have failed, they have NOT fired people. Closing Stadia did not lead to firing 90% of the "Stadia Team". Ditto 101 other initiatives and take overs. It was actually a joke on "Silicon Valley" (the TV show) that no one ever get's let go by tech giants. No matter if they are useless or incompetent.
So the head count is probably pretty bloated (I have no idea how many people are actually needed to run Google).
And that brings us to actual Activist Investors and last months lay-offs.
I think investors are finally calling for efficiency not growth. I think management are finally listening. I say finally not because I think I would have done it sooner, only because I think it makes sense in hind sight.
Tell that to Apple.
I went from startup world (mostly rails and frontend frameworks of the early teens) early in my career to google3 for about half a decade to the whole suite of public cloud stuff starting about a year ago and I really miss google's internal tooling.
It's not at all that any given component is much or at all better than any given public competitor. Indeed I think the article is right that on an individual component basis there are strong competitors to everything google has internally. But taken as a whole, I found it much easier to figure out what tool to use for a given job and get it working well for my task within google than it is with all the public tools. But it's an integration and analysis paralysis / paradox of choice and sales cycle problem, rather than a technical problem.
Do I go with a fully managed service or self-host? How well does the self-hosting work with my infrastructure? What about cost comparisons? Do these three tools work together at all? Do they support my particular sso setup? It's just so much crap before even getting to the point where the technical capabilities are important. And it is either prohibitively time consuming or just actually impossible to do any sort of objective analysis of different options, so everyone just guesses and cargo cults. "This seems to have the most mindshare in the community."
I experienced variants of some of these same problems with google internal tools, but there were easy and good answers to a large enough set of the questions that it always seemed a lot easier to get an answer and move on.
1. everything works reasonably well together and share the same cross cutting primitives (IAM, VPC).
2. you have service teams and support all under the same roof, so you get deep access to them and can get them to work together on solving your problem
3. you do not get a choice in the matter, so you do not have to waste time deliberating the choice. you also get rock bottom prices guaranteed.
this all made things much simpler to work with, much simpler to debug/get support for, and removed a lot of the churn we would've had if we had to re-evaluate services every N years.
Not really. There are many competing tools internally too, and you have to choose the appropriate one. One thing that's common though is they all integrate with everything else - monitoring, building, launching, codebase, documentation, etc. This makes it much easier to just make it work.
The way I see it, Google has four core cultural problems. They are all the natural consequences of having a money-printing machine called “Ads” that has kept growing relentlessly every year, hiding all other sins.
(1) no mission, (2) no urgency, (3) delusions of exceptionalism, (4) mismanagement.
For history, my first big contribution inside Google was criticized for being "not technical enough." (and that was fair because the technology was straight forward), however it had pushed through the organizational inertia of "don't change anything" across several groups to get into product. My contribution was to skillfully push things through the org without activating the organizational antibodies that resist change at all cost. But this was not a skill that was appreciated at the time by the people that mattered.
The second thing was this;
Any disagreement with the management chain is career risk, so always say yes to the VP, and the VP says yes to the senior VP, all the way up.
People who were there when I was often didn't know me personally, but did know "of" me by my calling out management on their bogosity. I was not so enamored with "working at Google" to bury my irritation with bad faith communications and actions. To the point where I got direct feedback from my management chain that they would have an easier time in "calibration" if I was not causing so much friction. :-)
I suggested they might get a lot less friction if they stopped doing the kinds of things they were being called out on. I have never been one of those "go along to get along" people in the presence of basically evil[1] people.
When people ask me if they should work for Google this is the litmus test I give them, "Can you suppress all of your feelings of justice or ethics in the face of senior leaders making unethical choices chasing promotion, market monopoly, or additional profits for the company?" If the answer is "no" then I suggest they look to work elsewhere.
[1] I know that reads hypercritically, I don't have a good neutral way of describing people who make decisions that help advantage them at the cost of disadvantaging others. Especially when that decision is unnecessary or frivolous.