> I will now point out that whether X makes it to production can become disconnected from the notion of whether X actually belongs in production.
... is the crux of her point.
So, um, who decides what belongs in production?
I generally like her essays but this has a weird senseless negative tone to it.
Look... I get it. We've all come across bozos. A lot of these bozos aren't the derpy clowns we (hope? wish?) they would be. No - they're usually very confident, very aggressive, and very very goal oriented. Yes it's absolutely infuriating watching them somehow gain the confidence of the C-level folks, obtain a lot of resources, and burn everything to the ground all while claiming plausible deniability. Wishing that they'd get filtered out by the hiring process is <insert idiom of improbability>.
The real culprit is your director of engineering. That person is making terrible decisions but is so good at avoiding accountability that you place your blame on the bozo. Hot take: if your company keeps hiring bozos then your executives are doo-doo. Leave the company. Oh, you can't because you're making 300k base + 2 years away from 800k in RSUs? Congrats. You're part of the problem (it's not your fault).
The same reason why you won't leave your company is the same reason Mr. Bozo gets a crack at "innovating" at that same company.
People like Rachael (who obviously love this work at their very core) are doomed to end up a jaded curmudgeon unless they 1) start their own company 2) join a tiny company. Big tech is hell. Make your money and GTFO.
>People like Rachael (who obviously love this work at their very core) are doomed to end up a jaded curmudgeon unless they 1) start their own company 2) join a tiny company. Big tech is hell. Make your money and GTFO.
Absolutely correct. This reflects perfectly with my professional career. In the early 2000s I worked at a place which started hiring "bozos" in frightening rate. They killed an internal project for CMS (I was the PM) and proceed to hire external agency for content management, because - "Nobody of our clients is willing to do this complicated technical stuff".
After a heated "debate" with the "bozos", management and CEO of the company, I quitted my well payed job.
I created my own company and hired all of my team. For the next 10 years most of my ex company clients came to me by simple word of mouth marketing.
> Yes it's absolutely infuriating watching them somehow gain the confidence of the C-level folks, obtain a lot of resources, and burn everything to the ground all while claiming plausible deniability.
Thanks for summarizing my biggest gripe with my current employer so clearly! It is going on, and all the senior all the engineers see it and there is nothing you can do to prevent it. Only make sure you stay away from whatever these bozo's fancy, as someone will have to clean up or take the blame when everything falls apart.
Anyone know an alternative GTFO? After a few years of complain/ignore/avoid if feel that is my only option.
> Look Mr. C-level, I built this amazing tool. Look at the graphs it produces. It allows _the_ user to customize parameters, track new things, and basically makes unicorns excrete rainbows.
> OMG, I don't know the first thing about how computers work, but I love colorful pictures and unicorns. You are amazing. And, this is the first time you tried this stuff?
> Yes, but these jerks over there, they won't put it on the web! They say it is insecure, risks revealing confidential data to the whole world, and can be used to infiltrate the company network.
> But these are the best colorful pictures I've seen. Off with their heads!
I agree. The article feels weird. To me the topic is complex.
Absolutely you can do it yourself, but that is also a common pitfall. The belief that something can just be built in x amount of seconds by yourself comes with several risks. Building is easy, committing to a lifecycle is hard and managing the technical debt is an important part of the job.
I'm confused why anyone would think this is about optics and not about performance.
If something Does Not Work Reliably - when it has no reasonable reason for not doing so - then it Does Not Belong In Production.
As for mature markets - useless and/or broken buildings happen all the time. Sometimes this is because a starchitect is good at making buildings that look incredible but bad at making buildings that don't have problems. (Calatrava, Frank Lloyd Wright, many more.)
Sometimes it's because the project is extractive and the goal is money at the expense of quality - or even usability. (Many housing estates and not a few McMansions.)
Both of these are different to a tech co hiring the equivalent of a minor starchitect to Make A Thing Happen, but getting something non-performant and/or impossible to maintain. The difference is that unless it's a greenfield startup Thing is likely to be easy to define and its performance should be easy to measure objectively with simple metrics like uptime, throughput, and so on.
They can look, but as you go up the scale of complexity, at a certain point, only the master tradesmen will have a clue whether the work is good or not.
Any high-skill industry has huge amounts of problems with this — rarely outright fraud, but lots of problems with "confidence men" pitching mediocre solutions to problems, and conning their way into running the full lifecycle of contracts based on this. Generally they'll deliver something, but it's usually something pretty mediocre that only kind of does the job. The further a thing is from a company's core competency (and thus, the likelihood that the executives have subject-matter expertise), the more likely this problem is.
People can raise objections all they want, but there's so much "poor faith" lying done in our world, that, as a boss, it's almost impossible to tell which things are libel, and which things are a desperate attempt to warn you something's wrong. Rigorous "professional bodies" like doctor's associations help immensely. These problems are at their worst in fields where things like that don't exist (such as software).
Actually I thought the way it ended was the best point. You can use the interview process itself to understand if you are about to work for one of these companies, which lines up with my experience well. Leet code grind? Expect a shit show on the inside. Which is fine if you are joining for money / friends (who work there) / prestige. But quality shop that knows what they are doing? Probably gonna ask things other than leet code questions.
“Building” is fun. Projects are fun. Governance and sustainment are not fun for builders and as an organization grows, you need more than builders.
It’s kind of a universal theme in any endeavor. I work in a very different place and have similar experiences. As a founding member of a “startup” within a massive bureaucracy, we grew from 4 people to 4,000.
Remember that people are different. My wife is a finance/auditor by trade - digging through balance sheets and controls to audit stuff is awesome work for her. Conversely, some of the details of how the sausage is made when you build a growing service just seems “wrong” based on her training/profession.
I think there is something important there that all people are different and have different goals/experiences/skills etc.
I would say that many companies corporate hierarchy enables/rewards people with political goals to end up in positions of power which slowly corrode the company. It also allows for "bozos" per OP language to propel up the ladder and espouse their ideas without any pragmatic experience or understanding to really solve the problems. People who present well and can play the political game but can't translate to execution.
The worst people for that are the finance people who are sometimes good at numbers, models, they think they understand everything (I.e. MBAs) and somehow accrue power. Once you company has a lot of those at the top level of leadership you are about to enter builder hell.
> Just imagine what companies would look like if their hiring processes filtered out this kind of charlatan instead of looking for whether they could do some dumb coding card trick in a 45 minute interview block. I think they'd look very different, and a whole bunch of people would have to find another industry to prey upon.
I would caution against hoping for a personality test that could weed out the phonies, though. "Behavioral interviews" are strange enough already, in my opinion, even if their goal might be admirable (e.g. identify the assholes).
Anywhere there is money and prestige to be had, you can either keep things small or inevitably suffer the phonies.
Honestly, I don't think it's that hard. I assume we're talking about fairly senior candidates here.
"Tell me about the most complicated projects you contributed a significant part of the design, implementation to, or ideally both."
Start with the problem they aimed to solve.
Probe for how they chose the solution, and what alternatives they considered.
Get into the details of specifically what artifacts they produced, and if part of a team what role they played.
Get them to describe the solution in technical detail.
Probe into trade-offs they had to make or compromises.
Ask what aspects they found novel or innovative.
Once they finish describing the system, ask if it was successful, and how they measured it.
If they were around after launch, how did they operate it, monitor it, what unexpected challenges they encountered that they didn't foresee.
By the way, all along it doesn't matter if the domain or technology they are describing to you is totally different from your own experience. A senior+ engineer should be able to explain their domain to someone equally technical in a different domain with some competency.
Finally, since you now understand the problem space and the solution yourself, ask them "How would you scale this out to 10-100X" (by some metric).
Altogether, this is a dense 40-45 minutes extremely well spent. And it weeds out the phonies and charlatans.
Some yellow flags that individually aren't a problem but when you start seeing multiple of them turn into a red flag:
* Someone who didn't question the requirements and just accepted them from upstream.
* Someone that wrote a lot of code on the project, but didn't make any of the tough decisions.
* Someone that avoids talking about what they did, and focuses too much on "we"
* Someone who did all the initial decisions but did none of the actual implementation work.
* Someone who describes how X solved the problem, but you realize that's the only option they really considered
* Someone who left before the project finished, or immediately after.
* Someone who isn't sure if the project was a success or didn't see it their responsibility to find out.
“Someone that avoids talking about what they did, and focuses too much on "we"”
This is very interesting. I naturally use “we” when discussing past products and achievements. I do this in recognition of the fact that things are very rarely designed and built in true isolation. Even principal engineers socialize their designs and thinking with colleagues, making small tweaks here and there or gaining additional confidence to move forward.
What would you say to someone who's starting their career and feels they would tick most if not all of those yellow flags?
I have to say this is an awesome thing to be reading as within a couple of months I'll be taking on my first major project at my new employer. Valuable insight to consider as I proceed.
* Someone who left before the project finished, or immediately after.
In my experience, people leave just after the big project ships. They've been focused on a task, working hard and are usually left with "now what?". It's interesting psychology.
In science these types of questions are part of the interview (either implicit or explicit), with maybe more added emphasis on future work and vision depending on the rank of the scientist.
We're not talking about some ordinary rank-and-file engineer here. The situation Rachel is talking about where the person is usually well-known among a significant fraction of their industry for their expertise in X. That level of seniority (principal level or beyond) is supposed to imply that they also know when -not- to use X and the responsibility to speak up, but sometimes either their belief in X is too strong ("When all you have is a hammer, everything looks like a nail.") or, in some cases I suspect, the monetary and career-enhancing incentives are high enough to overcome any unease they may have misapplying X. Then you get this situation where a mandate comes down for X is used where X is not really the appropriate thing to do.
One probably spent too much time debugging someone's shit mountain code base, the original author got to enjoy bog bonuses and bragging about X in conferences and getting hired by similar companies to build another X.
It doesn't have to be a malicious charlatan to pounce on value that has already been built, without them having actual consideration for future value that should be more rewarding and more sensibly worth working toward instead.
The corporate-climber-who-can't-build can often do this and nothing else, and they can sometimes even subconsciously exhibit some of the behaviors that end up with the same outcome, and it often can actually be unintentional.
> I will now point out that whether X makes it to production can become disconnected from the notion of whether X actually belongs in production. In one outcome, it's brilliantly executed and things grow. But, this is about the downsides, and in that scenario, it's a stinking pile of garbage and needs to be thrown out, but it gets forced in, anyway. [emphasis added]
She's left the positive case out for the charlatan comment, they wouldn't be charlatans (in this instance at least).
You'll see this sort of things a lot from Rachel's writing, but I tend to write it off as a stylistic choice. She kind of channels the BOFH [0] sometimes, I think :)
Filtering out the assholes is an accepted policy today but it didn’t just happen. Assholes were accepted and that sort of attitude wasn’t discouraged, until people started speaking up about it and others realized they probably had seen that dynamic. I see this article as a step in extending the anti-asshole policy to people who engage in this sort of behavior.
Can I tell you a perfect personality test that will always work and never fail? Absolutely not. Even the anti asshole processes don’t always work. But it seems a good thing that this sort of behavior is at least being described and reported on.
Every company that I’ve been in that had an anti asshole policy has meant that they don’t tolerate assholes at the line engineer level, since that’s reserved for management, especially director and up.
I guess it’s better, but the proof is in what the management team looks like.
I love the call out at the end. I believe much of the post is a slight at Google (and much of big tech), where building complicated crap is one of the fastest ways of promotion. Rachel is completely right, "big tech" has a problem of attracting politician types, who are focused on abusing systems for their own gain. You can hardly blame them though, the company as a whole is doing the same thing.
The incentives are misaligned at pretty much every level.
> "big tech" has a problem of attracting politician types, who are focused on abusing systems for their own gain
Not just attracts, but selects for them as well. The interviews are set up to select people who are morally ok with doing arbitrary and pointless work (months of leetcoding at home) in order to game the system.
It really is an incentive issue. People are given large amounts of money and promotions and raises are always lurking around. Performance review really incite you to promote your own work and play nice with people instead of acting like a team. It’s quite remarkable I find that companies like that still manage to be productive, but it seems to be working.
I will say that avoiding "builder culture" probably saved my job. I joined a company that had migrated from an open source identity management solution to a homegrown one and it had ran perfectly well in production for many years but was getting creaky and didn't offer the features that were needed for a SaaS offering. VPs gave me the option of determining whether we migrated to an identity vendor or built our own. I've contributed code to popular OIDC implementations on GitHub, I've written code in widely used open source JWT libraries and I felt extremely comfortable working in that space so my inclination was towards building.
The problem was to staff a quality engineering team was going to cost 5x the yearly price of the most expensive service and hiring other people in that space was proving to be very difficult and we didn't have time to take a year to build it and then start migration
In the end I chose to go with a vendor and a smaller team to support a migration. We migrated in less than 6 months and a year later we ended up getting contacted by law enforcement about some activity that was happening our system. The auditing system on the vendors side allowed us to quickly diagnose what exactly had happened and let us give data to folks who needed it.
Had that been our homegrown system, it would've been 100% an unmitigated disaster that took days to unravel. Had we decided to build it ourselves, there's no way we could've hit production as quickly as we did.
One unwritten trade off that's in addition, not a knock on your stance just an observation, is that teams that build one quality product can often build another. To some extent when you choose to build you are also choosing to build a culture that knows how to build. So you have to think about not just the next thing you need, but the five things after that. I've worked at companies that bought literally everything, and what I found is the engineers there knew a lot about how to plugin to these various systems, but when they needed to build something complex together they weren't able to do so very well, or at all. Which might be fine, but you should head in that direction with intention.
It really depends on how well the offering matches the need. My personal experience is that outsourcing core bits where vendors don't match exactly the needs ends up causing more overhead that if we had written the service ourselves.
I don’t know. I’ve been in the position of being an “X-builder” (I’m a dyed-in-the-wool compiler engineer), and I’ve been through something like what she describes, and nothing about the decisions involved seemed as obvious (or as malicious/egotistical) as she makes it sound.
Like, maybe I don’t quite want to build an X, but other engineers and managers have their own reasons why we should, and after we discuss/argue for a little while, I decide that, OK, fine, I don’t entirely agree but I’m willing to concede that maybe I’m wrong and give it a shot.
The thing we end up building is OK, but not stellar. We have an outage or two but they’re minor and not an ongoing headache. We have a few wins but it’s not the runaway success we hoped. Maybe we should have gone with Y instead of building our own X? Would it have been better that way? Am I a charlatan? I don’t know.
They're a builder, all right, as long as you want them to build X. Perhaps they have been building X at some other company, and now your company has hired them to come and do that same job over here instead.
Just imagine what companies would look like if their hiring processes filtered out this kind of charlatan
The company specifically filtered them in to build the X product they're renowned for building at other companies. The hiring process is working perfectly if the company decides they want X, hires a famous X builder, and that person then builds X in their role.
I think the point Rachel is alluding to is that the person should have realised X is inappropriate and said so, but if your world is X then it can be hard to see the broader picture. We've all tried to leverage the tools we know best to do jobs they're not perfect for rather than start from zero with something more appropriate, especially when there's a looming deadline or a lot of pressure from higher ups. To call someone a charlatan for doing that seems grossly unfair.
I think she is not specifically talking about the exact same person here. She specifies people putting themselves over the success of the team later in the text, right in front of the charlatan statement.
The x builder can have this affect, on the one end of the spectrum what you describes happens, on the other end of the spectrum charlatanerie happens.
An engineer is expected to speak up when told to construct X out of Y when Y is not good enough for it. He is personally responsible for the lives at stake. A software engineer is not expected to speak up when told to build X out of Y but to push through no matter what.
Wait, there is some disconnect here, the last paragraph does not follow at all.
To see it more easily, here is her post but with some specific, made up example instead of "X":
A chocolate factory wants to build a new dark chocolate line. They hire a very experienced industrial engineer, who just recently finished building car engine line for the Ford factory. However, once the new engineer started to work on chocolate line, they found it extremely boring and devoid of challenges. So they somehow convinced management that the chocolate factory needs to start producing car engines for their delivery trucks. This is obviously a bad idea, and many people point that, but the engineer has secured support of one of VPs, so the project goes ahead. Eventually the line is ready, new engines are produced and installed in all the new trucks. They proceed to break down frequently, the deliveries are late, customers are unhappy, and everyone is having a bad time.
Everyone is blaming engineer now and calling them "charlatan", but the engineer does not understand why: They really worked their ass off building the new engine line, working 80 hours a week to realize their dream. It was not easy -- the factory was only doing chocolate before, so switching to heavy machinery was quite a challenge! And the reason customers are unhappy is because of the management -- this project should have gotten way more resources! Then we could test more and iron out all the bugs instead of having to go to production too early.
Rachel's final paragraph puts the blame squarely on the engineer, with wishes that "hiring processes filtered out this kind of charlatan" and "whole bunch of people would have to find another industry to prey upon", but that engineer is not a charlatan -- they have a legitimate technical talent, and they used it to build real stuff. No hiring process can filter them out -- they will pass a technical interview and take-home test, they have great communication skills, they have a github page with open source code, and they can talk passionately about their past successful projects.
Instead, I'd put most of the blame on the management. In a large company, it is inevitable that someone will work on stuff that is useless or could harm the main product. Maybe because engineers lack perspective, or they overestimate themselves, or they just don't care about company's goals and want to do what they find fun. The company should have some mechanisms to prevent this -- maybe management should listen to feedback from engineers, or there should be technical committee, or at least people should not be rewarded when they ship a broken product. If the company has no such mechanisms, it will have problems as described in the blog.
That's a very reductive and limiting view of what an engineer does. Someone that accepts requirements, and outputs artifacts.
A senior/staff/principal engineer should be aware of the customer needs they are working to solve, and focused on the problem. Their engineering skills are a tool to fixing it, not itself the result.
In your example,
* The industrial engineer should have asked "Do you need me to build car engine machinery?" And if they said no, pursued a different job.
* Once on the job, if management said "let's start building car engines", he should have said "Why, how does this help our customers?"
Actually, in your example:
> However, once the new engineer started to work on chocolate line, they found it extremely boring and devoid of challenges. So they somehow convinced management that the chocolate factory needs to start producing car engines for their delivery trucks.
That's borderline unethical. If you're bored and devoid of challenges. Change your job, don't ruin your company.
You place the blame on managers and say "maybe management should listen to feedback from engineers", but that's exactly what happened in your scenario, and was the root cause of the problem! The manager listened to a poorly motivated, mis-incentivized, yet strangely persuasive engineer.
The problem was that neither the managers nor the engineer actually thought about the customers, or listened to them. Incidentally, maybe they would have found out that the customers actually really like the chocolate but it always get delivered too late warm and melted, and so actually investing in building bespoke engines for their trucks would have been correct in that situation. But it has to start from the customer, not the prior skills of the engineer.
Fix the process, and maybe neither the managers nor the engineer need to be fired.
I've worked with several Java evangelists in my career ... two stand out.
One had written several books on Java and was tapped to head up a new initiative. They "noped" out very quickly, "this isn't a project suitable for java" and recommended a couple of different architectures and platforms and went back to their less prestigious role.
The other was very active in the Java community and shoe horned Java into a very inappropriate product, costing the client $$$ and never reaching the MVP stage.
You could blame management for both of those situations; but one of those companies went on to a modicum of success in an acquisition because the engineer decided NOT to turn the chocolate factory into a car factory.
But what stands out in that happy story is engineering integrity of the highest calibre. The behaviour of an engineer who (correctly) bows out for reasons of technical principle is the peak of engineering as a discipline and goes a bit beyond mere competence.
That sort of exemplary behaviour can save bad management, but doesn't excuse it.
Readit News
... is the crux of her point.
So, um, who decides what belongs in production?
I generally like her essays but this has a weird senseless negative tone to it.
Look... I get it. We've all come across bozos. A lot of these bozos aren't the derpy clowns we (hope? wish?) they would be. No - they're usually very confident, very aggressive, and very very goal oriented. Yes it's absolutely infuriating watching them somehow gain the confidence of the C-level folks, obtain a lot of resources, and burn everything to the ground all while claiming plausible deniability. Wishing that they'd get filtered out by the hiring process is <insert idiom of improbability>.
The real culprit is your director of engineering. That person is making terrible decisions but is so good at avoiding accountability that you place your blame on the bozo. Hot take: if your company keeps hiring bozos then your executives are doo-doo. Leave the company. Oh, you can't because you're making 300k base + 2 years away from 800k in RSUs? Congrats. You're part of the problem (it's not your fault).
The same reason why you won't leave your company is the same reason Mr. Bozo gets a crack at "innovating" at that same company.
People like Rachael (who obviously love this work at their very core) are doomed to end up a jaded curmudgeon unless they 1) start their own company 2) join a tiny company. Big tech is hell. Make your money and GTFO.
Absolutely correct. This reflects perfectly with my professional career. In the early 2000s I worked at a place which started hiring "bozos" in frightening rate. They killed an internal project for CMS (I was the PM) and proceed to hire external agency for content management, because - "Nobody of our clients is willing to do this complicated technical stuff".
After a heated "debate" with the "bozos", management and CEO of the company, I quitted my well payed job. I created my own company and hired all of my team. For the next 10 years most of my ex company clients came to me by simple word of mouth marketing.
Thanks for summarizing my biggest gripe with my current employer so clearly! It is going on, and all the senior all the engineers see it and there is nothing you can do to prevent it. Only make sure you stay away from whatever these bozo's fancy, as someone will have to clean up or take the blame when everything falls apart.
Anyone know an alternative GTFO? After a few years of complain/ignore/avoid if feel that is my only option.
> Look Mr. C-level, I built this amazing tool. Look at the graphs it produces. It allows _the_ user to customize parameters, track new things, and basically makes unicorns excrete rainbows.
> OMG, I don't know the first thing about how computers work, but I love colorful pictures and unicorns. You are amazing. And, this is the first time you tried this stuff?
> Yes, but these jerks over there, they won't put it on the web! They say it is insecure, risks revealing confidential data to the whole world, and can be used to infiltrate the company network.
> But these are the best colorful pictures I've seen. Off with their heads!
I'm working in a semi big tech company and not making anywhere near that (in UK) should I cry myself to sleep ?
For context, making GBP175000/year or over would put you in the 99th percentile for the UK.
Source: https://www.gov.uk/government/statistics/percentile-points-f...
Dead Comment
Absolutely you can do it yourself, but that is also a common pitfall. The belief that something can just be built in x amount of seconds by yourself comes with several risks. Building is easy, committing to a lifecycle is hard and managing the technical debt is an important part of the job.
I'm trying not to build a house of cards.
Product Owner, or just management?
On a side note: in "mature" markets a bricklayer nor an architect, nor constructor can really build anything they want without anyone looking.
If something Does Not Work Reliably - when it has no reasonable reason for not doing so - then it Does Not Belong In Production.
As for mature markets - useless and/or broken buildings happen all the time. Sometimes this is because a starchitect is good at making buildings that look incredible but bad at making buildings that don't have problems. (Calatrava, Frank Lloyd Wright, many more.)
Sometimes it's because the project is extractive and the goal is money at the expense of quality - or even usability. (Many housing estates and not a few McMansions.)
Both of these are different to a tech co hiring the equivalent of a minor starchitect to Make A Thing Happen, but getting something non-performant and/or impossible to maintain. The difference is that unless it's a greenfield startup Thing is likely to be easy to define and its performance should be easy to measure objectively with simple metrics like uptime, throughput, and so on.
Any high-skill industry has huge amounts of problems with this — rarely outright fraud, but lots of problems with "confidence men" pitching mediocre solutions to problems, and conning their way into running the full lifecycle of contracts based on this. Generally they'll deliver something, but it's usually something pretty mediocre that only kind of does the job. The further a thing is from a company's core competency (and thus, the likelihood that the executives have subject-matter expertise), the more likely this problem is.
People can raise objections all they want, but there's so much "poor faith" lying done in our world, that, as a boss, it's almost impossible to tell which things are libel, and which things are a desperate attempt to warn you something's wrong. Rigorous "professional bodies" like doctor's associations help immensely. These problems are at their worst in fields where things like that don't exist (such as software).
All the customers who leave when your product/service goes to shit because that thing actually didn't belong in production?
> ... is the crux of her point.
No, the crux of her point is that the company is now suffering. If 'X' did belong in production, the company would now be blossoming.
It’s kind of a universal theme in any endeavor. I work in a very different place and have similar experiences. As a founding member of a “startup” within a massive bureaucracy, we grew from 4 people to 4,000.
Remember that people are different. My wife is a finance/auditor by trade - digging through balance sheets and controls to audit stuff is awesome work for her. Conversely, some of the details of how the sausage is made when you build a growing service just seems “wrong” based on her training/profession.
I would say that many companies corporate hierarchy enables/rewards people with political goals to end up in positions of power which slowly corrode the company. It also allows for "bozos" per OP language to propel up the ladder and espouse their ideas without any pragmatic experience or understanding to really solve the problems. People who present well and can play the political game but can't translate to execution.
The worst people for that are the finance people who are sometimes good at numbers, models, they think they understand everything (I.e. MBAs) and somehow accrue power. Once you company has a lot of those at the top level of leadership you are about to enter builder hell.
Success is not guaranteed, but that can be a well-earned element that at least greatly reduces the odds of failure when they start their own company.
What problem? Sounds more like a solution.
> Just imagine what companies would look like if their hiring processes filtered out this kind of charlatan instead of looking for whether they could do some dumb coding card trick in a 45 minute interview block. I think they'd look very different, and a whole bunch of people would have to find another industry to prey upon.
I would caution against hoping for a personality test that could weed out the phonies, though. "Behavioral interviews" are strange enough already, in my opinion, even if their goal might be admirable (e.g. identify the assholes).
Anywhere there is money and prestige to be had, you can either keep things small or inevitably suffer the phonies.
"Tell me about the most complicated projects you contributed a significant part of the design, implementation to, or ideally both."
Start with the problem they aimed to solve.
Probe for how they chose the solution, and what alternatives they considered.
Get into the details of specifically what artifacts they produced, and if part of a team what role they played.
Get them to describe the solution in technical detail.
Probe into trade-offs they had to make or compromises.
Ask what aspects they found novel or innovative.
Once they finish describing the system, ask if it was successful, and how they measured it.
If they were around after launch, how did they operate it, monitor it, what unexpected challenges they encountered that they didn't foresee.
By the way, all along it doesn't matter if the domain or technology they are describing to you is totally different from your own experience. A senior+ engineer should be able to explain their domain to someone equally technical in a different domain with some competency.
Finally, since you now understand the problem space and the solution yourself, ask them "How would you scale this out to 10-100X" (by some metric).
Altogether, this is a dense 40-45 minutes extremely well spent. And it weeds out the phonies and charlatans.
Some yellow flags that individually aren't a problem but when you start seeing multiple of them turn into a red flag:
* Someone who didn't question the requirements and just accepted them from upstream.
* Someone that wrote a lot of code on the project, but didn't make any of the tough decisions.
* Someone that avoids talking about what they did, and focuses too much on "we"
* Someone who did all the initial decisions but did none of the actual implementation work.
* Someone who describes how X solved the problem, but you realize that's the only option they really considered
* Someone who left before the project finished, or immediately after.
* Someone who isn't sure if the project was a success or didn't see it their responsibility to find out.
This is very interesting. I naturally use “we” when discussing past products and achievements. I do this in recognition of the fact that things are very rarely designed and built in true isolation. Even principal engineers socialize their designs and thinking with colleagues, making small tweaks here and there or gaining additional confidence to move forward.
“We” is absolutely not a red flag for me.
I have to say this is an awesome thing to be reading as within a couple of months I'll be taking on my first major project at my new employer. Valuable insight to consider as I proceed.
In my experience, people leave just after the big project ships. They've been focused on a task, working hard and are usually left with "now what?". It's interesting psychology.
The corporate-climber-who-can't-build can often do this and nothing else, and they can sometimes even subconsciously exhibit some of the behaviors that end up with the same outcome, and it often can actually be unintentional.
She's left the positive case out for the charlatan comment, they wouldn't be charlatans (in this instance at least).
You'll see this sort of things a lot from Rachel's writing, but I tend to write it off as a stylistic choice. She kind of channels the BOFH [0] sometimes, I think :)
[0] https://en.wikipedia.org/wiki/Bastard_Operator_From_Hell
Can I tell you a perfect personality test that will always work and never fail? Absolutely not. Even the anti asshole processes don’t always work. But it seems a good thing that this sort of behavior is at least being described and reported on.
I guess it’s better, but the proof is in what the management team looks like.
The incentives are misaligned at pretty much every level.
Not just attracts, but selects for them as well. The interviews are set up to select people who are morally ok with doing arbitrary and pointless work (months of leetcoding at home) in order to game the system.
They work because they have a quasi-monopoly on something, not because of efficient use of employees time.
The problem was to staff a quality engineering team was going to cost 5x the yearly price of the most expensive service and hiring other people in that space was proving to be very difficult and we didn't have time to take a year to build it and then start migration
In the end I chose to go with a vendor and a smaller team to support a migration. We migrated in less than 6 months and a year later we ended up getting contacted by law enforcement about some activity that was happening our system. The auditing system on the vendors side allowed us to quickly diagnose what exactly had happened and let us give data to folks who needed it.
Had that been our homegrown system, it would've been 100% an unmitigated disaster that took days to unravel. Had we decided to build it ourselves, there's no way we could've hit production as quickly as we did.
Like, maybe I don’t quite want to build an X, but other engineers and managers have their own reasons why we should, and after we discuss/argue for a little while, I decide that, OK, fine, I don’t entirely agree but I’m willing to concede that maybe I’m wrong and give it a shot.
The thing we end up building is OK, but not stellar. We have an outage or two but they’re minor and not an ongoing headache. We have a few wins but it’s not the runaway success we hoped. Maybe we should have gone with Y instead of building our own X? Would it have been better that way? Am I a charlatan? I don’t know.
They're a builder, all right, as long as you want them to build X. Perhaps they have been building X at some other company, and now your company has hired them to come and do that same job over here instead.
Just imagine what companies would look like if their hiring processes filtered out this kind of charlatan
The company specifically filtered them in to build the X product they're renowned for building at other companies. The hiring process is working perfectly if the company decides they want X, hires a famous X builder, and that person then builds X in their role.
I think the point Rachel is alluding to is that the person should have realised X is inappropriate and said so, but if your world is X then it can be hard to see the broader picture. We've all tried to leverage the tools we know best to do jobs they're not perfect for rather than start from zero with something more appropriate, especially when there's a looming deadline or a lot of pressure from higher ups. To call someone a charlatan for doing that seems grossly unfair.
The x builder can have this affect, on the one end of the spectrum what you describes happens, on the other end of the spectrum charlatanerie happens.
Also, for political reasons, nobody else can or will question it, so there's no safety mechanism.
This sounded interesting but I searched for her blog and I couldn't find it. Does anyone know which post this refers to?
To see it more easily, here is her post but with some specific, made up example instead of "X":
A chocolate factory wants to build a new dark chocolate line. They hire a very experienced industrial engineer, who just recently finished building car engine line for the Ford factory. However, once the new engineer started to work on chocolate line, they found it extremely boring and devoid of challenges. So they somehow convinced management that the chocolate factory needs to start producing car engines for their delivery trucks. This is obviously a bad idea, and many people point that, but the engineer has secured support of one of VPs, so the project goes ahead. Eventually the line is ready, new engines are produced and installed in all the new trucks. They proceed to break down frequently, the deliveries are late, customers are unhappy, and everyone is having a bad time.
Everyone is blaming engineer now and calling them "charlatan", but the engineer does not understand why: They really worked their ass off building the new engine line, working 80 hours a week to realize their dream. It was not easy -- the factory was only doing chocolate before, so switching to heavy machinery was quite a challenge! And the reason customers are unhappy is because of the management -- this project should have gotten way more resources! Then we could test more and iron out all the bugs instead of having to go to production too early.
Rachel's final paragraph puts the blame squarely on the engineer, with wishes that "hiring processes filtered out this kind of charlatan" and "whole bunch of people would have to find another industry to prey upon", but that engineer is not a charlatan -- they have a legitimate technical talent, and they used it to build real stuff. No hiring process can filter them out -- they will pass a technical interview and take-home test, they have great communication skills, they have a github page with open source code, and they can talk passionately about their past successful projects.
Instead, I'd put most of the blame on the management. In a large company, it is inevitable that someone will work on stuff that is useless or could harm the main product. Maybe because engineers lack perspective, or they overestimate themselves, or they just don't care about company's goals and want to do what they find fun. The company should have some mechanisms to prevent this -- maybe management should listen to feedback from engineers, or there should be technical committee, or at least people should not be rewarded when they ship a broken product. If the company has no such mechanisms, it will have problems as described in the blog.
A senior/staff/principal engineer should be aware of the customer needs they are working to solve, and focused on the problem. Their engineering skills are a tool to fixing it, not itself the result.
In your example, * The industrial engineer should have asked "Do you need me to build car engine machinery?" And if they said no, pursued a different job.
* Once on the job, if management said "let's start building car engines", he should have said "Why, how does this help our customers?"
Actually, in your example: > However, once the new engineer started to work on chocolate line, they found it extremely boring and devoid of challenges. So they somehow convinced management that the chocolate factory needs to start producing car engines for their delivery trucks.
That's borderline unethical. If you're bored and devoid of challenges. Change your job, don't ruin your company.
You place the blame on managers and say "maybe management should listen to feedback from engineers", but that's exactly what happened in your scenario, and was the root cause of the problem! The manager listened to a poorly motivated, mis-incentivized, yet strangely persuasive engineer.
The problem was that neither the managers nor the engineer actually thought about the customers, or listened to them. Incidentally, maybe they would have found out that the customers actually really like the chocolate but it always get delivered too late warm and melted, and so actually investing in building bespoke engines for their trucks would have been correct in that situation. But it has to start from the customer, not the prior skills of the engineer.
Fix the process, and maybe neither the managers nor the engineer need to be fired.
One had written several books on Java and was tapped to head up a new initiative. They "noped" out very quickly, "this isn't a project suitable for java" and recommended a couple of different architectures and platforms and went back to their less prestigious role.
The other was very active in the Java community and shoe horned Java into a very inappropriate product, costing the client $$$ and never reaching the MVP stage.
You could blame management for both of those situations; but one of those companies went on to a modicum of success in an acquisition because the engineer decided NOT to turn the chocolate factory into a car factory.
That sort of exemplary behaviour can save bad management, but doesn't excuse it.