Readit News logoReadit News
tikhonj · 2 months ago
> The smallest unit of software ownership and delivery is the engineering team.

I see where this is coming from, but it's also pretty sad. In my experience, it tends to create environments where engineers are second-class citizens compared to managers or product: we're just responsible for "delivery", but can't independently make any real decisions beyond a tiny scope. Our timespan of discretion becomes measured in days or weeks, while, on the best teams I've seen, it's measured in months, quarters or years.

It's counterintuitive, but you absolutely can have real individual owernship for engineers without creating single points of failure or brittle systems. It's a matter of having other sources of slack, encouraging quality work, and giving people a lot of room to fluidly adjust what they're working on when. Folks still have real ownership and can make longer-term decisions on their own, but they also collaborate in ad hoc ways and share tacit knowledge, so that somebody else can jump in, help out or even take over in a pinch. I'm being a bit vague, but all I can say is that I've seen this before, and I'll know it when I see it again.

In practice, the model I saw did end up with more rewriting than a normal team—but we were still way more productive overall, even accounting for the rewrites! Turns out that rewriting systems incrementally, in self-contained chunks, is an amazing way to both evolve the design and to build up instutitional knowledge and capacity. It looks like waste, but it's actually slack that is crucial to making your system as a whole more flexible, adaptable and resilient. (In fact, that's true of a lot of the "waste" top-down management systems try to reduce—I'm incresingly convinced that anybody trying to optimize software team "utilization" is actively sabotaging the work!)

dkarl · 2 months ago
People outside of engineering can't give engineers immediate credit for long-term decisions. They don't have the competency to know what to reward.

I'd go further and say that even within engineering, people outside of a team can't give immediate rewards for work whose long-term value is internal to the team, for the same reason: they don't know if the work you're doing is actually valuable or is just superficially similar to the kind of work that often does have long-term value.

If you're confident enough in your long-term decisions, and how you spend slack time, then you should be fine with being rewarded for delivery, since you expect your long-term work to pay off in future delivery and future rewards.

maxsilver · 2 months ago
> They don't have the competency to know what to reward.

I'd even take this a bit further, and say this is basically an argument that the engineering manager, engineering/dept VP, and CTO all need to be engineers or past-engineers themselves, so they actually do have enough competency to know what to reward.

matthewkayin · 2 months ago
I think that rewrites are an important part of how software is written, and it's an important part of being "agile", in the sense that you can go in and write a prototype that's coded very simply without much regard for long-term architecture, knowing that requirements likely will change and that you likely won't get the design right on the first go anyways.

Coding is like writing, in the sense that it's often faster to write a sloppy first draft followed by a better second draft than it is to agonize over getting the first draft right on the first go. The first draft is generative. Its purpose is not to be good but instead to let you get something built quickly and to let you explore the problem, so that you know what edge cases you'll need to account for in your final architecture.

But this still of working will never get through management because the moment you show them a working product, they'll tell you to ship it and won't give you a chance to rewrite.

I think the best way to solve this is to flatten the hierarchy. Get rid of the notion of managers who rule over engineers and give ownership of the code back to the engineers. Have the engineers and product "owners" make decisions together in a democratic fashion.

xyzzy_plugh · 2 months ago
I used to share this mindset, and I still agree that individual ownership is possible for engineers. Unfortunately many, many engineers simply do not want it. I would reckon most if not all engineers are comfortable with ownership at the team boundary, but many simply do not care beyond that. It's just a day job.

Individual ownership at the individual engineer boundary can breed distrust within a team or org, but often alienates team members who like their job but aren't trying to lead, at least with respect to what ownership entails. In this blended environment someone almost always ends up without agency. Sometimes no one gets agency. Who wants that?

It's surprisingly simple and effective by comparison to give a team agency and ownership, usually in part because of the dynamic of having a manager or lead to begin with.

Simply put, there are too many modes of failure at the individual level for software ownership to settle there comfortably: staffing changes, job security, career growth are the obvious ones, but the dysfunction (even in otherwise healthy orgs, there's always some amount) will find the shortest path to complete the circuit here.

I like to think of it like a gearbox. If you only have one gear, and you break it, or wear out all the teeth, then you don't get to go. If you have many gears, well, the ride may be uncomfortable at times, but you'll keep moving.

almosthere · 2 months ago
I personally think _most_ people should treat their jobs as a _day_ job - unless they have actual ownership in the company (beyond what would be a 50-100k payout at option time)
eikenberry · 2 months ago
In my experience this is mostly big-company vs. small company cultural differences due almost strictly to size and scaling. Small companies work best when individuals have ownership and large companies with team based ownership. They attract culturally like-minded people.
athoscouto · 2 months ago
Agreed that you can have real individual ownership. Not only that, I think that is the only way to be really "productive".

But I think that is beside the point.

Individuals are not fungible, but team members are - or at least can be, depending on how you structure your teams.

And as your org grows, you want predictability on a team level. Skipping a bunch of reasoning steps, this means having somewhat fungible team members, to give you redundancy.

The engineering parallel here is the tradeoff between resilience and efficiency. You can make a system more reliable by adding redundancy. You make a system more efficient by removing redundancy.

bsoles · 2 months ago
I blame Agile and the like for fucking up individual ownership by treating every engineer and every task interchangeable. You can't build expertise by working on a different type of task every sprint...
hnthrow90348765 · 2 months ago
>we're just responsible for "delivery"

Ownership has never gotten me anything but more headache. I'm just here to put the things on the pages.

We've got to charge for additional responsibility. Manager/executive pay scales with how many people they're responsible for, no sense in not giving developers that too.

Deleted Comment

inky-solver · 2 months ago
that does work in smaller places, but there are many reasons and many books about exactly why this does not scale or work at most organizations.

> I'm being a bit vague, but all I can say is that I've seen this before, and I'll know it when I see it again.

being vague does not work when talking about organizational design so telling people that they don't need to build teams around systems is a bit irresponsible

spimmy · 2 months ago
i'm struggling to see how what you are saying you value is any different from what i am saying i value (author here).
tikhonj · 2 months ago
possibly we're saying the same things in different ways, or maybe it's just hard to pin the difference down in a words

what do you mean by "the smallest unit of software ownership and delivery is the engineering team" in practice?

what's the largest scope of work some engineer can do entirely on their own recognizance?

sublinear · 2 months ago
It's extremely simple. Stop hiring/promoting into management who don't know the job. This is extremely common in every other industry, and I and legion of people will keep saying this.

It only seems like not a big deal while "good enough" is a really low bar.

Dead Comment

tikhonj · 2 months ago
I definitely agree that the best teams have cultures that make even normal engineers incredibly effective compared to the status quo. Managers definitely put too much stock in hiring and "engineering quality" compared to culture, trust, systems and processes.

> Any asshole can build an org where the most experienced, brilliant engineers in the world can build product and make progress.

But I have to wonder: if "any asshole" can build orgs like that, why don't they? I know of only a handful of orgs that actually manage to build strong teams of really strong engineers, and they're almost exclusively either trading firms or specialized/research-oriented teams. What's stopping everyone else?

I guess this comes down to the initial point in the article: what do we even mean by "productivity"? (Or, the word I'd prefer: "effectiveness"?) There are lots of orgs where only experienced folks can succeed because they're the best at working around and tolerating disfunction. I've seen performance review processes that—intentionally or unintentionally—selected not for some general technical strength/taste/etc, but for the willingness, ability and stamina to put up with management dysfunction. But, to me, that is very much not what I have in mind when I think "top engineer".

bluefirebrand · 2 months ago
> But I have to wonder: if "any asshole" can build orgs like that, why don't they

Well the supply of extremely talented engineers isn't limitless so you wind up competing for talent with companies much larger than you building much cooler stuff or the same stuff but paying more

MontyCarloHall · 2 months ago
> But I have to wonder: if "any asshole" can build orgs like that, why don't they?

Other posters have said money, but it’s also a function of time—how long can a company afford to wait to find the perfect “unicorn” engineer who excels in every aspect of the product but may take a long time to hire, versus hiring someone whose expertise only covers some of the product’s needs but can be found immediately?

Certain companies benefit disproportionately from unicorns, especially those in interdisciplinary fields. For instance, in quantitative finance, a single individual who’s an excellent 1. systems programmer, 2. mathematician, and 3. financial market expert will contribute a lot more than a team of three specialists in each of those domains. But it’s a lot faster to hire that team of three versus finding the rare individual who can supplant a whole team. This is also true in less exotic fields—it’s rare to find someone who’s truly a “full stack” web developer, with a deep understanding of networking protocols and Linux system administration and (cloud based) distributed systems and databases and caching services and frontend development & cetera. But the company that can afford the money and time to find these people will make a much better product than the company that cannot.

(Whether the product actually needs the quality commensurate with all that engineering firepower is an entirely other question.)

spimmy · 2 months ago
because there is, by definition, an extremely limited supply of the "best engineers in the world". if you can afford to pay top-10% salaries, and attract and retain top-10% engineers, more power to you.

maybe not literally "any asshole" can do it -- but it certainly asks more of your leaders to craft sociotechnical systems oriented towards learning and enablement. and i don't think that's a bad thing.

latchup · 2 months ago
There is more to choosing a job than money. No engineer likes being managed by incompetent assholes, and the best engineers also get to choose the best managers. And the bar for those is, sadly, quite low.
ChrisMarshallNY · 2 months ago
> if "any asshole" can build orgs like that, why don't they?

The biggest reason, is that most first-line and middle managers suck. They can't create and maintain a productive environment.

The cure is/was to pay heaps of money. People will put up with almost any kind of hell, for good pay.

sorokod · 2 months ago
Beyond budget constraints, those brilliant engineers may not be good team players.

Their brilliance may be in the way of finding the necessary compromises and doing the required but not intellectually challenging work.

tikhonj · 2 months ago
I dunno, almost all the brilliant engineers I've known have also been great team players and mentors. Not all, but I'd say it's pretty correlated.

The reason brilliant assholes stand out is one of those statistical paradoxes whose name I've forgotten: somebody who's an asshole has to be brilliant to succeed, while team players can get pretty far with a wide range of skill levels.

burningChrome · 2 months ago
>> I know of only a handful of orgs that actually manage to build strong teams of really strong engineers

The best teams I've been on work like a team in sports.

You have the guys who are really good, really sharp developers. They know all the ins and outs of the framework, but are insanely specialized with one thing, but do the majority of the heavy lifting. Then you have the mid-range guys who are more of the JOT guys. They know UI/UX, accessibility, front-end dev and some back-end stuff. Then you have all the entry level rubes. The guys you can give them something to do and they'll figure it out. They're usually learning as they go, but can handle tasks with some direction and hand holding. As the project runs on, they get more comfortable with the processes and tasks so they need less direction and hand holding.

Building teams is all about finding a good mix of people who compliment each other. Too many of the really sharp devs and they'll be arguing over everything. Get too many mid-range or entry level guys and it will slow down the whole project. You also have to have devs who are comfortable with their skill level and know what they're expected to be doing. Too many times I've been on teams where the mid-range guys start bumping heads with the senior devs. Lunch becomes a rage fest over what we should be doing better. They think they should a lead dev, not the guy they don't like.

The last thing is your senior/lead devs have to have the right attitude too. I've been around some insanely sharp lead devs, but they're complete assholes. They know everything, you know nothing. You use shit frameworks, they're always on the cutting edge and your an idiot because you like Angular not something bougie like Svelt.

The key in all of this is finding the chemistry that works. When you get that chemistry, you can capture lightening in a bottle and really build some amazing stuff. When it works, its the coolest thing. People are dialed in, they're enthusiastic about what they're doing. They're willing to work longer hours to make sure the product we're building is incredible. The team is happy, delivering and working on faster sprints and things just feel effortless.

AlotOfReading · 2 months ago
There are large parts of this I really like, but it's hard to overstate just how much I disagree with the idea that "The only meaningful measure of productivity is impact to the business". The natural result of this view is a focus on quantifiable changes and short term thinking. The greatest value of experienced engineers is in avoiding landmines that will destroy a project or even a company. It's difficult to quantify things that never happen, or communicate the value in avoiding them in terms that don't sound wildly hyperbolic.
Retric · 2 months ago
“Impact on the business” isn’t inherently a short term viewpoint.

Maximum impact is a long term proposition.

AlotOfReading · 2 months ago
It's not inherently a short term viewpoint, that's just the typical and natural result of adopting this view. Measuring long term impacts is hard. Attributing them is even harder.

Here's a real scenario. I worked for a company that evaluated by impact. They had a cellular modem with known issues in the field. A replacement was designed to fix those issues, but couldn't be backwards compatible. The cost to immediately upgrade the field units was high, so deployment was delayed to a cheaper, later time. One way of looking at this is that the decision saved millions of dollars and that argument was made. After the evaluation and before the deployment, a set of older field units failed in such a way that it made headlines across the country, which would have been prevented by the new units.

So, was the impact of those decisions negative the whole time in an unknowable way? Did their impact become negative as soon as the incident occurred? If the incident was still possible but hadn't occurred, would the impact be different?

People aren't good at evaluating things that haven't happened yet, so they'll tend to focus on the things they can see immediately in front of them. That incentivizes engineers to build things that have immediate short term impacts and discount long tail risks which can't be reliably evaluated.

tikhonj · 2 months ago
100%. I've taken to using intentionally fuzzy phrases to describe productivity/impact/effectiveness/whatever. Like, "in a holistic accounting, person X did a great job". It's not necessarily—in fact, necessarily not—easily quantifiable or legible, it requires some human insight and judgement, but so does any complex, creative endeavor. Trying to reduce management to metrics is inherently short-sighted.
nonameiguess · 2 months ago
I'm not sure I totally agree with measuring value by avoiding landmines or anything at all related to project management, but it definitely bugs me to see everything reduced to business impact. There are plenty of things in life that matter to individuals, to humanity, to the entire world at large, that have nothing to do with selling products for money. When I think of the engineers I revere the most, I don't think of titans of post-2001 Silicon Valley so much as John von Neumann, Robert Oppenheimer, Nikola Tesla, Leonardo DaVinci, whoever the hell built the Roman aqueducts and Egyptian pyramids, Babylonians and Mesoamericans who figured out how to predict eclipses.

Did these people have a business impact? I guess Tesla made Westinghouse a lot of money at one point, but that seems far from the most distinguishing thing that made him great at what he did. If anything, he was mediocre at business.

Even if we want to look at current titans of the computing industry, I admire the work done by orgs like Nvidia or humans like Geoff Hinton, but they also just got lucky that what they were doing for completely different reasons ended up benefiting so tremendously from the galaxy-scale data harvesting that has been going on due to the Internet becoming primarily ad-monetized, which they didn't know was going to happen. How many equally great engineers toiled in obscurity on dead ends but did equally great work? Doug Lenat was just as great an AI engineer, if not better, than Geoff Hinton. History just went one way and not the other, due to factors completely outside of the control of either of them.

jessitron · 2 months ago
ooh, good point.

You can build systems for efficiency, or build them to minimize disaster. It's really hard to see the negative impact on the business that was prevented.

esafak · 2 months ago
The company should be constantly striving to better quantify the unknown.
spimmy · 2 months ago
idk, avoiding landmines seems like pretty significant business impact
tikhonj · 2 months ago
it's real impact, but it requires folks to be able to confidently talk about counterfactuals—"we would have wasted X days if not for..."—which I've found to be really hard at most places

the exception was places where leadership already thought in the same terms about software quality/etc, which meant I didn't have to do much convincing :P

how would you build teams or structures to support that sort of holistic thinking about software?

rebeccaskinner · 2 months ago
I’ve been lucky enough to work on a number of teams in my career that are full of brilliant high performing people who are also kind, empathetic, and focused working together to deliver something to customers. These days I’m lucky enough to lead a team like that.

Counter to the article, in my experience these are the hardest teams to manage well because organizations typically aren’t set up to deal with them. Larger companies tend to lean into standardization and making things accessible to average engineers in ways that make high performing teams less effective and often demotivated.

I think this is an overly cynical approach that assumes that you can’t invest and grow people into exceptional engineers. In my experience you can if you are willing to invest in it, and the long term benefit of having more high performing engineers who aren’t being restricted from doing good work outweighs the cost of training and growing people.

narag · 2 months ago
Some good points, bad logic. The fact that you can't effectively measure something doesn't mean that something doesn't exist.

Individual productivity exists.

Maybe it's easier to measure groups' productivity? Probably.

"Business impact"? I don't think so, that later concept seems much more arbitrary. But feel free to look for the keys under the lamplight. If you choose that metrics, you're not going to retain many extra productive people anyway.

The old problem: judging the work of an expert is very difficult if you lack comparable expertise. I can give you advice, but I can't make you smart to accept it. How could you tell if I'm a genius or an overconfident asshole?

tikhonj · 2 months ago
the real problem is not that we can't measure productivity, but that we can't even fully define an abstract productivity metric—what does "productivity" even mean?

we can come up with some notion that talks about the net effect of an individual (the "wins about replacement" of programmers), but that tells us absolutely nothing about how any given individual achieves that; hell, the net effect of any given individual is presumably a function of the entire context and the rest of the org, not just of the individual!

alternatively we can try to define more direct notions of "productivity" even if we can't measure them, but those notions end up varied, multidimensional and, again, painfully context-specific—it's absolutely a useful thing to think about, but it does not let us pin down what a "top 1%" engineer "really" is, or even if that's a meaningful notion

sailfast · 2 months ago
A real genius can often explain their work without being an asshole, I think. So it’s fairly straightforward to tell the difference.

Deleted Comment

acedTrex · 2 months ago
> The best engineering orgs are the ones where normal engineers can do great work

I quite like this. It is absolutely true as well, not all teams can be 100% rockstars. How well do you enable the average dev to be good, maybe not GREAT but good and reliable.

lizardking · 2 months ago
Most great engineers were once good engineers. A healthy engineering organization helps its engineers on this path.
spimmy · 2 months ago
yes! love this. and many great engineers will go back to being "good engineers" as they pick up new skills. rinse and repeat++
ericvolp12 · 2 months ago
> Make it easy to do the right thing and hard to do the wrong thing.

This is basically the mantra of every platform team I've worked on. Your goal is to make the easy and obvious solution to engineers' problems the "right" one for the sustainability of software and reliability of services.

Make it easy to ship things that are reliable and manage distributed state well and can scale well and engineers will build better muscle memory for building software in that shape and your whole org will benefit.

This will never stop being true.

simonw · 2 months ago
Coincidentally, Charity Majors wrote one of my favorite essays about that too: https://charity.wtf/2018/12/02/software-sprawl-the-golden-pa...

> Assemble a small council of trusted senior engineers.

> Task them with creating a recommended list of default components for developers to use when building out new services. This will be your Golden Path, the path of convergence (and the path of least resistance).

> Tell all your engineers that going forward, the Golden Path will be fully supported by the org. Upgrades, patches, security fixes; backups, monitoring, build pipeline; deploy tooling, artifact versioning, development environment, even tier 1 on call support. Pave the path with gold. Nobody HAS to use these components … but if they don’t, they’re on their own. They will have to support it themselves.

Tade0 · 2 months ago
> Are you using golang, python, COBOL, lisp, perl, React, or brainfuck?

For years now I had this feeling that people confuse React with the entire frontend ecosystem, but dismissed it thinking that surely they're aware that there's a whole world of non-React frontend out there?