I like the article and I have been promoting a blameless culture for most of my career. To respond rapidly to failings, we need transparency about what has happened. The same is required to ensure it doesn't happen again.
I have seen a lot of incidents and I cannot think of one where a single person was to blame. Sure, one person ran a command or made a bad commit. However, someone else granted them access, someone trained them. A manager either reviewed the process performed or never considered a process was required. A lot of company cultures do not promote proper risk management in technical processes.
I get a lot of satisfaction from technical post-mortems. There is always something that could have been done, a process that could have been in place or a software/infrastructure change to mitigate the problem.
A couple of companies have worked for have had individuals that did not subscribe to this culture. They would want a name. I never knew why exactly. Maybe it was to block a pay bump or defer a promotion. As the tech lead or dev manager any team failings are my responsibility and in companies where bullets are fired I take them. I find this protects the team and helps a blameless culture thrive amongst engineers.
Generally, I find this culture leads to fewer incidents and problems as the openness when things are going wrong allows for faster response times and software/process changes in review.
I have worked years in a culture that attributes all mistakes to defects in the process. And the only possible individual mistake is not to follow the process. Process culture is just as bad as blame culture in some ways. It leads to ever more complex processes. The teams I've seen do best, the manager of the project will allow process to be informal and treated as best practices only. Then people are held accountable for bad results by putting them on tasks that can cause less damage. Good results mean more critical tasks. No single mistake kills a performance review or promotion, or makes one. We all make mistakes and get lucky. But over time reapeated mistakes or successes do have an impact.
Process culture can be nice in the ideal scenario where the processes can have huge parts automated or programmatically guard railed. However, when that isn’t the case, I agree the paperwork barriers can get dreadfully tedious and introduce their own opportunities for mistakes
I've not heard of the idea of blameless culture, so I enjoyed reading this from that angle.
One of my worst and most consequential work experiences was with the opposite type of culture.
There's lots to be said about it, but in reflecting on things I've often felt like one of the worst parts of that type of culture (
"blameful"?) was a profound lack of trust, a feeling that the blame belied some lack of real interest in improving the situation, as opposed to being punitive against certain individuals for unrelated reasons. That is, the individuals who were targets of the most blame as far as I could tell were generally being targeted for cliquey reasons that had nothing to do with performance or anything of that sort. Conversely, people in the "in group" were given a free pass for all sorts of serious problems. Getting a free pass wasn't really the problem, it was a sense that blame wasn't really about the ostensible blameworthy act, it was that it was being meted out as a kind of superficial leverage for some other thing.
Also, because of these types of issues, serious, legitimate administrative, communication, and other systemic problems never got addressed, because the blame was being used as a kind of social ostracizing mechanism, and the actual underlying problems weren't fixed in their entirety. So in many ways a lot of the problems got worse, not better.
In contrasts, in other environments (even the same place at a different point in time) where there's an emphasis on problem solving and figuring out what could be done differently by everyone, and assuming the best intentions and basic competence, trust underlies everything. You're motivated more because you believe that actions have fair consequences, and that everyone has each other's best interests in mind.
> He very likely has no idea how much impact he had on my career, but I’m very thankful for his patience with me that day.
There have been many folks like this, in my career. I am humbled and grateful.
I worked for a Japanese corporation, where they didn't "blame," per se, but did figure out if someone was Responsible. If so, that someone was expected to cop to it, and the team would then close ranks, and fix the issue. Lesson learned.
I do sometimes wonder if the (American) culture of blamelessness has led to a general lack of a feeling of responsibility among software engineers, leading to general carelessness. The idea of responsibility that becomes opaque as you move up the hierarchy sounds good, but I also expect that also has a lot to do with Japanese culture.
The US has a very strange cultural approach to apologizing and accepting Responsibility.
I know that part of it is lawyers and police. If you say "I'm sorry," in many contexts, that puts you on the hook, legally. Most lawyers will advise you to keep your mouth shut, and never, ever admit a thing.
I was taught to promptly admit when I'm wrong; regardless of the consequences. Also, to not cop to stuff that I'm not responsible for.
I have often had arguments with (American) people, where they are 100%, without-a-shred-of-doubt, wrong. Often, embarrassingly so, yet, they Will. Not. Admit. It.
I could bring the spirits of Carl Sagan and Stephen Hawking in, to show them they are wrong, and they will still never admit it.
It's actually kind of sad.
My ability to apologize worked extremely well, in the Japanese corporation. They respected it, and I was given a great deal of deference.
On the other hand, the Americans tended to be pretty much dicks about it. If I apologized, they usually tried to use it to gain some leverage, and bully me.
> I do sometimes wonder if the (American) culture of blamelessness has led to a general lack of a feeling of responsibility among software engineers, leading to general carelessness.
I don't think there's some general "(American) culture of blamelessness," and I think your comment might misunderstand the point of it.
My understanding about "blameless culture" is it's all about avoiding counterproductive things like scapegoating, blame-shifting, defensiveness, etc. Removing those things encourages the full cooperation of the team in resolving the issue, by removing the anxiety of punishment for being the guy who's being blamed. Everyone's supposed to take responsibility.
I don't know if there is an American culture of blamelessnes. Lots of variation, such a blanket statement needs more than confirmation bias.
With that said, I think your comment describes kiss-up and corporate fiefdom culture. There are plenty of examples of that. I wonder if non-American cultures are different in that respect.
One very key distinction is freedom to admit error vs "the baby is not ugly." I don't think blameless cultures are the norm, where people volunteer to write post portems and voluntarily submit after action plans..
This "blameful, not blameless" wording is a bit confusing to me. Are we blaming or not? "Blameful" isn't even a word, according to my web browser. The browser is showing red squiggles and so is my brain.
I think it'd be a little more potent to just say hey, there's a difference between saying someone is partially responsible for an issue, and pestering them about it (e.g. telling them they did a bad thing when they likely already know - I suspect that is what people want to avoid when they go "blameless"). Blame is blame, and disrespect is disrespect. I like to see them as orthogonal.
The article's fire suppression story doesn't even seem like a blame problem to me. It was a jumping to conclusions problem. But again, maybe I only think this because the definition of "blame" is being played with.
pestering them? What people are actually worried about is firing them, which still happens even at companies that pretend they know better. I heard from friends at Salesforce that this guy was fired, for instance. https://www.theregister.com/2021/05/19/salesforce_root_cause...
Most of all the author's gratitude toward the patient fire chief
whose leadership example he praises.
We could do with more of this in cybersecurity.
We're all surrounded by insecure software and defective devices
churned out by companies chasing a quick profit. We're mired in
complexity, non-determinism and broken systems. The experts who
designed them throw up their hands in confusion. And we're hamstrung
with laws and policies, which despite their laudable intentions only
beget twisted systems filled with perverse incentives and loopholes.
How do we solve this? Blame employees! And then humiliate them! Make
our cyber team into a punitive, even vindictive internal police force?
Karen from HR is sobbing into her cardigan sleeve because ICT caught
her out in a "Phishing drill". Now she'll have to go on a humiliating
"this is how a computer works" course - despite the fact she was
programming before her IT manager was born. No doubt she'll get her
own back in time.
As ICT and cyber teams we could learn a lot from more traditional
organisational culture.
I'd love to chat w/you about this re: cyber. I am at a software vendor in the incident response space. It's always good to know other folks in the industry and collaborate on ways to help improve what I'll rephrase as "the human element" in the broader cyber space.
I feel like people misunderstand what blameless culture is about?
Blameless does not mean it is ok to break stuff. Or that you are not supposed to feel shame. Or that doing shit job comes with no consequences.
Blameless culture was supposed to mean that it is more important to figure out why something broke and find solutions to fix it than pursue penalties for people who did it. That pursuing penalties and assigning blame tends to turn into this nasty political circus that frequently makes it impossible to figure out what really happened and if somebody is found to be responsible, it frequently is the person who is least adept at politics.
So we say: "The only way we know to be able to maximise improvements based on factual analysis is to stop clouding the process with politics of blame finding."
So this is more like getting away with murder on a technicality.
Not blaming is not the goal. It is the means to uncovering the truth and figuring the way to prevent it in the future.
I find most “philosophies” regarding work to be BS. Honestly most teams just get too big. People don’t know each other and this allows things to go unseen for too long. When people know one another and work together in real time, and have healthy relationships, they can then evaluate each other’s work, rely on each other, and generally get the job done with a minimum of fuss.
I fully understand that some people are not looking for that kind of thing at all. Many would rather be nameless cogs, and some would simply rather not do the work (technical or soft skills). That’s fine too. For that latter group, whatever the management buzz words de jour are are likely to be employed where you work sooner rather than later.
Ye when org. and team size grows "accountability" is not practical anymore since the decision makers have no clue who to blame in all but obvious cases. And the workers start to do what they get paid for, not what the nominal culture dictates.
My take is that it is probably better to accept bad workers in big orgs rather than trying to reach some kind of competence utopia, to get the incentives right.
Would you mind elaborating? In particular, do you refer to the technical people ("no need to take care, even if I mess up no one will blame me") or the managers ("those cowards just want to avoid being accontable for their mistakes by blaming the process")?
My last company had a culture of blameless post mortems, and also a strong culture of accountability, so definitely no one would fit the first definition. And the company was too small to have non technical managers.
My current company (big corporation) officially supports blameless reaction to incidents, but internally has a strong culture of cover-up and blame ping-pong, but I still cannot think of anyone who would believe "blameless means no accountability". It just comes from deeply ingrained habits (lots of people do their whole career inside the company, lots of them even getting their masters at the company's own "university". Behavior is hard to change when everyone comes out of the same mould). I also think it comes from inadequate processes: where in the previous company the postmortem was mostly a meeting done by and for the technical team, my new company has a strong culture of "escalation meetings", where the highest managers quickly get drafted in, which obviously has a strong influence on the capacity to conduct an open and blameless process.
> too many people think “blameless” means no accountability for doing bad work
No, that's still somewhat blameful.
If you had stuck with "blameless does not mean no accountability", I think you're right.
But the emphasis of the article specifically does not assume people do bad work on purpose:
> assume the individuals involved had the best of intentions, and either they did not have the correct information to make a better decision, or the tools allowed them to make a mistake.
I'd like to hear other takes on how accountability plays into blameless culture.
I did not know that arctic wolf had a blameless culture. In fact, I had the opposite expectation. The wolf usually represents a solo or selfish kind of thing. Which is technically not accurate to real wolf packs, but pop culture misleads about this.
I did know arctic wolf was big about being friendly to Trans and Autistic people, which I did like.
For me, long ago I was busy enough at work, I embraced blamelessness because I don't have time to waste on that crap. I immediately ended up with results where people who were expecting blame and didnt receive it were suddenly willing to get better. I learnt almost counter-intuitively or unexpectedly that you must have a blameless culture. Been blameless for at least a decade.
Props to arctic wolf, and well my current employer who is awesome.
Readit News
I have seen a lot of incidents and I cannot think of one where a single person was to blame. Sure, one person ran a command or made a bad commit. However, someone else granted them access, someone trained them. A manager either reviewed the process performed or never considered a process was required. A lot of company cultures do not promote proper risk management in technical processes.
I get a lot of satisfaction from technical post-mortems. There is always something that could have been done, a process that could have been in place or a software/infrastructure change to mitigate the problem.
A couple of companies have worked for have had individuals that did not subscribe to this culture. They would want a name. I never knew why exactly. Maybe it was to block a pay bump or defer a promotion. As the tech lead or dev manager any team failings are my responsibility and in companies where bullets are fired I take them. I find this protects the team and helps a blameless culture thrive amongst engineers.
Generally, I find this culture leads to fewer incidents and problems as the openness when things are going wrong allows for faster response times and software/process changes in review.
One of my worst and most consequential work experiences was with the opposite type of culture.
There's lots to be said about it, but in reflecting on things I've often felt like one of the worst parts of that type of culture ( "blameful"?) was a profound lack of trust, a feeling that the blame belied some lack of real interest in improving the situation, as opposed to being punitive against certain individuals for unrelated reasons. That is, the individuals who were targets of the most blame as far as I could tell were generally being targeted for cliquey reasons that had nothing to do with performance or anything of that sort. Conversely, people in the "in group" were given a free pass for all sorts of serious problems. Getting a free pass wasn't really the problem, it was a sense that blame wasn't really about the ostensible blameworthy act, it was that it was being meted out as a kind of superficial leverage for some other thing.
Also, because of these types of issues, serious, legitimate administrative, communication, and other systemic problems never got addressed, because the blame was being used as a kind of social ostracizing mechanism, and the actual underlying problems weren't fixed in their entirety. So in many ways a lot of the problems got worse, not better.
In contrasts, in other environments (even the same place at a different point in time) where there's an emphasis on problem solving and figuring out what could be done differently by everyone, and assuming the best intentions and basic competence, trust underlies everything. You're motivated more because you believe that actions have fair consequences, and that everyone has each other's best interests in mind.
There have been many folks like this, in my career. I am humbled and grateful.
I worked for a Japanese corporation, where they didn't "blame," per se, but did figure out if someone was Responsible. If so, that someone was expected to cop to it, and the team would then close ranks, and fix the issue. Lesson learned.
I know that part of it is lawyers and police. If you say "I'm sorry," in many contexts, that puts you on the hook, legally. Most lawyers will advise you to keep your mouth shut, and never, ever admit a thing.
I was taught to promptly admit when I'm wrong; regardless of the consequences. Also, to not cop to stuff that I'm not responsible for.
I have often had arguments with (American) people, where they are 100%, without-a-shred-of-doubt, wrong. Often, embarrassingly so, yet, they Will. Not. Admit. It.
I could bring the spirits of Carl Sagan and Stephen Hawking in, to show them they are wrong, and they will still never admit it.
It's actually kind of sad.
My ability to apologize worked extremely well, in the Japanese corporation. They respected it, and I was given a great deal of deference.
On the other hand, the Americans tended to be pretty much dicks about it. If I apologized, they usually tried to use it to gain some leverage, and bully me.
I don't think there's some general "(American) culture of blamelessness," and I think your comment might misunderstand the point of it.
My understanding about "blameless culture" is it's all about avoiding counterproductive things like scapegoating, blame-shifting, defensiveness, etc. Removing those things encourages the full cooperation of the team in resolving the issue, by removing the anxiety of punishment for being the guy who's being blamed. Everyone's supposed to take responsibility.
With that said, I think your comment describes kiss-up and corporate fiefdom culture. There are plenty of examples of that. I wonder if non-American cultures are different in that respect.
One very key distinction is freedom to admit error vs "the baby is not ugly." I don't think blameless cultures are the norm, where people volunteer to write post portems and voluntarily submit after action plans..
I think it'd be a little more potent to just say hey, there's a difference between saying someone is partially responsible for an issue, and pestering them about it (e.g. telling them they did a bad thing when they likely already know - I suspect that is what people want to avoid when they go "blameless"). Blame is blame, and disrespect is disrespect. I like to see them as orthogonal.
The article's fire suppression story doesn't even seem like a blame problem to me. It was a jumping to conclusions problem. But again, maybe I only think this because the definition of "blame" is being played with.
Most of all the author's gratitude toward the patient fire chief whose leadership example he praises.
We could do with more of this in cybersecurity.
We're all surrounded by insecure software and defective devices churned out by companies chasing a quick profit. We're mired in complexity, non-determinism and broken systems. The experts who designed them throw up their hands in confusion. And we're hamstrung with laws and policies, which despite their laudable intentions only beget twisted systems filled with perverse incentives and loopholes.
How do we solve this? Blame employees! And then humiliate them! Make our cyber team into a punitive, even vindictive internal police force?
Karen from HR is sobbing into her cardigan sleeve because ICT caught her out in a "Phishing drill". Now she'll have to go on a humiliating "this is how a computer works" course - despite the fact she was programming before her IT manager was born. No doubt she'll get her own back in time.
As ICT and cyber teams we could learn a lot from more traditional organisational culture.
Should I DM you from info on your site?
PS: your website is so great. Love it.
Would love to know how this was explained. I wonder if the author remembers how it was put.
Blameless does not mean it is ok to break stuff. Or that you are not supposed to feel shame. Or that doing shit job comes with no consequences.
Blameless culture was supposed to mean that it is more important to figure out why something broke and find solutions to fix it than pursue penalties for people who did it. That pursuing penalties and assigning blame tends to turn into this nasty political circus that frequently makes it impossible to figure out what really happened and if somebody is found to be responsible, it frequently is the person who is least adept at politics.
So we say: "The only way we know to be able to maximise improvements based on factual analysis is to stop clouding the process with politics of blame finding."
So this is more like getting away with murder on a technicality.
Not blaming is not the goal. It is the means to uncovering the truth and figuring the way to prevent it in the future.
I fully understand that some people are not looking for that kind of thing at all. Many would rather be nameless cogs, and some would simply rather not do the work (technical or soft skills). That’s fine too. For that latter group, whatever the management buzz words de jour are are likely to be employed where you work sooner rather than later.
My take is that it is probably better to accept bad workers in big orgs rather than trying to reach some kind of competence utopia, to get the incentives right.
My last company had a culture of blameless post mortems, and also a strong culture of accountability, so definitely no one would fit the first definition. And the company was too small to have non technical managers.
My current company (big corporation) officially supports blameless reaction to incidents, but internally has a strong culture of cover-up and blame ping-pong, but I still cannot think of anyone who would believe "blameless means no accountability". It just comes from deeply ingrained habits (lots of people do their whole career inside the company, lots of them even getting their masters at the company's own "university". Behavior is hard to change when everyone comes out of the same mould). I also think it comes from inadequate processes: where in the previous company the postmortem was mostly a meeting done by and for the technical team, my new company has a strong culture of "escalation meetings", where the highest managers quickly get drafted in, which obviously has a strong influence on the capacity to conduct an open and blameless process.
No, that's still somewhat blameful.
If you had stuck with "blameless does not mean no accountability", I think you're right.
But the emphasis of the article specifically does not assume people do bad work on purpose:
> assume the individuals involved had the best of intentions, and either they did not have the correct information to make a better decision, or the tools allowed them to make a mistake.
I'd like to hear other takes on how accountability plays into blameless culture.
Blameless does not mean "you scan screw up as much as you want". If you start being careless, you will be shown the door.
I did know arctic wolf was big about being friendly to Trans and Autistic people, which I did like.
For me, long ago I was busy enough at work, I embraced blamelessness because I don't have time to waste on that crap. I immediately ended up with results where people who were expecting blame and didnt receive it were suddenly willing to get better. I learnt almost counter-intuitively or unexpectedly that you must have a blameless culture. Been blameless for at least a decade.
Props to arctic wolf, and well my current employer who is awesome.