In his book, The Checklist Manifesto, Gawande reports that at the end of a big checklist pilot project for the World Health Organization (IIRC), participating surgeons were surveyed.
• One of the survey questions was, in essence: Would the use of checklists improve your surgical practice? Most of the surgeons responded no. This was consistent with the resistance that Gawande encountered: I don't need no stinking checklist; I'm a good surgeon!
• But the more-telling survey question was this: Would you want another surgeon to use a checklist if operating on you or one of your family members? Something like 93% of the surgeons surveyed responded ... yes.
I chalk this up more to management and not individual egos. Incentive structures configured by the vast majority of management in most organizations do not reward checklist development and more importantly checklist maintenance.
Bob: I'm developing and testing a checklist.
Management: Why is it taking sooo long!?!?? Why aren't you doing <<revenue/KPI-enhancing activity>>?
Versus:
Bob: I'm selling truckloads of surgeries.
Management: W000t! My manager will be ecstatic!
There is also a free rider problem with checklist development. Lots of workplaces love to somewhat or all-out pit employees against each other. Unless the entire team is creating and maintaining the checklist, the subset of the team creating/maintaining the checklist will always over time come out looking worse on KPI's that almost invariably never measure these kinds of activities.
The subset of the team that doesn't spend the time on creating/maintaining the checklist come out looking better on KPI's over time, and the checklists increase their relative standing against their checklist-creating peers.
Fix the incentive structures, and this problem fixes itself.
Just today, I was thinking the same thing, but about compile-time type checking. "You don't need it, because you're a good programmer? Great. But not everyone on your team is as wonderful as you. Do you want to be able to catch their errors at compile time?"
Wow. That's such a supreme indicator of overconfidence--the belief that I and I alone am so competent that I don't need safeguards.
I would not have hoped that a high percentage of surgeons have that trait. I wonder if I am guilty of it as well. Something to examine in the mirror in our future endeavors.
I like to use the term Dunning-Kruger instead of over-confidence or hubris to describe it. Everyone thinks they demonstrate above average skill. During the 2007-2009 financial crisis every single financial services firm justified their executive bonuses by claiming their personnel were above average. Very few car drivers consider themselves below average.
Before germ theory gained acceptance or was even understood, physicians and surgeons considered themselves above hand washing as a gentleman's hands are always clean. Ignaz Semmelweis discovered that washing hands and tools in chlorine solution reduced childbirth deaths. But he was unable to convince others.
In modern times doctors have answered a survey on the effect of pharmaceutical representatives visit. The overwhelming majority responded that they are not swayed by drug saleswomen, but that other docs are susceptible to choosing a prescription drug because of salesmanship instead of only efficacy & risk.
This probably got posted in reaction to the news that the author, Atul Gawande, was announced to be the CEO of a new healthcare company. https://news.ycombinator.com/item?id=17355346
I think a key element of checklists, and checklist discipline, is coordination between members of a team. What checklists allow you to do is have accountability across differences in experience/authority/expertise and also track a process over time.
In my work as a naval aviator (caveat, I'm still a student), we use checklists in the cockpit to coordinate all sorts of functions such as engine start, takeoff, landing and emergencies. These ensure that we start the engine the same way, verify we are ready for landing, and also handle emergency procedures in an approved way. When there are rank and experience differences in the cockpit these checklists help equalize the disparity, and let less experienced or junior members hold senior members accountable. Before each flight we also follow a briefing checklist, and that includes human factors such as any possible issues relating to illness/medication/stress/alcohol/fatigue/eating. Doing so ensures that there is always a place for people to vocalize anything that the team should know.
Previously I worked on a Coast Guard cutter, and we used similar checklists to ensure we were prepared to get underway, enter port, or complete a complex evolution like tow another vessel. For these checklists the main value was in verifying between multiple changes of team personnel that everything was being completed to meet the timeline. The checklist served as documentation that the person before you had collected the necessary information, made proper notifications, and started/finished processes.
It has baffled me when I hear that medical personnel dont use checklists for these exact reasons. They perform time critical, high stress procedures that have a fairly documented set of steps. They require a team of personnel that have different scopes of expertise, experience, and authority. There are frequently changes in the team's composition over the duration of the patients care, and it seems like the patient will be handed off between different teams that specialize in their own respective areas of patient care. For us in the military it is pretty easy for authority to mandate we use checklists, and the buy in come from accident investigations revealing that checklists were ignored or skipped. Is there an equivalent medical authority that can pass this information down, or does this have to emerge independently from each hospital system?
"The checklists provided two main benefits, Pronovost observed. First, they helped with memory recall, especially with mundane matters that are easily overlooked in patients undergoing more drastic events. . . ."
Indeed, benefits of checklists can be huge. this reminds me of a US Army study I read about some time ago which examined the effects of sleep deprivation and stress in demanding environments.
What they found was that, in general, the high-level strategic capabilities stayed intact -- the people could still analyze complex situations, make and execute battle plans, etc.
What was lost was the smaller, mundane functions, which could add up to critical failures, such as forgetting to refill one's water can or refuel the vehicle before going out, etc.
I've always hated my bad memory, and I learned a long while ago that simply trying to remember through sheer force of concentration did not work. It was as hopeless as praying for rain. Only external memory tools and strategies planned in advance can solve it for me.
I have to build a system to keep me from forgetting, because if I leave it up to my future self alone then he will forget 100% of the time.
I have a dumbass problem where after I've driven away from home I can't remember if I locked the front door or closed the garage door so I turn around and go back home to check. I've managed to fix this by saying out loud, "door locked - check" and "garage door down - check" when I've done it. Now I just need a checklist to remind me to do a checklist.
It might sound dumb to you but your solution is used by Japanese train engineers before leaving at a station and trains in Japan are really efficient and on-schedule.
People are generically bad at reliably setting states. The problem isn't so much that someone failed to do something (an action) but that they failed to notice that the action was not done (the state was wrong).
Often times people forget the "check" part of the term "checklist" and treat them as a list of things they must do. The result is less reliable because it is easy to skip a step, even if you end up ticking a box. A checklist involves two steps, first doing the thing (possibly from memory) and then consulting the checklist to see that the appropriate state is set.
This is a particular issue for programmers as programmers are constantly involved with lists of actions that are performed in sequence. As a result there is a tendency to let the list run the human in a way similar to how a computer runs a program. Computers are way better at states than humans are however...
My coworker just fixed an unset state of mine— I checked the box, but did not perform the related action, three weeks ago.
Thankfully, Trello keeps a log of most changes, so it was easy to see where in the process my eyes had glazed over. We’ll get around to automating those steps someday, too.
I am an avid checklist user. I have a number of complex processes which are repeated monthly, quarterly or yearly. When the office is calm and I am feeling good I don't need them and find myself ticking them off having already done the stage. When I am feeling less-good, the office is noisy and I keep being interrupted the check-lists are the reason I succeed.
Yet still I find almost complete resistance to them by colleagues. They seem to think it is somehow demeaning to reduce their work to a fixed process. Perhaps it takes the mystique out of it, and gives the idea that anyone could do it.
Only if the job really could be done by anybody. Pilots use checklists for basically everything, but I don't think there's a widespread belief that the pilot could now be replaced by J. Random Passenger. The point of a checklist isn't necessarily to completely remove any thought or training from a process, but to ensure that every critical step is performed in the proper order.
Does anyone know of a good cross-platform checklist app? There's a million and one todo list apps but I can't seem to find a good checklist one. I use my phone's memo app for checklists but it doesn't help me with staying consistent over time, and doesn't scale as I use the same app for note taking, ideas, a scratch space and so on.
Oh, and if you're thinking "I don't need no stinkin' checklists, what would you even use them for?" here's what I have currently:
- Code review key checks
- Code review smells and anti-patterns
- Out-the-door checks (got my keys, entry pass, grabbed lunch from fridge etc)
- Gym gear list
- Traits to work on a la Benjamin Franklin
- Self-care list for when I'm stressed/anxious (relief suggestions e.g. drink a glass of water, go for a walk, mindfulness, organise workload on paper, etc)
- Carry lists for a couple hobbies
I would likely have more but the poor form factor limits how practical it is to keep many lists.
A few thoughts from someone who has used checklists extensively for many years:
1. Use a spreadsheet. The flexibility to change the structure as you go, and to create a 2-dimensional checklist is great. All the other power of a spreadsheet is at your fingertips, including automation.
2. Visualization is extremely valuable, and I think necessary. If you have to read through the text of the checklist to understand the current state, it's too cumbersome (other than for small lists) and hard to process. If spreadsheet cells change color based on status, and if their location indicates, via visual memory, what they represent, then now you have a dashboard which tells you the state of the project at a glance.
3. Human beings are terrible at following and updating checklists. I don't know why - it seems easy in concept - but I've never met anyone who could do it without forcing themselves. And everyone starts instinctively skipping steps after awhile. Keep it as simple as possible, with as little detail and as few updates as are necessary to keep things going.
it's not so much the text formatting as the ability to set reminders, physically check off the list and have a record per-day/instance/whatever of completing the list, that kind of thing. The main reason I use checklists so much is because my memory is extremely bad, so prompts make a huge difference to me.
Readit News
• One of the survey questions was, in essence: Would the use of checklists improve your surgical practice? Most of the surgeons responded no. This was consistent with the resistance that Gawande encountered: I don't need no stinking checklist; I'm a good surgeon!
• But the more-telling survey question was this: Would you want another surgeon to use a checklist if operating on you or one of your family members? Something like 93% of the surgeons surveyed responded ... yes.
Bob: I'm developing and testing a checklist.
Management: Why is it taking sooo long!?!?? Why aren't you doing <<revenue/KPI-enhancing activity>>?
Versus:
Bob: I'm selling truckloads of surgeries.
Management: W000t! My manager will be ecstatic!
There is also a free rider problem with checklist development. Lots of workplaces love to somewhat or all-out pit employees against each other. Unless the entire team is creating and maintaining the checklist, the subset of the team creating/maintaining the checklist will always over time come out looking worse on KPI's that almost invariably never measure these kinds of activities.
The subset of the team that doesn't spend the time on creating/maintaining the checklist come out looking better on KPI's over time, and the checklists increase their relative standing against their checklist-creating peers.
Fix the incentive structures, and this problem fixes itself.
I would not have hoped that a high percentage of surgeons have that trait. I wonder if I am guilty of it as well. Something to examine in the mirror in our future endeavors.
Before germ theory gained acceptance or was even understood, physicians and surgeons considered themselves above hand washing as a gentleman's hands are always clean. Ignaz Semmelweis discovered that washing hands and tools in chlorine solution reduced childbirth deaths. But he was unable to convince others.
In modern times doctors have answered a survey on the effect of pharmaceutical representatives visit. The overwhelming majority responded that they are not swayed by drug saleswomen, but that other docs are susceptible to choosing a prescription drug because of salesmanship instead of only efficacy & risk.
This probably got posted in reaction to the news that the author, Atul Gawande, was announced to be the CEO of a new healthcare company. https://news.ycombinator.com/item?id=17355346
https://www.newyorker.com/magazine/2009/06/01/the-cost-conun...
https://www.newyorker.com/magazine/2015/05/11/overkill-atul-...
In my work as a naval aviator (caveat, I'm still a student), we use checklists in the cockpit to coordinate all sorts of functions such as engine start, takeoff, landing and emergencies. These ensure that we start the engine the same way, verify we are ready for landing, and also handle emergency procedures in an approved way. When there are rank and experience differences in the cockpit these checklists help equalize the disparity, and let less experienced or junior members hold senior members accountable. Before each flight we also follow a briefing checklist, and that includes human factors such as any possible issues relating to illness/medication/stress/alcohol/fatigue/eating. Doing so ensures that there is always a place for people to vocalize anything that the team should know.
Previously I worked on a Coast Guard cutter, and we used similar checklists to ensure we were prepared to get underway, enter port, or complete a complex evolution like tow another vessel. For these checklists the main value was in verifying between multiple changes of team personnel that everything was being completed to meet the timeline. The checklist served as documentation that the person before you had collected the necessary information, made proper notifications, and started/finished processes.
It has baffled me when I hear that medical personnel dont use checklists for these exact reasons. They perform time critical, high stress procedures that have a fairly documented set of steps. They require a team of personnel that have different scopes of expertise, experience, and authority. There are frequently changes in the team's composition over the duration of the patients care, and it seems like the patient will be handed off between different teams that specialize in their own respective areas of patient care. For us in the military it is pretty easy for authority to mandate we use checklists, and the buy in come from accident investigations revealing that checklists were ignored or skipped. Is there an equivalent medical authority that can pass this information down, or does this have to emerge independently from each hospital system?
Indeed, benefits of checklists can be huge. this reminds me of a US Army study I read about some time ago which examined the effects of sleep deprivation and stress in demanding environments.
What they found was that, in general, the high-level strategic capabilities stayed intact -- the people could still analyze complex situations, make and execute battle plans, etc.
What was lost was the smaller, mundane functions, which could add up to critical failures, such as forgetting to refill one's water can or refuel the vehicle before going out, etc.
Interesting lesson
I have to build a system to keep me from forgetting, because if I leave it up to my future self alone then he will forget 100% of the time.
https://www.allaboutlean.com/pointing-and-calling/
Often times people forget the "check" part of the term "checklist" and treat them as a list of things they must do. The result is less reliable because it is easy to skip a step, even if you end up ticking a box. A checklist involves two steps, first doing the thing (possibly from memory) and then consulting the checklist to see that the appropriate state is set.
This is a particular issue for programmers as programmers are constantly involved with lists of actions that are performed in sequence. As a result there is a tendency to let the list run the human in a way similar to how a computer runs a program. Computers are way better at states than humans are however...
Thankfully, Trello keeps a log of most changes, so it was easy to see where in the process my eyes had glazed over. We’ll get around to automating those steps someday, too.
Yet still I find almost complete resistance to them by colleagues. They seem to think it is somehow demeaning to reduce their work to a fixed process. Perhaps it takes the mystique out of it, and gives the idea that anyone could do it.
If you simplify your job, everybody could do it, and the salary and job security would go down.
Oh, and if you're thinking "I don't need no stinkin' checklists, what would you even use them for?" here's what I have currently:
- Code review key checks
- Code review smells and anti-patterns
- Out-the-door checks (got my keys, entry pass, grabbed lunch from fridge etc)
- Gym gear list
- Traits to work on a la Benjamin Franklin
- Self-care list for when I'm stressed/anxious (relief suggestions e.g. drink a glass of water, go for a walk, mindfulness, organise workload on paper, etc)
- Carry lists for a couple hobbies
I would likely have more but the poor form factor limits how practical it is to keep many lists.
1. Use a spreadsheet. The flexibility to change the structure as you go, and to create a 2-dimensional checklist is great. All the other power of a spreadsheet is at your fingertips, including automation.
2. Visualization is extremely valuable, and I think necessary. If you have to read through the text of the checklist to understand the current state, it's too cumbersome (other than for small lists) and hard to process. If spreadsheet cells change color based on status, and if their location indicates, via visual memory, what they represent, then now you have a dashboard which tells you the state of the project at a glance.
3. Human beings are terrible at following and updating checklists. I don't know why - it seems easy in concept - but I've never met anyone who could do it without forcing themselves. And everyone starts instinctively skipping steps after awhile. Keep it as simple as possible, with as little detail and as few updates as are necessary to keep things going.