The Unreasonable Effectiveness of Checklists
Dr. Peter Provonost had a problem. People were dying and — to borrow a line from Fight Club — not in the Sylvia Plath, Tibetan Buddhist, we’re-all-dying-so-get-used-to-it sense of the word.
No, this is hospital kind of death we’re talking. I mean death in all of its macabre horror. You know, the horror we cover up with euphemisms like “passing away” and pretend that white sheets and a sterile environment somehow make the notion of oblivion no longer panic-inducing. That kind of death.
And not the inevitable sort. Not of the “his body just gave out” or “there was nothing we could do” kind of death, although I’m sure plenty of attendings fell back on that convenient cliche. No, I mean preventable death. Death of the there-but-for-the-grace-of-unwashed-hands-now-I’m-dead kind. I mean the kind where you’re in the hospital for a routine procedure and some dumbass with 15 years of schooling forgets to wear gloves so now you’re profoundly, absolutely dead. That kind. The sort of death where if the average doctor had one more percentile of conscientious you wouldn’t be dead because he wouldn’t have killed you.
The sort of deaths that define why hospitals are a dangerous place.
That sort of death was Dr. Provonost’s problem. Mistakes were killing people at his hospital. Not some podunk care center, either, but critical care at Johns Hopkins.
So he did the obvious, boring thing. He implemented a checklist for one basic-but-still-error-prone-and-infectious procedure, inserting a central venous catheter, and everyone had to follow it. And this checklist of his wasn’t complicated. These weren’t instructions where, in order to understand them, you need to rack up the equivalent of the GDP of a small nation state in medical school debt. There were five whole things, and they boil down to two: clean yourself and the patient, wear a mask and gloves. Not super tricky, only-clever-people-know steps.
These were the hospital equivalent of brushing your teeth before bed and wearing deodorant. The absolute basics. Stuff everyone is supposed to do, but sometimes people forget. Except when you forget to wear deodorant at a hospital, it’s a lot worse than spending a day fretting over whether or not your crush has discovered that your natural smell is not coffee-cinnamon-woodland, but something decidedly funky. When you forget at a hospital, someone catches Legionnaires’ disease and dies forever.
And maybe you’re skeptical: “A checklist for five things? I can remember five things no problem. How many mistakes could doctors possibly be making?” (And that’s how I know you’re not a programmer.) But you wouldn’t be alone. Dr. Gawande, a surgeon at Brigham and Women’s Hospital in Boston told the New York Times, “It seemed silly to make a checklist for something so obvious.”
Except, you know, this stupid checklist of five whole things totally worked. After a year, the rate of infection on this specific procedure dropped from 11 percent to zero. By two years, it had saved the hospital 2 million dollars, prevented 8 unnecessary deaths, and avoided 43 infections. Consequently, the hospital implemented still more checklists – reducing the average ICU stay by half and saving 21 lives.
A 2009 study duplicated this success in 8 other hospitals: “its use improved compliance with standards of care by 65% and reduced the death rate following surgery by nearly 50%.”
Checklists are awesome.
How awesome are they? A brief review
The bulk of the evidence for the effectiveness of checklists comes from medicine and is relatively recent. While other disciplines, such as aviation and engineering, have long used checklists, they haven’t bothered to actually vet that they work. A 2002 study puts it this way:
Aviation safety … was not built on evidence that certain practices reduced the frequency of crashes (but) relied on the widespread implementation of hundreds of small changes in procedures, equipment and organization (to produce) an incredibly strong safety culture and amazingly effective practices. These changes made sense; were usually based on sound principles, technical theory or experience; and addressed real-life problems, but few were subjected to controlled experiments
This is less surprising when we consider when the pre-flight checklist was implemented. They’ve been a constant in the airline industry since 1937.
Even newer and emerging disciplines, like software engineering and quality assurance, have done little to empirically verify the effectiveness of checklists. A 2007 paper, “Best Practices in Code Inspection for Safety-Critical Software,” is a typical example. Though it focuses solely on the use of checklists to improve software quality, it presents no evidence on the actual effectiveness of checklists. Similarly, a 1999 review further calls using checklists to inspect software a “best practice,” but again assumes their efficacy.
Reassuringly, though, the evidence from medicine is near overwhelming. Checklists have been found effective in scenarios as diverse as oxytocin administration to pregnant mothers (which decreased the rate of cesarean delivery by a quarter), actually giving patients medication, measuring lipid levels, screening stroke patients, and improving the report quality of RCT trials.
This and more is captured in a 2012 review and meta-analysis, which finds that checklists reduce the risk of mortality by nearly half:
This review shows that with the use of the checklist the relative risk for mortality is 0.57 and for any complications 0.63.
It should come as no surprise, then, that checklists are cost-effective, with the ability to save hospitals anywhere between $103,829 and $2,671,253.
If you’re still not convinced, not only do checklists save lives and money, but they may also improve process efficiency and productivity:
Use of a “preflight checklist” in Kaiser Permanente Southern California’s operating rooms resulted in improved nurse retention as turnover decreased from 23% to 7%. Also, after implementation of Kaiser Permanente’s checklist, there was a decrease in the number of operative cases that were canceled or delayed.
So, if the question is “How awesome are checklists?” I’d say: pretty awesome.
Further Reading
- I have written before about the checklist I use for writing.
- For more on Dr. Peter Pronovost, check out the original NYTimes article or the book The Checklist Manifesto.
- Daniel Kahneman, Dan Lovallo, and Olivier Sibony have a nice checklist on checking and compensating for bias before making an important decision.
- Jesse Galef has written a checklist on dealing with procrastination.
- A bunch of people share their own checklists in this thread.