My dad headed up the redesign effort on the Lockheed Martin side to remove the foam PAL ramps (where the chunk of foam that broke off and hit the orbiter came from) from the external tank, as part of return-to-flight after the Columbia disaster. At the time he was the last one left at the company from when they had previously investigated removing those ramps from the design. He told me how he went from basically working on this project off in a corner on his own, to suddenly having millions of dollars in funding and flying all over for wind tunnel tests when it became clear to NASA that return-to-flight couldn't happen without removing the ramps.
I don't think his name has ever come up in all the histories of this—some Lockheed policy about not letting their employees be publicly credited in papers—but he's got an array of internal awards from this time around his desk at home (he's now retired). I've always been proud of him for this.
This isn't a failure of PowerPoint. I work for NASA and we still use it all the time, and I'll assure anyone that the communication errors are rife regardless of what medium we're working in. The issue is differences in the way that in-the-weeds engineers and managers interpret technical information, which is alluded to in the article but the author still focuses on the bullets and the PowerPoint, as if rewriting similar facts in a technical paper would change everything.
My own colleagues fall victim to this all the time (luckily I do not work in any capacity where someone's life is directly on the line as a result of my work.) Recently, a colleague won an award for helping managers make a decision about a mission parameter, but he was confused because they chose a parameter value he didn't like. His problem is that, like many engineers, he thought that providing the technical context he discovered that led him to his conclusion was as effective as presenting his conclusion. It never is; if you want to be heard by managers, and really understood even by your colleagues, you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I've seen this over and over again, and I'm starting to think it's a personality trait. Engineers are gossiping among themselves, saying "X will never work". They get to the meeting with the managers and present "30 different analyses showing X is marginally less effective than Y and Z" instead of just throwing up a slide that says "X IS STUPID AND WE SHOULDN'T DO IT." Luckily for me, I'm not a very good engineer, so when I'm along for the ride I generally translate well into Managerese.
I love it when some company gets one of the engineers to do a demonstration, you know you got an actual engineer because it will be the worst sales pitch you ever received. They will tell you in excruciating detail all the problems with their product. Recognize and cherish these moments for what they are worth, despite the terrible presentation it is infinitely more valuable than yet another sales rat making untenable promises.
It is something to do with that being the engineers actual job, to find and understand the problems with the product. so when talking to a customer, that is what tends to come across, all the problematic stuff. The good stuff that works, not important to them.
As a former sales engineer - it's more about setting expectations correctly. If the customer knows the shortcomings (as well as the benefits) and signs anyway it's usually a good partnership. If the customer finds them out after the contract, then it's the opposite.
it's truly a horrible thing that hearing the facts as they are, is considered excruciating. i'm very lucky that the company i've been working for for 6 years takes care of me exactly because i'm detailed, i say what i think when i think it, and have built a cult following amongst our customer base for being able to get to the meat of problems and solve them, or get at least on the path to resolution.
I was just reading a great discussion about how "academics use qualifiers as to how confident they are in the information" and you can see similar trends in spaces like hacker news.
But using "uncertain" language seems unconvincing to people outside of these types of cultures.
- bad communication possible in any medium
- pptx in NASA even today!
- issue is managers/SMEs communication differences
- issues with technical papers
- long
- boring
- vs word, excel, pdf...
(Next slide please)
Manager/SME Differences
- context vs conclusion
- tell a compelling story
- but give away the ending FIRST
- inherent personality differences
- motivations/incentives/mindsets
(Next slide)
Learning from disasters
- medium guides message and messenger
- blame tool - binary choice?
- presentation aide vs distributed technical artifact
I think the problem is that most people, especially non-engineers, are over-selling and over-promising all the time. Being honest about risks, issues, and short-comings makes a project or product look bad in comparison.
The most feasible way to get X done is saying "X is a great option, the risks are managable, and it's fairly quick". Then, it will unexpectedly take a bit longer, plus some unforeseen trade-offs will need to be made.
Agree and was going to say the same thing. Messages need to be created for a specific audience.
When I'm sending an email to non-tech mgrs that has a bunch of tech details like that slide, I typically separate more detailed stuff from the conceptual message:
Summary:
System performance is not good enough for go live.
If nothing else it's quite hard/uncommon to print out a PowerPoint and read it carefully in a quiet room by yourself, I do this with written stuff all the time.
The whole point of PowerPoint is to pander to people who can't or don't want to sit down and carefully read a report. They want to sit back and passively consume information like they were watching TV. The problem then isn't so much PowerPoint itself, but rather these quasi illiterate people being in decision making positions. That's the real problem, and PowerPoint is just a symptom.
>if you want to be heard... and really understood ... you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I question your premise. :J
I'm just kidding, that's interesting I'll have to think about applying that. I don't suppose that would translate over to blogging? The fear of course is that one makes a statement and the commentariat thinks the speaker is full of it for not having provided backup instead of questioning. Maybe it's dependent on what type of forum it is.
I feel that. I looked at the powerpoint from the post and I cringed because it looks just like the kind of thing I could have written, with half finished thoughts all over the place masquerading as something a decider could use...
This article (as it makes clear) owes it's analysis at least largely to what Tufte has written about the Challenger disaster (1986) and Columbia Disaster (2003). He wrote about the Columbia one more fully in the second edition of The Cognitive Style of Powerpoint.
Given that the link in the article to his report on his website is now broken, people might be interested in teh few page grabs that he has included in the "comments" on his site here[0].
See also the article that he has re-posted under the "comments" section on his page on the matter[1].
The full report (2003 edition, low-res) is available on ResearchGate. It appears to be a lawful copy, uploaded by the author himself. Fascinating reading, indeed.
That link is the chapter "The Cognitive Style of PowerPoint" from Tufte's book Beautiful Evidence, and it does mention Boeing's slides in the Columbia incident, but the main work that the author of this blog post cribbed (and failed to grasp) is a more detailed essay by Tufte called "PowerPoint Does Rocket Science: Assessing the Quality and Credibility of Technical Reports".
I don't see how this has anything to do with PowerPoint. There wasn't clear communication; the medium was completely incidental to that. They could have been writing on a chalkboard and had a communication failure, does that mean that chalkboards should be blamed in that case?
Because the medium is not conducive to dense amount of technical information that readers are expected to use to make or understand decisions. Other similar mediums like a chalkboard were not singled out because the problem was identified with PowerPoint specifically. And it wasn't a choice of mediums all with similar problems, but slides vs papers. From the article,
> “The Board views the endemic use of PowerPoint briefing slides instead of technical papers as an illustration of the problematic methods of technical communication at NASA.”
But the problem, if anything, was that too much dense information was conveyed at all. Based on the analysis in the post, of the engineers had replaced that slide with one that said "Don't go forward with reentry", that might have saved lives better than any change in medium. To be clear, I'm in favor of abolishing PowerPoint for any non-ephemeral use, but the problem here was focus and framing of the info.
Would it be better if you sent them a PDF document instead? There seems to be an assumption here that if you send the stakeholders a larger volume of information they will take the time to read it. Is that a valid assumption?
Feynman communicated the problem with the Challenger disaster using a rubber band and a glass of ice water.
I don't think PowerPoint is the problem in and of itself, but rather its use as a crutch to compensate for poor communication. Of course, even among scientists, few can count themselves at Feynman's level in terms of communication skills. Maybe this is a skill that NASA scientists need to brush up on, perhaps with Pluralsight courses or something? lol
Speaking of chalkboards, next time you have to give a presentation, bring a chalkboard and do your slides in realtime. Something about the visual show, auditory overload, and not least the novelty of the act makes it much more impactful and memorial than "another powerpoint that puts you to sleep"
White boards are... ok... better than powerpoint but still fail to sell it like a chalkboard does. I think it is the noise.
Yes, the noise (which I'd call "the sound") is a big factor.
I teach in a classroom that had blackboard that had stood the test of time for decades. When it was replaced with a whiteboard, things went downhill. The markers dry out quickly, without much notice, so that students often have trouble reading the material. And the whiteboards get harder to erase year after year.
I guess the advantage of whiteboards is that a variety of colours can be used. But some students have deficiencies of colour recognition, so that's not really helpful. (I never used coloured chalk, for the same reason. Maximal contrast is the key.)
And the noise. That click drag click of chalk. Students after the transition to whiteboards told me that they really missed that. It enlivened the lectures. And when students were writing down notes, they knew to look up when they heard the sound.
Back to the point about the "visual show" and doing slides in real time. Yes, yes, yes. Once in a while I need to show something on the projector. The moment I turn it one, I see students start to disengage.
PowerPoint gets used because it requires less effort from the audience. They sit back and zone out like couch potatoes. Scrap the PowerPoint and throw the technical report at the managers. Any of them who complain or otherwise don't read it are incompetent and should be fired on the spot.
I think it's a lot harder to have this particular type of communications failure if you're writing on a chalkboard. Imagine trying to write out that whole slide, it would take forever. If you really did have to present that information on a chalkboard, you'd be significantly more likely to write something along the lines of:
"We checked the test data: possible to damage tiles significantly"
"Foam that hit wing was way bigger than the tests"
Obviously you can miscommunicate via any medium, but I think the author's point here (which I agree with) is some mediums lend themselves to specific types of miscommunication.
Yeah, the choice to gloss over the point "our tests are not relevant" was a deliberate one. If it was in a paper you'd have big fancy graphs of the tests and you'd have to do your own work to compare the x axis against a mention of the actual scale in question in another paragraph. It's not as if they started with "Warning: even the 600X smaller bits we tested can damage the wing" and microsoft just kind of spontaneously grew a bunch of random stuff above the fold. It's a kind of chickenshit communication which you can do in any medium. The point they ought to be making is not dense or technical, it is so simple a child could understand.
This was an interesting article but it doesn't really provide solutions. I watched a few tech talks teaching a new API. Most slides were split, left side bullet poitns, right side either code or an image. As I was watching I was thinking "isn't this supposed to be almost the worst style"? but I was also thinking "I can't think of any way to do this better". It's an API. It requires examples. And it requires something describing what to concentrate on, what the example or image is showing.
I've been the plenty of great talks with just images, no words. But they fit the type of talk. I'm not sure an API talk would be better without bullet points. If you know of some to reference, please post links.
Tufte did make specific recommendations that one should prepare a real document that your audience can and should read, and that they would have in front of them during the meeting. I'm not sure how best to translate that to your API example.
What would make the most important point of that slide stand out any more in a "real document" than in a slideshow? If anything, I would expect it to be buried even more - a slide and limited-time presentation forces you to be concise, while in a document there tend to be few limits on length.
I would say the disaster occurred despite PowerPoint, not because of it. It's not clear to me at all why the slide author thought all that text was needed, when it seems to communicate almost nothing. If anything I would blame it on the culture around "real documents", where having more information is treated as better (probably because they serve multiple functions - to educate, but also as a record of activities), even if it makes it bloated and hard to read.
I found it surprising that the slide in the article uses Calibri, a typeface that wasn’t publicly available at the time. The original discussion confirms that the slide in the article is a recreation of the original one:
> The slide in the article has the same text, but is a recreation of the original (The Calibri typeface used wasn't part of PowerPoint until 2007).
> The original slide can be seen in the full report linked in the article:
No, that's not how the physics works. The foam is moving at the same velocity as the shuttle when it breaks off, and had a short time to accelerate(decelerate) before hitting the shuttle.
THANK YOU. I've seen the velocity of the space shuttle quoted as the speed that the foam had when it hit Columbia's wing so many times, and it bugs the crap out of me.
If that was the actual impact velocity of the foam, there would not be any doubt about whether the shuttle would survive reentry, that is if it even managed to make it all the way into space.
Readit News
I don't think his name has ever come up in all the histories of this—some Lockheed policy about not letting their employees be publicly credited in papers—but he's got an array of internal awards from this time around his desk at home (he's now retired). I've always been proud of him for this.
To folks out there: do the important work, not the glamorous work, and you'll not only sleep well, but you might actually matter as well.
Dead Comment
My own colleagues fall victim to this all the time (luckily I do not work in any capacity where someone's life is directly on the line as a result of my work.) Recently, a colleague won an award for helping managers make a decision about a mission parameter, but he was confused because they chose a parameter value he didn't like. His problem is that, like many engineers, he thought that providing the technical context he discovered that led him to his conclusion was as effective as presenting his conclusion. It never is; if you want to be heard by managers, and really understood even by your colleagues, you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I've seen this over and over again, and I'm starting to think it's a personality trait. Engineers are gossiping among themselves, saying "X will never work". They get to the meeting with the managers and present "30 different analyses showing X is marginally less effective than Y and Z" instead of just throwing up a slide that says "X IS STUPID AND WE SHOULDN'T DO IT." Luckily for me, I'm not a very good engineer, so when I'm along for the ride I generally translate well into Managerese.
It is something to do with that being the engineers actual job, to find and understand the problems with the product. so when talking to a customer, that is what tends to come across, all the problematic stuff. The good stuff that works, not important to them.
But using "uncertain" language seems unconvincing to people outside of these types of cultures.
Also of course the power dynamics.
In my mind, I'm thinking "so long as a meteor doesn't cause an extinction event," but the manager graciously pushes the target date back a week.
Manager/SME Differences
(Next slide)Learning from disasters
(Next slide)Questions?
I’ve found that most folks have no intention of improving their communication effectiveness. Everyone is much happier, blaming the audience.
[0] https://news.ycombinator.com/item?id=44202502
The most feasible way to get X done is saying "X is a great option, the risks are managable, and it's fairly quick". Then, it will unexpectedly take a bit longer, plus some unforeseen trade-offs will need to be made.
Agree and was going to say the same thing. Messages need to be created for a specific audience.
When I'm sending an email to non-tech mgrs that has a bunch of tech details like that slide, I typically separate more detailed stuff from the conceptual message:
Summary:
System performance is not good enough for go live.
Working on a few possible solutions.
Details (for those interested):
System x is connected to ...blah blah blah...
I question your premise. :J
I'm just kidding, that's interesting I'll have to think about applying that. I don't suppose that would translate over to blogging? The fear of course is that one makes a statement and the commentariat thinks the speaker is full of it for not having provided backup instead of questioning. Maybe it's dependent on what type of forum it is.
It would have been nicer if that had been the first sentence of that (interesting) comment.
Deleted Comment
Given that the link in the article to his report on his website is now broken, people might be interested in teh few page grabs that he has included in the "comments" on his site here[0].
See also the article that he has re-posted under the "comments" section on his page on the matter[1].
[0]: https://www.edwardtufte.com/notebook/new-edition-of-the-cogn... [1]: https://www.edwardtufte.com/notebook/the-columbia-evidence/
https://www.nasa.gov/history/rogersrep/v2appf.htm
The words "a safety factor of three" will live with me for every day of my life.
The full report (2003 edition, low-res) is available on ResearchGate. It appears to be a lawful copy, uploaded by the author himself. Fascinating reading, indeed.
https://www.researchgate.net/publication/208575160_The_Cogni...
<https://www.edwardtufte.com/notebook/powerpoint-does-rocket-...>
> “The Board views the endemic use of PowerPoint briefing slides instead of technical papers as an illustration of the problematic methods of technical communication at NASA.”
I don't think PowerPoint is the problem in and of itself, but rather its use as a crutch to compensate for poor communication. Of course, even among scientists, few can count themselves at Feynman's level in terms of communication skills. Maybe this is a skill that NASA scientists need to brush up on, perhaps with Pluralsight courses or something? lol
White boards are... ok... better than powerpoint but still fail to sell it like a chalkboard does. I think it is the noise.
I teach in a classroom that had blackboard that had stood the test of time for decades. When it was replaced with a whiteboard, things went downhill. The markers dry out quickly, without much notice, so that students often have trouble reading the material. And the whiteboards get harder to erase year after year.
I guess the advantage of whiteboards is that a variety of colours can be used. But some students have deficiencies of colour recognition, so that's not really helpful. (I never used coloured chalk, for the same reason. Maximal contrast is the key.)
And the noise. That click drag click of chalk. Students after the transition to whiteboards told me that they really missed that. It enlivened the lectures. And when students were writing down notes, they knew to look up when they heard the sound.
Back to the point about the "visual show" and doing slides in real time. Yes, yes, yes. Once in a while I need to show something on the projector. The moment I turn it one, I see students start to disengage.
PowerPoint gets used because it requires less effort from the audience. They sit back and zone out like couch potatoes. Scrap the PowerPoint and throw the technical report at the managers. Any of them who complain or otherwise don't read it are incompetent and should be fired on the spot.
"We checked the test data: possible to damage tiles significantly" "Foam that hit wing was way bigger than the tests"
Obviously you can miscommunicate via any medium, but I think the author's point here (which I agree with) is some mediums lend themselves to specific types of miscommunication.
https://norvig.com/Gettysburg/
I've been the plenty of great talks with just images, no words. But they fit the type of talk. I'm not sure an API talk would be better without bullet points. If you know of some to reference, please post links.
I would say the disaster occurred despite PowerPoint, not because of it. It's not clear to me at all why the slide author thought all that text was needed, when it seems to communicate almost nothing. If anything I would blame it on the culture around "real documents", where having more information is treated as better (probably because they serve multiple functions - to educate, but also as a record of activities), even if it makes it bloated and hard to read.
I found it surprising that the slide in the article uses Calibri, a typeface that wasn’t publicly available at the time. The original discussion confirms that the slide in the article is a recreation of the original one:
> The slide in the article has the same text, but is a recreation of the original (The Calibri typeface used wasn't part of PowerPoint until 2007).
> The original slide can be seen in the full report linked in the article:
> https://www.edwardtufte.com
No, that's not how the physics works. The foam is moving at the same velocity as the shuttle when it breaks off, and had a short time to accelerate(decelerate) before hitting the shuttle.