Out of Touch Screens
Chapter 23: Scrambling to figure out who to blame
Dear Friend,
In last week’s chapter I described what happened moments before a Navy Destroyer lost control and crashed. This week I describe the aftermath and why the investigation completely missed the real reason for the tragedy: bad design. As a designer by trade, this blind spot scares me. Our lives drift around on auto-pilot, surrounded and saturated by things designed by people we will never know. My motivation for writing User Zero is to expose the thinking (and lack of thought) behind systems we take for granted and equip you with tools (and hope!) for understand why things are the way they are. The McCain is a metaphor for you. Regain control before the ship crashes. Stay creative.
Your friend,
Ade
“Individual humans are now helplessly inarticulate in the face of the present crisis.” —Buckminster Fuller
The teenager at McCain’s controls was undoubtedly proficient with his smart phone, but he probably couldn’t explain why the touchscreen of the ship tripped him up. While ship captains are publicly disgraced, their photos publicized for all to condemn, the best excuse he could find was to blame his training. At his court-martial, the McCain’s commander, Alfredo Sanchez accepted blame for the 18-year-old’s actions. He said, “I did not put him in a position to succeed.” Why did Sanchez blame lack of training instead of the design of the ship’s controls? Couldn’t he see how the flaws of the equipment lead to destruction?
Without words to describe something, it is essentially invisible. Lack of design literacy has hidden the problem from the very people who are looking for answers. The crew, Navy’s leadership, independent investigators, and the media all lack the ability to describe, let alone fix the problems that caused McCain’s accident. Just as in the Columbia space shuttle tragedy, the life-saving data was hiding in dense, opaque, jargon-filled documentation. It is a deep fake, a hand stuck in a coconut, an intentionally blank document. It’s a con that nobody can detect.
In the aftermath of the investigations into the McCain accident, the Navy announced they would be abandoning touchscreens in their fleet and reverting back to the old physical controls. While normally I am opposed to the touchification of everything, I find myself in the odd position of defending the touchscreens.
The decision to revert to physical controls was not made because usability tests had discovered flaws in the touchscreens. It wasn’t the result of a design audit or suggestions by design experts. Instead, the decision seems to largely hinge on the results of a single survey. Admiral Bill Galinis described the process as “doing some fleet surveys and whatnot.” I hesitate to judge Galinis on a single quote, but this doesn’t sound like somebody who appreciates the importance of user research or design.
One of the distinct advantages of touchscreens over physical controls is that they can be iteratively improved. Most of the flaws of the touchscreens could be just one software update away. And yet, instead of iterating to a better design, the Navy will abandon touchscreens based on a preference test. Here is Admiral Galinis again,
“We got away from the physical throttles, and that was probably the number-one feedback from the fleet – they said, just give us the throttles that we can use.”
When surveyed, it should surprise nobody that responses overwhelmingly preferred physical controls over the digital screens. All they know is that the software was confusing, opaque, and required additional training. Given a choice between the existing experience and the old system, what would you expect sailors to say? Somehow the fact that software can be improved doesn’t seem to be an option. Even the survey was poorly designed, because the sailors weren’t asked whether they wanted better designed interfaces, their only option was what they had before or what they had today.
Of course you can’t go completely back to the old controls, the makeover will require a hybrid approach. As with the evolution of the Boeing 737 Max, careful compromises will be made as some aspects will be automated, some of the feedback will be simulated, and all of it will be digital. This puts the designers who are called upon to design the new controls (if they are even included in the process) in a tough position. Rather than address the true causes of user error (visibility into the system, error recovery, confusing UI, etc.) the new interfaces will be forced to combine the digital and physical. It is quite possible that their solution will end up with a situation similar to the big red button. The physical controls may be redundant, and unused. Meanwhile, the underlying design flaws could remain unaddressed.
To further pick on Galinis, listen to how much nuance his description of the Navy’s technology conveys:
“We really made the helm control system, specifically on the DDG 51 class, just overly complex, with the touchscreens under glass and all this kind of stuff.”
This apparent lack of design vocabulary scares me to the bone. Again, I want to give Galinis the benefit of the doubt, but this is one of the few quotes I can find where a leader within the Navy publicly acknowledges that maybe design played a part in the McCain accident. If there was a more insightful view of the role of design being evangelized by prominent voices within the Navy, trust me, I would amplify it. I’ve looked and it simply isn’t there. This is the dog that isn’t barking.
In the 57 page marine accident report filed by the National Transportation Safety Board (NTSB) titled Collision between US Navy Destroyer John S McCain and Tanker Alnic MC, the NTSB has this to say about touchscreens:
The NTSB concludes that the design of the John S McCain’s touch-screen steering and thrust control system increased the likelihood of the operator errors that led to the collision. Therefore, the NTSB recommends that the Navy ensure that the modernization of complex systems, such as steering and control systems within the IBNS, incorporates the design principles set forth in ASTM International Standard F1166, Standard Practice for Human Engineering Design for Marine Systems, Equipment, and Facilities.
Based on this glowing endorsement of the F1166, let’s see what the international standards are for touchscreens. On the right is page 157, which is nearly the entirety of its advice for designers implementing touchscreens. Remember, this is the international standard for human engineering design for boats.
The only other mention of touchscreens comes on page 156 in a table weighing the pros and cons of non-keyboard input devices. It lists the advantages as not needing a separate input device and being fast. The disadvantages are low resolution, a finger can block view, fingerprints on the screen, and it tires the arm.
I don’t need to explain how much touchscreens have evolved since 1988, but the F1166 hasn’t been changed in the 32 years since it was released aside from minor edits. Although it was revised in 2007 and 2013, it is fundamentally a relic of Pre-Windows 95 design patterns. To allow this document to remain as the only guidance cited for interface design is an invitation to disaster.
Two years after the accident, the National Transportation Safety Board (NTSB) released a report detailing the findings of their investigation. Conspicuously absent from their recommendations was any discussion about user interface design. Of their seven safety recommendations, six involve training. Rewritten in human-readable language they essentially said:
1. Let the computer drive, and only let humans take control in an emergency.
2. When your boat is out of control, use the radio to broadcast your emergency so other boats don’t hit you.
3. Steering and thrust are important controls, so you might want to write down instructions so that people can learn how to use them.
4. After you write those instructions, put them in the manual so we have something to point to next time this happens.
5. Train sailors, then make them take a test to prove they know how to drive the ship. Don’t forget to train them on steering and thrust.
6. There are existing standards and certification processes for ships. Use them even if they are outdated to the point of irrelevance.
None of these address design. There was only one recommendation that could potentially point at a problem with the design of the ship’s controls. I will quote this one verbatim. It said,
“Ensuring design principles in ASTM International Standard F1166 are incorporated when modernizing complex systems such as steering and control systems within the Integrated Bridge and Navigation System.”
If you were hoping the 228 page F1166 document will contain guidance based on sound design principles, I am sorry to disappoint you. Take for example the international standard’s recommendation for checkboxes.
13.13.8 Check Boxes—Check boxes shall have two states, selected and unselected. Check boxes shall be provided if a user must be able to select any number, including all or none, of a set of options. Users shall be able to toggle selected and unselected states on a check box using either a pointing device or the keyboard.
13.13.8.1 Check Box Appearance—Labels shall be provided for each set of check boxes and the style and orientation should remain consistent for groups of check boxes within an application and across related applications. Check boxes shall be arranged in logical order so that the most frequently used boxes are at the top or at the left, depending on how the boxes are oriented. See Fig. 137, “Check Boxes.”
Touch is not even listed as a method for toggling a checkbox because touchscreens didn’t exist when this spec was first written. I can’t confirm this, but my research leads me to believe that touch was the only method of input on the McCain’s controls because there doesn’t seem to be keyboards or a mouse at any of the ship’s stations.
I doubt that the F1166 document was consulted when the McCain’s controls were designed, but it is hard to see how the outdated recommendations could have saved the ship. Specification documents tend to be created less for guidance than they are to have something to point at when things go wrong. Specs make leaders feel good, they give lawyers something to point at to prove incompetence. True design is not about covering your butt, however. Design requires careful thinking and intentional decisions.
The checkbox that should have told the sailors whether or not the propellors were ganged technically got the job done, but was it the best way to indicate that controls are linked? Rather than redesign the interface with incomplete data, I would like to point out some of the questions that a designer should be thinking about in this tiny, but critical interaction. Rather than reference a specification document you need to put yourself in the user’s shoes. That sounds like:
How can I make it clear when the sliders are ganged? When unganged, how can I show the difference in thrust coming from the right and left? Could I give an indication that the mismatch is causing an imbalance that is pushing the ship right or left? Should this trigger a warning state, perhaps even broadcasting the information on the big screens at the front of the ship? Could a single button instead of a checkbox reduce the cognitive load of unlinking the propellors? Is there a symbol that could represent ganging and unganging so that a sailor could understand the action without knowing the term “gang?” Where should this be placed so it isn’t confused by surrounding buttons and checkboxes?
As a designer weighs these ideas, prototypes should be created to test the different potential solutions. Usability tests are crucial because they validate that the equipment will work in the real world, not simply in the mind of a fallible designer. When usability testing is skipped, which was almost certainly the case with the McCain’s interface, the risk of error is high.
Designers are often mistaken for decorators who simply make things look better. This misunderstanding can lead to the exclusion of designers from critical decisions and results in interfaces that don’t take into account the humans at the center of the computer interactions. It wouldn’t be surprising to learn that the McCain’s controls were created without a designer’s input at all. Without an understanding of design, an inspection of theMcCain’s user interfaces would conclude that everything functioned properly. The checkbox did its job, as did all the other sterile pieces of the interface.
We should remember that the goal is not better documentation of design rules, the goal is a better, safer end product. That is why I put so much emphasis on human centered design. As I review the documents, quotes, and recommendations post-McCain, what I am really looking for is evidence that there is a culture where design thinking is present. That is very hard to find.
The F1166 is the only design specification mentioned in the reports following investigations into the McCain accident.
Design criticism is completely absent from the 96 page Strategic Readiness Review. The 177 page Comprehensive Review of Recent Surface Force Incidents (CRR) comes closest to acknowledging the important role design plays in ship safety. In a section titled Human Systems Integration (HSI) and Human Factors Engineering (HFE) it admits:
“In IBNS, the selection of control types (e.g., discrete controls, such as physical levers, buttons, and knobs, versus touch screen controls), their spatial arrangement and density, as well as use of color schemes to clearly indicate out-of-normal conditions were inconsistent with best practices in industry for safety critical control panels.”
It goes on to confirm my suspicions that usability testing was neglected:
“For safety critical controls interfaces, issues like these should be prevented through upfront analysis of human-machine-interface requirements and validated through qualification testing in advance of equipment delivery. If thorough human factors assessments, land-based testing, and design qualification are considered too expensive or time consuming, then modernization of these controls systems should not be undertaken.”
And I applaud this insight even if it is written in lawyer-speak:
“When considering the effects on operator cognitive loading, ability to make decisions with an uncalibrated degree of trust in automation, and potential increases in frequency of error, even modernization only intended to improve reliability can have the opposite effect when the whole human-machine system is assessed.”
There is one potential glimmer of hope hidden in the CRR:
Naval Surface Warfare Center Dahlgren was identified as a center of excellence for human factors engineering within the Navy design community. Discussions with their leading experts revealed they had not been involved in specific reviews during modernization of Bridge system.
Apparently there is a Navy design community and conversations are happening that might lead to improved design literacy across the fleet. But while these statements finally hit the mark, they are only a few paragraphs lost among hundreds of pages of dense documentation. The vocabulary of design professionals doesn’t appear anywhere in the post-accident reporting. The term “user experience” isn’t used once. The term “usability” isn’t used once. The term “design” is used, but only in the generic sense. Design thinking appears to be the needle lost in the haystack.
Few of us will ever find ourselves steering a Navy destroyer. We will probably never sit in the cockpit of a Boeing 747. And yet every day we interact with technology with the capability to give and revoke life. Whether it is our car, our smartphone, or our vocabulary, we wield the tools that are changing our world. When we are lost in the opaque we are destined to be crushed in cars, disintegrated in planes, lost at sea–helpless and oblivious to how the tragedy came about. When our minds are in tune with our tools, user zero has the power to reshape even the darkest dystopia into something better.
We are entering the final pages of this book. Equipped with the lessons of User Zero, you will begin to see alternate realities where others can only accept the story as it is presented to them. Like the McCain tragedy, some people will see unexplainable incompetence while you see an intricate series of preventable design flaws. In essence you are seeing the world in higher fidelity, it is as if there is a map overlaying the world.
There is one last obstacle to overcome before User Zero concludes. The last step is for us to confront our ego. The hardest task has been saved for last. Are you capable of committing the final act of self destruction? Stay tuned for the next chapter.