Last week, a fascinating discovery caught my eye. “Astronomers detect an ‘ultracool’ brown dwarf star burning at roughly 800 degrees, or cooler than a typical campfire; object is the coldest star on record still emitting radio waves”.1 These kinds of elementary discoveries are what kindled my love of science and keep it burning even today. I find it amazing—truly I am at a loss for words—that we can still make such simple scientific advances. Another news article illustrates this idea: “Developing human embryos imaged at highest-ever resolution — Non-invasive imaging approach could lead to innovations in embryo screening”.2 I had no idea that such a simple problem had evaded solution. The photographs are wonderful, and they stir something even deeper than the first story in my heart and mind. This kind of joy is what I felt when I started in on the STPA Tutorial video. Discovery and learning appeal to many of us in this way. I imagine standing on an intellectual Pike’s Peak and looking over the landscape of this subject matter, something we’ve heard about for several years now. But now we get to see it with our own eyes.
For the past few weeks, I’ve used my treadmill running time to watch the STPA tutorial from the most recent Flight Test Safety Workshop in Wichita. I recommend it. It might also benefit you to watch the videos at a faster speed. (It turns out the human body can hear—and absorb what it hears—much, much faster than the human body can speak.) I didn’t realize I could change the playback speed until I was near the end of the first of three tutorials, probably a consequence of watching it on my phone. While you are on the FTSC website, under the Workshops tab there are other videos to watch too (https://www.flighttestsafety.org/2023-wichita-ks?start=15).
STPA means System Theoretic Process Analysis. It’s a topic we’ve covered in these pages before, and I encourage the reader to peruse or thoroughly review the previous articles. There is an incomplete list of these articles at the bottom of this column.
Darren McDonald opens the tutorial by introducing Daniel “Mirf” Montes, Lt Col, USSF, and Sarah “Poncho” Summers, Lt Col, USAF. Poncho gives some contextual background about STPA and Mirf gives an overview of several “systems” for which STPA has been applied, examples to motivate the discussion. Poncho then walks through an overview of the steps one would use in doing an STPA analysis. The second video is a flight test application: STPA Tutorial Part Two – “As Applied to a Boeing Automated Test Maneuver.” And the final video is something I haven’t watched yet.
These videos are meat and potatoes. Get ready to do some mental gymnastics, and if you intend to follow along in the exercise, you might actually break a sweat. STPA is not for the faint of heart.
But this is also a huge part of the value proposition of the Flight Test Safety Committee. Where else can you rub shoulders with PhDs who studied this topic in a convent and mastered the way (okay, I’m exaggerating a little bit)? Where else can you get feedback from your peers across the flight test profession, from both the military and civil domains? And if you have forgotten, the purpose of this newsletter is to point to the resources that the FTSC have compiled on their website, something Tom Huff did with STPA digital resources in his coverage of the STPA topic (see “For Further Reference” below).
I appreciated the depth of coverage of the material, but this is also the first weakness of STPA. It’s not something you can teach yourself, and because of its nature, if you try to apply it at your work, there are pitfalls awaiting you. The speakers try to fit a lot of material into a one day tutorial, but their expertise and passion for the subject are evident. Several of us reached out to the speakers for supporting commentary to this review but were unsuccessful. [UPDATE: Poncho responded to my inquiry here.]
I am still an STPA skeptic, and some of our past reporting has highlighted some of the reasons for my doubt. A second reason is even more simple: what is the value proposition of STPA? It seems like a lot of work, and I’m not sure I understand what it does that our existing methods do not do. I’m also certain that the intentional exclusion of probability from the modeling is a fundamental flaw. That is a topic for another time, but I will offer one point of order: we must understand the difference between statistics and probability and use our words carefully to say what we mean when talking about these topics. I believe this statement applies to all of our flight test endeavors. I do have a specific technical question though for anyone who has implemented STPA: How robust is the “analysis process”? What I mean is, if my system model is missing something (due to the ignorance or oversight by the test team about a new system under test) is the STPA tool strong enough or flexible enough to accommodate?
In closing, I recommend the videos and thank those who prepared the tutorial. It will make us all better, strengthening our intellect and safety in the process if we wrestle with it the way we ought.
For further reference
| FTSF 20-06, Trip Report – the vFTSW, by Pete Donath | FTSF 19-02, “It Didn’t Work” |
| FTSF 19-11, Chairman’s Comments, by Tom Huff | FTSF 19-03, Don’t Rule Out STPA, Douglas “Beaker” Wickert |
| Safety Planning Using Applied Systems Theory (with slides), by Daniel Montes |
Footnotes