Paper: The Psychology of Incident Investigations
This week's paper was yet another nerdsnipe. This is Sidney Dekker's The psychology of incident investigations: epistemological, preventive, moral and existential meaning-making. That's a mouthful of a title, but rather accurate since in this text Dekker looks at all four items and their respective relationships within any single incident review.
This paper starts with a reference to how incident investigations can be politically motivated in a general sense—about pursuing agendas and protecting interests. Different agencies with different purposes and priorities will, with two equally competent authors analyzing the same incident based on the same facts and material, come up with divergent stories and conclusions. He mentions, for example, two investigations of the same 1995 aviation incident. One was written the airline whose aircraft crashed in the mountains, and the other by the civil aviation authority of the country in which the accident occurred (and who employed the air traffic controller whose airspace this was in):
The main differences are obtained by either choosing to highlight or omit different aspects of the accident, and Dekker points out they on their own represent a call for "plurality and multidisciplinarity in the investigation process so that multiple voices are heard and given legitimacy", which should help make it more visible where lines of reasoning come from, for example.
Now back to the main thesis of the paper. The author mentions that psychological factors may as well be in play, and compete with each other to an even greater extent than the political ones:
Where the political explanation almost necessarily relies on competition between different parties or stakeholders, the psychological one suggests that competing meaning-making functions of accident inquiries can exist within single agents (e.g. an investigator) This can lead to possible compromise and contradiction in the narratives suggested by them even before they are submitted to political scrutiny and control.
The four terms in the title can be defined as follows:
- Epistemological: explain what happened (causes, effects)
- Preventative: explain how to avoid recurrence (ask for alterations)
- Moral: explain transgressions, and reinforce moral and regulatory boundaries (refer to norms and instruct)
- Existential: explain the suffering that occurred (outrage, demand improvements to avoid)
We'll visit them one by one.
Epistemological purpose
The Epistemological purpose of an investigation is to explain what happened. The assumption is that by digging deep enough, pulling on all the loose ends, you can get a fully analytical view of events. The explanation must be accurate, exhaustive, but also plausible by linking cause and effect in a believable way. Historically, this led to linear explanations, noting that effects and consequences required some amount of proportionality: broken parts can explain broken systems, and (if I can paraphrase) human error or loss of situational awareness can explain mishandling a situation.
Dekker points out that in complex systems, a lot of effects emerge from the interaction of multiple parts—often normal, functional ones—rather than broken ones. Additionally, there's a limit to what can be taken into account for any specific description, which leads him to say:
a 'good' epistemology of a complex system does not claim that one description is true and that all others are false, but rather that multiple descriptions can and must be made at the same time — partially overlapping and contradictory. In complex systems, there is no proportionality between cause and effect. And by extension, a complex epistemology does not commit to a Newtonian proportionality between cause and effect. Small 'causes' can be amplified hugely by the normal interactions and interconnections and multipliers in a complex system.
In short, because we can't get a full picture, a contrasting view of multiple perspectives tends to be better at properly describing what happened. Some of these view points may contradict each other, and many may sometimes be regarded as true at the same time. This, Dekker points out, may clash with the three other psychological purposes.
Preventative purpose
The next psychological purpose is the Preventative one, which is often seen as the most important one: making sure this does not happen again. The paper uses the terms explanatory variables for the things in the epistemological aspect of the report that explain the failure, and change variables for things that could be amended or changed to prevent the failure next time.
The psychological and epistemological purposes can often align well, but not always: a focus on explanatory variables can sometimes lock attention on specific instances of issues in a way that makes prevention more difficult. An example given is one of hand injuries in repairing megatrucks used in the mining industry:
Careful investigation of each instance revealed explanatory variables: the variety of places in the machinery where hands were most likely to get hurt, during which procedures, and with what shortcuts and improvisations that contractors applied. What these insights and subsequent interventions (e.g. more stringent procedural compliance demands) did not do was prevent the same injuries from occurring, nor did they reduce the injury frequency. [...] The change variable introduced in the business [that finally was effective] was to significantly invest in preventive maintenance — replacing parts and systems on specific (and short) intervals rather than running them to failure. Preventive maintenance takes places in hangars and workshops — controlled, well-lit, better-resourced and well-equipped environments where injury risk has traditionally been much lower. Explanatory variables, in other words, had little sway over the eventual change variable.
Basically, knowing what went wrong in a specific incident does not give you surefire improvements, and you may have to try many things before any works (which means you may have to have many repeat incidents).
Moral purpose
The moral purpose of an incident investigation is mostly one of maintaining boundaries on behaviour:
Reports [with a moral purpose] act as boundary-maintaining devices in the sense that they demonstrate where the line is drawn between behavior that belongs in the special universe of the group and behavior that does not
They highlight deviance to perform cultural work: transgressions being pointed out while reaffirming and clarifying rules or ways of accomplishing work can be useful to a group of practitioners. Incident reports of this kind will often make allusions (to professionalism, for example), denounce acts or deviations, and provide instructions.
Dekker has few positive words for this, saying that this type of language is often disguising judgments from bureaucratic enterprises patrolling their own boundaries as if they were epistemological causes. Rather than seeking to understand the mechanisms behind transgressions, the moral purpose helps reinforce and protect the organization's structure.
Existential purpose
The last psychological purpose is the existential one, which aim to give meaning to suffering. By identifying risk and making it understandable, it gives the hope of containing or controlling it. This is because from this point of view, suffering should not happen, and it should be possible to eliminate it (the author points at zero vision type programs as an example here). This approach tends to link suffering due to accidents to some moral calculus, even in preventative cases.
The issue here is that when you don't find human choices or transgressions, or that you can't find anything that's broken, the investigation becomes upsetting and feels broken even if all the epistemological rules were followed. Citing Snook's report on a friendly fire accident in Iraq in 1994, every time the investigators found something that seemed like a failure and started digging, he kept only finding people doing normal things as part of normal work:
[Not finding transgressions] becomes unsatisfying on all other counts. It may be a really good (epistemological) story, a story of complexity and emergence. But how do we control risk, how do we prevent recurrence? Whom do we hold accountable? And how do we explain the suffering caused by 28 deaths if there is no ‘cause’ to point to? Was that suffering in vain? It is in the conflict between these psychological purposes that accident investigations have to make meaning out of a bad outcome.
(note: I checked the sources and I believe the actual number of casualties was 26, not 28—I even quickly searched for things like 2 people dying out of grief/self-harm adding to the official figures and found nothing)
Conclusion
Dekker repeats that all psychological purposes can and do play a role. The problem, however, is that they are not all compatible with each other. The epistemological purpose may align well with the preventative one, though it sometimes turn it bothersome or shows prior interventions weren't effective. Specifically though, the epistemological purpose is in a far more direct tension with both the moral and existential purposes. If you are forced into more complex narratives, conflicting views, and away from more obvious broken boundaries, then there are few ways to hold people accountable, and few ways to reassure ourselves it won't happen again.
He concludes:
Perhaps the lesson is this. We should desist from seeking the meaning of suffering in the past, in explanatory variables. We should locate it in the future, in change variables. We should demand from investigations not a backward-looking, but a forward-looking accountability. We should seek solace not in trite (and surely false) assurances that ‘this will never happen again,’ but in an understanding that error and failure are inevitable by-products of pursuing success in a resource-constrained, goal-conflicted world.
At the very least, disambiguating the four psychological impulses that guide incident reviews might be a good starting place.