Paper: Ecological Interfaces: A Technological Imperative in High-Tech Systems?
This week's paper is from Jens Rasmussen and Kim J. Vicente, titled Ecological Interfaces: A Technological Imperative in High-Tech Systems?. This is a paper from 1990 that sort of explains the rationale before a much more cited framework written by the two same authors (in reverse order) that introduces Ecological Interface Design's (EID) theoretical background. EID is something borrows from a discipline called "ecological psychology" to analyze how people interact with their environment at the cognitive level, in an attempt to create interfaces in technical systems that feel intuitive and lets people improvise at a higher level without going against the system's goals.
This paper explains the borrowing they did, the reasons why, and gives an example of what it looks like in designing the interfaces of a nuclear power plant control panel.
The authors start by asserting that in the natural environment, humans evolved good ways to deal dynamically with their environment, both in terms of perception (understanding and classifying the features of elements of the environment) and of motor coordination. Some parts of it are hard-wired, and some parts of it require deliberate adaptation. This is subdivided into two further phases required when learning new tools as part of adaptation: exploratory behaviour, and an adapted phase once behavioural patterns have reached an equilibrium.
In a stable work domain with "moderate-size artifacts" (the example given is working with a radio receiver), the adaptation phase is transient and eventually it becomes a lot more automatic, cue-action based. They point out that in case of challenges, trial-and-error is usually fine: you don't need to know about the internals of tools, and the cost of errors is generally expected to be less than the value of the information you gain out of it.
In high-technology environments, however, most of the work ends up requiring conscious decision-making and planning. You can do trial and error, but because you have to continuously aim for acceptable work performance trade-offs and constantly looking for new opportunities to improve, you never reach that equilibrium. Continued adaptation is required from the workers. This is made worse in large-scale systems, which just grow the scope to people, property, and the environment, with various safety trade-offs. Because the systems are more complex, with worse possible consequences, faults can't properly be anticipated, and trial-and-error and surface features are no longer an acceptable way to develop skills, since they risk major accidents.
The challenge then is that practitioners in these environments will need to gain an understanding of the internal state and functional structure of the system:
In this way, advanced technology poses the following new requirements for interface design:
- It is mandatory that the internal functions of the system be represented at the surface, that is, the interface, in a way that they can be directly operated under familiar as well as unfamiliar conditions.
- In order to activate the normally very effective and reliable sensorimotor system, the representation used for the interface should support direct perception and manipulation of the internal control object as well as analytical reasoning.
- Design of interfaces can no longer be allowed to evolve empirically through trial and error, nor can they be based on general rules and guidelines applied by human factors experts; the interface design is an integrated part of the functional system's design and, like the latter, requires subject matter expertise.
- Finally, designers of systems must have an in-depth knowledge of the cognitive context in which the system is to be implemented, in order not to choose display forms that are in conflict with the established norms, mental models, and organizational structures as communicated by textbooks, drawings, and manuals of the profession and company practice.
That's a tall order.
The human perceptual system
As an underlying set of principles to hit the 4 points above, Rasmussen and Vicente turn to ecological psychology. The idea is to look into which types of system functions you need to expose to make more intuitive interfaces. Unfortunately this is all extremely contextual, and what they point out is that in more natural objects, most of the properties are available at the same time, and what we focus on depends on the tasks at hand.
Based on models I am eliding here (they represent the perceptive world at a conceptual level that does not attempt to explain how anatomical perception works), they create a sort of "grammar of action control", which lets them classify various affordances by levels:
- values (purpose, goals): survival, pleasure, altruism
- priorities (abstract functions): reward, comfort, privacy, danger, nutrition, etc.
- context (general functions): communication, warmth, support, punishment, eating, locomotion, etc.
- movement (physical process): sitting, cutting, lifting, swimming, throwing, barriers, etc.
- objects and backgrounds (physical form): layouts, surfaces, substances, etc.
These broad categories line up with others you'd find in a nuclear power plant (examples in category order: safety, controlling energy balances, coolant circulation and power conversion, adjust equipment parameters, and changing equipment). They state that the relationship between them is based on means-end hierarchies:
Selection of goals to pursue are related to perception of value features at the highest level, the planning of activities to perception at the middle levels, while the detailed control of movements depends on perception at the lowest level of physical objects and background.
They add that a single affordance can be used by multiple levels or values at once: higher-order context elements helps constrain how many lower-level elements are considered relevant. Here, the authors follow the argument made by ecological psychologists that natural and man-made environments are one and the same thing: whatever we build with tech is situated within the larger world, and we can frame human intervention an effort to change and expand its affordances.
Direct perception, invariants, and system design
Since the operators will need to improvise with the system to keep meeting its objectives, we should consider that "there is no clear-cut distinction between system design and operation." The problem solving done live in the moment requires considerations of the internal functioning and structure of the system.
One of the constraints here is the one of invariants. Complex systems contain them at all levels of abstraction:
The structure of the system is meant to provide a set of means (i.e., possibilities) for carrying out the higher-order goals. In addition, the invariant laws describing the operation of the system at lower levels provides further constraints that need to be considered. Respecting the intended invariants implied for normal functioning is essential, not only to maintain the desired functioning and production, but also to avoid serious accidents if changes in system structure or the behavior of individual components threaten the global invariants. [...] Consequently, operators of advanced systems are supposed to operate the system and to cope with disturbances by redesign of the operational regime in unforeseen situations so as to maintain the necessary conditions for the intended invariants of system behavior
In short, the idea here is for the interface to allow operators to take advantage of perception and action while reflecting the invariants to support problem-solving. In Ecological Interface Design, the representation must support dynamic switching among levels of focus and control, movements, and plans, simultaneously.
An example given as an analogy is music notation: sheet music can be read slowly and deliberately by new musicians who will painstakingly play notes one at a time. But the same sheet, by an adept musician, has multiple notes read at once ("higher-order chunks") and playing chords can be done directly without having to invoke thought: the higher-order chunking at the perceptual is accompanied by a chunking of movements. Similarly, the interface should be designed in a way that allows arbitrary chunking of its visual elements into higher-order routines that are themselves chunked. The key point here is that when this chunking happens, all the information at all levels is there.
This principle here is that higher-level information ends up being an aggregation of lower-level information; a hierarchical visual structure helps building skill by facilitating chunking, and flexibility is maintained by not limiting people to a single abstraction level at once. You want people to shift from visual signs and markers and toward patterns.
An Example
I'm gonna be honest, the example given is really not simple. It's the interface for a nuclear power plant, showing the temperature and cooling cycle for it. Here it is in all its glory:
To understand it, you have to be familiar with nuclear power plants, which I am not. But the authors do try to give an explanation for it, by breaking it into parts. First of all, the coolant cycle:
You can see the temperature scale to the left, and the coolant follows the circuit (T1, T2, T3, T4). So when you see that square in the full interface, it represents a temperature differential as the coolant flows from component to component (heating up in the reactor, and cooling down in the heat exchanger). This sort of explains the top-left area of the interface. For the bottom-right area, the following explanations are given.
The left side represents the water-boiling cycle. the left half of the curve is water, the right half of it is steam, and whatever is within the curve is a water and steam mixture. The diagram on the right right is the rankine cycle put together with its phase diagram. I frankly don't get most of it, aside from the authors pointing out that whatever is within the curve is safer, and what's outside of it tends to be a big danger because controlling temperatures there is difficult and accident-prone.
The key point of the explanation as far as I understand it is to show that the diagram is rooted into common theory and concepts for the operators. The core of the example, however, is geared towards the cooling cycle, where the two following figures are added, showing two point-in-time changes as coolant flow is rapidly increased within the system:
I had a bit of a hard time truly getting it, so I put all 3 of them into a sort of janky animation:
And I have to say I kind of get it. The square sizes for the coolant loops both show the change on each component (the reactor, exchanger, and super heater), and as time moves forward, you can see the cooling shift happen there, while also being able to track their relative positions. There's a bunch of lower level information, but I can imagine how the higher level motions support that "chunking" they speak of and which would let you shift attention use based on context. As the authors state it:
The display can be perceived at the level of flow of energy, of state of the coolant circuits, of the physical implications of temperature readings, and the like, at the discretion of the observer. At the same time, the display takes advantage of a standard format (i.e., a temperature-entropy plot) that is used by engineers [...] [T]his kind of graphic display can be interpreted at several cognitive levels. Since it is transparent (i.e., it represents that internal functional relationships which are to be controlled), it will support knowledge-based, analytic reasoning and planning. [...] [T]he faithful representation or the control object will invite the evolution of reliable "direct manipulation" skills.
The authors conclude that the ideas advanced by ecological psychology are of interest to cognitive engineers, and more should be done on that front. As far as I know, EID as a discipline is still going on to this day in a few critical industries, and has been particularly successful in domains where invariants are key—industrial process control or anesthesiology, for example. On the other hand, its influence isn't as great in intent-driven flows (heavy on information retrieval).
I can see the attractiveness of optimizing for "chunking" like they suggest, and it does make me wonder about the value of attempting to get ever more solid if not more opaque abstractions made in the name of hiding complexity in software.