Appendix D: Findings From the Wildland Firefighters Human Factors Workshop, 9551-2855-MTDC

Findings From the Wildland Firefighters Human Factors Workshop

Appendix D—Keynote Presentations

Naturalistic Decision Making and Wildland Firefighting*

Gary Klein, Ph.D., Klein Associates Inc.
582 E. Dayton-Yellow Springs Road
Fairborn, OH 45324
(513) 873-8166
August 8, 1995

The Recognition-Primed Decision Model describes what people actually do when they make difficult decisions. This has many implications for training and helping people make decisions under stressful situations. It can also help explain the factors behind bad decisions.

The standard method of decision making is the rational choice model. Under this model, the decision maker generates a range of options and a set of criteria for evaluating each option, assigns weights to the criteria, rates each option, and calculates which option is best. This is a general, comprehensive, and quantitative model which can be applied reliably to many situations. Unfortunately, this model is impractical. People making decisions under time pressure, such as fire fighters, don't have the time or information to generate options and the criteria to rate each option.

The rational choice model is also too general. It fits each situation vaguely, but no situation exactly. The worst news is that in studies in which people have been asked to follow the rational choice model exactly, the decisions they come up with have been worse than decisions they make when they simply use their own experience base. This model is of little value to training because it does not apply to most naturalistic settings or to how people actually make decisions when faced with complex situations under time pressure. Decision aids which have been produced to assist with the application of the rational choice model have been largely ineffective. Because of these drawbacks, a field emerged called Naturalistic Decision Making (see Table 1). This field emerged because governmental sponsors such as NASA, FAA, the military, and others, realized that they had spent a lot of money and built decision models that did not work in the field. They wanted to get away from building analytical models which didn't work when they were brought into action. Naturalistic Decision Making uses expert decision makers, and tries to find out what they actually go through in their decision making process.

Positive Features	Contrasts
Studies people with expertise Tries to describe Takes a broad focus Task context: field settings Time pressure Shifting conditions Unclear goals Degraded information Subtle cues and patterns Team interactions Organizational constraints High stakes Focus on cognitive processes Relies on Cognitive Task Analysis	Studies novices Tries to evaluate Takes a narrow focus Task context: laboratory settings Ample time Stable conditions Stated goals Precise information Clear inputs Individual tasks Individual tasks Low stakes Focus on analytical strategies Relies on performance measures

Instead of restricting decision making to the "moment of choice," experts are asked about planning, situational awareness, and problem solving to find out how these all fit together. This model is used to understand how people face decisions in shifting and unclear situations and under high stakes. Team interactions and organizational constraints with high stakes are also used as factors. For years, researchers had been simply asking college sophomores what they would do given a set of options, and a clear goal. For Naturalistic Decisionmaking research, experts are asked to size up actual situations, using all cues and constraints to set goals and make decisions.

The first study I performed to generate models and training recommendations for decision making under pressure and certainty was a study for the Army. The Army Research Institute wanted some data on decision making in real, stressful situations, and I thought that urban firefighters would be a good example of people who had become experts at making such decisions. We studied commanders who had about 20 years of experience, and studied the most difficult cases they had. Of the cases we studied, there was an average of five changes in the fire and in the way it had to be handled. About 80 percent of the decisions were made in less than a minute. As we started the study, we found that each expert firefighter told us that they had never made any decisions. They explained that they simply followed procedures. But as we listened, we realized that in each case, there was one option which they thought of quickly. They evaluated that one option, and if it seemed viable, they went ahead with it.

We began to wonder how they came up with that first option and how they were able to evaluate one option without others for comparison. The strategy used by the firefighters is the basis for the Recognition-Primed Decision (RPD) Model (see Figure 1). The first level consists of a simple match, where decision makers experience a situation and match it to a typical situation with which they already have experience. Because of this, they know what to expect. They know what's going to happen, they know what the relevant cues are, what the plausible goals are, and a typical action. They are able to do all of this because of their experience base. Experience buys them the ability to size up a situation and know what is going on and how to react. That's what decision researchers weren't learning when they studied college sophomores who didn't have an experience base.

[image] Chart showing the three levels of the Recognition-Primed Decision model.
Figure 1—Recognition-Primed Decision model.

An example of the first level of the RPD model is a firefighter I interviewed early in the process. He explained to me that he never made decisions. After trying to press him on the issue, I asked him to describe the last fire he was in. He told a story of a fairly conventional fire. He described parking the truck, getting out his hoses, and going into the house. I asked him why he went into the house instead of simply working from outside, as I would have been tempted to do. He explained that he obviously had to go in because if he attached it from the outside, he would just spread it deeper inside the house. He took into account the nature of the fire, the distance of the house from other buildings, and the structure of the house. But, even while he was attending to these conditions, he never saw himself as making a decision. He never experienced that there was another option. He immediately saw what needed to be done and did it.

The second level of the model includes diagnosing the situation. On this level, expectancies are violated. The firefighter is trying to build a story to diagnose the event, and when evidence doesn't fit the story, the firefighter has to come up with a new scenario which fits the new evidence. There is still no comparing of options.

On the third level, decision makers evaluate the course of action they have chosen. Originally, we weren't sure how people could evaluate single options if they had no other options to compare it to. As we looked through the materials we were getting, we found that a decision maker would evaluate an option by playing it out in his/her head. If it worked, they would do it, if it didn't, they would modify it, and if modifications failed, they would throw it out. In the incidents we studied, commanders simply generated each option and then evaluated it for viability. Usually the first option an experienced firefighter generated was a viable option, but they also understand that they should simply be satisfying, not optimizing. They will not necessarily pick the best option. They will pick the first one which is possible and involves minimal risk. The first viable option is chosen and improved upon, if necessary. It is not compared with all other options to see which one will be best. As soon as it is deemed viable, it is chosen and applied.

Naturalistic Decision Making has implications for training. Decision training needs to teach people to deal with ambiguous, confusing situations, with time stress and conflicting information. Situation awareness, pattern matching, cue learning, and typical cases and anomalies can be taught by giving people a bigger experience base. Training could teach decision makers how to construct effective mental models and time horizons and how to manage under conditions of uncertainty and time pressure.

Methods for providing better training include changes in such things as ways of designing training scenarios. Another strategy is to provide cognitive feedback within After-Action Reviews. This would do more than point out the mistakes which were made in an exercise. It would be an attempt to show decision makers what went wrong with their size up, and why. Another method would include cognitive modeling and showing expert/novice contrasts. This would be done by allowing novice decision makers to watch experts. Novice decision makers would also benefit by learning about common decision failures. On-the-Job Training should be emphasized rather than simply assuming that once the traditional training is finished, decision makers are ready to begin to function proficiently. Test and evaluation techniques and training device specification could also be improved. All of these might have an effect on the ability of firefighters to deal with stressful situations.

Why is it that people do make bad decisions? I looked through a database of decisions to identify reasons behind bad decisions. We came up with 25 decisions which were labeled as poor. Of those, three main reasons for bad decisions emerged. By far, the most prominent reason was lack of experience. A smaller number of poor decisions were due to a lack of timely information. The third factor was a de minimus explanation. In this situation, the decision maker misinterprets the situation, all the information is available, but the decision maker finds ways to explain each clue away, and persists in the mistaken belief.

The problem of lack of experience has many effects (see Figure 2). Inexperienced decision makers lack the understanding of situations to be able to see problems and judge the urgency of a situation, and properly judge the feasibility of a course of action. These are skills which could be developed to improve decision making.

[image] Chart showing how poor decisions come about.
Figure 2—NDM factors that———> poor decision outcomes.

The field of Naturalistic Decision Making research is more appropriate than traditional decisionmaking models for understanding how crisis managers, such as firefighters, handle difficult conditions such as time pressure and uncertainty. We have broadened our focus from the moment of choice, to take into account situation awareness, planning, and problem solving. By so doing, we have gained a stronger vantage point for understanding errors and for designing training interventions.