How can decisions be automated? - Chapter 8

Why do decision-support systems fail?

Many decision-support systems are a fail. The developers think that people will welcome decision aids that make it easier to conduct decision analyses. The developers often can’t believe that decision makers are too stupid to appreciate all the benefits of such decision aids. But decision makers are not rejecting the aids. They are rejecting the mindset that keeps churning out these kinds of systems. Decision makers are rejecting the mentality that idealizes reflective, analytical thinking and marginalizes automatic, intuitive thinking, instead of blending the two kinds of thinking.

Those who advocate for a balance between the automatic (intuitive) system and the reflective system are continually challenged to ensure that their organizations value their employees’ tacit knowledge.

Once the balance shifts, expertise may begin to wither away. We identified four approaches to decision-support systems:

1. The decision is made by the decision maker alone.

2. The decision maker is helped by a support system or an algorithm.

3. The decision-support system has the final say, with inputs from the operators.

4. The decision-support system makes the entire decision on its own.

Some risks in using decision-support systems under complex conditions.

The last two of these four approaches cede decision control to the equations. They are best suited to well-ordered domains that have enough structure to get the systems organized, enough stability to make the programming economical, with not too much complexity or inter-dependencies, little need for tacit knowledge, and clear criteria for good decisions.

Algorithms aren’t used where flexibility is demanded, because they are carefully tailored to narrow conditions. In addition, algorithms can get in the way when we need people to develop and sustain skill and expertise.

Algorithms can get in the way when we need people to develop and sustain skill and expertise. When the algorithms are making the decisions, people often stop working to get better. The algorithms can make it harder to diagnose reasons for failures. As people become more dependent on algorithms, their judgment may erode, making them depend even more on the algorithms.

Another disadvantage of using decision-support systems is that we shouldn’t replace human judgments with algorithms if we want to make discoveries. Even in fairly chaotic environments, human intuition may provide clues about the way different factors relate to one another. Replacing human judgment with algorithms reduces opportunities for discovery.

When are decision-support systems rejected?

If decision biases are pervasive and crippling, then various decision-support tools should improve performance. Decision researchers often formulate various kinds of decision support systems and tools to help people make better judgments and avoid biases.

Decision Analysis uses decision trees or influence diagram to structure the process. Techniques such as Multi-Attribute Utility Analysis should help us combine information about our preferences.

Yet people don’t use these systems. Again and again, researchers sadly report that people in the field rejected their systems.

Researchers who design decision aids don’t worry about most of these issues. Their goal is usually to help people eliminate inconsistencies. Decision analysts worry about inconsistencies because the analysts have been able to demonstrate that people aren’t as consistent as statistical formulas.

Yates et al found a problem. They found that decision aids make the decision process unnatural and difficult. The decision aids are trying to change and improve on the way we think, not to support or amplify our thinking. We are responsible for the outcomes, not the analyst or the method.

People who develop decision support systems find it convenient to simplify the complexities that surround so many difficult decisions. The developers may get the details right, but may miss which way the wind is blowing.

What is the essence of project Phoenix?

Decision makers have been studied as carefully as weather forecasters. They make predictions every day. They make short-range predictions, long-range predictions. They are very accurate in making 1-2 days predictions. A problem is that good weather forecasters have one big drawback: They cost money.

By building in decision support systems, organizations can get away with hiring relatively inexperienced forecasters and still hope for the same level of quality. That’s why Canadian forecasters are ordered to use large-scale programs providing Numerical Weather Predictions. These are computer based systems to generate forecasts.

This system isn’t perfect. What was project Phoenix? Experiments to look if skilled forecasters did a better job on their own, so without a automated forecasting system, or that the unskilled people using an automated forecasting system scored better.

The result was that unskilled forecasters had to go through training to upgrade their skills, because that was better than relying only on the system.

Snellman (1977) used the term ‘metereological cancer’ to warn about the trend for forecasters to cease being professionals and instead rely on computer-generated information.

When decision support systems are designed by people who are insensitive to tacit knowledge, their devices and strategies are likely to interfere with expertise rather than support it.

If we try to balance the human-as-hazard model with the human-as-hero model, and to balance the automatic, intuitive system with the reflective, analytical system, we should have more of a chance to create decision support systems that will get used.

Many decision-support systems are a fail. The developers think that people will welcome decision aids that make it easier to conduct decision analyses. The developers often can’t believe that decision makers are too stupid to appreciate all the benefits of such decision aids. But decision makers are not rejecting the aids. They are rejecting the mindset that keeps churning out these kinds of systems. Decision makers are rejecting the mentality that idealizes reflective, analytical thinking and marginalizes automatic, intuitive thinking, instead of blending the two kinds of thinking.

Those who advocate for a balance between the automatic (intuitive) system and the reflective system are continually challenged to ensure that their organizations value their employees’ tacit knowledge.