When is more less? - Chapter 9

What does claim 4 cover?

We often have to make sense of highly uncertain situations, but uncertainty isn’t always reduced by gathering more and more information. Sensemaking is not just a matter of connecting the dots. Sensemaking determines what counts as a dot.

Our mind aren’t computers. They don’t connect dots in the same ways that computers do. Uncertainty is caused by not having the information we need. Therefore, adding more information will reduce uncertainty. That looks simple enough for this claim:

Claim 4: We can reduce uncertainty by gathering more information.

There are different types of uncertainty. Sometimes we are uncertain because we don’t have the information we need. That’s the type of uncertainty that claim 4 covers. Sometimes we have the information, but we don’t know if we can trust it. Sometimes we trust it but it conflicts with other information we also believe. And sometimes we believe it, but we can’t figure out what it means.

Claim 4 covers only the first type of uncertainty, which stems from missing information. When we are faced with the other types of uncertainty, adding more information may not help at all.

If I’m going to doubt the accuracy of any information I receive, adding more information just gives me more to doubt. If I believe in a data point but it conflicts with others, then adding data may add to the conflicts instead of resolving them. And if the situation is too complex, adding more data may increase the complexity , not decrease it.

A useful way to think about uncertainty is to distinguish between puzzles and mysteries. A puzzle is easily solved with the addition of a critical data point. Example: Where is Osama Bin Laden?

A mystery isn’t solved by critical data. It requires more analysis, not more data. If we want to know what the future will bring to Iraq, no data point will give us the answer. No amount of data will eliminate our uncertainties about whether China is an potential business partner of the United States or an inevitable military, political and commercial threat.

Claim 4 aims to solve puzzles, not mysteries. Mysteries emerge from ambiguous and complex situations. Mysteries require sensemaking. Adding more data doesn’t necessarily improve success in resolving mysteries. Well-ordered situations abound with puzzles.

What does the saturation point show?

Too much information can make things worse. As we add more and more information, the value of each successive data point gets smaller and smaller while the strain of sorting out all the information keeps increasing.

We may reach a point where the additional information gets in our way. We would do better to stop gathering more information before this point, but most of us keep seeking more data. We become data junkies.

The initial data are helpful, but we then reach a saturation point. After that little is gained by the additional data, which may create more than less uncertainty. Stewer et al. found that, after a point the more information the forecasters had, the worse their reliability and skill in using the information.

Result experiment Omedei et al. (2005) came to this conclusion in a study of firefighters: The team with incomplete information performed better than the teams with detailed information.

Therefore, even though the studies are nice showing that people reach a saturation point and their performance gets worse when we drown them in too much data, in real-world settings experts usually can protect themselves. They self-select which types of data to seek, whereas the novices try to use all the data.

The biggest danger of claim 4 (that we can reduce uncertainty by gathering more information) is that it invites a mindless search for more data. It invites a search for anything that might be relevant. It invites abuse, encouraging us to move past our saturation point. Even when we control our own searches for information we tend to gather more than we need, and the extra information can get in our way.

The saturation point, the point at which our attempts to resolve uncertainty by getting more information become counterproductive, is technically referred to as information overload. Sutcliff and Weick argue that information overload reduces our ability to take a good perspective on events, or to identify which cues are the most relevant, because we gets distracted by all other cues.

Information overload isn’t just having too much information. After all, we already have more information than we can absorb, but it doesn’t bother us as long we can skillfully redirect our attention. Information overload, the saturation point, becomes problematic when we have trouble focusing our attention to the relevant signals.

That’s what happens when people interpret claim 4 as an invitation to turn on the information spigot in order to reduce uncertainty. They start by collecting more and more data, then they complain that they have too much. It is never perfect. Not too much, not too less

We can label this as the Goldilocks Effect. The Goldilocks effect assumes that there is a ‘right’ amount of information, that we can know what it is, and that we can aim for it the way we might set a thermostat.

None of these assumptions holds. There is never a right amount of information.

People sometimes try to filter strategies to get the right amount of information. Information filters let us design a search that returns mostly relevant material. The idea of setting filters comes from the Goldilocks effect.

Goldilocks effect: A belief that perhaps we can find just the right amount of information.

We would be better off if we stopped worrying about getting the right amount of information and instead tried to see the meaning in the data that we do have.

Examples: 9/11 and Pearl Harbor.

Why? Could the events of Pearl Harbor and 9/11 have been thwarted by additional information? Perhaps, not necessarily. At the time, additional cues would have been ignored, lost in the noise and suppressed by the disbelief.

If the United States had been able to double the amount of information it had, the extra information would have doubled the messages that weren’t truly relevant, and the connections between all these messages would have gone up exponentially. Adding more information wasn’t going to prevent these events unless there was also a way to analyze the messages better.

All these examples show how decision makers ignore weak signals or explain them away when the signals didn’t fit with the prevailing mindset. The data was there, but the sensemaking broke down.

What are unintended consequences?

We will gather more data. The more the uncertainty, the more strenuous the data gathering. We won’t stop until our certainty disappears or becomes exhausted, whichever comes first.

Because of the potential for overload, we will have to invest in informational technology to collect and categorize and analyze all the data. And we’ll have to store the data in a way that will let us retrieve what we need instantaneously. (as long as we remember it and the label we used to store it). We will also have to develop intelligent filters to figure out what is relevant and what we can ignore.

Replacement

The claim that we can reduce uncertainty by gathering more information is sometimes accurate but sometimes misleading, especially under conditions of complexity. It is accurate for cases in which uncertainty stems from a lack of data, as opposed to conflicting or ambiguous data.

When dealing with a mystery, instead of a puzzle, we enter the realm of complexity. Under complex conditions we need to manage uncertainty more than we need to reduce it. To manage uncertainty we have to know how to seek and prioritize information. The replacement for the claim: In complex environments, what we need isn’t the right information, but the right way to understand the information we have.

Claim 4 emphasizes the quest for more information, whereas in facing mysteries people need to focus on the ways they are interpreting and anticipating events. We have to focus on understanding, not collecting!

Conclusion Chapter 9: If information is good, more information is better, pushing us past the saturation point. We devote more energy, resources, and funding to gathering more information than sorting it out. Organizations put resources into acquiring and cataloging more information rather than into improving the way the information gets shared and integrated.

The next chapter will examine the process of sensemaking. Sensemaking is understanding situations by finding connections between the data.

Claim 4: We can reduce uncertainty by gathering more information.

There are different types of uncertainty. Sometimes we are uncertain because we don’t have the information we need. That’s the type of uncertainty that claim 4 covers. Sometimes we have the information, but we don’t know if we can trust it. Sometimes we trust it but it conflicts with other information we also believe. And sometimes we believe it, but we can’t figure out what it means.

Claim 4 covers only the first type of uncertainty, which stems from missing information. When we are faced with the other types of uncertainty, adding more information may not help at all.