The Confidence Trap: Balancing the Proof Burden in the Climate Conversation

RFF’s Roger Cooke discusses the confidence trap and how current conversations about climate change misallocate the burden of proof. Cooke was a lead author of the chapter on risk and uncertainty in the Intergovernmental Panel on Climate Change’s Fifth Assessment Report.

RESOURCES: What is the confidence trap, and how can it influence our thinking?

ROGER COOKE: The confidence trap is a cognitive illusion. It’s like an optical illusion but involves the brain instead of the eyes. A cognitive illusion tricks you into thinking things are there—when, in fact, they are not.

Let’s imagine, for example, that John is feeling really good and he says, “Well, today will be a great day. I’m highly confident that I won’t get a parking ticket. I have high confidence that it won’t rain and I also have high confidence that my portfolio will go up and that our team will win today.” His partner Jane says, “Does that mean you have high confidence in no ticket, no rain, your portfolio going up, and a win?” To which John replies, “Huh? That’s just what I said.” Jane says, “No, it isn’t. Here, I have a fair die with six faces and I’m really confident that if I throw it once I won’t see a six. I’m also really confident if I throw it a second time I won’t see a six, and a third time, and a fourth time. But should I be confident that if I throw it four times I will never see a six? Think about it. The probability of not seeing a six on four throws is about 50–50. That’s not high confidence.”

That illustrates the confidence trap: saying, that if you are highly confident in this and highly confident in that and highly confident in the other—then you are highly confident in this, that, and the other. The natural language makes it very difficult to distinguish between the two, but they are very, very different.

RESOURCES: Basically, each event may have a high probability, but the probability of these events happening together can be much smaller.

Climate Change and the “Confidence Trap”

RFF Senior Fellow Roger Cooke discusses how the confidence trap distorts the debate about how to respond to the threat of climate change.

Listen to the Podcast

COOKE: Correct. When you put it like that, the trap is obvious. However, the natural language—by conflating confidence in each element separately and confidence in all components together—is herding humanity into confidence traps in droves. You see it everywhere. The Intergovernmental Panel on Climate Change (IPCC) addresses this in its Fifth Assessment Report (Chapter 2; see Box 2.2): “Natural language is not adequate for propagating and communicating uncertainty.”

RESOURCES: How do these issues of uncertainty and cognitive illusions play into discussions about climate change, and our understanding and characterizations of climate science?

COOKE: Broadly speaking, the IPCC has done a monumental job of cataloging the current state of the science on climate change and defining where scientific consensus lies; but it does this in very lengthy assessments that report, generally speaking, high confidence in this and that and the other. This presents two dangers. First, it tends to downplay scientific disagreement where consensus is weaker. Second, it invites us to fall into the confidence trap in thinking that high confidence in each is the same as high confidence in all.

We can find an example of downplaying disagreements in the IPCC’s Fifth Assessment Report, where it covers equilibrium climate sensitivity (ECS), the amount of warming eventually attained by a doubling of carbon dioxide in the atmosphere (Chapter 12; see Box 12.2): “ECS is positive, extremely unlikely less than 1°C (high confidence), and very unlikely greater than 6°C (medium confidence).” The attention is directed to the (rather wide) interval in which the scientific consensus bounds equilibrium climate sensitivity. On one hand, a value of less than 1°C is good news for business as usual. On the other, highly regarded climate scientist David Archer says “a climate change of … 5–6°C, would be catastrophic to human civilization” (The Long Thaw, Princeton University Press; p. 95).

That eventuality has a likelihood of less than 10 percent (in the IPCC’s calibrated language), which is affirmed with only medium confidence. That’s rather short shrift for a catastrophe to civilization. Would you put your children on an airplane in which authorities had medium confidence that the likelihood of a crash was less than 10 percent?

The confidence trap encourages climate change deniers to gerrymander the proof burden. Let’s go back to the 2012 presidential election for an example. Mitt Romney famously said “My view is that we don’t know what’s causing climate change on this planet. And the idea of spending trillions and trillions of dollars to try to reduce CO2emissions is not the right course for us.” Romney placed the proof burden entirely on the other side—implying that he will sit and wait until the climate science is thoroughly sorted out before making a decision. He doesn’t have to prove anything. Science communicators who have fallen into the confidence trap are all to inclined to accept this lopsided proof burden.

RESOURCES: People opposed to action on climate change accuse proponents of overstating their confidence. Are you saying that this diverts the discussion from climate to confidence?

COOKE: That’s exactly right. The confidence issue is a red herring that impedes due consideration of potentially very harmful scenarios about which we have no consensus but that cannot be ruled out.

RESOURCES: So this desire for certainty, which we see in the political system, could in fact be causing us to be somewhat reckless, rather than alarmist?

COOKE: Yes. Within the narrative of the IPCC, the confidence trap reinforces the tendency to focus on the things about which we agree and suppress discussion of the things that might be and about which we do not agree.

RESOURCES: If scientists disagree, how might public misconceptions about uncertainty and scientific consensus skew broader understanding and debate?

COOKE: We must help the public understand that scientists are supposed to disagree. If Copernicus hadn’t disagreed with Ptolemy, we would still be computing planetary epicycles. The fact that scientists do disagree is healthy and good. It shows that the science isn’t there yet and that we have to consider decisions under uncertainty, which, as noted, political leaders are loathe to do.

In April, physicist Steve Koonin wrote an op-ed in the Wall Street Journal proposing a “Red Team/Blue Team” approach for climate science. The idea is that two scientific teams would take a published scientific report intended to inform international climate policy; the red team would be charged with critiquing the report and the blue team charged with rebutting that critique. Now what’s wrong with that? There’s something fundamentally misguided about this setup. The blue team has to defend something. The red team has to defend nothing. It’s like a soccer game in which the red team’s goal has been removed, The red team needs no defense and can only score. The blue team can’t score and can’t possibly win.

If we refuse to make any decisions until all uncertainty is removed, whoever gets stuck with the burden of proof is going to lose. If the IPCC has to defend the idea that continuing with current levels of greenhouse gas emissions is going to do great harm to humanity, the IPCC will lose. They can’t prove that because it’s uncertain. Likewise, if the other side has to prove that we can go on with current levels of emissions and nothing bad will happen, they will lose. Whoever gets the burden of proof loses. The Trump administration has laid the burden of proof entirely on climate scientists and relieved climate deniers of any proof burden.

RESOURCES: It seems like a very complicated and tricky thing to characterize. Is the system facilitating misguided doubt? What is your advice to policymakers, the media, the IPCC—or to anyone who has to communicate about the uncertainties related to climate change and how we should go about responding to those uncertainties?

COOKE: We must move on all fronts. First, we must not let Trump and the climate deniers gerrymander the burden of proof. We must insist on burden sharing rather than burden shifting.

Second, we must insist on considering the full range of climate uncertainty. Chapter 15 of the National Academies’ recent Review of the Draft Climate Science Special Report on potential climate surprises should be required reading for every editor, pundit, and talking head on the climate circuit. We need to infuse awareness that the Trump administration’s lack of climate policy forces us into a lottery with medium confidence that the probability of disaster is less than 10 percent.

Finally, we must promote and advance science-based quantification of uncertainty. Every time I read that the IPCC or some other climate committee used their own expert judgment, I recall our review of recent studies on structured expert judgment (Ed. note: Keep an eye out for a related article by Colson and Cooke—“Expert Elicitation: Using the Classical Model to Validate Experts’ Judgments”—in a forthcoming issue of the Review of Environmental Economics and Policy). Experts quantified their uncertainty about things from their field that are known post hoc. The hypothesis that an expert is statistically accurate would be rejected at the 5 percent level for nearly three quarters of the 320 experts in the studies we reviewed. About half would be rejected at the 0.5 percent level. When unvalidated expert judgment is invoked to produce 90 percent confidence bands for a number of unknown quantities, we should not assume that 90 percent of the true values will fall within those bounds.

A report out earlier this year from the National Academies of Sciences, Engineering, and Medicine explicitly encourages science-based uncertainty quantification and provides guidelines.

We need to help the IPCC raise its game—and the game does need raising. RFF’s initiative on the social cost of carbon will help address this need by updating and improving current estimates.

We also need to reach popular culture in a way that will help people think differently about the challenge of climate change­—to consider all angles. There’s a scene from the movie Pearl Harbor that might resonate in terms of the climate debate. The scene depicts US military personnel on the morning of December 7, 1941, before the bombing. Forward radar station operators see hundreds of blips on their screens. One says, “It might be an attack. We have to warn the base.” The commanding officer replies, saying that he needs confirmation before waking the base on Sunday morning. In a few minutes’ time, smoke becomes visible rising over the harbor. Another officer says, “There’s your confirmation.”

That’s the message we need to get out there. The reality is that we have to make decisions about climate change before all the facts are in. We don’t want to get our confirmation as we did at Pearl Harbor.