Cracking the Case of the Vanishing Air Pollution Data, with Eric Zou

Notable Quotes

• Opportunities to adjust behavior on air monitoring days: “The Environmental Protection Agency—the government agency who oversees [air quality monitoring]—publishes this calendar that tells state and local governments and the polluters, ‘These are the days that we’re going to do the monitoring.’ The information is in the public domain … You might wonder whether that creates opportunity and incentive for people to behave strategically in response to that rule.” (7:51)
• Local governments want to avoid penalties for poor air quality: “When a county [is deemed noncompliant with Clean Air Act standards], it reduces the manufacturing sector’s productivity, because they have to install more abatement technologies—scrubbers and things like that. There are labor transition costs … It’s pretty costly to the states and the county government to have these kinds of regulatory violation statuses.” (15:42)
• Strategies to prevent missing data: “The Environmental Protection Agency, in their implementation of the Acid Rain Program—that was the cap-and-trade program for power plants to treat emissions for SO₂—actually has this very stringent rule … The rule is, if the availability of data falls below [a certain standard], what they’re going to do is impute the missing value with the maximum in the past couple of days or week or so. That creates a very strong incentive for people not to strategically miss monitoring. The data compliance rate in the Acid Rain Program is very high.” (27:12)

Top of the Stack

• “Unwatched Pollution: The Effect of Intermittent Monitoring on Air Quality” by Eric Zou
• Fort Lee lane closure scandal
• “Next-Generation Compliance: Environmental Regulation for the Modern Era” by Cynthia Giles
• Indians & Energy: Exploitation and Opportunity in the American Southwest edited by Sherry L. Smith and Brian Frehner

The Full Transcript

Daniel Raimi: Hello, and welcome to Resources Radio, a weekly podcast from Resources for the Future, I'm your host, Daniel Raimi. Today, we talk with Eric Zou, assistant professor of economics at the University of Oregon. Eric has published fascinating work on how air pollution monitors work or don't work, as the case may be, to detect harmful levels of air pollution in the United States.

Using data from satellites and ground-based monitors, his work has uncovered how local actors, particularly local governments, may be manipulating air quality data to avoid penalties under the Clean Air Act. It's a mystery of missing data and we'll seek to solve it in today's episode. Stay with us. Okay. Eric Zou from the University of Oregon, thank you so much for joining us today on Resources Radio.

Eric Zou: Thank you, Daniel. Thanks for having me.

Daniel Raimi: Yeah, it's a pleasure. I've been a fan of your work for quite some time, it's really great to meet you. We're going to talk about some really fascinating work you've done on air pollution and monitoring of air pollution, but before we do that, can you tell us and our audience how you got interested in environmental issues in the first place?

Eric Zou: Yeah. Thanks Daniel. I was thinking about this, and a big part of the way I was drawn into this is about people. And so I was a research assistant to David Molitor and Nolan Miller at the University of Illinois while I was a student there, and these are just people that are really smart, my career role models. And so I worked with them, and I really liked the topic. They were working on stuff like the effect of ambient temperature and pollution on health, so they were working with Medicare administrative data, and that's honestly how I got interested in the field.

When I graduated, I went to Cornell’s Dyson School, and I met people at Oregon as well in conferences. People show you that there are many more projects that you can do that haven't been done, things like that. It's also a great community that I enjoy to be in. So I think a big part of that, honestly, is the people part. And then, of course, you need to really enjoy what you do. So I think a part of that is just the fact that I come from China, and if you think about pollution really, the issue, that was really first order. So I think that's the main thing that gets me going.

Daniel Raimi: That's great. It's interesting, we've had several guests in recent months who grew up either in mainland China or in Hong Kong, and every one of them has actually mentioned the air pollution issue in China as a motivating factor for their careers.

Eric Zou: Yeah. It was really bad, if you think about like 10 or 20 years ago when I grew up. I actually grew up near a chemical plant where my grandparents worked, and there's a huge stack and chimney just a couple of miles away. It’s funny, I looked at that stack every day, and I didn't think of that as a problem.

Now I think back, I think I got respiratory issues when I was a kid, and I'm pretty sure that part of that is attributed to the proximity to the plant. So it was a huge problem. It's also changing, which makes it interesting, that the government is changing its attitude towards pollution regulations and how much pollution they want to tolerate and things like that, so it's a big topic in China as well.

Daniel Raimi: Yeah, absolutely. And where in China was it that you grew up?

Eric Zou: Shanghai, it's a coastal city.

Daniel Raimi: Yeah. Shanghai, I've heard of it. So I imagine we could talk about China for the next half hour, but we're mostly going to talk today about your work that's based in the United States. And you have lots of cool papers on lots of interesting topics, but as I mentioned, we're going to focus today on this issue of air quality monitoring and the way that companies sometimes behave strategically, and maybe governments sometimes behave strategically to evade some of the monitoring efforts that the federal government has set up.

But before we get into those details, I think it'd be really helpful if you could give us just a baseline understanding of how air quality is monitored under the Clean Air Act in the United States and how that monitoring regime might create some opportunities for companies or governments to be strategic about this issue of pollution.

Eric Zou: Okay, great. I'll say what I know. So I think the best way to think about it is that you can pull up your phone, and then you click on the weather app, and there's going to be something called the Air Quality Index, so AQI. So it might tell you, for example, that the air quality in your city currently is 50, fine, like a green code or something. The way that data was obtained is from ground air pollution monitoring stations. Think about that as a shelter, it has a fence around it, but it's a shelter with air pollution monitoring devices in it.

And so the way it works, for example, is that if people want to measure how much particle pollution is in the air, what the device does is that it pulls air into it, and the particles get deposited on a Teflon filter, and then a technician is going to go inside the shelter and take the filter out and do lab analysis and figure out how much particulate matter is on that filter, and then they divide that by how much air has been drawn through the device to calculate what's the concentration of pollution in the air. And that data gets streamed then into your cell phone.

About the research on air quality monitoring, the stuff that I looked at is the fact that in some places, the way they monitor air quality involves intermittency. So for example, it's not true that everywhere in the United States that they monitor air continuously every day. So a part of the paper is about the fact that in many places, they monitor particulate matter on a once-per-six-day basis. So it's like I monitor it today, I’ll wait for five days until I do it again.

So you can see why they want to do this, because, first of all, just the way I described the monitoring, it was pretty costly. You need to do lab analysis, and there's going to be engineers and scientists involved, so you want to save some costs. On the other hand, you want to be statistically representative, so once per six days is pretty smart, because if you do something, and it's a per-six-day basis, over the long run, you get, hopefully, representative statistics because nothing else is really changing on a six-day basis. If you do once per seven days, then you're always on Monday and Tuesday and Wednesday, and that's not what you want.

And then regarding your question on incentive, one important part is that the Environmental Protection Agency (EPA), so the government agency who oversees this process, also publishes this calendar that tells, basically, state and local governments and of course the polluters, ‘These are the days that we're going to do the monitoring or the testing.’ And so the information is in the public domain. As a researcher and an economist who thinks about incentives a lot, you might wonder whether that creates opportunity and incentive for people to behave strategically in response to that rule.

Daniel Raimi: Yeah, for sure. And just one quick clarification, you mentioned PM earlier, that's particulate matter, and I think it refers to particulate matter that's 2.5 microns or smaller, is that right?

Eric Zou: Yeah, exactly. PM2.5 is a technical word, but yeah, that's exactly what you said it is.

Daniel Raimi: Perfect. So let's talk about this first paper that you have called “Unwatched Pollution: The Effect of Intermittent Monitoring on Air Quality,” so exactly what we're talking about. How do you investigate whether companies are being strategic about when they pollute?

Eric Zou: So just to be precise, the paper actually doesn't directly look at companies. Remember there's these once-per-six-day monitoring cycles, and the idea is that although the monitors are only operating once per six days, we have independent measures from the satellite that are on almost every day, so the satellite doesn’t obey the ground intermittent monitoring rule. The idea is to use a satellite measure of air quality and look at places that had monitors and compare days when the monitors are on versus the days when the monitors are off.

And the reason I say it's not directly looking at firm emissions is because this is looking at ambient air quality. So it's the condition of air quality in the atmosphere rather than just what's coming directly off of the stack, so I want to be careful there. But the thing that I discover is that, because we have this once-per-six-day monitoring cycle, you can look at what the pattern looks like in a typical six-day pattern.

What you see is that from the satellite's point of view, air quality is pretty stable across those six kinds of days, except for a drop on the day when the federal monitoring is scheduled. And then to the extent, like I said, that there should not be any reason for anything basically to exhibit a once-per-six-day pattern, we would think that these differential levels in air quality is attributable to strategic polluter behavior. That's what the paper argues.

Daniel Raimi: Great. And so, like you just said a moment ago, to enforce these Clean Air Act regulations, the EPA does not monitor particulate matter coming out of specific smokestacks or specific industries or specific factories, so it's an ambient measurement. So if it's an ambient measurement, would any specific factory reduce its pollution on the day of monitoring? It's a little confusing, and so you investigate some other explanations for why this might be happening. What do you hypothesize and then what do you find?

Eric Zou: Yeah, that's a great point. So there's definitely this tragedy of the commons problem. So, if you have a lot of polluters, why would they coordinate? I think one point that I try to make in that paper is, if you think about the principal-agent problem, the way we think about it usually is, the federal EPA is the principal, and then you have the ones they regulate, which are individual plants. But actually, if you look at the structure of the Clean Air Act regulations, there's this layer of, I think, agents who are like state or county managers.

So the way it works is that if air quality in the state or county jurisdiction exceeds the regulatory standards, the state and the counties also bear the cost of that. So I think that's important to think about too. So one way I illustrate that is, I looked at the issuance of smog alerts. It's an extreme example, but I think gets at the point that I want to make, which is, so these are things like public warnings and things that state and local government can issue to ask people to reduce driving, stay indoors, don't burn anything.

In California, it's called “Spare the Air.” In Chicago, it’s called “smog alerts.” And if you just look at the timing of when those alerts are issued and you align that timing with the once-per-six-day schedule that I mentioned earlier, you can see a pile up of issuance exactly on the day when the federal monitoring is scheduled. So on those days that everybody knows that monitoring is scheduled, you have more warnings to the public that encourage people to reduce emissions.

So I think that illustrates to me that there is this principal-agent problem where the state, because they were bearing the cost, they also have an incentive to make sure the air quality and things are in check. And so when we talk about coordination, it's not just among the plants, it might also involve the people who also bear the costs, which are the state and local government.

Daniel Raimi: Yeah. It's so interesting to think about those smog alerts going up, and maybe it's not because the air is particularly bad, but because local officials might want to avoid some punishment that they get when the air is at the baseline quality that it might otherwise be. And before we go to the next question, it would be really helpful to know, what are some of the punishments that state or local governments might face, or the consequences that they might face if, let's say, the EPA finds that they have particularly bad air quality on those monitoring days?

Eric Zou: The way it works, as far as I understand, is: So, let's say the area has a bunch of air quality monitors, and if the recorded data says that the recorded air quality is above the regulatory standards, then what the state has to do is they have develop something known as the State Implementation Plans, or SIPs. Actually every state has to develop those, but if your state has a nonattainment county, then your SIP is just going to be much more detailed and has a lot more clauses and things that tells the federal government exactly what you're going to do to bring air quality back into compliance.

So what kind of things are you going to do for each plant to bring them back into compliance? There's a lot of costs. There's a line of research in environmental economics that specifically looks at this. So they found, for example, that when a county gets a nonattainment status, it reduces, for example, the manufacturing sector's productivity, because they have to install more abatement technologies—scrubbers and things like that. There are labor transition costs, because things are more costly and there's costs in the labor market too. It reveals that it's pretty costly to the states and the county government to have these kinds of regulatory violation statuses.

Daniel Raimi: Yeah, that makes sense. So interesting to think about the different actors here. Let's dig in a little bit deeper on this issue of local or state governments behaving strategically. There's another really interesting paper you have on this topic that's called “What's Missing in Environmental Self-Monitoring?” And it starts on a bridge in New Jersey, so let's all travel in our minds to a bridge in New Jersey in 2013. So what happened on that bridge and what happened to the air quality monitor that was near the bridge around the same time?

Eric Zou: Yeah. Thanks Daniel for talking about this. It's a work in progress or a working paper that we developed that—instead of looking at strategic polluting behavior, like the paper I just talked about—this one is thinking about strategic monitoring, which is more directly on the state and government side. And I mentioned this was a team effort, so this is also authored by Yingfei Mu, who was a Master's student at Oregon and is moving to Johns Hopkins’ Economics Department to do her PhD, and also Ed Rubin. Ed was my colleague, an assistant professor at Oregon, and he was a Berkeley graduate.

So the Bridgegate thing, as you mentioned, was a motivation to our paper. So this is incident, I think, back in September 2013. What happened is that there was this traffic study that occurred on the George Washington Bridge that takes you normally from Fort Lee in New Jersey to Manhattan. And on that day, there was a traffic study, they said, that closed two of the three main lanes on that bridge and caused a massive traffic jam and gridlock. And people later figured out it was not a traffic study, but it was political retribution. So if you Google “Fort Lee lane closure scandal,” you'll find a Wikipedia article that describes it.

And the thing that we notice is that there was this complaint by environmental groups and journalists who found out that there is this Jersey City firehouse, and there were air quality monitors on the rooftop of that firehouse near the bridge that was inoperative. So it was off for an extensive period of time, and it missed out the entire peak of pollution generated by the traffic jam. So that was suspicious to many people, and then many people petitioned the regional EPA to investigate this issue.

And they actually did an investigation, and the result, I think they say. is because of a wireless router or malfunction or some kind of equipment malfunction. So it's weird because I don't think that's—I can't say precisely—but I think that might not be the reason, because if there were multiple devices on the rooftop in that place, and it seems like the PM monitors is the only one that's down.

Daniel Raimi: So just it's basically fishy results.

Eric Zou: Yeah, it was pretty suspicious to some people.

Daniel Raimi: Interesting. And so the basic storyline, if I understand it correctly, and correct me if I'm wrong, is that there's this political fight, some lanes are closed down to create a traffic jam as an act of political retribution, and at the same time that that's happening, mysteriously, this air quality monitor turns off. And so you investigate whether analogous or somewhat similar issues or incidents have occurred in different parts of the country, and what do you find?

Eric Zou: So like I said, the motivation is really that we're probably never going to know what happened for that monitor in that incident, but it's possible, we found, to have some statistical evidence on whether strategic monitoring behavior actually occurred. So is it true that we actually observed that the monitors are more likely to be down when the state and local government expect air quality to be bad?

So the way we did the study is that, I think the idea is simple, we are going to measure expectations of bad pollution using the pollution alerts, the warnings. The way it works is that state and local governments have these prediction algorithms and models. These are numerical forecasting models that tell them what air quality is probably going to look like in the next several days, and they often issue alerts when they expect air quality to be bad.

So we use that as an expectation and we look at monitors and say, "Okay, is it true that when the state and local government expect pollution to really deteriorate, you see that the monitors are more likely to be off or malfunction or something like that?" And we actually started with the monitor on the Jersey City firehouse because that was one of the ones that we think are interesting.

And actually, if you look at the pattern, it seems true that the Fort Lee city issued about 21 of these alerts across the years we studied, and on average, we look at the pattern, it's true that near the days that the government issues alerts, you see that the availability of data, the PM2.5 data starts to drop and goes to the lowest point exactly on the day when the alerts are issued, and then come back to the normal level. So that's one main picture that we show.

And then what we do is that we repeat this exercise along with some statistical ways to do inference and test that across all the monitors we can possibly test, meaning that these are the monitors that are supposed to be on every day, and they are in areas that also have pollution alerts. Basically, what we do is that we generate a list of monitors that appear to show similar behavior. So their data availability drops whenever the local government expects bad pollution.

Daniel Raimi: And is the hypothesis that local governments or someone is turning off these monitors or making them malfunction, again, for the purpose of avoiding violations of the Clean Air Act that would trigger these more detailed state implementation plans that have economic consequences? Is that what you suspect is going on?

Eric Zou: That would be the assumed or the implied mechanism or reason. The alternative, of course, is that there is something that's just mechanical going to go wrong when the air quality gets bad for those monitors. We investigated this in multiple ways, and we don't think that's going to be the case. One of the reasons is just that the levels of pollution that we see in the United States are not going to be so bad to trigger, we believe, equipment malfunction. So it's very likely to be something that's not mechanical.

Daniel Raimi: It would be a pretty serious design flaw if your pollution monitor failed to work when there was pollution around. So let's go to one more question before we go to our Top of the Stack segment, which is thinking about next steps or implications from some of these findings.

As you mentioned earlier in our conversation, we have satellite data now that can do a good job monitoring wide swaths of land. We have other tools like the PurpleAir network that I'm a part of, you have hundreds and maybe thousands of monitors all around the country monitoring particulate matter data in real time. So are state or federal policymakers starting to use any tools like this or other tools to try to get around the strategic behavior that seems to be cropping up?

Eric Zou: Yeah. That's a good question. So like in the previous project, the first thing that we do every time we got this kind of results is that we actually talked to people in the EPA and let them know that we have a set of findings that appear to be pointing to some kind of issue with the way that the monitoring is done. And I think that one thing that's really important that I learned from the process is that as researchers, we can identify these issues and patterns, but at the decisionmaking level, you never know what complications they face. So there are a lot of legal barriers and then practical issues that we are just unaware of.

I don't want to say what exactly the government should do to erase that problem, but like you said, in the paper, we also proposed ways that this thing potentially could be alleviated. So for example, I think a simple thing that we proposed in the paper is simply, do not just ignore the data that are missing. So we think that the root of the problem is that when the regulators calculate how much pollution you have in your area is that they only use the available data and just ignore what's missing. When you calculate an average and when you ignore the missing days, what you are implicitly doing is you're just imputing those missing days as the overall average.

And then the point of the paper is that that might be wrong, because if missing data were not random, then you're underestimating the pollution value. So one thing you can do, for example, you can impute those values with something that's more, probably, closer to the truth. So we can use nearby monitors that are actually functioning, you can use PurpleAir, things like you mentioned, so these are grassroots ways, alternative ways to monitor air quality. So figuring out ways to impute those values to deter strategic monitoring is something that's useful.

There are actually ways to do this. So, the Environmental Protection Agency, in their implementation of the Acid Rain Program—so, that was the cap-and-trade program for power plants to treat emission for SO₂—they actually have this very stringent rule. This is a completely different context, so here, we’re talking about continuous emission monitoring from a device installed on each stack that measures exactly how much pollution you're emitting.

And so there the rule is, if your data capture rate, the availability of data falls below, let's say, 90 percent, what they're going to do is that they're going to impute the missing value with the maximum, let's say, in the past couple of days or week or so. So that creates a very strong incentive for people not to strategically miss monitoring, and the result appears to be good. The data compliance rate in the Acid Rain Program is very high, and I think it's way above 90 percent or something like that. So there are definitely ways to do data substitution and imputation to, I believe, to deter this type of behavior.

Daniel Raimi: Yeah. That's really interesting. Well, I mentioned we could have, again, many more questions and conversations about maybe the barriers that the EPA faces to deploying some of these things, the legal and technical barriers, but we are just about out of time. So let's go to our last question that we ask all of our guests, which is to recommend something that you've read or watched or heard lately that you enjoyed. It can be related to the environment, or maybe not related to the environment, just whatever you're interested in these days.

And I'll start with a book that I am about halfway through. It's from 2010, so I'm a little late to it, but it's really fascinating. It's a series of essays and case studies about Native Americans and energy in the US Southwest. The book is called Indians and Energy: Exploitation and Opportunity in the American Southwest. It's edited by Sherry Smith and Brian Frehner. It's an academic book, but it's really accessible to anyone, I think.

It tells just really fascinating stories about the relationship between different tribes in the Southwest and energy development, whether that's hydropower dams that displaced some Native Americans, whether it's oil and gas development, coal mining, uranium mining, and it explores all these fascinating issues, and it really gets at the complex issues of the benefits and the substantial environmental damages associated with energy development in Indian country. So I'm really enjoying it, and if you're interested in energy issues, I think others would too. But how about you, Eric, what's on the top of your stack?

Eric Zou: So I apologize that out of work, I don't read that many environmental books, but I will recommend something that I found very cool and important lately. So, this is actually a blog post. So if you Google the Environmental and Energy Law program at Harvard Law School, there's this blog post by Cynthia Giles. So Cynthia, she was the assistant administrator at the Environmental Protection Agency's Office of Enforcement and Compliance Assurance.

She's currently a guest fellow at Harvard Law School’s Environmental and Energy Law program, and she has this amazing blog post called Next Generation Compliance: Environmental Regulation for the Modern Era. It's a compilation of things, like the incidents that she saw while she was at the EPA enforcement office, incidents of things like pollution leaks and what are the enforcement challenges that the EPA faces, what are some of the problems and the remedies, and I think it's a really cool read.

So she includes different parts. So for example, part one is about rules with compliance built in, part two is about how non-compliance with environmental rules is worse than you think, and then she also talked about things like climate change in water regulation and things related to that.

So I would recommend it, first because it's a great read. And the other is that if you're interested in let's say research and compliance and enforcement in environmental economics, this is like a gold mine that I highly suspect if a graduate student were to dig into what Cynthia has written, there's going to be multiple papers of research that can come out of it. So I would really recommend that to the audience of this podcast.

Daniel Raimi: Great. That sounds fascinating. So anyone out there looking for new paper ideas, that sounds like a great way to start. Once again, Eric Zou from the University of Oregon, thank you so much for coming on the show today and helping us understand these really fascinating issues around air quality and monitoring. We really appreciate it.

Eric Zou: Thank you. Thank you for having me.

Daniel Raimi: You've been listening to Resources Radio. Learn how to support Resources for the Future at rff.org/support. If you have a minute, we'd really appreciate you leaving us a rating or comment on your podcast platform of choice. Also, feel free to send us your suggestions for future episodes. Resources Radio is a podcast from Resources for the Future. RFF is an independent, nonprofit research institution in Washington DC.

Our mission is to improve environmental, energy and natural resource decisions through impartial economic research and policy engagement. The views expressed on this podcast are solely those of the podcast guests and may differ from those of RFF experts, its officers, or its directors. RFF does not take positions on specific legislative proposals. Resources Radio is produced by Elizabeth Wason with music by me, Daniel Raimi. Join us next week for another episode.