Reporting bias and the “increase” in weather events in the US

Guest essay by Alberto Z. Comendador

In a recent article I discussed the apparent increase in tornadoes in the US since systematic reporting began, in the early 50s.

I showed how, if one looked at the year-on-year change in temperatures, there was no correlation with the change in tornado counts. The advantage of using year-on-year changes is that the factors that could lead to an observation or reporting bias are almost completely absent: the population of a state, coverage of Doppler radar, etc. will change very little in that timeframe.

So it appears that the increase is due to improved/expanded reporting, not because there are in fact more tornadoes. This is essentially uncontroversial: NOAA gives a similar explanation on its website, though they get around the observation bias with a different method.

Today I want to look at the other weather events NOAA counts. These are:

• Hail, since 1955
• Thunderstorms since 1955, too – on paper. In practice there were almost no events reported until 1995, so that’s what I’ll show here
• 29 other event categories since 1996. As you can imagine these run the whole gamut, from reasonable to mystifying (‘winter weather’)

There are two ways to look at the change in the number of events. One is what we could call the long-term method: simply drawing a chart like the one above for tornadoes. The other is the short-term method, which is what I did in the previous article: checking if event counts rise when temperatures increase, and if they decline when temperatures fall.

I’m especially interested in the recent events because observation bias is supposed to be stronger the farther back one goes in time. In other words, one should see a very strong bias comparing 2015 with 1955, but perhaps not with 1995. Additionally, in such a short period of time there couldn’t have been a strong warming; the lower-48 US had virtually the same temperature in 1995 as in 2014. It seems reasonable to expect that weather events would react more to year-on-year swings, which sometimes exceeded 2ºF (1ºC), than to any ‘trend’.

The results show a strong observation bias in the recent events too – meaning all those NOAA reports since 1995 or 96. Using numbers:

• The long-term method understates 2 event categories: lightning and heat. (Yes, NOAA tracks instances of ‘heat’)
• Both methods are in agreement in another 10 events
• For 17 events, the long-term method overstates

Put other way: for 17 event classes, the apparent ‘trend’ one could plot on a chart is probably overstating the real relationship between event counts and temperature. This rises to 19 if one includes the old events, hail and tornadoes.

(I excluded another event, high surf, as it shows a few hundred incidents per year – except for 2009, when there are over 13,000 occurrences. I’m not sure what to make of that. Besides, it almost always happens in Hawaii – and the NOAA temperatures I’m using refer only to the lower 48. To be strict I should have excluded all events happening in Hawaii and Alaska from the count, for the same reason, but it won’t make much difference; for example, in 2015 there were 57,000 events reported but these two states accounted for only about 1,000.)

Here I’m going to show some examples. I will not show every weather event because a) many of them are irrelevant and b) the post would have 64 charts.

Hail: okay, this event started to be reported in the 1950s so bias is to be expected. Still, just looking at the chart it’s obvious that the relatively recent increase has to be mostly due to expanded reporting. Does anyone think hail events multiplied by four or five in the nineties?

Looking at year-on-year changes the correlation coefficient (r2) is 0.069, or for all purposes zero.

Blizzards are another good example. The chart shows stable numbers or, if one excludes the first two years (which have very high figures), an increase.

Can higher temperatures be associated with more blizzards? In fact, the years in which temperatures increase have less blizzards, while those when temperatures decline have more of them (as common sense would indicate). The correlation since 1996 is -0.55 including all years (p-value = 0.011).

Here I show the plot excluding the first two years, for consistency. Still the correlation is -0.50 and the p-value is quite low (0.041), so the association between increased temperatures and decreased blizzards seems robust.

As for thunderstorms, there seems to be no correlation with temperatures (r2 = 0.039). But again a simple plot would appear to show an increase over time.

NOAA also tracks something it calls winter weather – really. I’m not sure what exactly they include here, but looking at a plot you’d think we’ve been seeing a lot more winter of late…

Obviously, the year-on-year chart shows a negative rather than positive relationship between winter and temperature. Correlation = -0.43, p value = -0.06.

Flash floods appear to be going through the roof…

… when in fact the relationship is negative, with a correlation of -0.32. (The p-value, 0.18, suggests this is just noise, i.e. no real relationship).

Wildfires also seem to be increasing:

But there is virtually no correlation (0.057).

Heavy rain is supposedly exploding too:

But the correlation is again negative: -0.08

Conclusions, and a question for readers

Using event counts is useless for most weather events. It may make sense for the largest (e.g. hurricanes) as these are unlikely to be affected by any reporting bias, but for wildfire, hail, tornadoes, and so on it’s dead wrong.

Another measure of the impact of weather events on the economy is needed. One such measure could be the proportion of losses as a percentage of GDP; if you follow the debate perhaps you’ve come across this chart, or a similar one.

Now, Roger Pielke plots insured losses, which makes sense if one wants to be more or less sure the losses are real (if not, one would have to assume the ‘losses’ are equal to whatever the government decides to spend after a disaster). But there are problems when using this data over a long time frame:

a) As the world develops, a greater share of assets will be insured. If insured losses are growing faster than overall losses, there will be an upward bias in the chart.

b) There aren’t convincing reasons to expect weather disasters, as a percentage of GDP, to be stable. If technology (buildings, early detection systems, etc.) improves, then perhaps we should expect it to decline. This will create a downward bias.

There are probably more biases that I cannot think of right now, but you get the point. The red line in that chart doesn’t necessarily mean the weather is becoming better over time, nor would it mean the weather is getting worse if it trended upwards. It simply means weather disasters cost less as a % of GDP, a fact that may or may not be due to better weather.

It occurs to me that using the year-on-year change in disaster losses is a way to get around the ‘drift’ or bias created by technology improvements, economic growth, etc. So my question is, does anyone know where the actual figures on weather-related losses are? The chart says the source is ‘Munich Re’ but I cannot find the numbers going back to 1990 on that website.

One last thought…

Whether weather disasters/events are increasing or decreasing seems to me an interesting question by itself. But the simple fact that there are more or less weather events does not mean one climate state is preferable to another.

After all, there must be pretty few weather events in Antarctica.

References:

NOAA event database here

NOAA temperatures here

Files as used, along with code, here

The files on NOAA’s webpage still seem to be missing a lot of info – including, crucially, the column that describes the event type. The files I uploaded to Dropbox do include that info, but they’re missing 2016 as I downloaded them a few months ago.