Reporting bias and the “increase” in weather events in the US

Guest essay by Alberto Z. Comendador

In a recent article I discussed the apparent increase in tornadoes in the US since systematic reporting began, in the early 50s.

image

I showed how, if one looked at the year-on-year change in temperatures, there was no correlation with the change in tornado counts. The advantage of using year-on-year changes is that the factors that could lead to an observation or reporting bias are almost completely absent: the population of a state, coverage of Doppler radar, etc. will change very little in that timeframe.

So it appears that the increase is due to improved/expanded reporting, not because there are in fact more tornadoes. This is essentially uncontroversial: NOAA gives a similar explanation on its website, though they get around the observation bias with a different method.

Today I want to look at the other weather events NOAA counts. These are:

  • Hail, since 1955
  • Thunderstorms since 1955, too – on paper. In practice there were almost no events reported until 1995, so that’s what I’ll show here
  • 29 other event categories since 1996. As you can imagine these run the whole gamut, from reasonable to mystifying (‘winter weather’)

There are two ways to look at the change in the number of events. One is what we could call the long-term method: simply drawing a chart like the one above for tornadoes. The other is the short-term method, which is what I did in the previous article: checking if event counts rise when temperatures increase, and if they decline when temperatures fall.

I’m especially interested in the recent events because observation bias is supposed to be stronger the farther back one goes in time. In other words, one should see a very strong bias comparing 2015 with 1955, but perhaps not with 1995. Additionally, in such a short period of time there couldn’t have been a strong warming; the lower-48 US had virtually the same temperature in 1995 as in 2014. It seems reasonable to expect that weather events would react more to year-on-year swings, which sometimes exceeded 2ºF (1ºC), than to any ‘trend’.

The results show a strong observation bias in the recent events too – meaning all those NOAA reports since 1995 or 96. Using numbers:

  • The long-term method understates 2 event categories: lightning and heat. (Yes, NOAA tracks instances of ‘heat’)
  • Both methods are in agreement in another 10 events
  • For 17 events, the long-term method overstates

Put other way: for 17 event classes, the apparent ‘trend’ one could plot on a chart is probably overstating the real relationship between event counts and temperature. This rises to 19 if one includes the old events, hail and tornadoes.

(I excluded another event, high surf, as it shows a few hundred incidents per year – except for 2009, when there are over 13,000 occurrences. I’m not sure what to make of that. Besides, it almost always happens in Hawaii – and the NOAA temperatures I’m using refer only to the lower 48. To be strict I should have excluded all events happening in Hawaii and Alaska from the count, for the same reason, but it won’t make much difference; for example, in 2015 there were 57,000 events reported but these two states accounted for only about 1,000.)

Here I’m going to show some examples. I will not show every weather event because a) many of them are irrelevant and b) the post would have 64 charts.

Hail: okay, this event started to be reported in the 1950s so bias is to be expected. Still, just looking at the chart it’s obvious that the relatively recent increase has to be mostly due to expanded reporting. Does anyone think hail events multiplied by four or five in the nineties?

image

Looking at year-on-year changes the correlation coefficient (r2) is 0.069, or for all purposes zero.

image

Blizzards are another good example. The chart shows stable numbers or, if one excludes the first two years (which have very high figures), an increase.

image

Can higher temperatures be associated with more blizzards? In fact, the years in which temperatures increase have less blizzards, while those when temperatures decline have more of them (as common sense would indicate). The correlation since 1996 is -0.55 including all years (p-value = 0.011).

Here I show the plot excluding the first two years, for consistency. Still the correlation is -0.50 and the p-value is quite low (0.041), so the association between increased temperatures and decreased blizzards seems robust.

image

As for thunderstorms, there seems to be no correlation with temperatures (r2 = 0.039). But again a simple plot would appear to show an increase over time.

image

image

 

NOAA also tracks something it calls winter weather – really. I’m not sure what exactly they include here, but looking at a plot you’d think we’ve been seeing a lot more winter of late…

 

image

Obviously, the year-on-year chart shows a negative rather than positive relationship between winter and temperature. Correlation = -0.43, p value = -0.06.

image

Flash floods appear to be going through the roof…

image

… when in fact the relationship is negative, with a correlation of -0.32. (The p-value, 0.18, suggests this is just noise, i.e. no real relationship).

image

Wildfires also seem to be increasing:

image

But there is virtually no correlation (0.057).

image

Heavy rain is supposedly exploding too:

image

But the correlation is again negative: -0.08

image

Conclusions, and a question for readers

Using event counts is useless for most weather events. It may make sense for the largest (e.g. hurricanes) as these are unlikely to be affected by any reporting bias, but for wildfire, hail, tornadoes, and so on it’s dead wrong.

Another measure of the impact of weather events on the economy is needed. One such measure could be the proportion of losses as a percentage of GDP; if you follow the debate perhaps you’ve come across this chart, or a similar one.

Now, Roger Pielke plots insured losses, which makes sense if one wants to be more or less sure the losses are real (if not, one would have to assume the ‘losses’ are equal to whatever the government decides to spend after a disaster). But there are problems when using this data over a long time frame:

a) As the world develops, a greater share of assets will be insured. If insured losses are growing faster than overall losses, there will be an upward bias in the chart.

b) There aren’t convincing reasons to expect weather disasters, as a percentage of GDP, to be stable. If technology (buildings, early detection systems, etc.) improves, then perhaps we should expect it to decline. This will create a downward bias.

There are probably more biases that I cannot think of right now, but you get the point. The red line in that chart doesn’t necessarily mean the weather is becoming better over time, nor would it mean the weather is getting worse if it trended upwards. It simply means weather disasters cost less as a % of GDP, a fact that may or may not be due to better weather.

It occurs to me that using the year-on-year change in disaster losses is a way to get around the ‘drift’ or bias created by technology improvements, economic growth, etc. So my question is, does anyone know where the actual figures on weather-related losses are? The chart says the source is ‘Munich Re’ but I cannot find the numbers going back to 1990 on that website.

One last thought…

Whether weather disasters/events are increasing or decreasing seems to me an interesting question by itself. But the simple fact that there are more or less weather events does not mean one climate state is preferable to another.

After all, there must be pretty few weather events in Antarctica.


References:

NOAA event database here

NOAA temperatures here

Files as used, along with code, here

The files on NOAA’s webpage still seem to be missing a lot of info – including, crucially, the column that describes the event type. The files I uploaded to Dropbox do include that info, but they’re missing 2016 as I downloaded them a few months ago.

Advertisements

46 thoughts on “Reporting bias and the “increase” in weather events in the US

  1. I wonder what the increase was in low energy tornadoes since the increased use of Doppler radar.

      • Dr. Pielke destroyed all of these memes years ago. The 1950s, where all the memes start, were also relatively quiet as far as weather related disasters go.

    • I wonder what the increase has been in the number of people employed as weather forecasters, and what the increase has been in the number of weather reporting programs or spots/updates/breaking news snips.

      CO2 leads to an increase in global average birth rates, which leads to an increase in weather-forecaster employment, which leads to an increase in extreme weather events. There, I solved that. (^_^) … in a lot fewer words than our featured writer.

  2. Educational piece. Lots of food for thought on perspective.

    Now for the politics. Albert Gore Jr. Was VP of the USA for 8 years during the Nineties. I don’t have to remind anyone here that his fanaticism regarding AGW manifested itself a decade betore that. Bill Clinton gave Gore a wide brief on climate. Gore repaid him by standing with Clinton through his many scandals.

    It is not surprising that observation bias would be discounted by agencies overseen by Gore. Political science comes from the top down.

    • troe
      “Political science comes from the top down.”
      Political something beginning with ‘S’ comes from the top down.
      Fixed.

      Auto – not too keen on participating in the old Chinese curse – ‘May you live in interesting times!’.

  3. An awful lot of the so called ‘observational bias’ ie events and frequencies never before reported (or the records lost) is nothing more than the common statistical error resulting from assuming that ‘no event reported – or recorded – or archived’ means that the event(s) didn’t happen. Clearly that assumption is NOT true and probably never has been true. For instance, just because no hail events were recorded in Kansas in 1926 doesn’t mean there were none. Neither does it mean there was more than one! The same is true for many other categories. “no data” does NOT mean zero events and it is wrong to assume that it does. Sometimes it is necessary to simply state “I cannot correlate the frequency of events today with those of xxx because there is no usable data”.

      • ATheoK April 11, 2017 at 6:43 am

        Thanks for the link. I almost always prefer the KJV (and I speak no Hebrew) but in this case the modern translation seems more apropos.

        … but he seemed like a comedian in the eyes of his sons-in-law.

    • As an archaeologist friend of mine is fond of saying:
      “Absence of evidence is not evidence of absence.”

    • Hottest Year Evah! = Low Tornado Count

      Isn’t that the exact opposite of what the promoters of CAGW are claiming? They say hotter weather means more and larger storms on land and sea. It’s not happening now. Maybe that means they are wrong, or maybe it’s not really the “Hottest Year Evah!”, they are just pretending it is. I think it’s the latter.

      • I think it’s both – they’re wrong (colder climate produces bigger temperature DIFFERENTIALS, and THAT is what causes the most violent weather, NOT the highest “average” temperatures, which generally translate to higher temperatures in the coldest/driest air masses, thus reducing the poles vs. equator temperature differentials) AND it’s not really the “hottest year evah” (because the data is crap, polluted with both UHI and “adjustments” riddled with confirmation bias).

  4. I have an article by Gore, long before he became VP, where he is advocating government involvement in developing the internet.
    I recently discovered an item in which he is trying to help destroy the Digital Audio Tape industry as a government agent.

      • Garymount

        Assume that scribbling is meant sarcastically. So densely written it is difficult to tell. Try to follow along. Politicians have agendas. You can verify that on the internets. Those agendas drive policy and funding. Policy and funding strongly influence the recipients of same.

        Albert Gore was in a position of great influence over U.S. science policy and funding. He used that position along with others to drive policy and funding. Not that difficult to understand. AGW has been a very potent driver of policy, funding, and science.

        Maybe you can scape up the IQ to refute that.

      • And Gore didn’t really have any opposition to his human-caused global warming meme back then because we weren’t spending large sums of money on it then, which causes controversy, and most people looked at it as an effort to reduce pollution because that was the way it was presented.

        Gore definitely had a big influence on the science government was pursuing at the time.

    • What’s your point. He advocated many things. Some good some not so good. Mostly over promised and under delivered

  5. The “extreme” parameter needs to be well defined. The problem is that historical weather is so very subjective. That’s why people created instrumentation (eg. thermometer) based on physical definitions.

    Here is an “official” look at a parameter called a “heat index” with a plot vs time since 1895.
    https://www.epa.gov/climate-indicators/climate-change-indicators-high-and-low-temperatures

    For the USA, the hottest decade of all time is obviously the 1930s.

    Another researcher, or insurance actuary, or historian or accountant might consider historical death rates due to weather to be another useful parameter. The average farmer has a better idea of weather risks than the average climate scientist, and anyone who has lived in a temperate climate knows that cold weather is more dangerous than hot weather.

    • bw
      Ow wow – the link is emphatic – the 1930s had some interestingly extreme weather.
      And, apparently dated to August 2016 – before the non-election of HRC . . .
      What can this mean?

      EPA hedging bets?
      Or doing real science?
      I certainly have no insight into their motivation . . . . . . .

      Auto

  6. As “one who was there,” I can testify that the NWS reporting of weather events, especially via Storm Data, varied greatly from Region to Region and even intra-regionally in the 1970s and ’80s. It still does, of course, albeit to a much lesser degree because of greater oversight. But there is no question in my mind that reporting became more standardized and, of course, more scrutinized, during the 1980s and beyond. In fact, IMO there are some offices which underdo it and others which overdo it, and the preparer’s subjectivity creeps in to skew data somewhat. Look at the variability of how flash floods are treated from office to office.

    Also, one must account for advances in communication and technology from the 1990s to the current time, and the increased reporting of storm spotter networks, contributions from remote sensing tools such as Doppler radars and their subjective use by individual Storm Data preparers, etc. It seems very unlikely to me that efforts to account for these significant changes will accurately allow for comparison to prior decades, and certainly not back to the 1950s and ’60s.

    To me, it seems that weather events occur in cyclic fashion on the order of a couple or few decades; but, it also seems to me that these U.S. weather cycles have led to pretty similar weather conditions today compared to those a few or even several decades ago. If one looks at historical records, such as Ludlum’s works and English and European accounts of stormy periods and even specific storms, many areas of the U.S., North America, and Europe experienced much harsher conditions a few centuries ago during the LIA, with more severe winters and larger magnitude summer severe storms and floods. But, these fairly long period cycles, whatever their genesis and cause, will play out over a long time. We’re in the middle of the Modern Warm Period now, and it’s pretty obvious that the less severe conditions and less overall storminess that we enjoy, with decadal ups and downs, are to our great benefit.

  7. Wildfires are not a weather event. There was a recent study (I believe mentioned here at WUWT) showing that number of wildfires are directly related to incursion of human habitation into forested areas. More people., more houses. more fires started by human activity.

    Weather related fires are those attributed to lightning strikes — more lightning the more likely there will be fires. Here in the northeast US, few lightning strikes cause fires — they just kill individual trees.

    Almost NONE of these stats — which “number of xxxx” have any reliability as one can almost never discover what they are REALLY counting and if the counting parameters have remained the same from year to year. They are, simply, not scientific in any way, they are gross generalities and prone to fads.

    Your wildfires graph, by the way, does not “seem to be increasing” — it is flat with one exceptional year after 1998/99. It shows the characteristics of a stat in which reporting begins at some point in time and ramps up as different reporting agencies come on line.

    • “Kip Hansen April 10, 2017 at 7:55 pm

      Almost NONE of these stats — which “number of xxxx” have any reliability as one can almost never discover what they are REALLY counting and if the counting parameters have remained the same from year to year. They are, simply, not scientific in any way, they are gross generalities and prone to fads.

      Your wildfires graph, by the way, does not “seem to be increasing” — it is flat with one exceptional year after 1998/99. It shows the characteristics of a stat in which reporting begins at some point in time and ramps up as different reporting agencies come on line.”

      Kip makes two excellent points. Alberto Z. Comendador does tangentially mention both points; especially the observation ability technical point.

      To throw a bit of a curve into the topic; I’ll add in social contexts to the mix.

      1) I suspect that if you add in several viewing technologies you’ll find additional correlations.
      – 1a) The proliferation of satellite and cable television.
      – 1b) As viewership increased, a desire to report “local” disturbances increased.
      – 1c) Alternative weather reporting channels developed to profit off of the increased weather views.

      2) Bad Weather is an incredible social driver for large parts of the populations.
      – 2a) Just a warning that bad weather is approaching drives large numbers to available weather outlets to learn about approaching weather.
      – 2b) Once bad weather arrives, significant numbers of people want to know what is happening to others.
      – 2c) There also arises a competitive desire to share local weather experiences.
      – 2d) Broadcast stations; “weather channel”, “wunderground” fan these desires by giving blurbs and images nationwide coverage.

      Correlating increases in weather broadcasting stations recognizes a slightly different version of increased or improved technical capabilities for spotting severe weather.

      However the social revolution that those technical improvements cause should not be overlooked.

      Especially since increases in social participation are triggers for improved official weather recognition.
      Citizens who do not see weather they personally noticed, tend to pester their elected representatives regarding weather recognition capabilities.

  8. Short term and long term relationships between specific weather events and global temperature can be different.

    For example, long term correlations of Atlantic hurricanes and north-central/northeast Pacific hurricanes (since detection of each got good due to satellite coverage) with global temperature don’t show any statistically significant trend, although eventually a warming world is expected to make these showing up as getting worse. (I don’t expect by much, because global warming is mostly a little overstated in terms of what already happened and greatly overstated in terms of what climate models predict, and is happening and will happen less in the tropics than for a global average.) Global temperature is rising, although somewhat unsteadily. Linear trends of hurricane counts are not statistically significantly different from flat.

    But there seems to be a significant positive short term correlation between N-central & NE Pacific hurricanes and global temperature, and a significant negative short term correlation between Atlantic hurricanes and global temperature. (No, I did not actually make and check out scatterplots, but this is what I expect.) The reason is that El Nino raises global temperature, raises ocean temperature where Pacific hurricanes form, and increases hurricane-disfavoring wind sheer in the Atlantic, and La Nina does the opposite.

    • I expect that with tornadoes, the main long term correlation with global temperature is shifting the time of year of greatest tornado incidence slightly away from summer, slightly towards cooler times of the year. And not much change of tornado count long term when global temperature makes a long term change. (Although I expect global warming to make tornadoes worse in the northernmost parts of Canada that get them in a season that peaks there in the summer.)

      As for short term correlation of tornadoes with global temperature, such as can be discerned from a scatterplot over a time period whose linear trend of temperature has a change under, around or even twice the standard deviation from the linear trend: I expect small negative correlation. La Nina favors lower global temperature and more tornadoes. However, La Nina’s favorability to more tornadoes is irregular because there are other factors where irregularity of weather affects formation rate of tornadoes.

      La Nina favors more tornadoes, but with small consistency. One reason is that La Nina favors the late winter / early spring main storm track in the early spring to be a northerly one typical of later in spring, which subjects more of “tornado country” to the warm-sector parts of storm systems that can produce tornadoes in the “early season”. Also, it is notable that the two greatest tornado outbreaks in US history since 1950 happened in April after a La Nina winter from extratropical cyclones whose strength was favored by La Nina making global temperature cooler, because global temperature changes less in the tropics and more in the Arctic, and horizontal temperature gradient is the #1 ingredient for fueling the kinds of storms that produce major tornadoes, even though vertical temperature gradient and heat and humidity are essential. One of these two great April tornado outbreaks happened in 1974, and hit Indiana hard due to La Nina favoring April tornadoes where they usually peak no sooner than May – while most April tornadoes tend to be more south than Indiana.

      • “Albert Gore was in a position of great influence over U.S. science policy and funding. He used that position along with others to drive policy and funding.”

        That must be changing because we are not getting major tornado outbreaks.

      • Here’s what I meant to say:

        “because global temperature changes less in the tropics and more in the Arctic, and horizontal temperature gradient is the #1 ingredient for fueling the kinds of storms that produce major tornadoes,”

        That must be changing because we are not getting major tornado outbreaks.

  9. The frequency of flash floods is affected by factors other than climate change, and even other than longer term weather variations such as the Atlantic Multidecadal Oscillation and multidecadal oscillations in the Pacific (such as PDO, and multiyear-smoothed ENSO which may be loosely linked to AMO). A significant factor is change of land use. Replacing forest with anything else, and replacing any kind of vegetated rural land with urban development, usually increases flash floods.

  10. The natural variability is common on weather events. The interpretation using the data after 1950 may mis-guide the reality. It is worthwhile to look the data from prior to 1900. We have seen the extreme unusual events similar to now before 1900.

    Dr. S. Jeevananda Reddy

  11. No radar, no computer models, only land line phones, the Old Weather Bureau before the Severe Storms Center in the 1940’s and 1950’s….even the 1960’s there were very few tornado warming, few storms reported and we even only had three or four still pictures of tornadoes from previous years and only two films of actual tornadoes. As a kid TV weathercaster and then a fledgling Meteorologist I felt tornadoes were exceeding rare events. By the mid 1980s it had all changed. Computer models created severe storms probability forecast which easily turned into watches. Television keep telling people to watch for tornadoes and people kept seeing them. The numbers increased dramatically. Then came the age of the cell phone with cameras and sightings, phone reports, videos kept climbing and the age of Doppler radar added to the tornado warnings exponentially. Were there actually more tornadoes. I doubt very, very seriously.

    • Absolutely correct, John, and you’d know the history of this better than anyone.
      Back in 1857, Lorin Blodget, in his classic “Climatology of the United States” (text at http://quod.lib.umich.edu/cgi/t/text/text-idx?c=moa&cc=moa&view=text&rgn=main&idno=AGH8863.0001.001 ) with the information on hand at the time, wrote:
      “They occur over every part of the United States where the rain fall is abundant, and at the seasons of its greatest abundance. There are none on the great plains so far as known …”
      The Great Plains are now known, of course, as Tornado Alley.
      Reporting makes all the difference. I was on some storm detection & forecasting experiments in the 1980s and 1990s, and in my role as a “storm chaser” was solely responsible for several observed and confirmed tornadoes. In other words, this one storm spotter added five or so tornadoes to the graphs in the 1980s and 1990s. Multiply that by God knows how many storm chasers & spotters are out there now and you have your increase in reports. There were very few storm chasers before 1970.
      Also having a mechanism to collect tornado reports adds to the number. I have personally spotted tornadoes in Panama and Malawi that aren’t on any record because neither place had any means of recording the report. The US didn’t have a coherent mechanism until the Weather Bureau opened the storm forecasting center in 1953 or so. Before then reports were gleaned from newspaper reports, which we know from current reporting standards can be hit or miss.
      While cruising for tornado papers (a cathartic and rewarding activity) I found this paper about tornadoes in Turkey
      http://journals.ametsoc.org/doi/pdf/10.1175/MWR-D-13-00364.1 or
      http://journals.ametsoc.org/doi/full/10.1175/MWR-D-13-00364.1
      and Figure 1 from the paper

      shows an even more extreme rise in tornado reports that has to be due to the same factors we see in the US.

      • “There were very few storm chasers before 1970.”

        Now we have so many people chasing tornadoes that they collide with each other while doing the chasing. I’ve heard storm chasers complain that there are now too many amateurs out chasing tornadoes and are making traffic situations more dangerous. I think you can actually pay some “storm chaser companies” to take you along with them. People come from all over the world to do it.

    • I almost died in Tropical Storm. All that footage they have of people being rescued from their cars in knee deep water? After our VW Bug hit a tree and stopped, I thought what I saw in front of us was a guardrail at the end of an alley. I opened the door to walk us over to get back on a road. I couldn’t touch bottom. Found out later it was a guardrail at the the top of a bridge. No cell phones to record our rescue.

      The IMPRESSION given is that such weather events have increased or are, somehow, more intense.
      They are not. They are just better documented.

      PS If you’ve ever been to the Air Force Museum in Dayton Ohio, the P-61 Black Widow they have on display was a part of this weather project in 1947.
      http://www.weather.gov/iln/ThunderstormProject

      • Another PS.
        That tropical storm was in Texas in September of 1973. I think it’s name was Ilya. I also think it was the first one recorded to make landfall twice.
        (It’s center hit land, made a little loop out to sea and then came onshore again.)

    • Rudyard Kipling’s “Mowgli” stories are among my childhood favorites. I’m pleased that you were able to
      work in such a reference to a climatology blog.

    • “Extreme weather in pictures
      Tue, March 28, 2017
      The weather across the world is getting wilder and weirder, causing chaos, death and destruction around the globe.”

      Can this be any less representative of our current weather and the world today? I don’t think so.

  12. Up until recent decades (maybe the 90s) only certified NWS and military weather observers, state police and certified storm chasers tornado observations would qualify as “confirmed” tornadoes. All others would be relegated as “unconfirmed”. It’s easy to see why there is such a spike

  13. I have strong misgivings with categorizing storm severity with economic damage as this has little to no correlation with the storms actual energy. As populations continue to expand and migration concentrates populations into even more dense spots development has expanded into previously “marginal zones”. All too often the areas affected by sever weather events are in marginal development zones such as river bottoms, flood plains, steep unstable hillsides, and below sea level. Compound this observation with the economic value of such marginal dwellings will not amount to much actual economic damage. While a tornado that decimates a highly populated urban center such as Atlanta GA make the headlines, a similar weather event in “Bugsplat AR” may not even make it on the charts.

  14. http://www.cbc.ca/news/technology/climate-change-more-turbulence-1.4064783

    Johanna Wagstaffe is now promoted to little helper of Bob McDonald Quriks and Quarks on CBC…

    “A new study conducted by atmospheric scientist Paul Williams, of the University of Reading in the United Kingdom, suggests that an increase in carbon dioxide concentrations in our atmosphere could cause changes in the jet stream over the North Atlantic flight corridor, leading to a spike in air turbulence.”

    From the abstract:

    Here, we use climate model simulations to analyse the transatlantic wintertime clear-air turbulence response to climate change in five aviation-relevant turbulence strength categories. We find that the probability distributions for an ensemble of 21 clear-air turbulence diagnostics generally gain probability in their right-hand tails when the atmospheric carbon dioxide concentration is doubled.

    https://link.springer.com/article/10.1007/s00376-017-6268-2

    Solid data based stuff…

Comments are closed.