Do You Want More Complex Tornado Warnings??
Guest essay by Mike Smith, CCM
This is the story of a bad idea: probabilistic storm warnings.
Under the current storm warning system, you are either in a storm warning or not. For example, in the radar illustration below, the red area is a tornado warning which is nested in a yellow severe thunderstorm warning (large hail or damaging winds).
So, if you are in the tornado warning, your weather radio or tornado siren goes off, or the TV cuts in to let you know. You immediately go to shelter. That’s it; that’s all a member of the public has to know.
Unfortunately, the meteorological research community has been in love with the idea of probabilistic weather forecasts (“20% chance of rain”) for decades and now wants to extend that concept into tornado and other storm warnings. In their minds, probabilities of flash floods and tornadoes (to name two) are “more scientific” than “deterministic” (yes or no) warnings.
What might a probabilistic tornado warning look like? Below is an example:
presented yesterday to an American Geophysical Union meeting
Instead of “Take Cover!” (or not) as with the current system, your local television station will show you a graphic similar to the above that will update as often as every one minute.
Instead of, “Go to the basement!,” you will learn you have a 70% chance of tornado in the next 15 minutes, a 40% of a tornado in the next 30 minutes and a 10% chance of a tornado in the next 40 minutes. And, the probabilities you are given will change at least every five minutes and, perhaps, as often as every one minute. Really.
This is a terrible idea that will cost lives.
Why is this a bad idea? Let me count the ways:
- Surveys (here, among many others) show the public does not adequately understand what a “20% chance of rain” officially means (other than 80% is more than 20%). Those probabilities (and it rains far more often than a tornado occurs) have been around for a half-century.
- If we have not been able to educate the public over a half-century as to the meaning of probability of precipitation, there is virtually no hope of educating them to learn that a 10% chance of a tornado in 30 minutes is actually high.
- Unquestionably, you should go to the basement or take other shelter if there is a 40% chance (which is very high) of a tornado in the next 20 minutes. I confidently forecast this 40% number will cause mass confusion (deadly when dealing with life-threatening weather) and far fewer will take shelter than do now — and, that deaths and injuries will almost certainly rise.
- With the colors and numerical probabilities constantly changing, they will be nearly impossible to convey on television and online in a meaningful way. In the above illustration, part of the area in the tornado’s path is green. Green is a color associated with safety, not danger.
- Commercial radio and weather radio will not be able to convey this type of warning at all.
- What do you do with tornado sirens?
- These probabilities will be uncalibrated. Because tornadoes are so rare at a given location, there will be no way to know if a 55% chance of a tornado is actually meaningful or if it is just a number.
I first heard about this concept at a meeting in Norman, Oklahoma, five years ago this month. I, and others, raised these concerns then and since but those concerns have fallen on deaf ears. To give a simple example, I’ve mentioned that green is not a color that should be used in tornado warnings. As recently as yesterday (see above illustration), it still was.
What makes this even worse is that the existing tornado warnings continue to decline in accuracy — a topic we have discussed several times on this blog; if you want a recent example of a missed warning of a major tornado, go here.
Before we make any more changes to the warning system, we need to arrest the problems that have caused existing tornado warnings to be less accurate.
The tornado warning system has been a magnificent scientific accomplishment worthy of a Nobel Prize. The idea to create probabilistic tornado warnings needs to go away and the excellent brainpower of those involved should be used to fix and improve the existing warning system.
Mike Smith has been doing weather for decades, and has written a fantastic book about the state of weather warnings in the United States, I highly recommend it.
Warnings: The True Story of How Science Tamed the Weather
AW comment:
Mike asks What do you do with tornado sirens?
The answer? Run them probabilistically. If there’s a 40% chance of a tornado, run the sirens at 40% volume of course.
Imagine if somebody in the met community suggested that like I just did, but I did so entirely sarcastically.
That’s the kind of off the rails thinking that “probabilistic tornado warnings” is. This new warning concept is worse than a bad idea. It will lead to even faster warning fatigue. Plus, how would one respond to a “40% chance of a tornado”??? Why force people to make such decisions? It’s madness.
What conditions give rise to a warning ? Is there some formula or is it a human judgement based on weather conditions?
For years they thought that eventually you could predict weather with more and more computer power and models. Eventually there were hard proofs given that classical turbulence models like Navier-Stokes etc are incomplete, basically they all break down because of quantum mechanics which seeming small manifests at the larger scale.
In layman terms there is a lot of energy actually being carried in quantum domain as QM spins etc (like greenhouse gas effect) which isn’t even measured in classical physics and as it comes in and out of the classical world it makes the predictions impossible. In layman terms you sort of view it as classical chaos.
So what changed is they realized and accepted you will only ever be able to make probability based predictions with classical models. So once you realize that the rest falls out you have to make arbitrary cutoffs such as when you blow a warning siren.
So the idea the “meteorological research community has been in love with the idea of probabilistic weather forecasts” is fundamental wrong. The community simply have accepted the fact they will only ever have probabilistic weather forecasts on their models and hence you will have to make arbitrary choices.
It is human judgement based primarily on interpretation of radar signatures, but also on weather conditions (analysis of the near-storm environment).
If you are lucky enough to have trained spotter reports, then that is factored in as well.
I agree with your assessment and add…..repeated warnings of any probability level with no incidents of tornadoes will give people a false sense of security leading to ignoring when they shouldn’t. Truth is now that weather prognosticators have been brainwashed with modeling as a valid tool with accuracy they’re looking for more applications.
40% chance?
Sounds like odds are against a tornado.
I agree completely…most people have no idea what to make of such numbers and probabilities.
On the other hand: “There is a tornado on the ground 10 miles southwest of your location, moving northeast”, sounds like I need to get to the safest possible location I can find, and do it fast, as in right now.
…and your “other hand” is correct. Mike is pushing out misleading information along with an advertisement for his book with one motive only. (Hint: It’s not public safety.)
1. Rob, you’ve got a reputation for disliking anyone’s [meteorological] opinion but your own. It’s on display here.
2. Here’s some background: A lead time of six minutes recently as compared with 13-15 minutes before is a less useful warning. A 20 point decline in issuing a tornado warning before one appears (POD=probability of detection) against a 5 point improvement in false alarm rate (FAR) is a less accurate warning however you define it. http://www.mikesmithenterprisesblog.com/2017/04/less-accurate-tornado-warnings.html
3. If you actually read the article carefully, you see that it was ME not Mike, that posted up the book link to Amazon. My decision and not his, period. I get a small percentage from any book sale, just like thousands of other bloggers.
You really should apologize to Mike Smith for making assumptions and claims that aren’t true. In this case, it’s you pushing misinformation.
Rob,
#1. Anthony contacted me yesterday to request permission to re-blog my piece. Otherwise, it wouldn’t be on his blog.
#2. While I appreciated him doing so (like thousands of others, he apparently enjoyed reading “Warnings”) Anthony added the information about my book without my knowledge. It is NOT on my original posting (you can confirm that here: http://www.mikesmithenterprisesblog.com/2018/12/do-you-want-more-complex-tornado.html ). Of course, if you don’t like my book, you are free to write your own and attempt to get it published.
So, I will gladly accept your apology.
Merry Christmas and Happy New Year, Rob.
we had tornado watches and tornado warnings.
As a kid, ‘watch’ meant sit on the front porch and watch the sky. The warning siren was down the street.
It meant get into the basement. Jeez what corner of the basement? can’t remember now.
That’s about all I needed to know. Actionable intelligence.
Actionable intelligence – How about, “Move to a position where you can be in a tornado shelter within one minute if necessary.”
If a tornado is likely, it would be stupid to be plowing in the back forty.
I can imagine that there are better ways to give tornado warnings. How about, “Going to the beach today is the equivalent of playing Russian Roulette.”
Actionable intelligence – it’s probably the most effective way to communicate. Confusing people with some kind of sophisticated sounding word salad is probably the least effective.
More stupidification of science.
Nonsense. This is the exact opposite. They are giving the public more information.
Currently there is still a probabilistic forecast for tornadoes at the expert level and
then it gets dumbed down to a ‘yes’ or ‘no’ for the public.
It is transferring the decision expertise from the (presumably well-trained) meteorologist to the completely untrained public. Worse, it gives the public a deluge of uncalibrated numbers (i.e, there is no way to know if a 60% chance of a tornado will produce six tornadoes AT THAT LOCATION for each ten warnings).
One way to look at this, which is natural for me, is that a 60% chance means that there are 60 ways for the event to happen and 40 ways for it not to happen, per incident. A warning is one incident, at least where I live, for one thunderstorm capable of producing a tornado. Of course, one thunderstorm can produce multiple tornadoes. Generally, here, the thunderstorm that produces multiple tornadoes has them one at a time. That said, the general public has a really poor understanding of this and the media reporting does not help.
Ok.. so just how does the 20% chance of rain work? Is it by the day or hour? The hourly forecast always gives the same percentage as the day. But after 10 hours standing outside facing 20 % hourly odds, you have had 90+ percentage of said rain having fallen.
Can someone please enlighten me?
Well before that the weather reporter told you it was going to rain or not based on his cutoff of the probability and he either got it right or wrong. So you got to blame him when it was wrong some percentage of the time and most did.
So does it make a difference if they give you a percentage or the weather reader gets it wrong some percentage of the time … obviously you are happier blaming someone 🙂
My father always expressed it this way:
“20% chance of rain over 60% of the area 30% of the time.”
Make of that what you will…. 🙂
Mr. Layman here.
Several years ago I asked the NWS person to whom I sent our monthly precipitation, via email, a similar question.
The gist of what I remember of his reply (which may apply to your question) was that a long range forecast, say, ten days out, that there is a 30% chance of rain means that there is a 30% chance that it will rain somewhere in the forecast area.
About three days out, a 30% chance of rain means that it will rain in 30% of the forecast area.
So, if the forecast say’s there is a 90% chance today but it didn’t rain in the 10% of the forecast are where you live but it did rain in the the rest of the 90%, they nailed it.
(Be thankful it didn’t on your parade. And keep your eye on the radar.)
I might have misunderstood or mis-remembered what he said. I’m open to correction/clarification.
My personal problem is I don’t have a basement. The Hill Country of Texas does get tornadoes fairly regularly, at least as far as one can tell by TV weather warnings since I moved there 13 years ago.
So what I do is watch the TV weather, which will go to continuous coverage in severe thunderstorm events, which happened about once a year, and see just how far away the front is at any given time. Ideally, I would prefer a TV hookup in the nonexistent basement, but any warning is going to be either/or, that is, there can be a tornado in your area, or their isn’t the right conditions. But, for me, the warning would do precisely nothing, as there is no “safe place” anywhere near that I know of.
I think there are these steel cylinder things that are about size of a phone booth that you can bolt to your garage or concrete floor that will protect you. Saw an ad for them.
Yeah, but the risk of any tornado actually hitting here is somewhat better than my winning the lottery. I did buy a lottery ticket, once, so it is non-zero. But it is quite low.
An issue is of building codes, which should encourage not building on slabs, but then again, the costs involved are probably not worth the benefit.
Safest possible location is relative…the most interior room of the building you are in, which may be a bathroom or a bedroom closet.
Basically put as many walls as you can between yo and the outside.
Close drapes if you have time (and drapes), and stay away from windows.
In a pinch, I have heard of people surviving by doing such things as getting into a bathtub and putting a mattress over the top of it.
Shield small children with your body.
And get a helmet. Protecting your head from blunt force trauma is a big deal and a helmet isn’t a huge financial outlay. I’ve seen old school meteorologists also say to get a pair of safety goggles to cover your eyes. It’s not bad advice and odds are that you can find some uses for them outside storms anyway like home improvement projects.
It would be the choice of the bath or the laundry room. neither of which has a window, but does have an outside wall.
I like the concept of adding two sceptic tanks instead of one, one for use and a second configured as a tornado shelter for decades. Then, when necessary converted back to its original purpose.
Just be sure you pick the correct one….
Run the opposite direction of the nearest trailer park.
I;d say go outside and look for one….that’s what I would do unless it’s nighttime
Not a bad idea, but one should also be aware than even during daytime, they can be right up on you and yet invisible…whether due to be rain wrapped or low clouds and/or trees blocking the view.
They usually make a lot of noise, so that is a clue.
Also, cell phones can be set to receive emergency text alerts…and if this feature is disabled on one’s phone, one may be unknowingly depriving oneself of potentially life saving information.
“Existing tornado warnings are less accurate than in the past.”
Evidence please.
There is LOTS of evidence, including the NWS’s own accuracy figures.
Here is a good place to start: http://www.mikesmithenterprisesblog.com/2017/04/less-accurate-tornado-warnings.html
I’m not sure what problem a probabilistic system solves?
None. It’s their new toy.
That includes me, Ms I’m sort of obsessed with weather (being originally a Brit).
Can anyone explain it to me? And why do weather reports not just include a link to an explanation, anyway?
I don’t know for sure either, but I would guess that it just means that the computer models used by weather forecasters show it raining at a location 20% of the time, which sounds like a means of hedging a prediction to deflect criticism when it goes wrong; it’s not the model’s fault even if it was a 95% rain prediction and it didn’t rain – it just happens that the real world turned out to fall into that 5% slice of the theoretical world.
I suppose you could test the models’ predictive capacity by putting all the model’s 20% rain predictions into a bin, testing each prediction by scoring a “yes” if it did rain and a “no” if it did not, and showing that once a threshold number of scores have been made, the “yes” rate floats randomly around 20% as more scores are added to the testing procedure. You do that for every probability step size (5%, 10% etc.) to measure the model’s accuracy.
I have yet to see an explanation with a weather report of what 40% chance, say of rain or tornadoes means.
40% of an area will get rain at some point, it will rain all over the area for 40% of some time period, 60% of the area won’t get any rain, some parts of the area will get enough rain to equal the average rainfall in the area, there may be repeated periods of rain over 40% of an area…….
You get the idea.
I find the following kind of forecast useful:
There are likely to be periods of rain beginning around 9 am and likely increasing. Most of the area will experience heavy rain overnight.
or:
light showers are possible until about noon. The afternoon should see cloudiness breaking up with some sunshine increasing.
I have no real experience of tornados in central PA. They are rare. One small one did appear to hit the town I live in about thirty years ago, but the damage was limited.
when I was in grade school in St. Paul MN tornadoes were not uncommon.
The closest thing I ever saw was sitting on the porch with thunderclouds everywhere, a big, long green cloud rolled across the sky.
Michael Lewis’ (Moneyball, Liar’s Poker, The Big Short) newest book, The Fifth Risk, includes an interesting history and explanation of current tornado forecast and warning methods.
According to NOAA, in 2018 there were nine tornado deaths. Five in watch areas, one in severe thunderstorm watch areas, one close to a watch area (15 minutes or 25 miles) and two where there was no watch in effect.
https://www.spc.noaa.gov/climo/torn/fatalmap.php
Forecasting is hard. Getting people to act on a forecast is harder.
What percentage of residents in NYC would even know there was a watch or warning? Living in their concrete jungle doesn’t require paying much attention to the weather. Rural folks (fly-over country- deplorables) pay attention, farmers listen to the weather first thing every morning (right after the commodity report), and school districts are fully aware (bus kids etc). Rural folks can even see the sky when they look up. The urban/rural split in Western countries is enormous, and getting wider.
We are also at risk from alarmist warnings. In Oz over the last few years, if you listened to or read weather and other warnings, you’d imagine we were being continually hammered with massive and extreme events. I’ve only lived here for 17 years, but nothing seems to have changed.
Right now we have a ‘zombie cyclone’, I kid you not. It’s being touted as a rare event, but it’s just a cyclone that crossed a small strip of land at the top end of Australia, reached the bay on the other side and continued to develop. It’s not surprising at all.
Even more bizarrely, a storm in Melbourne where it rained a bit had apparently ‘smashed’ Melbourne, and is also apparently caused by this zombie cyclone 3 or 4 thousand kilometres (2 or 3 thousand miles) away! Not only is it a zombie, apparently it has effects thousands of km away!
All this causes people to ignore warnings because very little really devastating happens. When something does happen, people will ignore the crys of “Wolf!”
And looking at the national radar, I imagine a lot of farmers are dancing in the fields because of getting some rain at last, anyway.
People who listen to BOM or any Australia media source for a weather forecast has rocks in their head. I see no reason to listen to any of them.
‘Fake-weather’.
How does this get translated into SAME format for the NOAA alert radios?
As a former operational weather forecaster/warner for NOAA, I feel somewhat able to address this issue. Firstly, there are already too many false alarms, especially outside of the Midwest. Doppler radar has brought out the fact that many thunderstorms rotate or display a bit of rotation, thus causing warners to issue more (tornado) warnings than otherwise would be issued. Secondly, one of the hardest jobs was to NOT issue a warning. There is a line between issuing valid warnings, and issuing too many false alarms which lead to the “cry wolf” syndrome. In the heat of battle, it’s difficult enough to decide what needs a warning (severe thunderstorm for a cell, a line, a line with an embedded tornado, a tornadic cell, or a cluster of tornadic cells). It’s likely that this probabilistic function would be automated, as a lot of the warning process has become over the years, but even so, this adds unnecessary confusion and, yes, danger that action won’t be taken by people who need to take a safety action. I knew warners who issued too many products within a short period of time, and this added too much complexity to the recipients (media, emergency management, law enforcement, and the public at large) Sure, updates need to be sent, but too much information is as bad as not enough, possibly worse.
Thus, I must almost completely agree with the poster. What might be done is to issue a tornado warning, cone shaped for decreasing-probability purposes, and provide the actual probability numbers in a code only to emergency mangers and law enforcement, but convey the warning area as a definitive warning for public action purposes. We learned in the old days that there were two parts to a warning: the issuance of the warning, and action on the part of the recipient of the warning. If there is confusion as to whether to take action or not, the warning is not useful, or at least not as useful as it could be.
This probabilistic warning for short-fused events is a bad idea, likely based on the increasing nerd-ism and group-think in NWS/NOAA administration. For longer-gestating events such as regular forecasts and flood watches and hurricane, sure, probabilities are a fine and time-tested way to convey uncertainty, but in a short-fused and rapidly changing environment, it’s ridiculous. I and my retired colleagues are glad we’re retired, and with good reason!
I grew up in Tornado Alley. A “Warning” meant that a tornado had been spotted – there was really a tornado outside. People got to shelter.
Then I moved further East. Here, a Tornado Warning means that “conditions are favorable for a tornado to develop” – or what I used to know as a “Watch.” Nobody does anything here when there is a Tornado “Warning.” And why should they; there is no tornado!
So I guess my point is that even the current system is screwed up.
It apparently didn’t post the first time. Here is the evidence that NWS tornado warnings, nationwide, are less accurate than ten years ago: http://www.mikesmithenterprisesblog.com/2017/04/less-accurate-tornado-warnings.html
Agreed, Mike Smith!
I stopped paying attention weather probabilities over twenty five years ago.
Our weather report for New Orleans was 30% chance of thunder showers.
The day was humid, relatively still, hazy with some cloud cover.
So, I and a friend went fishing, less than 15 miles away; in the same weather forecast area.
There, our day was hellish with one severe thunderstorm after another hammering us and dragging our boat & anchor hundreds of yards. When lightning got to be very frequent as individual cells passed over us, we pulled anchor and headed to the dock.
The two hours we spent anchored on a bayou just off the Mississippi River were hellish for a nearly 100% thunderstorm slap down.
Twenty five minutes after leaving the pull-out, we were back in New Orleans with hazy, humid, still air and no thundershowers. My wife told me that they did not have a single thunderstorm.
Zero percent thunder storms in New Orleans combined with 90+% thunder storms where we were fishing = 45%; slightly over NWS’s 30% prediction. Must be good enough for government work.
This past week, NWS started off the Nor’easter prediction with slight chance of light snow.
As the storm gathered energy and heavy snow was falling fast, NWS finally increased their probability percent of snow to 60%. We got almost eight inches of snow for their “slight chance of snow”.
Making it very likely that most of America disbelieves and ignores NWS probability percentages.
Words that put fear into every human:
“Hi! Were from the Government and we are here to help you.“
More scary:
The Blues Brothers, Red White and Blues
It’s clear to me from personal experience that it’s already the case that there is no singular definition for probabilistic forecasts. Having lived for years in Portland, OR and then subsequently in the Houston TX area it’s very clear to me that a forecast like ‘80% chance of rain’ means two different things in those two places. In Portland that forecast means rain over the entire region, fairly equally, 80% percent chance that it will happen, 20% chance of no rain for anybody. In Houston it almost always means thunderstorms, 80% chance you’ll find yourself under one. Except sometimes it doesn’t; sometimes, usually in winter, it means Portland style rain, something your kind of expected to know on your own. So I guess the forecast means two things just in Houston alone come to think of it. Consistency? Fugedaboudit.
I have been in the unfortunate position of needing to know the exact probability that I will get rain in order to assess whether I can do some work on the house that needs up to a week of dry weather.
So, I look at these forecast which show this:
“9am light rain … 20%” and I wonder what on earth does it mean.
The most obvious forecast is that the probability of getting ANY rain in the ONE hour specified is 20%. Therefore if there are 5 hours of 20% chance of rain, then the probability of not getting rain should be 1-0.8^5 (or 70%).
However, there is another interpretation. That is that the 20% figure means that at any SINGLE time within the one hour period there is a 20% chance of rain, but if those showers are very short lived, that could mean the actual chance of SOME rain is something like 80%.
The distinction may not matter to a commuter who has to decide whether they need a brolley for the 5min wait for their their train (which is who the forecasts are aimed at). But if you are just about to paint your house and ANY rain could spoil the paintwork, the fact it will likely be dry 80% of the time doesn’t matter, but whether there will be ANY rain.
But, another interpretation is that there is a 20% chance of light rain, and perhaps an ADDITIONAL 10% chance of heavy rain and 5% of absolutely torrential downpour – which makes a 35% chance of some rain.
However, weather doesn’t occur in isolated hourly packets. We tend to get fronts and therefore, when predicting 2-3days in advance,much of the prediction is based on the assessment of when and where the front goes. So, within the front there may be a 50% chance of rain in any hour, and what the 20% may mean is that there’s only a 40% chance that the front will pass over our location.
If this is what the 20% actually means, then if you are working on things that need extended periods of dry, it’s much better to get a “40% chance of a front coming over” rather than the silly “20% chance of rain”. Because during the several hours a front passes over there is a very high chance of some rain. So the actual chance of some rain is much closer to 40% rather than the hourly prediction of 20%.
… and in Case you wonder why I’ve thought about this so much … it rains a lot in Glasgow and recently I’ve been spending a lot of time waiting for days “without” rain.
That probability of rain calc was inverted. But … I just to explain more, it’s not just paint. Instead most is specialist render which can take up to a week to dry here. I can cope with “light rain”, but I cannot cope with extended periods of light rain nor shorter durations of heavy rain.
And just an observation…the UK Met Office forecast has proven to be a better match to actual weather than the lot the BBC are currently using. However, even the Met Office cannot predict rain with a high level of certainty just a few hours ahead. And beyond three days … the forecast is so unrelated to actual weather that it’s useless.
I remember a wonderful Wizard of Id cartoon from a number of years ago, along these lines:
King to Wizard: “What’s the forecast for today?”
Wizard: “There’s a 60% chance that 75% of your kingdom will experience a 40% chance of showers during half of the day.”
King: “If you’re wrong, there’s a 100% chance that you’ll lose your head.”
I notice that the author did not give credit to the author/presenter of the probabilistic graphic in this article. Was it used with the AGU and/or author’s permission? What about the uncaptioned warning graphic? Did the author create that himself?
Hi Stephanie,
The figure is U.S. government and thus in the public domain. It is from a paper, as stated in my piece, presented Tuesday at the US AGU meeting.
That said, I did not use her name because I did not want this to come across as a personal attack. There are many scientists publishing probabilistic warnings. My piece was published out of concern about the deteriorating situation w/r/t current U.S. tornado warnings and the additional issues that would arise if probabilistic warnings are adopted.
And, to anticipate your next question, the evidence that tornado warnings are deteriorating in quality is here: http://www.mikesmithenterprisesblog.com/2017/04/less-accurate-tornado-warnings.html and, in Kansas, just this past summer, we had a major (EF-3) tornado hit a town with no warning of any kind. http://www.mikesmithenterprisesblog.com/2018/06/the-truth-of-non-warning-of-eureka.html
Absolutely correct.
What’s needed is more time and precision before a warning is issued so the people being warned can take cover.
(Better precision would reduce “warning fatigue”.)
That won’t be achieved by burning more Green on the Tree-Ring fire.
It might be achieved by applying real science to understanding the weather of our world.
PS “The Storm Channel” has been using something they call a (I think) TORCON Scale. Sounds very similar to this “probabilistic tornado warnings” stuff. Both are USELESS when it comes to actually helping the people on the ground.
Many drivers are killed by hail when they run in front of the house at hail warning
to bring the vehicle parked under a tree into the garage.