Reposted from The Cliff Mass Weather Blog
Tuesday, February 25, 2020
Why Should the U.S. Be the Leader in Numerical Weather Prediction?
During the past several years, I have written a number of blogs bemoaning the third or fourth place status of U.S. numerical weather prediction, with suggestions on how we could regain leadership.
But I am often asked: why should we worry that the European Center is way ahead? Why don’t we simply acquire their forecasts and forget about the whole business?
Well, I believe there are powerful, compelling reasons why the U.S. should regain its status as the best in the world in operational numerical weather prediction. Let me give you a few:
1. There is no reason to expect that forecasts made by the European Center (ECMWF) and the UKMET office, the current world leaders, are the best that can be achieved. Properly using its huge resources, U.S. numerical weather prediction can be much better.
I am not saying this as a speculation. This is an area with which I have great familiarity–and there are a number of ways that we can improve upon the ECMWF and UKMET approaches, including not repeating a few of their mistakes or missed opportunities. We could produce far superior forecasts.
2. The U.S. has the largest weather research community in the world– no nation or groups of nations is even close. Thus, we have the scientific infrastructure and expertise to be the best.
The National Center for Atmospheric Research in Boulder
Numerical weather prediction also leans on expertise in computer sciences and access to advanced computer technologies. The U.S. is far ahead in these areas.
3. Many Nations And Companies Depend on U.S. Numerical Weather Prediction and Cannot Afford the ECMWF or UKMET Forecast Products. Same with U.S. universities.
The ECMWF and UKMET office charge big bucks for access to the forecast output. Like hundreds of thousands of dollars a year for private sectors firms wishing access. Many nations and companies cannot afford to pay the high fees. In contrast, U.S. agencies have a policy of making our model forecasts available at no charge-— greatly helping poorer countries, in what can considered a form of foreign aid. The free access also helps new weather start-ups and companies who can’t afford expensive European forecast products.
University research, such as at the University of Washington, depend on the free model grids from the National Weather Service for research and to develop next-generation local prediction systems. ECWMF grids…at 100,000 a year or more..are beyond our financial reach. Thus, the quality of U.S. academic research depends on the quality of NOAA/NWS models.
4. Only U.S. Numerical Weather Prediction Can Service All U.S. Needs
International centers, like the European Center, do global prediction, but they aren’t interested in running high-resolution and specialty weather prediction models over the U.S. Only U.S. weather entities (mainly NOAA/National Weather Service) will do that. We need to be the best for our own good.
Virtually all weather modeling centers are moving towards or using Unified Modeling Systems, in which the same forecasting model works on all scales. So if you are going to have the best model, it will serve both global and local uses.
5. U.S. Numerical Weather Prediction Research and Operation is Spending More Money Than Any Other Nation or Groups of Nation.
I mean spending five to ten times as much as the Europe or the UK. For that price we should be the best. Unfortunately, we are currently wasting huge amounts of resource with large number of redundant efforts. That needs to change. The U.S. taxpayer is already paying to be the best, they might as well get their money’s worth.
6. Global Weather and Climate Prediction are Converging
Global weather prediction and climate prediction are converging towards virtually identical modeling systems: coupled global atmosphere/ocean/crysphere (ice/snow)/land surface models. Furthermore, weather and climate systems are moving together to higher resolution. Such modeling systems are obviously most easily tested for weather and seasonal forecasts. So if the U.S. gives up leadership in the weather domain, it will inevitably do the same in the climate domain. Not good.
7. Operational Weather Prediction is a Key Testbed for Evaluating Physical Understanding of the Atmosphere.
The best way to test physical understanding of the atmosphere is to “stress test” the science by including it in operational models that are run several times each day. Thus, operational modeling can greatly foster science discovery and understanding. If the U.S. gives up global modeling to the ECMWF or others, we would inevitably weaken the scientific infrastructure of the nation.
The Bottom Line: The U.S. can and should be the leader in numerical weather prediction. Giving up such leadership inevitably leads to poorer forecasting for the nation, the undermining of the U.S. scientific infrastructure, and would be damaging to the private sector and lower-income nations dependent on U.S. forecast models.
But the UK’s Met office has a bigger, more powerful and more expensive (£1.2 billion) supercomputer
https://www.metoffice.gov.uk/about-us/press-office/news/corporate/2020/supercomputer-funding-2020
I think they are only using it to mine bitcoins, the too-sensitive climate data is handled “manually”
The advantage of doing it on a supercomputer is (a) It sounds really convincing and (b) There are limited other people who have one to check the result you just have to make sure the model is so complex doing nothing it can only run on a supercomputer.
I have seen no detail behind their estimate that it will produce benefits worth 19 times its cost. I’m sure they think they can get away with making the vacuous claim and no-one will get to challenge it properly. But if it really did have so much bene6fit, surely banks would be crawling over each other to fund such a project?
On balance, I think that Cliff Mass gives compelling reasons why the US should NOT do what he’s asking.
We’re paying for the best, so let’s get it, he says. Maybe we should stop spending so much taxpayer money.
We’d keep giving the info away for free, he says. Stupid socialist! Why for free?
Weather models & climate models are converging, he says. But, climate models are CRAP! I don’t want weather forecasting models becoming crap.
All in all, I think he’s dead wrong.
I remember, that some longtime weatherpredictions of the MET Office has been as wrong as possible.
Long-term weather prediction = climate. And you can’t predict climate….
Long term can be as short as 7 to 10 days. That’s still weather.
The further out you go the less accurate forecasts get because we don’t know current conditions with perfect accuracy.
The cases I remember were for about 6 month, a warm, rainy winter was announced, it was one of the coldest, and a hot dry summer shouldt come, it was cold and rainy.
I’m not sure if the source of these forecasts was CRU
I recall that they also changed from publishing their 10 year “projections/predictions” to 5 year ones. It was obvious to all observers that this was because
a) they were so quickly proved wrong
b) they probably also figured out that people were getting tired of the promised “Mediterranean Climate” never arriving in the UK.
That aside, perhaps the US should aim for supremacy in this area, but only after the stable has been cleaned. Nobody can hope for the best when the whole discipline is utterly corrupted by politics.
“ Global weather prediction and climate prediction are converging towards virtually identical modeling systems“.
“ some longtime weatherpredictions of the MET Office has been as wrong as possible“.
“ Long-term weather prediction = climate. And you can’t predict climate….“.
Got it in one, Krishna Gans, Dodgy Geezer.
As I’ve said many times before, the climate models they are using are weather models not climate models. For climate they have to turn the models upside-down. Bottom-up can work for weather, can’t ever work for climate.
Unfortunately for us taxpayers, bottom-up models are much more expensive to run, so there is every incentive (in the form of very large government grants) for the climate “scientists” to keep going with their current version of diabolically expensive futility.
If I have a look at weather forecast model output and compare it with what may be climate forecast model output, I can’t imagine that both have the same angel of view. They look at complete other and different patterns.
Btw, climate is about 30 year weather, but that’s not what the climate models are working on.
Or will you, Mike Jonas tell me, weather models use proxies ?? 😀
So it’s all some sort of international p*ssing contest? Include me out.
I will trade you. Shut down the US government funded GCMs. And you can have all the weather predictions you want.
I really hate it when people throw out Acronyms without defining them first. Luckily for me after finding about 70 definitions I quickly realized GCM obviously means Golf Course Management.
Ted
And to make things worse, some people equate the acronym to Global Circulation Model, while others intend Global Climate Model, all without specifying which they mean or defining what they mean.
Here’s a link to the WMO obituary for Edward Norton Lorenz. So, that seems to make it official that accurate long term predictions are a fantasy.
Throwing a more powerful computer at the problem just produces more garbage faster. If modelers want bucks for a bigger supercomputer, it is contingent on them to PROVE that Lorenz was wrong. By that, I mean actually mathematically prove. Hand waving doesn’t count.
Entirely different problems.
The problems pointed out by Lorenz does not make better 7 day forecasts impossible.
Because?
Cliff
“5. U.S. Numerical Weather Prediction Research and Operation is Spending More Money Than Any Other Nation or Groups of Nation. I mean spending five to ten times as much as the Europe or the UK. For that price we should be the best. Unfortunately, we are currently wasting huge amounts of resource with large number of redundant efforts. That needs to change. The U.S. taxpayer is already paying to be the best, they might as well get their money’s worth.”
Once this is addressed then talk about more TAXPAYER money!
Cliff, Pat.
This is the key issue. Focus on this issue alone. The industry has become fragmented, bloated, special interest groups. It needs a complete overhaul. The US operators have lost their way.
Good Lord! What a self-centered article! Let us look at the points you make. I put a paraphrase of your position in quotes below:
1 – “Europe does not have the best that can be achieved. We could do better.” But we haven’t, have we? Our track record is poor. And, if money is short, it makes better sense to improve the best in the world rather than start again from a low capability.
2 – “We’re largest, so we could be best.” See the answer to 1) above.
3 – “Many people depend on what we give away free.” Looks like they are getting the service that they are paying for. But why is that a reason for improving the US forecasts? It looks as if there is a marketplace for low-quality free services and high-quality expensive services. If an improved service is going to cost more, who is going to pick up the bill? The US taxpayer?
4 – “Only the US will do specialist US prediction.” Not at all. Europe will do specialist US prediction, if you pay them. And I suspect that will be a cheaper option than expensively duplicating all of Europe’s work.
5 – “We are spending 5x-10x as much as Europe, and wasting most of it.” Pardon? We spend an order of magnitude more and produce worse forecasts? This sounds like an argument for shutting down the US exercise and moving in with Europe – not spending even more on something we do not seem to be good at. Why not work with the UK if they can do things better – we took the A-10 from them, for instance…
6 – “Weather prediction is becoming Climate prediction.” So? Weather prediction is worth money, and we already do it. Climate prediction may never be possible, and if it is it will be available for the world. Why should we have to do it on our own?
7 -“It will aid fundamental science.” But fundamental science is already freely available internationally. Volta and Faraday worked with electricity, Newton with forces; the US is quite capable of making use of their discoveries even though they were Italian and British. If a new field of knowledge is opened up, I am sure the US can absorb it quite easily.
This sounds more like a plea for ‘more research funds’ rather than a proposal for putting the US at the forefront of a new science. We already massively overspend compared to Europe, and we have less to show for it. I suggest that we cut the Met financing down to a maximum of twice the European levels and insist that it will be cut further unless the output improves.
The NWS “predictions” more than a couple weeks in the future are notoriously too warm. They like that — many people are so clueless that that is all they remember even when the time comes & it’s not nearly as warm as predicted.
A-10? I think you mean the AV-8B.
I sit corrected. I knew it was one of them gawd-danged ‘A’ airyplanes. Senior moment!
Nice summary Dodgy Geezer. But perhaps it’s not all doom and gloom …
As Pat says, “ we are currently wasting huge amounts of resource with large number of redundant efforts” – and that sounds like competition, the foundation of the US economy. Like democracy, it seems messy, wasteful and ineffective compared with a command system, but in the end it’s by far the best.
I do suspect that a lot of the redundancy in the system is the wrong kind of competition – competition for government grants rather than competition of ideas – but maybe some of it is genuine. I hope so.
The beatings will continue until the morale improves?
Why Should the U.S. Be the Leader in Numerical Weather Prediction?
The people in charge don’t want that for the same reason they don’t want (& haven’t achieved for 40+ yrs) better/more refined estimates of transient & equilibrium climate sensitivity — they like the inflated results they have right now. Scientific obfuscation.
Why are we seeing a rapid decline in the number of quality weather ground stations around the world to monitor (or even prove?) the possible extinction of life on earth?
If we wanted to get to the truth, the surface globe (land and sea) would have been saturated by now with automated self-calibrating ground stations.
Climate Science seems to be completely at odds with all other scientific endeavors. Normal rigorous science involves gathering more and better data when studying systems (especially when studying complex systems over time).
The obvious answer would appear to be that the facts don’t matter in Climate Science. Hiding data and corrupting data and the failure to gather accurate data is all highly ANTI-Scientific…not merely Unscientific.
“2. The U.S. has the largest weather research community in the world.” A climate research community, maybe.
The “community” is not serious. I believe – please prove me wrong – that models do use a latitude-longitude grid, not only highly inhomogenous, but with singularities at poles. A serious research would start with defining a well-behaved grid, maybe based on an icosahedron. The grid should be adopted by all “researchers”.
Snowfall is notoriously hard to predict.
Knowing that going in, the weather forecasters are being raked over the coals for the total bust of the latest storm predicted for Chicago and environs.
Three days out, it was possible heavy snow.
Two days out, it was almost assured heavy snow.
One day out, the storm track was not following the models.
It turned out to be 2-3 inches on “warm” ground.
I read a “quote” that said something like, “we haven’t missed a forecast this bad in a long time”.
u.k.(us)
It has been my experience that the predictions for temperature are reasonably dependable. However, precipitation, of either type, often has a high rate of false positives.
Cliff Mass is a proponent of spending huge sums of other people’s money, to improve the piss poor performance already achieved by spending huge sums of other people’s money. Hmmmm…. Now, where have we heard that kind of circular ‘reasoning’ advocacy before? “Socialism is the cure for every problem, including the problems created by socialism.” Spending other people’s money is the common factor, or the large intersection area on their common Venn diagram.
Cliff Mass acknowledges (paraphrasing)’“We are spending 5x-10x as much as Europe, and wasting most of it.‘, then argues for more of other people’s money to improve the profligate waste already occurring??? Truly unbelievable! The answer is not only “NO!” but “Hell NO!” This is a perfect example of bureaucratic ‘Waste, Fraud, and Abuse’ that should immediately be addressed by budget cutting for duplicative efforts and redundancies followed by consolidation and reassignment of assets, products, authority, and funding.
I urge each US citizen reading this blog to contact their federal Representatives and Senators and inform them of this area of profligate bureaucratic waste, long overdue for consolidation and budget cutting, as a necessary prerequisite to achieving US competency in weather modeling and forecasting. If ever a bureaucratic ox needed goring, this bull needs the budget cutting horn put to it!
People ignore history. They see huge advances achieved by democracy and private enterprise, and somehow think that the next advance can be achieved by fiat with public money.
How much money are we talking about here? Or are you talking about redirecting the money already allocated?
I agree, I think the U.S. should be the best.
You may have a look Kachelmann weather
It worth a deeper look – Enjoy !
The UK is spending £1.2bn. Their previous setup cost £97m. To make a “worthwhile” improvement you might therefore be looking at say $15-20bn. Still want to play?
https://www.hpcwire.com/2020/02/24/cray-to-provide-noaa-with-two-amd-powered-supercomputers/
The United States’ National Oceanic and Atmospheric Administration (NOAA) last week announced plans for a major refresh of its operational weather forecasting supercomputers, part of a 10-year, $505.2 million program, which will secure two HPE Cray systems for NOAA’s National Weather Service to be fielded later this year and put into production in early 2022.
A 60 second video from the Royal Society no less. Could not believe it!
https://youtu.be/n4e5UPu1co0
The Royal Society made a blunderous(not blunderbuss) climate change clip.
Yes, CO2 and other gases do re-rediate infrared traveling spaceward. However, the amount of energy radiated mostly goes through “clear” areas in the atmosphere and isn’t radiated back to the ground.
The best estimates of “global warming” I have read are less than 1.5°C. which is in line with the trend in actual measurements via satellite and weather balloons.
The graphical presentations of CO2 increase, temperature change, ocean heat content, sea level rise, snow cover, and ocean ice cover are ludicrous. None of the scales, spacing, date range, and format allow for any visual comparison, yet alone a quantitative measure. They all reflect the actual 0.8°C rise in air temperatures from 1819, 1969, 1960, 1970, and 2013 with various degrees of accuracy.
In other words, carefully crafted propaganda, not information. They don’t point out the obvious contradiction in the temperature graph from 1850- a flat period, a rise period, a drop, ending in a rise. Both rises in temperature are well within the error of the measures.
They also don’t even mention that temperature is not a measure of energy, which is what drives the climate. It is more or less a measure of the frequency of vibration(liquids and solids) of molecules or the speed of atoms or molecules in a gas.
The use of average temperatures actually erases any information in temperature measurements. This has been shown by serious studies that demonstrate that most of the rise in air temperatures is due to an increase in the minimum temperature during a day, not the highest temperature.
Philo: “…that temperature is not a measure of energy, …”
Then explain the kinetic theory of gas.
As a Brit I am pleased to know that Americans think we have something better than you, apart from beer.
But The UK Met office has made some awful predictions about what we can expect over a certain summer of winter. I also have a holiday place in North Wales, in shadow of Mount Snowdon. They can’t get the weather right more than a few hours ahead some days.
Having said all that, and having worked in the US, I do think Americans tend to believe that throwing money at things will solve problems quicker than sitting down and thinking for a bit (over a beer at room temperature, as God intended).
Haaa, Haaa, Haaa!
I like the hat, and as far as weather predictions I rely on it several times a day and it changes not quite as much as the headline on Breitbart News. Very helpful.
Any model incorporating any assumption that greenhouse gasses have anything to do with our climate are doomed to failure.
Our climate is completely unpredictable beyond the short term, since it is primarily controlled by the amount SO2 aerosol emissions in the atmosphere from random volcanic eruptions, and the burning of fossil fuels.
Further, Alan MacRae has recently proven the CAGW hypothesis to be false.
Short-term climate??
Curious George:
“Short-term climate??”
Probably 3 years, or less, depending upon the frequency of volcanic eruptions, and changes anthropogenic SO2 aerosol emissions.
Burl Henry – most of what you say is correct, but not, I think, your “Our climate is completely unpredictable beyond the short term“. Climate is a long term thing, it doesn’t have a short term. There are strong climate patterns, both regional and global, over various (long and longer) timescales, which I am quite sure would allow successful climate forecasting. But in the current scientific climate (npi) it isn’t happening. The first thing that needs to change is the mindset in which the study of climate and the study of weather can use the same techniques.
We see, what climate predictions are able to forecast. 😀
Was only one forecast we heard from so called climate forecasters right ?
Can’t remember one only.
Mike Jonas:
Unless we are able to predict when the next volcano will erupt, we can make no predictions about what the climate will be like, even 3 years from now.
There could be a hiatus, with no eruptions in between, and temperatures would rise.
Or there could be series of large eruptions, and temperatures would seriously decrease.
Or more probably, a VEI4 eruption, with some predictable minor cooling.
Some improvement could be made if all volcanoes known to have had a VEI4, or larger, eruption were instrumented and monitored for an impending eruption,
We do have the ability to control the amount of anthropogenic SO2 aerosols in the atmosphere, and this could be used to adjust our climate to some extent, increasing them during warming periods, and decreasing them during cooling periods.
With the correct inputs, supercomputers and AI might be able to extend the predictability range beyond what is currently possible.
crysphere? lol!
Assertion: The U.S. has the largest weather research community in the world– no nation or groups of nations is even close.
Conclusion: Thus, we have the scientific infrastructure and expertise to be the best.
Analysis: Does not follow.
Respectfully, US FedScience is big but big ≠ best. It doesn’t even represent potential. In fact Centuries of US innovation indicate that the opposite is true. 17th Century Citizen Scientists. Mavericks. Small teams. Skunk Works. Scaled Composites. SpaceX.
US Big Weather suffers from “too big.” Think about this scenarion. A brilliant weather scientist or engineer comes up with a great idea. “Instead of more petaflops we can spend 1/50th as much on incorruptible data collection.” Welcome to permanent unemployment.
Being the ‘leader in numerical weather prediction’ would be a worthwhile ambition if we were dealing with well-understood phenomena that just needed appropriate algorithms programmed into more powerful superdupercomputers.
Unfortunately, while weather (and climate) are well-documented, particularly in recent times, actual understanding of the relevant phenomena remains poor. This means that more computing power will merely result in the generation of more detailed bullshit, faster. GIGO in spades!
If you really want to throw more resources at this ‘problem’, advocate for programs that have a realistic prospect of improving understanding.
In the meantime, graciously allow others to have the dubious glory of generating more glitteringly worthless predictions.
Other than pushing at the limits of computer engineering, I see very little return on more accurate 1 to 2 week weather forecasts. We could achieve a lot more by investing in adaptation – removing structures from hurricane-prone coasts and flood plains. If you are going to live in such a place – no government money for you to rebuild. Increase the safety of building codes. Investing in better 24-hour weather prediction and severe storm tracking, detection, and warnings are a better us of the money.
I see no value at all in trying to improve climate prediction through building bigger models. We need to understand the math, physics, and inter dependencies before go spending more on advanced models. Basic research into these areas is a much better return-on-value.
We need to funnel money away from bureaucratic political pot holes and into different Universities that will bring in fresh eyes, minds, and not have tunnel vision caused by political leaders. Take money from NASA and NOAA, and use it to fund new University departments. Let them conduct the new leading edge research. These old government organizations are dying from new age social justice ideologies – let them die. Just like capitalism let’s old inefficient companies go bankrupt, we need to cut funding to old government agencies.
The biggest reason to not just use the ECMWF forecasts is that the most accurate predictions arise by using the consensus of different NWP forecasts. As much as the GFS lags in skill, the combination of GFS & ECMWF forecasts are superior to using ECMWF alone.