Reposted from the Cliff Mass Weather Blog
U.S. global weather prediction run by NOAA/National Weather Service is now in fourth place among national centers and FAR behind what one would expect from the world-leading U.S. weather research community.
Why?
The answer is clear: the vast U.S. weather community does not work together effectively in developing weather prediction models and transitioning research to operations.
NOAA has highly competent and motivated weather modeling researchers in its several labs and in the National Weather Service. The center of U.S. academic research in meteorology and modeling is located at the National Center for Atmospheric Research (NCAR) in Boulder, Colorado, an entity run by consortium of U.S. universities (UCAR: the University Corporation for Atmospheric Research). The U.S. Navy also does research and development on weather prediction models in places like the Naval Research Lab in Monterey. But with all these labs and researchers, the U.S. cannot field a world-leading weather prediction model.
Here is the issue:
- NOAA/NWS has its own models and is developing its own global prediction system called UFS (Unified Forecast System).
- UCAR/NCAR has its own models and is developing its own global prediction system called SIMA (System for Integrated Modeling of the Atmosphere).
- The military (mainly the U.S. Navy) has its own models and developing its own global prediction system.
There are all trying to do the same thing. Spending many tens of millions of dollars. None are state of the art.
The separate development of the same type of global prediction model has been a disaster for the U.S.:
- NOAA/NWS lacks the innovation, manpower, and ideas of the academic community.
- The academic community work lacks sufficient resources and misses the effective transfer of its research to societal needs.
- The Navy modeling system is lagging behind NOAA in forecast skill, with obvious implications for national security.
Thus, both government and academic efforts fall short of being state of the art in global weather modeling and prediction. Progress is slowed. Tens to hundreds of millions of dollars are used poorly. The costs to society are huge. You care about global warming? State-of-science numerical weather prediction is a first line of defense against severe weather.
Turf Battles, Ego, and the Inefficiency of Big Bureaucracies
How do I put this discreetly and without offending anyone?
Big governmental bureaucracies are not known for efficiency and innovation. Surely, I know of exceptions to this statement, but from what I have seen, folks get comfortable in safe government jobs. Jobs in which excellence is not heavily rewarded and where failure in a Civil Service position does not risk job loss. Government bureaucracies tend to expand over time, with little trimming of inefficient branches or activities. In NOAA model development and responsibility for model development are split over several, often competing, offices. And frequently they don’t want to help each other or work together, attempting to protect their piece of the pie and turf. Let me emphasize that I have great respect for NOAA researchers–they are working in a broken system.
And scientists in major government-supported labs (like NCAR) can also become content to play in their own scientific sandbox, seeing little reason to work with others.
Then there are the human frailties of scientists and scientific administrators. Too many times I have heard NCAR weather modelers dissing NOAA folks, and sometimes the other way around. At a meeting to discuss working together on a major joint effort (the EPIC modeling center), an important university lab administrator said disparaging remarks about his counterpart in the National Weather Service. Such toxic interactions make cooperation more difficult.
Why Isn’t Competition Working?
When I write blogs on this topic, inevitably someone brings up the point of competition? Isn’t it good to have many U.S. groups trying to do the same thing? My answer:
- We have a lot of competition going on right now in the U.S. weather modeling community, and it doesn’t appear to lead to a superior approach.
- Global weather modeling is one of the most complex tasks our species attempts: simulating the future from the molecular to the global scales, requiring complex data assimilation, model development, and statistical post-processing. Weather models (often coupled to ocean, ice, wave and land-surface models) encompass hundreds of thousands to millions of lines of code. They require the largest computers on the planet. A HUGE effort. We didn’t have three Apollo programs in the 1960s. Instead the complex problem was divided among groups around the nation. But in weather prediction we have three groups attempted to do something even more complex.
- There are certain key problems that need to be solved by everyone, such as in model physics. Only by dividing up the problem can we attend to all the issues. Having three or more teams trying to do the same tasks, inevitably leads to important work not being done well or at all.
- Money and science/technology talent are limited.
- There is plenty of international competition, with other groups working on global prediction (like the European Center)
Congress Knows There is a Problem. They Proposed a Way Forward.
Problems with U.S. global weather prediction are well known and has been covered extensively in the media (e.g. here) and in this blog.. Congress has had several hearings on the subject (one in which I testified last year) and they have passed legislation providing funding to NOAA for supporting weather prediction research and development.
Importantly, Congress, with full bi-partisan support, passed new legislation that set up an “Earth Prediction Innovation Center” focused on “advancing weather modeling skill, reclaiming and maintaining international leadership in the area of numerical weather prediction, and improving the transition of research into operations.”Congress intended the EPIC center to be a robust, independent entity that would bring together the entire U.S. weather research community to build the best weather prediction system in the world. To FINALLY, break through the divisions and duplication that are crippling U.S. efforts. And Congress gave NOAA millions of dollars to make things happen.
But as I shall describe, some in the NOAA bureaucracy has installed roadblocks in the way of EPIC.
NOAA is about to make a big decision regarding EPIC: will NOAA work with the UCAR/NCAR–the weather academic community– to create it a viable center that will transform U.S. weather prediction? Or will the current stagnation continue?
Leaders in my community testifying in Congress about U.S. weather prediction
NOAA’s Big Decision
NOAA is about to make a decision that will determine whether U.S. operational numerical prediction is second or third rate, or will become the best in the world.
As noted above, Congress has authorized and funded an EPIC center that would promote cooperation and innovation, and gave the responsibility to NOAA to make it happen.
NOAA put out a request for proposals (RFP) for EPIC, but instead of following the will of Congress to create an independent center that would bring the entire community together to create the best global modeling system in the world, they altered the effort into a support services contract with NOAA. Science-development or scientific research are not even in the document. Bringing the community together is not in the document. Innovation is not in the document. NOAA retained complete control. The intent was pretty transparent.Now a major hope for many of us was that UCAR/NCAR, which represents the research community, would win the EPIC contract, reducing the chasm between NOAA and the academic community (UCAR/NCAR) and bringing the two groups together. NOAA personnel made EPIC as unattractive as possible to UCAR/NCAR, but miracle upon miracle, UCAR/NCAR did put in a proposal for EPIC, hoping that a small start might grow into the national center that Congress had in mind. And let me be frank, there are others in UCAR/NCAR that want EPIC to fail or go elsewhere, allowing them to continue to do business in the same way.
During the next few weeks NOAA will decide on who gets the EPIC contract–and this decision may well decide the future of U.S. weather prediction. UCAR is the obvious choice, with deep experience in model development, relevant software engineering, and in supporting outside users. No one is even close to their capabilities and experience.
This figure shows the skill of the U.S. global modeling versus other major groups
If NOAA selects UCAR/NCAR, the U.S. weather modeling community can follow a new, and far better path. But if NOAA selects some beltway bandit enterprise or some consortium of less experienced academic institutions, then the future of U.S. operational prediction will not be bright.
In many ways, this is NOAA’s last chance to get global modeling right. Private sector firms, like IBM, are starting to building global modeling capacity. The competition (like the European Center) are surging ahead. NOAA. should be able to create a global modeling system that is the best in the world….not for some jingoist or nationalistic reason… but because better weather prediction would be of great benefit to humanity, providing a potent tool to protect mankind against a range of weather threats.
HT/Cam_S
I have an idea. Instead of all that money wasted on NASA, NOAA, and DOE Climate Dowsing models and their large teams, re-direct it all to improving Operational Forecasting. (not really a novel idea I know, but it needs to be said again and again to drive home the point that funding the climate scam diverts resources away from real science and real needs).
Of course then the forecasting sk1ll of the climate-turned-weather model teams would be held to a standard, unlike the current fraud on science (and the tax-payers) called the CMIP process. Basically today the climate dowsers just keep cranking ever higher resolution garbage climate divinations with zero accountability on correctness, as long as they produce an alarmist output for a socialism-enabling political agenda.
This thread is very misleading to say the least. NCAR (Dave Williamson) acquired the ECMWF weather model from ECMWF so the dynamics were identical as were the numerics (the pseudo spectral method) until very recently. So the only difference was the tuning parameters (physics). And although there are different varieties of physics parameterizations, there are only a few standard ones of each. ECMWF would not provide their parameterizations to NCAR so they had to use some of the standard ones. So to state that NCAR would be a better choice than NOAA is complete nonsense as even NOAA uses the same dynamics and numerics.
Now the interesting thing is that the spectral method has long been touted as the most accurate numerical method and that is why almost all global weather forecast and climate models followed the herd. But recently ECMWF switched to a different grid and numerical method. The obvious question is why would they do that. The simple answer is that with the advent of parallel computers, the global communications required by the pseudo spectral method does not lend it to work efficiently on parallel computers. The new finite difference scheme only uses local communications and works well on parallel computers. So ECMWF has now admitted that a lower accuracy numerical method is sufficent, i.e., that the major source of forecast error is not the accuracy of the numerical method but the error in the parameterizations and the observational data. We pointed out that the accuracy of a finite difference
method was more than sufficient as compared to the pseuo spectral method especially when the accuracy of the spectral method was reduced by several orders of magnitude because of the excessive dissipation (continuum error) necessary to overcome the use of discontinuous forcing :
Browning, G.L., J.J. Hack, and P.N. Swarztrauber (1989). “A comparison of
three numerical methods for solving differential equations on a sphere.” In:
Mon. Wea. Rev. 117, pp. 1058–1075.xcessive dissipation necessary to overcome the use of discontinuous forcing
Now all global numerical weather and forecast models are based on the primitive (hydrostatic) equations. It has now been mathematically proved through the Kreiss Bounded Derivative Theory that this is not the correct continuum dynamical for large scale atmospheric flows:
The Unique, Well Posed Reduced System for Atmospheric Flows:
Robustness In The Presence Of Small Scale Surface Irregularities
G. L. Browning
Dynamics of Atmospheres and Oceans, September 2020
Dedicated to my mentor and colleague, Heinz-Otto Kreiss
Now the obvious question is how can these models obtain seemingly useful results given the wrong dynamical system of equations. I have provided a simple example on Climate Audit that shows
that one can obtain any answer one wants if one is allowed to choose the forcing even eith the wrong system of equations. This is exactly what the modelers have done, i.ee. tuned the forcing to
provide reasonable results for several days. However as the equations are wrong and the obs and parameterizations have large errors, the models go off the rails in a few days. So the modelers insert
new observation every 6-12 hours in an attemp to put the models back on track.
As to the errors that are used to distinguish between models, they are not standard mathematical
relative mathematical errors because that would show how bad the forecasts really are . Instead
they compare the forecasts against just using persistence or climate. This adds large terms in the denominator of the error formula and makes things seem much better than they are. In a manuscript
by Sylvie Gravel that investigates the accuracy of the Canadian model versus obs over the U.S.,
the relative mathematical model error reached 50% within a day and a half. The largest source
of error during this time was the artificial boundary layer dissipation (parameterization) used to keep
the surface velocity from growing too large too fast.
Jerry
Competition only works when there is punishment for failure.
In the market, failure results first in loss of market share which means loss of profit. If the failure continues, then loss of the entire business can be expected.
In government, failure means nothing. It can even result in getting a bigger budget. You have to catch up with the others after all.
If there is no punishment for failure, then it isn’t a true competition.
The usual government department goes the failure was due to lack of funding or lack of guidance either way it’s not their fault they are just lifelong civil servants.
It is called “diversity”, not “competition” (or, God forbid, “competence”).
You know, it never occurred to me previously but seeing those two words together: it would appear they have the same root. Hmm, wonder why?
From Latin “competere.”
Punishment for failure, and reward for success. Neither are at play here.
Unlimited (fake) money does not create (real) talent. Never has, never will. In matters of truth only real talent matters. But, truth is not the topic and thus the fake money prevails.
FWIW, as an operational forecaster in CO, when there has been a difference 3-7 days out between the GFS & EC, the GFS has won the battle pretty much every time over the last year + since the GFS upgrade. And the GEM (Canadian) & ICON (German) rarely beats either.
The situation doesn’t seem quite as dire as the author supposes, at least from my perspective.
I will say the GFS has had its successes of late, certainly when compared to the GFS of old. The Euro seems to have dropped in forecast skill, especially 3 or 4 weeks out. Of the two, I’ll take the GFS 10 days out anytime.
What is totally abysmal are the CPC forecasts. They can’t predict what general direction the Sun will rise in every morning. A total waste of taxpayer money.
Not for at least 2 of the Gulf storms this summer headed to Texas and Florida. For those, GFS was still less accurate than the other models in predicting the landfall location days out.
NASA is providing a family of forecasts for aerosols and gases that I’ve found exceptionally valuable in identifying what’s in the aerosol layers detected by my twilight photometers. See https://gmao.gsfc.nasa.gov/
De-fund and disband NOAA, and start from scratch with new people in charge.
When a bureaucracy gets old enough, it no longer serves its original purpose but instead becomes a political entity that just wastes money. NASA is a good example, so is the EPA. We should build a per-ordained lifetime into these agencies and periodically consider starting with new people. The FBI and CIA are also good candidates for starting over to get the political hacks out.
The reason our weather prediction services are so bad is group-think. People who have invested 10 or 20 years into an approach are loath to even consider something else. They are now the senior people who hire the new people, and guess what they are looking for in the new people? Agreement with the old approach.
Normally I would be in favor of moving this research into our universities but they have become so bereft of anything other than political correctness and social justice that I no longer consider them of much use… Perhaps put all of this underneath the Physics and Math departments where real science (and real math) still occasionally occurs.
Typical government agencies angling for funding. Either combine or admit varying approaches are desired although combining is rarely acceptable to those involved and usually forced causing eliminations. That’s where politics step in an science loses.
I took one meteorology class 50 years ago. The professor claimed then that the theoretical models we studied could predict weather with great accuracy but lacked only 1) the global data of millions of temperature and pressure sensors 2) the compute power to process the models with realtime data.
It would seem his claim at the time was a bit exaggerated.
Anyone with knowledge of these models? How is CO2 concentration or GHG radiative forcing factored into the forecast? If not, how can weather in 2100 be predicted with better accuracy than today.
As far as I know there are no calculations for CO2 or radiative forcing by greenhouse gases used in “weather predictions”. They rely on things that can actually be measured like temperature, pressure, humidity and wind flow. They use historical behaviors and attempt to simulate those for regions so that similar measurements produce similar predictive patterns. It’s all rather understandable and testable, and therefore has shown improvements over the last 40 years.
There is a math theory out there that suggests there is a limit to how far out weather predictions can be made with any accuracy – about 14 days. In this time the slightest error in initial conditions can produce completely different results.
This is all unlike “Climate Prediction”, which is based on untested hypotheses, full of guesses that have no real basis, and cannot be tested repeatedly over a short time to improve it. Add to that the distortion of actual measurement data to fit the models and you have a system which is doomed to failure. A simple symptom of this mess you can actually check for yourself is to look at how far the error margin on “Climate Prediction” has improved over 40 years…it basically hasn’t. This demonstrates no significant new knowledge has been gained in how the models should work.
I am sure the same mathematical limit applies to climate prediction as to weather prediction – just over a different time scale. It would be interesting to see someone work that out. It’s likely to be in the 20 +/- 10 year time frame. It depends on how much “inertia” the climate really has to resist change – the greater the inertia the more predictable it should be. They just need to give up on the whole “CO2 drives everything” idea to start making progress.
You beat me to it. Competition is useless unless there are real stakes and real consequences for losers.
Wikipedia has a very partial list of monumental software project failures. Depending on who’s doing the accounting, most large software projects fail! I can’t find a good link with just the statistics. Most articles also bloviate about their self serving theories that explain why most projects fail.
One of the great successes of the computer world is Linux. It’s the operating system that pretty much runs everything from tiny embedded computers to the biggest super computers. If you have an Android phone, you’re running Linux.
Linux is open source and, for most of its history, run by a benevolent dictator, Linus Torvalds. Although he has recently softened a bit, he is famously not diplomatic. In spite of that, there is no lack of developers willing to contribute their work to Linux. Strangely, I suspect Linus is actually easy to work with, just don’t expect him to pull punches if your work is crap. That contrasts with Richard Stallman who was unable to get enough developers to contribute their work to his operating system.
If I needed a giant software project built, I would look for someone with a history of success running large open source projects. I would be gobsmacked if you could find such a person who also possessed an MBA. 🙂
Back to your original point … the open source model is actually competitive in that many developers are trying to get their work into the project. The best code survives.
Torvalds shepherds the kernel not Linux.
You can’t have one without the other!
Building an operating system is completely different from building a science-based predictive model. An operating system is designed to behave in a particular way and anything outside of that is a bug (or feature if people are too lazy to fix it).
I am unsure one could build a working climate model using open source techniques. One could write it from a design, but how the heck do you design it? This is where you need real physicists how understand real physics to write various concepts – then you could use open source to build it.
Then you have to be able to discard hard work because it simply does not work and move on to another idea…that is where the so called “climate scientists” are abject failures.
You realize that any new model will be designed to increase Climate Change Hysteria, right?
That the political goal of propaganda for economy destroying Climate Change regulations will be the major purpose for the new models, right?
You don’t seriously believe that Congress & the Biden administration will be interested in anything else, right?
Biden on free airfare program to GITMO. THE CIA Farm, Dominion server obtained by Delta Forces in Germany shows Trump WIN by over 400 Electoral votes. Plus over 70 million votes. Bonus, Gina gone. She be ….??? How can Biden put full body weight on broken ankle? Will Biden’s other ankle also need boot to give broken ankle break from boot? Something to watch for. Did not McCain ankle boot switch legs? What does GPS have in common with ANKLE BOOT ??? Enjoy The Show. Will Dominion bring down The House?
My experience in consulting at government labs is that typically after 30 years they become hobby shops. DARPA seems to not suffer the hobby shop problem. They have clear goals but do not micromanage. When the goal is reached or not reached the project is defunded. The learnings are saved. New projects may be spawned from the results of failure or success. Working for them is somewhat brutal but intellectually rewarding. Non producers end up on overhead because no project will buy their time, and are soon gone. Then again, they are really motivated to succeed by the foreign sword dangling overhead.
Cut your losses. Buy into the Brits or the European systems.
Can someone please explain the content of graphs.
What are:
GFS
ECM
CMC
FNO
UKM
CFSR
GFSV16
Who supplied the data for each series (I assume a model) and what it represents?
Is there a graph showing the long term accuracy from each source (average compared to observation, error range & mean squared error, how far ahead to be 90% of time within 1C or 2F of obs or whatever standard you want to apply) ?
Thanks.
I’m not sure complex, complicated weather models are that useful. They lack resilience in the face of changing inputs. They have too many in-built biases and assumptions. Look at the wide discrepancy in the hurricane tracking models. There isn’t just one that is right, there are multiple models with different outputs that people try to “average” out to guess at the actual path. They don’t really get to the correct path until the darn thing actually gets close to land. Even then they can miss terribly.
In high school science 55 years ago we had a project for weather forecasting using the weather maps out of the newspaper showing high/low pressure points, frontal boundaries, and the jet stream, etc. We could track that on a daily basis and it did a pretty good job of forecasting several days ahead what kind of weather we were going to see. We didn’t try to forecast actual precipitation amounts but we could guess pretty accurately if it was going to rain/snow or not. As the weather map changed over time it was easy to integrate it into our project. We didn’t need thousands of temperature data points to do any of this. I *still* depend on tracking those simple weather maps for my own personal edification.
Just an observation from an old man who sometimes had to decide whether to mow the hay field or wait.
So Congress’ solution to the ineffective redundancy of three weather modeling groups is to create a fourth? How very Governmental of them.
Meanwhile the Chinese: “China Is Launching Weather-Control Machines Across An Area The Size Of Alaska”
https://www.forbes.com/sites/trevornace/2018/05/10/china-is-launching-a-massive-weather-control-machine-the-size-of-alaska/?sh=77564b2b6315
The Chinese don’t need any weather control machine. They are already defeating the west by getting them to destroy their economies and standard of living due to “Climate Change”
So, there are no meteorologists in this group of “experts”?
Well, then, I’ll stick to the Old Farmers Almanac and the Farmers Almanac, because they both have a long history of generalized accuracy. Both indicated a late spring in my area this past year and both were right: no trees leafing out until AFTER April 30, and late snows were included in the actual results.
Three bureaucracies are not going to be made more accurate by adding a fourth. Just pull back, get some meteorologists in there – really good, and accurate people, who know their job quite well – and stop trying to predict ‘WEATHER’ 30 years out, because that is just ridiculous. And THAT is what these people are trying to do. The fact is that WEATHER is short term and is ruled by chaos, not computers, and CLIMATE is long-term, also ruled by chaos and not computers.
They can write all the software they want to but it is not going to make an even vaguely accurate forecast any further out than one week. AND even that fails to take into account chaos events like a derecho erupting out of nowhere (July 2020) and damaging huge swaths of property. AS an example, the 1974 tornado swarm that started in the lower Midwest, gained strength and created more and more twisters (up to 178) as it moved northeast and destroyed property, and killed people.
https://www.smithsonianmag.com/smart-news/how-148-tornadoes-one-day-1974-changed-emergency-preparedness-180962706/#:~:text=On%20this%20day%20in%201974,or%20damaged%20thousands%20of%20homes.
That was NOT predicted. It literally came out of nowhere, just as a derecho (straight-line high wind) came out of nowhere last summer and rampaged through Chicago’s western suburbs in to the city, causing huge damages.
When they can’t even predict these things with any accuracy, how am I supposed to believe that they can make ANY kind of accurate prediction for the long term?
Thought question, if computer capacity had never developed, where would we be? No Monte Carlo, boosted trees, roulette wheels, etc? In marine fields that I have watched for many decades, these replaced descriptive works, necessary to know what you have. Some said that models were a barrier to understanding, but these were just called old horse and buggy types. Elaborate statistical programs have become the norm, used for complicated environments where duplication is difficult, which is being somewhat realized. And then there are climate drivers, some biological modelers don’t even know about freezes, just read one such paper. Some don’t even know or care what happens at night. And then there is the “Precautionary Principle,” have to prove it wasn’t due to humans, some have practiced that.
Does the US Military “Own the Weather”? “Weaponizing the Weather” as an Instrument of Modern Warfare?
By Prof Michel Chossudovsky
Global Research, January 15, 2020
Global Research 12 September 2017
Environmental modification techniques have been available to the US military for more than half a century.
The issue has been amply documented and should be part of the climate change debate.
The U.N. Climate Conference (COP 25) met in Madrid with Delegates from nearly 200 countries. The focus was on Greenhouse gas emissions.
Under the 1992 United Nations Framework Convention on Climate Change, “every country on earth is treaty-bound to “avoid dangerous climate change”, and find ways to reduce greenhouse gas emissions globally in an equitable way.”: A narrow consensus which focusses on the nefarious impacts of CO2 emissions (from fossil fuel) on World temperature.
What has casually been omitted from the COP debate is the manipulation of climate for military use.
The broader issue of environmental modification techniques (ENMOD) must be addressed and carefully analyzed. It should also be understood that the instruments of weather warfare are part of the US arsenal of weapons of mass destruction (WMD) and their proposed use by the US military against “enemies” constitutes not only a crime against humanity but to put it mildly a threat to planet earth.
In this essay I am providing the reader with direct quotes from a publicly available 1996 US Air Force document on the use of environmental modification techniques which indelibly provide evidence that the threats are real and must be addressed
https://www.globalresearch.ca/does-the-us-military-own-the-weather-weaponizing-the-weather-as-an-instrument-of-modern-warfare/5608728
NASA’s Van Allen Probes Spot Man-Made Barrier Shrouding Earth.
“Humans have long been shaping Earth’s landscape, but now scientists know we can shape our near-space environment as well.”
https://www.nasa.gov/feature/goddard/2017/nasas-van-allen-probes-spot-man-made-barrier-shrouding-earth
So, the reason US models run hot and see CO₂ weather everywhere is because Civil Service employees are not motivated and unable to work together?
Where the leaders demand that their opinions and beliefs drive program requirements on their employees, should be blamed; Mass lays the blame on workers doing the programming.
Federal turf wars do not happen because the employees want turf wars.
Indeed, most line employees view those other departments as potential employment locations.
Turf wars happen because their superiors are paid by the sizes of their turfs. i.e., how many employees they direct, size of their expense budget and their capital budget responsibilities (buildings, equipment, satellites).
Not, because they fielded a better or accurate weather prediction model.
After nearly thirty years working in the Federal Government, these problems are not from the line employees unless their superiors make it their problem.
Wrong cart, wrong horse.
Don’t think I mind DoD (Navy) running its own WX prediction operation, as the roles and missions are sufficiently divergent from the two Civil entities to retain. After all, DIA is routinely blowing the CIA / NSA out of the water in intel production these days, and has been doing so for years. Cheers –