Claim: Google NeuralGCM AI Might Improve Climate Models

Essay by Eric Worrall

“… Google adds machine learning to climate models for ‘faster forecasts’ …”

The secret to better weather forecasts may be a dash of AI

Google adds machine learning to climate models for ‘faster forecasts’

Tobias Mann 
Sat 27 Jul 2024  // 13:27 UTC

…

In a paper published in the journal Nature this week, a team from Google and the European Centre for Medium-Range Weather Forecasts (ECMWF) detailed a novel approach that uses machine learning to overcome limitations in existing climate models and try to generate forecasts faster and more accurately than existing methods.

Dubbed NeuralGCM, the model was developed using historical weather data gathered by ECMWF, and uses neural networks to augment more traditional HPC-style physics simulations.

As Stephan Hoyer, one of the crew behind NeuralGCM wrote in a recent report, most climate models today make predictions by dividing up the globe into cubes 50-100 kilometers on each side and then simulating how air and moisture move within them based on known laws of physics.

NeuralGCM works in a similar fashion, but the added machine learning is used to track climate processes that aren’t necessarily as well understood or which take place at smaller scales.

…

Read more: https://www.theregister.com/2024/07/27/google_ai_weather/

The abstract of the study;

• Article
• Open access
• Published: 22 July 2024

Neural general circulation models for weather and climate

Dmitrii Kochkov, Janni Yuval, Ian Langmore, Peter Norgaard, Jamie Smith, Griffin Mooers, Milan Klöwer, James Lottes, Stephan Rasp, Peter Düben, Sam Hatfield, Peter Battaglia, Alvaro Sanchez-Gonzalez, Matthew Willson, Michael P. Brenner & Stephan Hoyer 

Abstract

General circulation models (GCMs) are the foundation of weather and climate prediction^1,2. GCMs are physics-based simulators that combine a numerical solver for large-scale dynamics with tuned representations for small-scale processes such as cloud formation. Recently, machine-learning models trained on reanalysis data have achieved comparable or better skill than GCMs for deterministic weather forecasting^3,4. However, these models have not demonstrated improved ensemble forecasts, or shown sufficient stability for long-term weather and climate simulations. Here we present a GCM that combines a differentiable solver for atmospheric dynamics with machine-learning components and show that it can generate forecasts of deterministic weather, ensemble weather and climate on par with the best machine-learning and physics-based methods. NeuralGCM is competitive with machine-learning models for one- to ten-day forecasts, and with the European Centre for Medium-Range Weather Forecasts ensemble prediction for one- to fifteen-day forecasts. With prescribed sea surface temperature, NeuralGCM can accurately track climate metrics for multiple decades, and climate forecasts with 140-kilometre resolution show emergent phenomena such as realistic frequency and trajectories of tropical cyclones. For both weather and climate, our approach offers orders of magnitude computational savings over conventional GCMs, although our model does not extrapolate to substantially different future climates. Our results show that end-to-end deep learning is compatible with tasks performed by conventional GCMs and can enhance the large-scale physical simulations that are essential for understanding and predicting the Earth system.

Read more: https://www.nature.com/articles/s41586-024-07744-y

Reading the main study, they appear to be claiming adding neural net magic sauce produces better short term weather predictions and climate predictions.

The researchers attempted to test their model by holding back some of the training data, and using their trained neural network to product weather forecasts based on real world data which had never been seen before by the AI. They also discuss how their model predictions diverge from reality after 3 days – “At longer lead times, RMSE rapidly increases owing to chaotic divergence of nearby weather trajectories, making RMSE less informative for deterministic models”, but claim their approach still performs better than traditional approaches, once that chaotic divergence is taken into account.

I’m a bit dubious about putting faith in the predictive skill of neural net black boxes. History is littered with scientists who followed all the steps the authors described, only to see the neural net diverge wildly from expected behaviour on demonstration day. I would have preferred if they made more effort to reverse engineer their neural net, to tease out what it actually discovered, if anything, to see if it discovered new atmospheric physics which can be used to create better deterministic white box models.

In 2018 Amazon suffered a serious embarrassment when one of their neural nets went off the rails. They tried to use neural nets to filter tech candidates, but they discovered the neural net exhibited bias against women candidates. The neural net had noticed most of the candidates were men, and inferred it should discard applications from female candidates based on their gender.

Anyone who thinks there is any basis to that Amazon neural net gender bias needs to pay a visit to a tech shop in Asia. Somehow in the West we are convincing our girls from a young age they are not suited for tech jobs. The few women who make it through this Western cultural filter, women like Margaret Hamilton who led the team which programmed the Apollo guidance computer, more than demonstrate their ability. Hamilton is credited with inventing the term “software engineering”.

Neural Nets are incredibly useful, they can identify relationships which aren’t obvious. But that ability to see non-obvious patterns in data carries a severe risk of false positives, seeing patterns which don’t exist. Especially when analysing physical phenomena, if you don’t try to reverse engineer the inner workings of your magic black box neural network once it has allegedly demonstrated superior skill, you just don’t know whether what you are seeing is genuine skill or a subtle false positive.