This paper was published in late 2017, and we didn’t notice it then. Today thanks to a tip from Dr. Willie Soon, via Willis Eschenbach, we notice it now. The paper is open access. See PDF link below.
Seeding Chaos: The Dire Consequences of Numerical Noise in NWP Perturbation Experiments
Studying changes made to initial conditions or other model aspects can yield valuable insights into dynamics and predictability, but are associated with an unrealistic phenomenon called chaos seeding that can cause misinterpretations of results.
Perturbation experiments are a common technique used to study how differences between model simulations evolve within chaotic systems. Such perturbation experiments include modifications to initial conditions (including those involved with data assimilation), boundary conditions, and model parameterizations. We have discovered, however, that any difference between model simulations produces a rapid propagation of very small changes throughout all prognostic model variables at a rate many times the speed of sound. The rapid propagation seems to be due to the model’s higher-order spatial discretization schemes, allowing the communication of numerical error across many grid points with each time step. This phenomenon is found to be unavoidable within the Weather Research and Forecasting model even when using techniques such as digital filtering or numerical diffusion.
These small differences quickly spread across the entire model domain. While these errors initially are on the order of a millionth of a degree with respect to temperature, for example, they can grow rapidly through nonlinear chaotic processes where moist processes are occurring. Subsequent evolution can produce within a day significant changes comparable in magnitude to high-impact weather events such as regions of heavy rainfall or the existence of rotating supercells. Most importantly, these unrealistic perturbations can contaminate experimental results, giving the false impression that realistic physical processes play a role. This study characterizes the propagation and growth of this type of noise through chaos, shows examples for various perturbation strategies, and discusses the important implications for past and future studies that are likely affected by this phenomenon.
From the conclusion:
Here the phenomenon of chaos seeding was discovered within the WRF grid point model, although other studies with different grid point modeling systems suggest the issue is more generalized since common numerical schemes are the cause. Spectral models likely suffer the same issue of chaos seeding given their inherent, instantaneous communication of perturbations across the entire modeling domain. Thus, chaos seeding within perturbation experiments appears to be a universal modeling problem. In turn, our hope with this study is to bring an awareness to this relatively unknown issue to the field of atmospheric sciences, and other fields where chaos seeding may plague perturbation experiments, such that attempts can be made by researchers to remove potential misinterpretations from their work. From a predictability perspective, chaos seeding presents an intrinsic limit on the predictability of certain features since even if nearly all sources of error can be removed in a numerical weather forecast, any tiny error in any limited part of the domain will rapidly seed the entire model grid with other tiny errors, which will subsequently evolve wherever the atmosphere supports rapid perturbation growth. Ensemble sensitivity and EOF analysis were two techniques presented here that have the potential to mitigate chaos seeding in perturbation experiments toward distinguishing realistic processes, and we hope that these and other new techniques can be used to ensure that chaos seeding does not harm the integrity of modeling experiments in a variety of scientific disciplines.
This paper shows there is an issue in short term numerical weather models that span hours to days. Since we know chaos amplifies with time, one onders how much of lomg term climate modeling is affected by this.
The PDF of the paper: https://journals.ametsoc.org/doi/pdf/10.1175/BAMS-D-17-0129.1