Meandering Through A Climate Muddle

Guest Post By Willis Eschenbach

One reason that I’m always hesitant to speculate on other peoples’ motives is that half the time I have no clue about my own motives.

So … for the usual unknown reasons and with the usual unknown motives, I got to thinking about the GISS climate model known as the “GISS GCM ModelE”, or as I call it, the “MuddleE”.

Like many such climate muddles, it was not designed before construction started. Instead, it has grown over decades by accretion, with new parts added, kluges to fix problems, ad-hoc changes to solve new issues, and the like. Or to quote one of the main GISS programmers, Gavin Schmidt, in a paper describing the ModelE:

The development of a GCM is a continual process of minor additions and corrections combined with the occasional wholesale replacement of particular pieces.

For an additional difficulty factor, as with many such programs, it’s written in the computer language FORTRAN … which was an excellent choice in 1983 when the MuddleE was born but is a horrible language for 2022.

How much has it grown? Well, not counting header files and include files and such, just the FORTRAN code itself, it has 441,668 lines of code … and it can only run on a supercomputer like this one.

The Cray ecoplex NOAA GAEA supercomputer used for modeling. Gaea was funded by a $73 million American Reinvestment and Recovery Act of 2009 investment through a collaborative partnership between NOAA and the Department of Energy.

So I thought I’d wander through the GISS model and see what I could find. I knew, from a cruise that I’d taken through the MuddleE code two decades ago, that they’d had trouble with “melt pools”. These are the pools of meltwater that form on top of the seasonal sea ice. They are important in calculating the albedo of the sea ice. In my previous cruise, I’d found that they’d put a hard time limit on the days during which melt pools could form.

This leads me to a most important topic—the amazing stability of the climate system. It turns out that modern climate muddles have a hard time staying on course. They are “iterative” models, meaning that the output of one timestep is used as the input for the next timestep. And that means that any error in the output of timestep “J” is carried over as an error in the input into timestep “K”, and on ad infinitum … which makes it very easy for the muddle to spiral into a snowball earth or go up in flames. Here, for example, are a couple of thousand runs from a climate model …

Figure 1. 2,017 runs from the climateprediction.net climate model.

Notice in the upper panel how many runs fall out the bottom during the control phase of the model runs … and that has never happened with the real earth.

So I wrote up a computer program that searches through the 511 individual files containing the 441,688 lines of computer code for keywords and word combinations and the like, and when it finds a match it lists the file number and the line number where the keyword appears, and prints out the line in question and the surrounding lines, so I can investigate what it’s found.

To avoid the climate model losing the plot and wandering away from reality, as a programmer you have two choices. You can either fix what’s wrong with the model that’s making it go off the rails … or you can do what the MuddleE programmers did with the melt ponds. You can simply put a hard limit, basically an adjustable guardrail, that prevents the muddle from jumping the shark.

It appears that they’ve improved the melt pond code because I can no longer find the hard limit on the days the melt ponds can form. Instead, they’ve put in the following code:


C**** parameters used for Schramm sea ice albedo scheme (Hansen)                       
 !@var AOImin,AOImax           range for seaice albedo                       
 !@var ASNwet,ASNdry           wet,dry snow albedo over sea ice              
 !@var AMPmin                  mininimal melt pond albedo                
       REAL*8 ::                                                             
 C                         VIS   NIR1   NIR2   NIR3   NIR4   NIR5            
      *     AOImin(6)=(/ .05d0, .05d0, .05d0, .050d0, .05d0, .03d0/),        
      *     AOImax(6)=(/ .62d0, .42d0, .30d0, .120d0, .05d0, .03d0/),        
      *     ASNwet(6)=(/ .85d0, .75d0, .50d0, .175d0, .03d0, .01d0/),              *     ASNdry(6)=(/ .90d0, .85d0, .65d0, .450d0, .10d0, .10d0/),                  *     AMPmin(6)=(/ .10d0, .05d0, .05d0, .050d0, .05d0, .03d0/)

What this code does is to put up hard limits on the values for the albedo for sea ice and melt ponds, as well as specifying constant values for wet and dry snow on the sea ice. It specifies the limits and values for visible light (VIS), as well as for five bands of the near infrared (NIR1-5).

What this means is that there is code to calculate the albedo of the sea ice … but sometimes that code comes up with unrealistic values. But rather than figure out why the code is coming up with bad values and fixing it, the climate muddle just replaces the bad value with the corresponding maximum or minimum values. Science at its finest.

Here’s a comment describing another bit of melt pond fun:


 C**** safety valve to ensure that melt ponds eventually disappear (Ti<-10)
         if (Ti1 .lt.-10.) pond_melt(i,j)=0.  ! refreeze  

Without this bit of code, some of the melt ponds might never refreeze, no matter how cold it got … gotta love that kind of physics, water that doesn’t freeze.

This is what the climate modelers mean when they say that their model is “physics-based“. They mean it in the same sense as when the producers say a Hollywood movie is “based on a true story”

Here, for example, is a great comment from the MuddleE code (the “c” or the “!” in a line indicates a comment):


 !@sum tcheck checks for reasonable temperatures                         
 !@auth  Ye Cheng/G. Hartke                                              
 !@ver   1.0                                                             
 c ----------------------------------------------------------------------
 c  This routine makes sure that the temperature remains within           
 c  reasonable bounds during the initialization process. (Sometimes the  
 c  the computed temperature iterated out in left field someplace,       
 c  *way* outside any reasonable range.) This routine keeps the temp     
 c  between the maximum and minimum of the boundary temperatures.        
 c ----------------------------------------------------------------------

In other words, when the temperature goes off the rails … don’t investigate why and fix it. Just set it to a reasonable temperature and keep rolling.

And what is a reasonable temperature? Turns out they just set it to the temperature of the previous timestep and keep on keeping on … physics, you know.

Here’s another:


c  ucheck makes sure that the winds remain within reasonable         
c    bounds during the initialization process. (Sometimes the computed
c    wind speed iterated out in left field someplace, *way* outside
c    any reasonable range.) Tests and corrects both direction and     
c    magnitude of the wind rotation with altitude. Tests the total    
c    wind speed via comparison to similarity theory. Note that it     
c    works from the top down so that it can assume that at level (i), 
c    level (i+1) displays reasonable behavior.

… when the climate muddle goes off the rails, and the wind is blowing five hundred miles per hour, don’t look for the reason why. Just prop it up, put it back on the rails, and keep going …

Then we have a different class of non-physics. These are tunable parameters. Here’s a description from Gavin Schmidt’s paper linked to above:

The model is tuned (using the threshold relative humidity U00 for the initiation of ice and water clouds) to be in global radiative balance (i.e., net radiation at TOA within ±0.5 W m−2 of zero) and a reasonable planetary albedo (between 29% and 31%) for the control run simulations.

In other words, the physics simulated in the climate muddle won’t keep the modelworld in balance. So you simply turn the tuning knob and presto! It all works fine! In fact, that U00 tuning knob worked so well that they put in two more tuning knobs … plus another hard limit. From the code:


 !@dbparam U00a tuning knob for U00 above 850 mb without moist convection     
 !@dbparam U00b tuning knob for U00 below 850 mb and in convective regions    
 !@dbparam MAXCTOP max cloud top pressure

Finally, all models are subjected to what I call “evolutionary tuning”. That is the process whereby a change is made, and then the model is tested against the only thing we have to test it against—the historical record. If the model is better able to replicate the historical record, then the change is kept. But if the change makes it work worse at hindcasting the past, it’s thrown out.

Unfortunately, as the stock brokers’ ads in the US are required by law to say, “Past performance is no guarantee of future success”. The fact that a climate model can hindcast the past means absolutely nothing about whether it can successfully predict the future. And this is particularly true when the model is propped up and kept from falling over by hard limits and tunable parameters, and then evolutionarily tuned to hindcast the past …

What else is going on? Well, as in many such ad-hoc projects, they’ve ended up with a single variable name representing two different things in different parts of the program … which may or may not be a problem, but is a dangerous programming practice that can lead to unseen bugs. (Note that FORTRAN is not “case sensitive”, so “ss” is the same variable as “SS”.) Here are some of the duplicate variable names.


SUBR	identifies after which subroutine WATER was called
SUBR	identifies where CHECK was called from
SUBR	identifies where CHECK3 was called from
SUBR	identifies where CHECK4 was called from

ss = photodissociation coefficient, indicies
SS = SIN(lat)*SIN(dec)

ns = either 1 or 2 from reactn sub
ns = either ns or 2 from guide sub i2 newfam ifam dummy variables

nn = either nn or ks from reactn sub
nn = either nn or nnn from guide sub
nn = name of species that reacts, as defined in the MOLEC file.

ndr = either ndr or npr from guide sub
ndr = either nds or ndnr from reactn sub

Mo = lower mass bound for first size bin (kg)
Mo = total mass of condensed OA at equilibrium (ug m-3)

ks = local variable to be passed back to jplrts nnr or nn array.
ks = name of species that photolyses, as defined in the MOLEC file.

i,j = dummy loop variables
I,J = GCM grid box horizontal position

Finally, there’s the question of conservation of energy and mass. Here’s one way it’s handled …


C**** This fix adjusts thermal energy to conserve total energy TE=KE+PE 
       finalTotalEnergy = getTotalEnergy()                               
       call addEnergyAsDiffuseHeat(finalTotalEnergy - initialTotalEnergy)

Curiously, the subroutine “addEnergyAsDiffuseHeat” is defined twice in different parts of the program … but I digress. When energy is not conserved, what it does is simply either add or subtract the difference equally all over the globe.

Now, some kind of subroutine like this is necessary because computers are only accurate to a certain number of decimals. So “rounding errors” are inevitable. And their method is not an unreasonable one for dealing with this unavoidable error.

However, twenty years ago I asked Gavin Schmidt if he had some kind of “Murphy Gauge” on this subroutine to stop the program if the energy imbalance was larger than some threshold. In the real world, a “Murphy Swichgauge” is a gauge that will give an alarm if some user-set value is exceeded. Here’s what one looks like

Without such a gauge, the model could be either gaining or losing a large amount of energy without anyone noticing.

Gavin said no, he didn’t have any alarm to stop the program if the energy imbalance was too large. So I asked him how large the imbalance usually was. He said he didn’t know.

So in this cruise through the code 20 years later, once again I looked for such a “Murphy Gauge” … but I couldn’t find one. I’ve searched the subroutine “addEnergyAsDiffuseHeat” and the surrounds, as well as looking for all kinds of keywords like “energy”, “kinetic”, “potential”, “thermal”, as well as for the FORTRAN instruction “STOP” which stops the run, and “STOP_MODEL” which is their subroutine to stop the model run based on certain conditions and print out a diagnostic error message.

In the ModelE there are 846 calls to “STOP_MODEL” for all kinds of things—lakes without water, problems with files, “mass diagnostic error”, “pressure diagnostic error”, solar zenith angle not in the range [0.0 to 1.0], infinite loops, ocean variables out of bounds, one STOP_MODEL that actually prints out “Please double-check something or another.”, and my personal favorites, “negative cloud cover” and “negative snow depth”. Hate it when those happen …

And this is all a very good thing. These are Murphy Gauges, designed to stop the model when it goes off of the rails. They are an important and necessary part of any such model.

But I couldn’t find any Murphy Gauge for the subroutine that takes excess or insufficient energy and sprinkles it evenly around the planet. Now, to be fair, there are 441,668 lines of code, and it’s very poorly commented … so it might be there, but I sure couldn’t track it down.

So … what is the conclusion from all of this?

Let me start with my bona fides. I wrote my first computer program over a half-century ago, and have written uncountable programs since. On my computer right now, I have over 2,000 programs I wrote in the computer language R, with a total of over 230,000 lines of code. I’ve forgotten more computer languages than I speak, but I am (or at one time was) fluent in C/C++, Hypertalk, Mathematica (3 languages), VectorScript, Basic, Algol, VBA, Pascal, FORTRAN, COBOL, Lisp, LOGO, Datacom, and R. I’ve done all of the computer analysis for the ~1,000 posts that I’ve written for WUWT. I’ve written programs to do everything from testing blackjack systems, to providing the CAD/CAM files for cutting the parts for three 80′ fishing steel boats, to a bidding system for complete house construction, to creating the patterns for cutting and assembling a 15-meter catenary tent, to … well, the program that I wrote today to search for keywords in the code for the GISS ModelE climate model.

So regarding programming, I know whereof I speak.

Next, regarding models. On my planet, I distinguish between two kinds of models. These are single-pass models and iterative models. Single-pass models take a variety of inputs, perform some operations on them, and produce some outputs.

Iterative models, on the other hand, take a variety of inputs, perform some operations on them, and produce some outputs … but unlike single-pass models, those outputs are then used as inputs, which the model performs operations on, and the process is repeated over and over to give a final answer.

There are a couple of very large challenges with iterative models. First, as I discussed above, they’re generally sensitive and touchy as can be. This is because any error in the output becomes an error in the input. This makes them unstable. And as mentioned above, two ways to fix that—correct the code, or include guardrails to keep it from going off the rails. The right way is to correct the code … which leads us to the second challenge.

The second challenge is that iterative models are very opaque. Weather models and climate models are iterative models. Climate models typically run on a half-hour timestep. This means that if a climate model predicting say 50 years into the future, the computer will undergo 48 steps per day times 365 days per year times 50 years, or 876,000 iterations. And if it comes out with an answer that makes no sense … how can we find out where it went off the rails?

Please be clear that I’m not picking on the GISS model. These same issues, to a greater or lesser degree, exist within all large complex iterative models. I’m simply pointing out that these are NOT “physics-based”—they are propped up and fenced in to keep them from crashing.

In conclusion, a half-century of programming and decades of studying the climate have taught me a few things:

• All a computer model can do is make visible and glorify the under- and more importantly the misunder-standings of the programmers. Period. If you write a model under the belief that CO2 controls the temperature … guess what you’ll get?

• As Alfred Korzybski famously said, “The map is not the territory”. He used the phrase to poetically express that people often confuse models of reality with reality itself. Climate modelers have this problem in spades, far too often discussing their model results as if they were real-world facts.

• The climate is far and away the most complex system we’ve ever tried to model. It contains at least six subsystems—atmosphere, biosphere, hydrosphere, lithosphere, cryosphere, and electrosphere. All of these have internal reactions, forces, resonances, and cycles, and they all interact with all of the others. The system is subject to variable forces from both within and without the system. Willis’s First Rule of Climate says “In the climate, everything is connected to everything else … which in turn is connected to everything else … except when it isn’t.”

• We’ve only just started to try to model the climate.

• Iterative models are not to be trusted. Ever. Yes, modern airplanes are designed using iterative models … but the designers still use wind tunnels to test the results of the models. Unfortunately, we have nothing that corresponds to a “wind tunnel” for the climate.

• The first rule of buggy computer code is, when you squash one bug, you probably create two others.

• Complexity ≠ Reliability. Often a simpler model will give better answers than a complex model.

Bottom line? The current crop of computer climate models are far from being fit to be used to decide public policy. To verify this you only need to look at the endless string of bad, failed, crashed-and-burned predictions that have come from the models. Pay them no attention. They are not “physics-based” except in the Hollywood sense, and they are far from ready for prime time. Their main use is to add false legitimacy to the unrealistic fears of the programmers.

And there you have it, a complete tour of a climate muddle.


Here in the redwood forest, it’s my birthday.

I’m three-quarters of a century old. And I’ll take this opportunity to thank all of my friends both in real life and on the web for two things.

The first is the endless support that I’ve gotten for my life, my writings, and my research. Everything of value in my life I’ve learned from family and friends. I owe profound thanks to the encouragement and support of people like Anthony Watts, Steve McIntyre, Roy Spencer, William Gray, Charles the Moderator, Viscount Ridley, Bishop Hill, Judith Curry, of course my gorgeous ex-fiancee, and many others for all that you have done. No way to mention everyone individually, but you know who you are, and you have my thanks.

The second thing I’m thankful for is the endless checking and public peer-review of my work. I love writing for WUWT because I know that a) whatever I write about, someone out there has done it for a living and knows more than I do about it, and b) whatever mistakes I make won’t last longer than a couple of hours without people pointing them out. This has been immensely useful to me because it has kept me from following false trails and wasting weeks or years going down blind alleys based on mistaken beliefs. Keep up the good work!

So what does a man do at 75? Well, after doing absolutely nothing today, tomorrow I’m going back to climbing 20 feet (6m) up a ladder to pressure-wash shingles with my new pressure washer … hey, do I know how to party or what?

My very best to all,

w.

Again I Ask: When commenting, please quote the exact words you’re responding to. And if you’re commenting on something not on the thread, provide a link. It avoids endless misunderstandings.

Get notified when a new post is published.
Subscribe today!
4.9 59 votes
Article Rating
262 Comments
Inline Feedbacks
View all comments
February 18, 2022 12:25 pm

Very nice, Willis and Happy Birthday, as well! I appreciate your including ‘Figure 1’ in your post, as it clearly shows the potential, at least for this particular GCM, to veer off into terra incognita. Interesting then, that when Pat Frank demonstrated that large iterative propagation errors in GCMs implied that commensurately large error bands should be placed around their temperature predictions, he was told by the modelers that the GCMs were incapable of attaining such temperatures.

Reply to  Frank from NoVA
February 18, 2022 4:26 pm

the GCMs were incapable of attaining such temperatures.” A typical modeler objection Frank, revealing that they don’t know the difference between a statistic and a physical temperature.

Reply to  Frank from NoVA
February 18, 2022 7:04 pm

Pat Frank has repeatedly written that it isn’t temperature extremes he is talking about, it is the uncertainty in saying anything in particular about the temperatures.

My apologies to Pat Frank if I am misrepresenting his work.

wadesworld
Reply to  AndyHce
February 18, 2022 10:17 pm

Although I’m completely incapable of understanding the math, as I recall despite his best efforts, none of the climate scientists arguing with him ever indicated they understood his point.

Reply to  wadesworld
February 19, 2022 9:10 am

You’re right, ww, and that remains true of them today.

Reply to  AndyHce
February 19, 2022 9:10 am

You are correct, Andy, thanks.

Tom Abbott
Reply to  ResourceGuy
February 19, 2022 2:35 am

Definitely keep your Vitamin D up. It helps with so many bodily functions. And doctors say just about every covid patient that has a hard time with the disease also has a low Vitamin D count.

Tom Abbott
Reply to  Willis Eschenbach
February 20, 2022 12:05 pm

I was doing that, too, until my new doctor told me I had toxic levels of Vitamin D in my system. Keep an eye on your levels.

Reply to  Tom Abbott
February 19, 2022 10:27 pm

That’s why COVID, and just about any cold/flu, disappeared during the summer time.

Ron Long
February 18, 2022 12:36 pm

Yea, Willis, but did you ever do Fortran punch cards? Happy birthday from another 75!

Fraizer
Reply to  Willis Eschenbach
February 18, 2022 2:53 pm

Learned the hard way to always pencil in a sequence number in the corner of punch cards.

Thank goodness those days are gone.

Carlo, Monte
Reply to  Willis Eschenbach
February 18, 2022 5:49 pm

A fat Magic Marker was an indispensable item in the card-deck jockey’s toolbox.

Herbert
Reply to  Willis Eschenbach
February 19, 2022 9:19 am

Willis,
You have probably read John Scarne on Cards.
In the 1920s as a young man he met with Arnold Rothstein the gambling czar in New York.
Scarne took a fresh card pack,, opened it, took off the cellophane and riffle shuffled the Cards.
He then dealt Rothstein the 4 Aces face up.
When Rothstein asked how he did that, Scarne said it was easy if you practiced 10 hours a day for a decade!

Alan M
Reply to  Fraizer
February 18, 2022 4:12 pm

Or the pencilled diagonal line across the stack

Mr.
Reply to  Fraizer
February 18, 2022 4:34 pm

I had to handle trays of cards for each run on the mainframe, so putting sequence numbers on thousands of cars wasn’t an option.

But a diagonal marker line drawn across the top of the tray card deck at least gave us an approximation of what went where after a butter-fingers episode.

Rud Istvan
Reply to  Mr.
February 18, 2022 5:49 pm

That is what I always did for my thesis.

ETHAN BRAND
Reply to  Willis Eschenbach
February 18, 2022 3:51 pm

Pah…punch cards!…BASIC punch tape…a bit of a horder..can still lay my hands on a pile of punch cards for a fortran program I wrote for a college professor trying to prove that lowering rat body temperature would increase life expectancy. Data did not support, so we “modeled” the data to show it did…and received further funding. 1976…nothing has changed….🙂

John Dilks
Reply to  Willis Eschenbach
February 18, 2022 4:38 pm

I never dropped them, but I did reuse Mark Sense Cards by turning them around and punching them. I was a broke college student. It worked out for me until the day came that a computer operator thought I had my deck facing the wrong way inside of my JCL cards. I ended up with a box of computer paper with thousands of errors and my program only had about 60 lines. I also had a note from the Computer Science Department thanking me for showing them an error in their code that was suppose to restrict student output.

MarkW
Reply to  John Dilks
February 18, 2022 5:47 pm

My favorite story is of a program that had an infinite loop with only a single command in it. Form Feed.
I’m told that the paper arced all the way across the room and a full box of paper emptied in about 20 seconds.

Reply to  Willis Eschenbach
February 18, 2022 7:07 pm

I did, once

Gerald the Mole
Reply to  Willis Eschenbach
February 19, 2022 4:13 am

Happy Birthday Day young man, I am eight years in front. By the way it used to be said that if you used paper tape you had to work in a high building with stairs that had a hole down the middle so that you could untangle the tape by dropping one end down the well! Some installations even had a tape winder permanently installed at t the top of the stairs. Happy days.

Paul Hurley (aka PaulH)
Reply to  Willis Eschenbach
February 20, 2022 6:19 pm

I have a punch card story from my university days in the late 1970’s. In one compsci course we were studying PL/1. Late one school night I was wrapping up my PL/1 work on a campus terminal, and I wanted a printout to take home. The mainframe system had a command line language for creating printouts the looked, as I recall, like this:

copy filename to LP at building

where building is the location of the line printer I want to use. So, I ran the command and went to the admin building to collect my printout and then catch the last city bus home. I arrived but didn’t find the printout. Oh well, I’ll be back tomorrow morning. At my first compsci class the next morning, I was confronted by a student worker at the admin building who was rather annoyed at what I had done! Why? Instead of typing LP for Line Printer, I typed CP for… Card Punch! That command generated 2 or maybe 3 boxes of punch cards of my PL/1 code, which someone had to box by hand, just for me. Oops!

But hey, they didn’t have a Murphy gauge.

ResourceGuy
Reply to  Ron Long
February 18, 2022 1:20 pm

I define cruelty as being assigned a Fortran project with punch cards at a point when they were already becoming obsolete. It was also a sign of outdated university resources.

MarkW
Reply to  ResourceGuy
February 18, 2022 5:49 pm

Ever try using a teletype as a user interface?

D. J. Hawkins
Reply to  Ron Long
February 18, 2022 3:02 pm

Somewhere I have a deck of punch cards for a very simple crude oil distillation program I wrote in college. No doubt the rubber bands securing them have rotted away. I’d have to be very careful digging them up, assuming the spirit moves me to.

Rich Lambert
Reply to  Ron Long
February 18, 2022 3:14 pm

About 10 years ago I happened to mention punch cards to some young coworkers. Of course, they had never heard of them.

Carlo, Monte
Reply to  Rich Lambert
February 18, 2022 4:27 pm

Or ferrite core memory planes…

Reply to  Rich Lambert
February 18, 2022 7:09 pm

Hollerith cards

Jan de Jong
Reply to  Ron Long
February 18, 2022 4:25 pm

I remember paper tape and closets with subroutine rolls. That was just before those modern cards.

Reply to  Jan de Jong
February 18, 2022 5:23 pm

Paper is too fragile for military applications so we used mylar tape.

Best part of the project was when we finished a new release and inspectors descended to perform “software quality assurance”. The tape was inspected to see that the punched holes were in spec. and the tape had the prescribed length of blank leader and trailer.

I kid you not!

jeffery p
February 18, 2022 12:55 pm

But science!

Rick C
February 18, 2022 12:56 pm

Happy B-Day Willis. I’m 3 years younger, but FORTRAN was my first computer language followed by Basic and C/C++, but then my career path changed and I started hiring programmers instead of programming.

Anyway, many years ago I was provided with a FOTRAN program that written by a prominent PhD to carry out some quite complex chemical process calculations. When I reviewed the code I found a line that essentially said “if X< 0 then X = 0” with no explanation. X represented a quantity of a particular compound so a negative value was, of course, impossible. Curious, I removed the line and ran some sample data and was surprised to find in many cases the value calculated was substantially negative, not just a rounding artifact. It took a while but I tracked down the error which was causing the problem. I sent a note to the original author but never got an acknowledgment.

Since then I’ve seen lots of code that contains obvious fudge factors and constraining functions that just cover up what would be obvious errors. The climate-gate “Harry ReadMe” file demonstrated the poor practices that appear to be common in GCMs. Your post lays bare how bad the situation is with these climate models, but it is not surprising to me.

MarkW
Reply to  Rick C
February 18, 2022 5:54 pm

I used four different languages while in college, ASM80, Fortran, Pascal and PL/M. My first job out of college I worked for a small company that made EPROM gang programmers, I had to redesign the circuitry, layout the circuit boards and re-do much of the program which was written in 6805 assembler.

Robber
February 18, 2022 1:00 pm

Great work as always Willis. But it seems that in that 440,000 lines of code you didn’t find the key line that reads delta T=fn(CO2)

February 18, 2022 1:01 pm

Willis, congratulations with your special birthday!

The crux of the climate model matter indeed is in:

The fact that a climate model can hindcast the past means absolutely nothing about whether it can successfully predict the future.

Even the hindcast of several multi-million dollar models was not better than of a simple EBM (energy balance model) on a spreadsheet. That was what Robert Kaufmann found, see:
https://www.researchgate.net/publication/228543322_A_statistical_evaluation_of_GCMs_modeling_the_temporal_relation_between_radiative_forcing_and_global_surface_temperature

Here his comment on RealClimate in their better days when discussions were allowed (2005):
https://www.realclimate.org/index.php/archives/2005/12/naturally-trendy/comment-page-2/
and look for Kaufmann around the 6th comment.

Tom Abbott
Reply to  Ferdinand Engelbeen
February 19, 2022 2:51 am

“The fact that a climate model can hindcast the past means absolutely nothing about whether it can successfully predict the future.”

Especially when “the past” the models are hindcasting, the bogus, bastadrized global temperature record, is science fiction.

February 18, 2022 1:24 pm

As well as using the same variable name to represent different things, the names themselves don’t seem to be that descriptive, which makes it tough for the poor person trying to read the code.

I suppose that if you’re a climate model expert mysterious variable names like ‘nnn’ might mean something to you. I know, naming things is hard, but so is gleaning meaning from bad ones.

Joe Crawford
February 18, 2022 1:33 pm

If I remember correctly (I’ve got about 5 years on you), for government contract work we use to bid about 3.5 lines of code per man-day. That included design, coding and testing (i.e., both function and system testing). We assumed around 10 to 15 errors per thousand lines of code into function test and expected around 1 tr 2 errors per thousand out of system test. Using your count of 441,668 line of fortran code in the GISS GCM it won’t be ready for prime time until they’ve spent something like 500 man-years on it in development and testing. Without function or system testing, but giving them a little credit for fixing or kludging the errors they might have stumbled on, I would expect no better than 5 to 10 errors per thousand in the code as it sits today.

As you said: “All a computer model can do is make visible and glorify the under- and more importantly the misunder-standings of the programmers. Period. ” I have to agree. It, and the others like it, sure aren’t fit for prime time.

Reply to  Joe Crawford
February 18, 2022 7:13 pm

As compilers and error reporting improved, debugging generally went much faster.

Joe Crawford
Reply to  AndyHce
February 19, 2022 7:52 am

We usually had pretty strict requirements for system integrity, where the system and each module thoroughly validity checked each input, and for data integrity, where the system guaranteed to either report a mathematically correct answer or an error code and never unintentionally modified data. Meeting both of these requirements usually increased the design time, the size of the code and the testing by up to an order of magnitude over providing just the functional code as the GCMs appear to do. We also found that the man-power estimates for the project were pretty much independent of the programming language and compiler used, which were normally either specified by the user or matched to the hardware.

Vuk
February 18, 2022 1:35 pm

OT- attention if you use Chrome browser, email from Norton antivirus co :
“Google announces zero-day in Chrome browser  
What Happened?
Google confirmed in an official blog post that a new Zero-Day vulnerability (CVE-2022-0609) has been found in Chrome browsers and is being abused by the hackers.
You can read about the specifics of the vulnerabilities on Google’s blog site.
What should you do?
Google has pushed out the latest Chrome update (version 98.0.4758.102) which will roll out over the coming days/weeks.”

Editor
February 18, 2022 1:43 pm

Thanks w, for another excellent insight. In this case, that the core of a climate model, and the part that needs a supercomputer to run, is a random-number generator. The parameterisations simply provide the (very narrow) ranges that limit the random numbers. All you actually need are the parameterisations and a home computer. No 400,000 lines of code and no supercomputer are needed.

February 18, 2022 1:57 pm

Happy birthday, Willis, with great appreciation for your contributions here at WUWT and elsewhere.

You say, “Curiously, the subroutine “addEnergyAsDiffuseHeat” is defined twice in different parts of the program … but I digress. When energy is not conserved, what it does is simply either add or subtract the difference equally all over the globe.”

This is an important point. How much of an adjustment is it? NOAA’s GFDL model CM4.0 includes the atmosphere model AM4.0. There is a two-part paper describing the model. For Part II, this Supporting Information pdf quantifies the effect of using such an approach in the treatment of energy conservation. See p.2 top. (link pasted below)

The dissipation of kinetic energy in this model, besides the part due to explicit vertical diffusion, occurs implicitly as a consequence of the advection algorithm. As a result, the dissipative heating balancing this loss of kinetic energy cannot easily be computed locally, and is, instead returned to the flow by a spatially uniform tropospheric heating. This dissipative heating associated with the advection in the dynamical core in AM4.0 is ∼ 2 W m−2.”

2 W/m^2 seems like an awful lot of heat energy to spread around because it cannot easily be calculated locally. Especially so in an iterative model. And also because this amount of heat is so much larger than the year-by-year changes in GHG forcings being investigated.

https://agupubs.onlinelibrary.wiley.com/action/downloadSupplement?doi=10.1002%2F2017MS001209&file=jame20558-sup-0001-2017MS001209-s01.pdf

February 18, 2022 1:59 pm

Fortunately Earth’s climate is not an iterative model in FORTRAN.

The energy balance on Earth is a function of two very powerful temperature controlled processes.

Here is something to confirm. Turn off the nuclear reactors in the Sun and Earth and determines what happens to the oceans. I get 35,000 years for the first 2000m to form ice. Point here, sea ice has a huge bearing on the global energy balance. Where does it figure in the “greenhouse effect”.

However sea ice plays a small role in the energy balance compared with high level atmospheric ice over the oceans that kicks into gear when the atmospheric water reaches 45mm. Clouds are a function of surface temperature and ocean surface temperature is limited to 30C by the persistence of the cloud. Parameterising clouds is unphysical and definitely not “based on physics”

From what I have seen written about climate models, I believe they consider that excess surface heat can find its way into the deep oceans. That is unphysical because it is impossible to warm deep ocean water from the surface. Warming oceans is indicative of reduced evaporation not surface heat input.

bill young
February 18, 2022 2:06 pm

Happy Birthday Willis and may you enjoy many more. Your posts are always interesting and thought provoking.

Rud Istvan
February 18, 2022 2:08 pm

Happy birthday WE.
BTW, an easy and inexpensive way to ‘fireproof’ your shingles is to coat them in a colorless solution of diammonium phosphate (DAP, an inexpensive fertilizer) mixed with a clear water based latex sealant. Maybe that is what you are buying, branded. Simple to home brew. 5% by weight in solution suffices.That is the chemical (usually artificially colored purple or orange for visual drop application reasons) used to fight forest fires. Great flame retardant.

February 18, 2022 2:17 pm

Thanks, Willis, for a very systematic and detailed demolition of a GISS GCM Muddle.

Philip
February 18, 2022 2:25 pm

it’s written in the computer language FORTRAN … which was an excellent choice in 1983 

I respectfully disagree. Fortran was outdated by then. There were better languages around.
Fortran was used by “researchers” because they were not really interested in computing, beyond being able to use computers to solve things that were beyond their pocket calculators, and to make use of the large number of libraries available. Particularly statistical analysis libraries. Because, as we all know, statistics is hard. Getting it right is even harder.

People writing the code didn’t really understand what they were doing. Fortran was easy to learn (well, easy to come up with something that compiled, and printed a number at the end. The alternatives were harder to write a program which would actually compile, and run to completion without some sort of runtime error).

What you describe here is typical of stuff I have seen – patches to try to make broken code work. The real problem can be as simple as a programming error, it can be a runtime error where an arithmetic overflow goes undetected, the algorithm is incorrectly coded, or that the algorithm being used is actually wrong/inappropriate.

The programming mistakes are hard to detect, after the fact in an unstructured language, and even more so in half a million lines or so of that language.

Basically, not fit for purpose.

My favorite story about blind use of other’s code comes from the same era, 1980’s.
I had my computer science degree and was doing some graduate student work for the computer science department to earn a few pennies. Part of that work was running a “surgery” for undergrad computer science students to bring along their sick programs for a bit of help diagnosing the problem. One day, a girl I had never seen before arrived and asked if I could help with her program.

At that time, we were still using punch cards, and teaching using Algol60 as the programming language.

She plopped a lineprinter listing on the table and unfolded it.
I had honestly never seen anything like it. It was a solid block of code. No spaces, no indentation, just 80 columns wide, solid code.

At the end was the compilation error message:

“Mismatched Begin-End”

For those that don’t know, Algol60 uses the key words BEGIN and END to “bracket” together sections of code.

For a computer science student, I would have refused point-blank and told them to go back and re-enter it, with proper indentation to reflect the structure as they had been taught.

I asked a few questions – she was doing one of the “soft” sciences for her MSc. She had some data to analyze and he supervisor had handed her a book and pointed to a listing for a program to do what she needed. He told her that Algol60 was insensitive to spaces and indentation, so she didn’t need to worry about that. She just sat down at the card punch and started typing, ignoring spaces.

I explained that it was going to be painful because of the lack of any sort of structure, but we would have a go anyway.

I explained the technique of using a pen to draw lines between BEGIN-END to see what matched up with what.

45 minutes later, we found it. I was pretty much expecting it would be bad enough that she would have to re-type the whole thing again, but it turned out that it could be done by removing three cards from the deck, and replacing those with five.

It compiled. It ran.

I didn’t dare ask if the answer was correct. But she was happy with it, and so was her supervisor.

wadesworld
Reply to  Philip
February 18, 2022 10:26 pm

Fortran programmers later moved to SAS. Their tunnel vision knows no limits. When my university computing department needed to create some Helpdesk ticketing software, the modelers decided that SAS was the perfect environment in which to create a Helpdesk ticketing program, because, you know, SAS was the right answer for everything.

dk_
February 18, 2022 2:27 pm

Excellent post.

Until the software source code, compilation records, the input data, runtime conditions, and runtime records are made available for qualified, independent reviewers, no software should be “trusted” any more than one would a casual stranger.

At some level, is there anything that isn’t “physics based?”

Reply to  dk_
February 18, 2022 7:24 pm

RPGs

Disputin
Reply to  AndyHce
February 19, 2022 3:10 am

Rocket Propelled Grenades?

February 18, 2022 2:41 pm

Happy birthday

Richard Williams
February 18, 2022 2:49 pm

Best wishes Willis for a Happy Birthday, from middle England. Thank you for your good science, good sense and good humour.

Mike Thies
February 18, 2022 3:08 pm

Thirty minute iteration. Hmmm?

No wonder there are so few lines of code to model at least six interrelated subsystems whose natural iteration is one solar year.

It’s climate, not weather.

Tom.1
February 18, 2022 3:30 pm

Willis- You and I are the same age, and I also have a technical background, but otherwise, my life is pretty mundane compared to yours. I enjoy your posts so much. My first exposure to Fortan, or any programming was in college. We were still using card decks of course, and I recall a class problem solving a quadratic equation using the Newton-Raphson method or something like that. I moved on from there to industry where I became acquainted with linear programming, which we used to optimize the inputs and outputs of petroleum refineries. They also used punch cards and were probably even less accurate than today’s climate models. They were of course iterative, since they used the linear programming algorithm, but they were extremely dependent on some initial assumptions. There was no recursion then based on a run output. I suppose somebody might have come up with an automated key punch machine that would give you a new deck based on your just completed run, but we did not have that. As time went on, computers became powerful enough that you could add recursive calculations into which you incorporated non-linearities, and you could model a refinery down to the gnat’s ass in about two minutes. That was fun. Now I spend my time programming, which I have always loved, but the effort is not for refineries, but for automated trading systems for stock index futures. I do it all day, every day, just about. It’s fun.

Regarding your analysis of climate models, do people in the climate change business ever get exposed to your ideas? I feel like they should. If they do, what do they say? I’d really like to know what a Michael Mann, for instance, has to say about your post.

Mr.
Reply to  Tom.1
February 18, 2022 4:37 pm

Mann just blocks or deletes any information that doesn’t help push “the cause” (as he terms his climate “science”).

Tom Abbott
Reply to  Mr.
February 19, 2022 3:13 am

Yeah, the Climate Change Charlatans say “why should we turn over our data to you when all you are going to do is try to find something wrong with it?”

My question is: Why should we spend Trillions of dollars trying to fix CO2, when we can’t get confirmation of the “data” that says we need to fix CO2?

Reply to  Tom.1
February 18, 2022 7:27 pm

It seems hard to believe that anyone would want to know what MM says about anything.

Tom Abbott
Reply to  Tom.1
February 19, 2022 3:06 am

“Now I spend my time programming, which I have always loved, but the effort is not for refineries, but for automated trading systems for stock index futures. I do it all day, every day, just about. It’s fun.”

You sound like my nephew. He worked at the Citadel in Chicago working on automated stock trading systems. He has since retired at the age of 40.

MarkW2
February 18, 2022 3:41 pm

What a fabulous post. As someone who builds and uses computer models I find it unbelievable how so-called climate ‘scientists’ can be taken in by the idea that modelling past events somehow means it’s possible to model the future at anything like the precision being claimed.

Yes, the models are useful but we know they’re all wrong when it comes to predicting the future. What a bunch of conmen these people have become.

Tom Abbott
Reply to  MarkW2
February 19, 2022 3:14 am

“What a fabulous post. As someone who builds and uses computer models I find it unbelievable how so-called climate ‘scientists’ can be taken in by the idea that modelling past events somehow means it’s possible to model the future at anything like the precision being claimed.”

They are not modeling past events. They are modeling science fiction.

I’m amazed that climate scientists can be taken in by a bogus, bastardized global surface temperature Lie.

They should try to hindcast reality instead. The U.S. regional chart (Hansen 1999) is representative of the real global temperature profile. Unmodified, regional temperature charts from all over the globe have the same basic temperature profile as the U.S. chart. None of them look like the bogus, bastardized Hockey Stick chart lie.

comment image

February 18, 2022 3:47 pm

Happy birthday, Willis. 🙂

And thanks for the outstanding post. There’s nothing that illustrates the truth of a matter better than getting down to the details.

And without wanting to be self-serving, “any error in the output becomes an error in the input” exactly describes the logic at the center of “Propagation of Error …

Thanks again, Willis, for giving us all the gift of an outstanding analysis.

Clyde Spencer
February 18, 2022 3:52 pm

What this code does is to put up hard limits on the values for the albedo for sea ice and melt ponds, as well as specifying constant values for wet and dry snow on the sea ice.

I’m a little surprised that they use such a simplistic approach. The melt ponds will have a specular reflection that varies with location, (i.e. latitude), and with the day and time of day. The specular reflections vary from about 10% at a 60 deg angle of incidence to 100% at 90 deg. While snow, and to a lesser extent ice, is a diffuse reflector, one should use the bi-directional reflectance distribution function, and integrate over the hemisphere to obtain the total reflectance. Even snow has a strong forward reflectance lobe for low sun angles because of a tendency for the snow flakes to be sub-parallel. Therefore, the atmospheric scattering and absorption will vary depending on the angle of the outbound reflected rays.

If they can’t handle clouds, then they must be parameterizing the melt water ponds as well.

Reply to  Clyde Spencer
February 18, 2022 7:29 pm

snow exist over latitudes in a very uneven manner

Clyde Spencer
Reply to  AndyHce
February 19, 2022 8:43 am

That too!