Guest Post by Willis Eschenbach
A learned man was arguing with a rube named Nasruddin. The learned man asked “What holds up the Earth?” Nasruddin said “It sits on the back of a giant turtle.” The learned man knew he had Nasruddin then. The learned man asked “But what holds up the turtle”, expecting Nasruddin to be flustered by the question. Nasruddin simply smiled. “Sure, and as your worship must know being a learned man, it’s turtles all the way down …”
I’ve written before of the dangers of mistaking the results of the ERA-40 and other “re-analysis” computer models for observations or data. If we just compare models to models and not to data, then it’s “models all the way down,” not resting on real world data anywhere.
I was wondering where on the planet I could demonstrate the problems with ERA-40. I happened to download the list of stations used in the CRUTEM3 analysis, and the first one was Jan Mayen Island. “Perfect”, I thought. Middle of nowhere, tiny dot, no other stations for many gridcells in any direction.
Figure 1. Location of Jan Mayen Island, 70.9°N, 8.7°W. White area in the upper left is Greenland. Gridpoints for the ERA-40 analysis shown as red diamonds. Center gridpoint data used for comparisons.
How does the ERA-40 reanalysis data stack up against the Jan Mayen ground data?
Figure 2. Actual temperature data for Jan Mayen Island and ERA-40 nearest gridpoint reanalysis “data”. NCAR data from KNMI. Jan Mayen data from GISS.
It’s not pretty. The ERA-40 simulated data runs consistently warmer than the observations in both the summer and the winter. The 95% confidence intervals of the two means (averages) don’t overlap, meaning that they come from distinct populations. Often the ERA-40 data is two or more degrees warmer in the winter. But occasionally and unpredictably, ERA-40 is 3 to 5 degrees cooler in winter. Jan Mayen’s year-round average is below freezing. The average of the ERA-40 is above freezing. The annual cycle of the two, as shown in Figure 3 below, is also revealing.
Figure 3. Two annual cycles (Jan-Dec) of the ERA-40 synthetic data and Jan Mayen temperature. Photo Source
The ERA-40 synthetic data runs warmer than the observations in every single month of the year. On average, it is 1.3°C warmer . In addition, the distinctive winter signature of Jan Mayen (February averages warmer than either January or March) is not captured at all in the ERA-40 synthetic data.
So that’s why I say, don’t be fooled by people talking about “reanalysis data”. It is a reanalysis model, and from first indications not all that good a reanalysis model. If you want to understand the actual winter weather in Jan Mayen, you’d be well-advised to avoid the ERA-40, or February will bite you in the ice.
The use of “reanalysis data” has some advantages. Because the reanalysis data is gridded, it can be compared directly to model outputs. It is mathematically more challenging to compare the model outputs to point data.
But that should be a stimulus to develop better mathematical comparison methods. It shouldn’t be a reason to interpose a second model in between the first model and the data. All that can do is increase the uncertainty.
In addition, due to the fact that both models involved (various GCMs and the ERA-40) are related conceptually (being current generation climate models), we would expect the correlations to be artificially high. In other words, a model’s output is likely to have a better fit to another related model’s output than it does to observational data. Data is ugly and has sudden jumps and changes. Computer model output is smooth and continuous. Which will fit better?
My conclusion? The ERA-40 is unsuited for the purpose of validating model results. Compare model results to real data, not to the ERA-40. Comparing models to models is a non-starter.
Regards to everyone,
w.
[UPDATE] Several people have asked about the sea surface temperatures in the area. Here they are:
Figure 4. As in Figure 2, but including HadSST sea surface temperature (SST) data for the gridcell containing Jan Mayen. SST data from KNMI
Figure 5. As in Figure 3, but including HadSST sea surface temperature (SST) data for the gridcell containing Jan Mayen. SST data from KNMI
Note that SST is always higher than the Jan Mayen temperature. This is not true for the ERA-40 reconstruction model output.
John V. Wright
“Willis, if you did not exist we would have to invent you.”
Wouldn’t that be rather counterintuitive for he’d just be artificial, much like a computer model, so he’d be constantly off the charts and therefor much inline with the rest of the computer models out there, you know groupthink and all that. :p
Perhaps we could just write a model.
Simple explanations that make a scientific truth self evident for everyone are the best ones. Feynman was the master and Willis is in the same league. “If you can’t explain something to a first year student, then you haven’t really understood it.” Also, note his famous Space Shuttle demonstration with icewater and the 0-ring. Willis always seems to clear out all the obfuscation, smoke and mirrors and flimflam when he explains something to us. Perhaps that should be ‘smoke and mainframes’…
“Turtles all the way down” reminded me of reading Ken Wilbur which features the following version:
Ken Wilbur from a Brief History of Everything
There’s an old joke about a king who goes to a Wiseperson and ask how come it is that the world does not fall down?
The Wiseperson replies, “The Earth is resting on a lion.” “On what, then, is the lion resting?” “The lion is resting on an elephant.” “On what is the elephant resting?” “The elephant is resting on a turtle. On what is the turtle resting on ? You can stop right there, Your Majesty. It’s turtles all the way down
Turtles all the way down, holons all the way down. No matter how far down we go, we find holons resting on holons resting on holons. Even subatomic particles disappear into a virtual cloud of bubbles within bubbles, holons within holons, in an infinity of probability waves. Holons all the way down.
Thanks Willis for a telling post. Current climate scientists would rather rely on models than the horribly spiky and uncooperative real data that is observed. So using one or more sets of model data to confirm the output of the final model gives results more in line with theory. The fact that observational data refute this is seen as a travesty of our inability to do real world observation right.
Bit like modern physics really, where one has to be able to suspend all you know from your senses to even start to understand what the current ‘dream of the day’ really means, but of course it must be correct because a convoluted bunch of maths proves it.
Science has got itself into a really bad state!
Reanalysis. Homogenisation. Why don’t they call it for what it is? Fraudulent!
Willis, nice one! Crystal clear and succinct, as usual. The NZ Maori version of Nasruddin is Maui, a sort of Polynesian imp/everyman who features in all the important myths there.
I suspect Maui may have had a sneaky hand in helping the NZ Climate Coalition to best NIWA’s UEA-related myths about how the temperatures had mysteriously risen over the last century in NZ while nobody was looking.
SteveE says: UMMM Steve from what I read on that last paragraph the volcano heats the surrounding water to 30C you might want to re-read that.
How easy is it to write a program to check the reanalysis against each station data to try to locate and rank the worst offenders for consistent +/- bias [I suppose in terms of the mean difference as a fraction of the standard deviation of the measured data?]?
The purpose of the reanalysis is to interpolate the data onto a uniform grid. Are there bound to be these types differences at the measured points or are there interpolation methods which can make sure that the interpolation field values converge to the data at the locations of the measurements? I wonder how easy it is to try to do that.
To be fair to the re-analysers, I suspect it is not an easy task at all – and they cannot force AGW to emphasise the technical issues when the data is used as though it is observational data. I reckon that the documentation/publications/presentations from the guys who do the re-analyses might mention the technical difficulties and biases that come out from their imperfect and developing methods.
SteveE says..,
The water temperature will be different from the air temps. Anthony states sea water temps as just above freezing, not air temps. Is the air at the station on the island warmer than the air over the water surrounding it? I don’t think you can assume it is.
I suspect that the Jan Mayen weather is difficult to model. The surrounding ocean sometimes freezes, but the ice might be absent for many years like in recent years. The climate is harsh. Wind speeds up to 70 m/s have been recorded. Intruments get packed with snow and even sand. The surrounding mountains cause local wind and Föhn effects. The weather station has been moved 9 times since 1921.
There are descriptions and some metadata for the station in “Stasjonshistorie for norske meteorologiske målinger i arktis” by E. Steffensen, P.Ø. Nordli, I. Hanssen-Bauer. I can translate excerpts if anyone is interested. I can be reached at steinar@latinitas.org.
paulID says:
March 8, 2011 at 6:27 am
SteveE says: UMMM Steve from what I read on that last paragraph the volcano heats the surrounding water to 30C you might want to re-read that.
————-
It says “During an eruption, the sea temperature around the island may increase from just above freezing to about 30 degrees Celsius (86°F).”
In other words the water temp is just above freezing before the eruption. There fore it’s not surprising that the point data is not representative of the grid cell data. The grid cell includes large amounts of ocean which would increase the average temperature of the grid cell in the winter which is exactly what is observed on the graph.
Willis’ analysis is flawed as he’s comparing point data and averaged grid cell data and surprised that they don’t match.
Big FAIL.
@ur momisugly SteveE
So you think the model better fits sea temperatures around Jan Mayen? Possibly, but I have doubts.
For example, the modeled temps hit -15 which is impossible (unless Jan Mayen becomes ice-bound). The averages hit -4 each year which means the sea freezes over every year (maybe it does, but I don’t think so). Also the range in temperatures is 10 C, which seems large.
Of course we don’t want to confuse sea temperatures with air temperatures – but I would like to see real air temps over the sea near Jan Mayen as well before making a conclusion. It could be that the model fits better to Jan Mayen than the surrounding ocean.
There are certain things you can model on a computer as long as you have an known and acceptable error margin and there’s some way to validate what’s it’s predicting against real world data. Climate is very definitely not one of those things.
http://thepointman.wordpress.com/2011/01/21/the-seductiveness-of-models/
Pointman
Love the “cheese spread” comment. This Christmas my man and I tried eating the IMITATION “cheese product”. Yes, you read that right. It was an imitation of a cheese product. It didn’t taste like cheese and it didn’t melt like Velveeta. Sounds like the computer product that was developed from Hansenized Jone-ish imitation data and then used to check the model output. When compared to real data, you find yourself wondering exactly which part of the cow that product came from. For sure, it had none of mother’s milk in it.
Grant Hillemeyer says:
March 8, 2011 at 6:36 am
http://wattsupwiththat.com/reference-pages/atmosphere/
Have a look at the Global – Two Meter Temperature, particularly around islands such as Iceland and Svalbard and tell me which is warmer, the area over the sea or the area over the land.
The weather station is 10 m. a. s. l. and quite close to the coastline:
http://dokipy.met.no/projects/iaoos-norway/janmayen-stasjon2.jpg
I would expect the temperature to be very close to the SST in the area. If anything it should be slightly higher in summer since the area is dark lava which absorbs sunlight.
Admittedly the station is close to the airstrip, but since it is a gravel strip with similar albedo to its surroundings and there is only 8 flights a year to the island, this is hardly a major concern.
Willis, this:
reminds me of Joel Shore. Here is why. Joel is always claiming that sceptics are creationists.
A prominant scientist is said to have been giving a rebuttal to normal creationist arguments and was expounding on the creation of life and perhaps the earth. An old man stood up in the back of the audience to ask a question, and when his turn came, he said: “I don’t believe this malarkey! Don’t you know that the world is help up by a giant turtle?”
The scientist responded with: “So, what is holding up the turtle?”
The old man then said: “You are very clever young man, but it’s turtles all the way down!”
I suspect that this is one of those apocryphal stories that goes around. It has a certain appeal.
Hi Jit,
I’m suggesting that the model has a better fit for the whole grid cell rather than just the land based measurements that are shown in the graphic. The air over the ocean would be warmer than that over the land as you can see from the teh link below, you can see the difference along the Aleutian islands off of Alaska. Over the sea the temps are warmer than over the land. When you average this out over the whole grid cell you get a warming bais when compared to purely land based measurements.
They could generate temperature maps of the lower troposphere and compare directly to the satellite measurements which are gridded, but for some reason they don’t.
SteveE wrote:
“Grid cell data is not point data and so a direct comparison isn’t an easy thing as this article shows.
If you compare one point on land to a grid cell that include 10′s of square km of ocean you’d expect there to be a difference as the graphs clearly show. The grid cell is an average of the whole area and so won’t reflect the exact value for the point data.
The averaging will remove the small scale hetrogenity in an area like this, however the larger scale picture will still be valid. The key is looking at the scale you are modelling to try and keep the hetrogenity that is important. In this case the tiny island of Jan Mayen isn’t important when trying to model global temperatures.”
and
“As the last paragraph says; “the sea temperature around the island may increase from just above freezing” which is what the average for the modelled data is, +0.3C.
The point data for the island that is covered in snow and ice is unlikely to be representative of the whole grid cell which is modelled composed mostly of ocean.”
This is a valid point, with regards to Jay Mayen island, which sparked my curiousity about the actual grid size of the ERA-40 grid. The link to this paper
http://goo.gl/0Jk4G
suggests that the ERA-40 grid size is 100km x 100km.
So I think that the key question is how well is the ocean temperature known to within such a grid size as the oceans constitute about 71% of the earth’s surface?
The completed ARGO ocean temperature/salinity measurement system
Wiki: http://goo.gl/7v2Kx
has a nominal gridding of about 300km,
Wiki: http://goo.gl/sED5J
the actual distribution being quasi-random as the sensors are floating about and are carried by ocean currents.
Wiki: http://goo.gl/JeOx8
So one question that arises is what are the systematic errors in interpolating down to 100km^2 from 300km^2 which is a not insignificant factor of nine in grid area.
As the ARGO project was completed in Nov 2007, it’s unlikely that the ERA-40 model has much merit, if any, the further back in time one goes before this completion data as interpolating over 1000’s of km of ocean data without taking effects of local variation in ocean temperature (ocean currents, interactions with the atmosphere, etc) into account.
From the above two points, I suspect that much of ERA-40 is GIGO.
http://www.coaps.fsu.edu/~maue/extreme/gfs/current/x_t2m_012.png
Forgot to include the link on my previous post.
SteveE says: March 8, 2011 at 5:11 am
Grid cell data is not point data and so a direct comparison isn’t an easy thing as this article shows. … The grid cell is an average of the whole area and so won’t reflect the exact value for the point data.
True enough, but my observation was intended to be general, not specific to this island. Where did the ERA-40 reanalysis data come from? It was “homogenized up” from the point data, yes? So it is legitimate to estimate grid area from point data, but not to re-check climate model grid calculations back to that point data?
I am proposing that we close the loop:
1) Point data -> Area grid
2) Grid values -> Climate Model
3) Climate Model calculated value -> Point Data
The Harry_Read_Me file should have convinced you that there may be significant errors introduced even at step 1). Poor Harry was not even able to re-create the previous version from his predecessors programs and data.
At Step 3), there must be some reason to expect the point and calculated grid values to be at least somewhat related (::chuckle::). After all, GISS extrapolates Arctic data out 1200 km. Surely it is less of a stretch to compare inward within a grid cell? There may even be value in comparing to each point value where there are multiple points within a grid. The beauty of the multi-point comparison is, some may be higher, some lower, some even out of phase, but the overall comparison gives you a numerical yardstick for ‘how am I doing’.
The world of humans is becoming confused with numerous paradigms. It’s as if no one actually speaks the same language anywhere on the globe, or even on the ISS. Have we reached too high? Is someone up there mad at us? Or.. perhaps… maybe we have exceeded our own limits, and no one even noticed we were in the Land of Babel once again. One day I fear we will walk away from all that we have built, dazed, confused, dejected. And the sorry part was we were soooooo close. (Well… closer than the last time anyway;-)
PS: I know! You can’t understand what I’m talking about.
Since the Thor of Norse mythology and the Thor of Marvel Comics have strong correlation, Thor exists.
We have metadata from two sources. The Marvel data only goes back to 1962 but the Norse Data spans back to pre-industrial times.
As others have observed a comparison is being made between the model results for a grid square and measurements from a small island within that grid square.
It is not likely that the gid square REAL data match the island data, a better comparison may have been with the satellite data for the grid square and the model.
The range of annual variation seems to be quite closely matched, 11degC in the real data and a 0.3deg average, there are actually several averages for different periods around for the Jan Meyer island data, the early measurements by the Austrians in the 1800s are by far the lowest. The actual data certainly confirms the warming and its magnitude over the global increase as predicted from AGW theory that higher latitudes will show greater effect.
The ‘Elephants all the way down’ story is usually attributed to Betrand Russell and a little old lady theophist of limited rationality but great certainty.