Guest Post by Willis Eschenbach
Previously, we discussed the errors in Levitus et al here in An Ocean of Overconfidence
Unfortunately, the supplemental information for the new Levitus et al. paper has not been published. Fortunately, WUWT regular P. Solar has located a version of the preprint containing their error estimate, located here. This is how they describe the start of the procedure they describe which results in their estimates:
From every observed one-degree mean temperature value at every standard depth level we subtract off a climatological value. For this purpose we use the monthly climatological fields of temperature from Locarnini et a. [2010].
Now, the “climatology” means the long-term average (mean) of the variable. In this case, it is the long-term average for each 1° X 1° gridcell, at each depth. Being a skeptical type of fellow, I though “how much data do they actually have”? It is important because if they don’t have much data, the long-term mean will have a large error component. If we don’t have much data, it increases the size of the expected error in the mean, which is called the “standard error of the mean”.
Regarding the climatology, they say that it is from the World Ocean Atlas 2009 (WOA09), viz: ” … statistics at all standard levels and various climatological averaging periods are available at http://www.nodc.noaa.gov/OC5/WOA09F/pr_woa09f.html “
So I went there to see what kind of numbers they have for the monthly climatology at 2000 metres depth … and I got this answer:
The temperature monthly climatologies deeper than 1500 meters have not been calculated.
Well, that sux. How do the authors deal with that? I don’t have a clue. Frustrated at 2000 metres, I figured I’d get the data for the standard error of the mean (SEM) for some month, say January, at 1500 metres. Figure 1 shows their map of the January SEM at 1500 metres depth:
Figure 1. Standard error of the mean (SEM) for the month of January at 1500 metres depth. White areas have no data. Click on image for larger version. SOURCE
YIKES! In 55 years, only 5% of the 1° X 1° gridcells have three observations or more for January at 1500 metre … and they are calculating averages?
Now, statistically cautious folks like myself would look at that and say “Well … with only 5% coverage, there’s not much hope of getting an accurate average”. But that’s why we’re not AGW supporters. The authors, on the other hand, forge on.
Not having climatological data for 95% of the ocean at 1500 metres, what they do is take an average of the surrounding region, and then use that value. However, with only 5% of the gridcells having 3 observations or more, that procedure seems … well, wildly optimistic. It might be useful for infilling if we were missing say 5% of the observations … but when we are missing 95% of the ocean, that just seems goofy.
So how about at the other end of the depth scale? Things are better at the surface, but not great. Here’s that map:
Figure 2. Standard error of the mean (SEM) for the month of January at the surface. White areas have no data. Click on image for larger version. Source as in Fig. 1
As you can see, there are still lots and lots of areas without enough January observations to calculate a standard error of the mean … and in addition, for those that do have enough data, the SEM is often greater than half a degree. When you take a very accurate temperature measurement, and you subtract from it a climatology with a ± half a degree error, you are greatly reducing the precision of the results.
w.
APPENDIX 1: the data for this analysis was downloaded as an NCDF file from here (WARNING-570 Mb FILE!). It is divided into 1° gridcells and has 24 depth levels, with a maximum depth of 1500 metres. It shows that some 42% of the gridcell/depth/month combinations have no data. Another 17% have only one observation for the given gridcell and depth, and 9% have two observations. In other words, the median number of observations for a given month, depth, and gridcell is 1 …
APPENDIX 2: the code used to analyze the data (in the computer language “R”) is:
require(ncdf)
mync=open.ncdf("temperature_monthly_1deg.nc")
mytemps=get.var.ncdf(mync,"t_gp")
tempcount=get.var.ncdf(mync,"t_dd")
myse=get.var.ncdf(mync,"t_se")
allcells=length(which(tempcount!=-2147483647))
zerocells=length(which(tempcount==2))
zerocells/allcells
hist(tempcount[which(tempcount!=-2147483647)],breaks=seq(0,6000,1),xlim=c(0,40))
tempcount[which(tempcount==-2147483647)]=NA
whichdepth=24
zerodata=length(which(tempcount[,, whichdepth,1]==0))
totaldata=length(which(!is.na(tempcount[,, whichdepth,1])))
under3data=length(which(tempcount[,, whichdepth,1] < 3))
length(tempcount[,, whichdepth,1])
1-under3data/totaldata
APPENDIX 3: A statistical oddity. In the course of doing this, I got to wondering about how accurate the calculation of the standard error of the mean (SEM) might be when the sample size is small. It’s important since so many of the gridcell/depth/month combinations have only a few observations. The normal calculation of the SEM is the standard deviation divided by the square root of N, sample size.
I did an analysis of the question, and I found out that as the number of samples N decreases, the normal calculation of the SEM progressively underestimates the SEM more and more. At a maximum, if there are only three data points in the sample, which is the case for much of the WOA09 monthly climatology, the SEM calculation underestimates the actual standard error of the mean by about 12%. This doesn’t sound like a lot, but it means that instead of 95% of the data being within the 95% confidence interval of 1.96 * SEM of the true value, only about 80% of the data is in the 95% confidence interval.
Further analysis shows that the standard calculation of the SEM needs to be multiplied by
0.43 N -1.2
to be approximately correct, where N is the sample size.
I also tried using [standard deviation divided by sqrt (N-1)] to calculate the SEM, but that consistently overestimated the SEM at small sample sizes
The code for this investigation was:
sem=function(x) sd(x,na.rm=T)/sqrt(length(x))
# or, alternate sem function using N-1
# sem=function(x) sd(x,na.rm=T)/sqrt(length(x) - 1)
nobs=30000 #number of trials
sample=5 # sample size
ansbox=rep(NA,20)
for (sample in 3:20){
mybox=matrix(rnorm(nobs*sample),sample)
themeans=apply(mybox,2,mean)
thesems=apply(mybox,2,sem)
ansbox[sample]=round(sd(themeans)/mean(thesems)-1,3)}
P. Solar says:
>>
Hmmm, says: Using a Student’s t- table, degrees of freedom = #samples -1 = 3-1=2. The t-distribution from the table @ur momisugly 2 degrees of freedom and .05/2= .025 is 4.303.
What does the table say for #samples =1 ie zero DF? That is the dominant case here.
>>
Hmmm says:
>>
P. Solar that puts it off the charts! Basically you know nothing about your confidence interval with only 1 data point or 0. It tells you that you need way more buoys…
>>
Exactly. So what this study is doing is trying to use statistical methods for evaluating measurement uncertainty where no statistics are available for most of the data because it’s too spare.
The authors recognise the problem but carry on anyway by substituting statistics from the next level up , variation between the averages of adjacent cells.
Imagine a region of 5 x 5 cells, 24 of which are very well sampled and all have a std deviation around , say, 1 degree. The central cell has just one reading. In the absence of any better information, let us assume that real temperature variations in all these cells are similar. What do the statistics of those cells tell us about the accuracy of our problematic single measurement.
Absolutely nothing.
The SD of those cells are the sampling errors of those cells , they do not even hint at how representative out single reading is of its cell.
The most we could get from heavily sampled surrounding cells would be the total range of individual readings. This could give a first approximation to the range of temperature variation in our central cell. Thus half this range would be a guess at the +/- sampling error in the single reading.
Note this has nothing to do with the SD of sampling within the surrounding cells and even less to do with the SD of variations *between* the cells which is used in the study.
The pretence at estimating the confidence levels is just that, thought it’s hidden deep in pseudo-mathematical mumbo jumbo that would confuse all but the most determined and qualified reader.
It’s not my job to redo this work for them, but it’s clear from all they leave out that a more realistic uncertainty estimate would be at least an order of magnitude greater than that which they present.
Willis says @ur momisugly April 25, 2012 @ur momisugly 11.04 pm
Dear Willis
Thanks, I appreciate your input and it is always valuable to get another perspective.
Incidentally, I found a particularly valuable source of seawater thermodynamic equations for sea water properties. It looks like these are the source equations for the tables I provided earlier. This document can be found at:
http://dx.doi.org/10.5004/dwt.2010.1079
Just input the above in Goggle and pick the PDF entitled “Thermophysical properties of Seawater”. In the back of this paper are ten tables containing familiarly straight forward equations for calculation the thermodynamic properties of seawater. These equations may be a more usable source of data than are the tables.
I reviewed your response and generally agree; however, a closer look at the NOAA’s salinity data for shallow depths only re-enforces my doubts. More on this in a minute.
First, a question, where you state:
Would I be correct in stating the standard deviations in the CY 2009 chart you provided (July @ur momisugly 500 meters) would indicate the average standard deviation at a given location is below 0.05 psu – as opposed to being the upper limit of greater than 0.3 psu provided in the chart?
Eyeballing the CY 2009 standard deviation of annual salinity at 600 m (as opposed to the July chart for @ur momisugly 50o meters) it looks like a CY 2009 standard deviation of about 0.02 psu would be approximately correct for a random location. This would create a variation in the enthalpy calculation of:
Enthalpy variation due to salinity alone at 600 meters = 0.7 /39.45 * 0.02 = 0.035%
My figure is roughly in line with your 0.04% estimate. So, I would generally agree with your assessment at the 600 meter depth.
A further check of the 2005 to 2011 salinity anomalies at the 600 meter level suggests the low standard deviation pattern you alluded to is common within individual years and generally between years. This provides additional perspective with which to concur with you regarding the enthalpy calculation in the 600 meter range. (With caveats to be discussed below).
For data see: http://www.nodc.noaa.gov/cgi-bin/OC5/SAL_ANOM/showfiganom.pl?action=start
However, I also noted that NOAA’s salinity anomalies vary considerably in the 50-200 meter range – particularly between individual years. For example, if one picks the 100 meter chart, fixes ones gaze at particular spot, and then scrolls through the years – one can easily encounter 0.3 to 0.4 psu changes in the average annual salinity at a given location from one year to the next. Lets say 0.3 psu for the purpose of this discussion.
Now I suppose one could assume that the average of these shifts will even out with time. But… an examination of NOAA’s 100 meter 1955-1959 pentadal salinity anomalies compared to those for 2007-2011 show there really isn’t a consistent pattern. For example, the 2007-2011 data shows the Atlantic Ocean with a widespread increase in salinity where as the 1955-1959 data shows much lower levels. A similar reversal shows up in the northern latitudes. And the center of the Pacific Ocean is relatively saline in 2007-2011 and relatively fresh in 1955-1959 . So I’m having a tough time buying into the use of the climatology as a substitute for actual salinity data.
Keep in mind, that the increased salinities showing up in the Atlantic, Pacific, and Arctic Oceans in 2007-2011 would actually lower the energy storage potential of those oceans (at the same temperature). This occurs because increased salinity decreases the seas enthalpy at the same temperature.
To re-gear my perspective, I repeated my previous calculations using a new condition of 22C at a depth of 100 meters. I get as follows:
At 22C and a salinity of 30 psu the enthalpy of salt water = 88.4 kJ/kg
At 22C and a salinity of 40 psu the enthalpy of salt water = 87.0 kJ/kg
The change of enthalpy with a unit of salinity = 1.4 kJ/kg- psu.
At 22C and a salinity of 35 psu the enthalpy of salt water = 87.68 kJ/kg (not rounded)
Enthalpy variation due to salinity alone at 100 meters = 1.4 /87.68 * 0.3 = 0.48%
The short version is that the potential for systematic errors in calculating heat content are relatively high near the surface and drop with depth. Using a climatology salinity value as a substitute for the actual values would be OK if the Oceans were stable with no long-term variations in salinity at a given location with time and depth… but I don’t see this in the raw data. I see significant variations in both salinity concentrations and ocean anomalies patterns with time – particularly at shallow depths.
So how does this perspective relate to Levitus? Levitus’s Figure 2, shows that the bulk of his proposed energy storage is occurring in the ocean’s upper layers – at precisely the depths where the greatest chance for systematic error exists. And the potential for systemic error decreases with depth – along with Levitus’s decreased storage – until about 1550 meters below which the apparent storage virtually ends. This 1550 meter point is also the depth where salinity appears to become relatively constant with over wide areas.
To summarize, I see where you are coming from with regard to a low potential for systematic error at great depth. However, I still see potential to systemic error begging at shallower depths and decreasing with depth. In the end it I think is really comes down to a matter judgment. If one believes that any variations in salinity at shallow depths will, statistically, follow a uniform pattern with time. Then there is no need for concern. However, if one sees shifting patterns, apparent oscillations, and/or changing trends in ocean salinity; then this would lead to a different perspective. Personally, I think it would be better to simply take the time and energy to calculate the Ocean Heat Contents more rigorously… to simply to remove doubt.
Best Regards,
Kforestcat
Many thanks, Kforestcat. It’s an interesting analysis.
The problem, I suspect, is that for many of the temperature measurements, they don’t have any contemporaneous co-located salinity data. After all, not every ship doing temperature soundings was also taking samples for salinity. This is particularly true back in the day when there was no way to measure salinity electronically, it had to be done with a chemical analysis.
So that’s likely why they are using the climatology for salinity, it’s all they’ve got. And I can live with that, it’s just one of the many things that increase the error in any analysis.
My problem is that I don’t see a term for that in the error analysis … they take the climatology as a given, near as I can tell they don’t consider it … but it definitely increases the likelihood of a systematic error.
w.
All
Subject: Typos/errors in my previous post.
Regarding my April 25, 2012 post at 4:49 pm.
I found typos and calculation errors. Corrections are bolded and listed below as follows:
The Change in enthalpy with salinity @ur momisugly 10C and 600 meters should be listed as:
0.07 kJ/kg-psu (Not 0.7 kJ/kg-psu)
Enthalpy variation due to salinity alone at 600 meters = 0.07 /39.45 * 0.02
= 0.0035% (Not 0.035%).
The Change in enthalpy with salinity @ur momisugly 22C and 100 meters should be listed as:
0.14 kJ/kg-psu (Not 1.4 kJ/kg-psu)
The potential for systemic error @ur momisugly 22C & 100 meters remains = 0.14 / 87.68 * 0.3 = 0.48%
My conclusions remain the same.
So, sorry for this and other typos (extra words, wrong words, etc.) I suffer from a form of dyslexia that makes writing/editing difficult. Normally, I set aside documents and edit/error check the next day.
Regards,
Kforestcat
Willis
I think the biggest problem with Levitus’s error estimates is the huge reduction in sample size caused by autocorrelation. I analysed the 1955-1995 trend estimate for the Levitus 2005 0-3000m dataset last year. A simple regression on the overlapping pentadal estimates gives a trend of 0.3294 and a standard error of 0.0370, both in 10^22 J/yr, with a t-value of 8.902 and a probability of the actual trend being zero of only 7.80x 10^-11. But the AR(1) autocorrelation coefficient is over 0.89, reflecting inter alia the data being over overlapping pentads. That reduces the efective degrees of freedom in the regression from 38 to 2 (using the (1+AR1)/(1-AR1) Quenouille correction). On that basis, the standard error is 0.16 not 0.037, implying a 9% probability that the true trend is no more than zero.
I doubt if the trend error position is that much better for Levitus 2012, since it presumably uses much the same pre-Argo data as Levitus 2005.
Nic Lewis says:
April 27, 2012 at 2:52 pm
Couldn’t agree more, and it’s just one issue among a bunch.
My thanks to you,
w.
Willis & All
Searching around for a more accurate source of thermodynamic properties for seawater I came across the software for the TEOS-10 (Thermodynamic Equation Of Seawater – 2010). The FREE software includes a library of the TEOS-10 subroutines. The software can be found here:
http://www.teos-10.org/software.htm
The new equations of state include thermodynamics properties of: seawater, ice, and humid air. These equations were endorsed in 2009 by UNESCO in Paris for all its member states as an international standard for oceanography, to be valid from 2010 on.
I down loaded the VB version over the weekend and managed to import it into Excel. It take some minor tweaking to get it to work in Excel (VBA vs VB 6 issues). Mainly this involves: 1) entering the module “Convert_0_Mdl”; 2 ) going to the function “Read_gsw_data”; 3) commenting out the line “f = App.path” (VB 6 function); and manually adding the file path name as in the following example:
‘f = App.path (commented out)
f = “C:\Users \SeawaterProperties\”
The source code provides examples with which to check calculations. So there is a way to verify the code is working properly. Versions are available for Mathlab, Fortran, C, and VB.
I also recommend The site GSW Toolbox contents because it provides links to web pages which describe the thermodynamic theory behind each of the functions. It also provides descriptions of each equation, references, input/output descriptions, and example calculations: See here:
http://www.teos-10.org/pubs/gsw/html/gsw_contents.html
Once installed, it makes getting seawater data a breeze. Note that is also includes a database for converting preformed salinity to absolute salinity. Basically, just input the preformed salinity, sea pressure, latitude, longitude and out pops the absolute salinity. Please note, in this library, the enthalpy of seawater is calculated from the absolute salinity.
Regards,
Kforestcat
Fascinating stuff, Kforestcat, many, many thanks.
w.