UPDATE: 08/07 2:PM PST MLO responds with improvements to the CO2 data reporting
Approximately 24 hours after I published my story on the January to July trend reversal of CO2 at Mauna Loa, the monthly mean graph that is displayed on the NOAA web page for Mauna Loa Observatory has changed. I’ve setup a blink comparator to show what has occurred:
For those who don’t know, a blink comparator is an animated GIF image with a 1 second delay consisting only of the two original images from NOAA MLO. Individual image URLS for: August 3rd ML CO2 graph | August 4th CO2 Graph
Now the there is no longer the dip I saw yesterday. Oddly the MLO CO2 dataset available by FTP still shows the timestamp from yesterday: File Creation: Sun Aug 3 02:55:42 2008, and the July monthly mean value is unchanged in it to reflect the change on the graph.
[UPDATE: a few minutes after I posted this entry, the data changed at the FTP site] here is the new data for 2008:
# decimal mean interpolated trend
# date (season corr)
2008 1 2008.042 385.37 385.37 385.18
2008 2 2008.125 385.69 385.69 384.77
2008 3 2008.208 385.94 385.94 384.50
2008 4 2008.292 387.21 387.21 384.46
2008 5 2008.375 388.47 388.47 385.46
2008 6 2008.458 387.87 387.87 385.51
2008 7 2008.542 385.60 385.60 385.25
and here is the 2008 data from Sunday, August 3rd:
2008 1 2008.042 385.35 385.35 385.11
2008 2 2008.125 385.70 385.70 384.85
2008 3 2008.208 385.92 385.92 384.38
2008 4 2008.292 387.21 387.21 384.59
2008 5 2008.375 388.48 388.48 385.33
2008 6 2008.458 387.99 387.99 385.76
2008 7 2008.542 384.93 384.93 384.54
Here is the MLO data file I saved yesterday (text converted to PDF) from their FTP site.
Here is the URL for the current data FTP:
ftp://ftp.cmdl.noaa.gov/ccg/co2/trends/co2_mm_mlo.txt
I have put in a query to Pieter Tans, the contact listed in the data file, asking for an explanation and change log if one exists.
UPDATE 08/05 8:55AM PST I have received a response from MLO:
Anthony,
We appreciate your interest in the CO2 data. The reason was simply that
we had a problem with the equipment for the first half of July, with the
result that the earlier monthly average consisted of only the last 10
days. Since CO2 always goes down fast during July the monthly average
came out low. I have now changed the program to take this effect into
account, and adjusting back to the middle of the month using the
multi-year average seasonal cycle. This change also affected the entire
record because there are missing days here and there. The other
adjustments were minor, typically less than 0.1 ppm.
Best regards,
Pieter Tans
UPDATE 08/05 4:03PM PST
I have been in dialog with Dr. Tans at MLO through the day and I’m now satisfied as to what has occurred and why. Look for a follow-up post on the subject. – Anthony
UPDATE 08/06 3:00PM PST
A post-mortem of the Mauna Loa issue has been posted here:
http://wattsupwiththat.wordpress.com/2008/08/06/post-mortem-on-the-mauna-loa-co2-data-eruption/
– Anthony
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Mr IceFree (15:19:29) refers to a New Zealand website. The article carried there is one of many pieces of fine analysis by Prof John Brignell which are collected on his own website. It’s one of my faves and well worth a read:
http://www.numberwatch.co.uk
On the topic of this thread, I must again expose my ignorance and naivety for your delectation. It comes as a great surprise to me that two sets of results are not habitutally released, one being the raw data and the other being the data as adjusted/smoothed. I don’t doubt Dr Tans’ sincerity in the adjustments he makes but we all learn by exposing our analyses to scrutiny and inviting others to comment. Others might well learn from what we have done and we might well learn from suggestions made. What I do not understand is why there is no standard practice of releasing raw data as well as adjusted data in all fields where public policy might be affected by the results (subject, where appropriate, to issues of intellectual property rights and commercial sensitivity).
Found an interesting letter at iceagenow
Dear all,
FRAUDULENT SCIENCE
My advice to climate alarmists is that now is an appropriate time to start planning your exit strategy. The whole IPCC/UNFCCC edifice is about to disintegrate.
I described these events in my recent memos. My position during all these years has been very simple. I could find no evidence of unnatural changes in the officially published hydrometeorological records.
Not only is there no believable evidence in the data to support climate alarmism, but the evidence refutes the IPCC’s claims and completely undermines its position.
What is the future of climate alarmism and its associated research? There is none. The globe is cooling, the glaciers are advancing and Bangladesh is not being inundated by rising sea levels. Public interest is falling and the media are becoming more critical. The possibility of nations reaching agreement on meaningful actions to control, let alone reduce, their undesirable emissions is receding by the day. The basic science underlying the IPCC’s position is being eroded away, stone by stone.
If the alarmists try to follow the adaptation route, they will be squashed underfoot by civil engineers and applied hydrologists.
There is only one remaining option. Abandon ship.
Kind regards,
Will
Will Alexander is Professor Emeritus in the Department of Civil and Biosystems Engineering, University of Pretoria, South Africa; Honorary Fellow, South African Institution of Civil Engineering; and was a member of the UN’s Scientific and Technical Committee on Natural Disasters from 1994 to 2000.
This guy Will Alexander is for real. Here is a poor country’s view of climate catastrophism.
http://www.fcpp.org/pdf/FB051%20Will%20Alexander%20Climate%20Change%20and%20Africa260706%20with%20picsdraft%20edited%20-%20erin.pdf
@frank L
Suppose the equipment failed on July 20th and the failure caused the loss of the monthly data to date the issue becomes one of sloppy data safeguarding.\
What people don’t seem to be getting is that the original July 2008 data was incorrect because of this loss of data. The mean of the last 10 days of July cannot be representative of the mid-month mean which is the standard for all the other months. This is why it stood out like a sore thumb. It was wrong and had to be corrected.
The only real discussion is if the method used for the correction is represents a valid reconstruction/adjustment of the missing data. The exact method he used is still rather vague, so until there is a clear understanding of the process it is pointless to speculate bias or motives.
When the global July CO2 finally comes out, we will get a better picture of the accuracy of adjustment of the Mauna Loa July CO2 mean.
I was employed by a chemical plant for 20 years. If we had pulled the stuff that I see posted on this site regarding testing, we would have been sent to prison by the EPA. Correct me if i am wrong, but are not world decisons bases on this CO2 data? Gee Wizz, I have zero faith in the temp data, chemical anazlysis and every other graph that is out there.
(Must have missed closing some links properly. Is there a way to preview before posting?)
REPLY: Sorry there is no preview… the code was so messed up I gave up trying to edit, try posting again – Anthony
REPLY 2:You can just put in the links without trying to put in any tags or titles. That is the safest if not coolest thing to do.~charles the moderator.
Now we know why the earth is not warming as much as the models predict. Could it be the effect of water vapor and clouds? Negative feedback? The latest program on PBS, Nova series was just aired. The main subject was “solar dimming”. The link to the transcript: http://www.pbs.org/wgbh/nova/transcripts/3310_sun.html
But, the real subject was AGW and James Hansen was the major interviewee. (What a surprise!)
Hat tip to Fat Bigot for the link [from the “Laws” page]:
Langmuir’s Laws of bad science
1 .The maximum effect that is observed is produced by a causative agent of barely detectable intensity, and the magnitude of the effect is substantially independent of the intensity of the cause.
2. The effect is of a magnitude that remains close to the limit of detectability, or many measurements are necessary because of the low level of significance of the results.
3. There are claims of great accuracy.
4. Fantastic theories contrary to experience are suggested.
5. Criticisms are met by ad hoc excuses thought up on the spur of the moment.
6. The ratio of supporters to critics rises to somewhere near 50% and then falls gradually to zero.
It appears that we are currently transitioning from #4 to #5.
[more on Langmuir here. More laws here.]
[I forgot to close the tag after ‘FatBigot.’ I am a bad person. Sorry. I will try to do better, swear.]
Another great site out of New Zealand is
https://www.predictweather.co.nz/#/home/
These guys have taken a beating for their veiws, I Truly beleive that the sun
the moon and climate on earth is cyclical, and history and facts, not models
will lead us to predict long term trends.
Lucia,
It would be interesting to know how many readings required adjustment because they didn’t fall within the bound of what is expected. If these are commons maybe the adjustment aren’t required after all.
Nite owl,
I saw the graph. That is extraordinary.
Mike
Radar:
I would think an erroneous upspike would no doubt be pulled into line as readily as a down-spike. Otherwise the CO2 measurement might appear sensitive to other local factors such as the volcano. It is also possible that the first 21 days of data were discarded because they did not fit the criteria for what would normally constitute fair data. Is that possible rather than the equipment being the cause of the missing data?
Why does the line take such a pristeen shape? It looks wrong from the start, if indeed the change is supposed to be influenced by human activities. Sorry if these remarks are old hat for some but I would like to understand this a little better.
One more question, Why is it a good thing if the adjustment renders an even smoother line? It’s hardly a stretch to see the trend before the smoothing. Has this to do with masking the seasonal variation? Surely that is interesting in itself since it is supposed to originate from the growing v dormant season. Why should one assume as remarked above that an erroneous spike or trough is simply instrument or measurement error? I would very much like an answer from anyone who has the time, thanks.
I’m having a problem with those suggesting Dr. Tan’s forthrightness in providing an explanation should settle the argument in their favor.
Let’s look at the overall picture:
1. NASA and it’s affiliates have consistently presented “challenging” numbers
2. Hansen and his GISS refuse to provide any information including raw data, how and why they manipulate data, or any magical formula used in manufacturing his numbers
3. The SST numbers have mysteriously disappeared from the media’s discussion on AGW
4. Atmospheric numbers not coming out as expected are ruled as invalid or of such little value to be of no consequence
5 Just the other day, one of those affiliates presented the rough draft of an article on predicted climate related disasters using an obviously doctored photo of a house partially under water.
6. And now this fellow Dr. Tan is supposed to be our Knight in Shining Armor and provide a valid explanation for the Mauna Loa number manipulations?
I think not. Given the obvious corruption in NASA, there is NO WAY they would allow one scientist to bring a halt to there lying, deceiving methods and ways. He would be immediately ostracized and banned from all meaningful operations.
Jack Koenig, Editor
The Mysterious Climate Project
http://www.climateclinic.com
Even if Dr. Tans’ adjustment is reasonable and defensible on its face, it is still a bit troubling. Would a similar adjustment have been made if the CO2 numbers were higher than expected? Somehow I doubt it. And if not, it has the potential to introduce a bias into the numbers.
One of the cardinal rules of statistical analysis is that you choose your criteria BEFORE you see the data. Otherwise it’s very easy to fool yourself (and others) into thinking your results are significant.
By analogy, Dr. Tans should have carefully chosen an averaging method IN ADVANCE and then stuck with it.
In climate science, this principle seems to have been thrown out the window.
Thanks all. Hope this one works better…
A few months ago, I started plotting the change in both CO2 and RSS/UAH lower troposphere anomalies for the same months in adjacent years (sorry if this is old hat, but I haven’t seen this particular comparison done before). CO2 is constantly increasing, but by different amounts from one year to the next for the same month.
http://i36.photobucket.com/albums/e7/niteowl496/co2_mm_mlo_and_temps_13420_image-1.gif
http://i36.photobucket.com/albums/e7/niteowl496/co2_mm_mlo_and_temps_14682_image-1.gif
There seemed at first to be something there, so I tried a simple 13-month rolling average of these changes to smooth it out.
http://i36.photobucket.com/albums/e7/niteowl496/co2_mm_mlo_and_temps_13420_image-2.gif
http://i36.photobucket.com/albums/e7/niteowl496/co2_mm_mlo_and_temps_14682_image-2.gif
It’s not a strong predictor of amplitude nor duration of the cycles, but one thing does seem to jump out. The trend line of temperature consistently switches direction anywhere from 6 to 18 months PRIOR to a corresponding switch in the direction of CO2 change (i.e. temperature leads CO2).
Based on this, I wouldn’t expect the amount of CO2 change in future months to start increasing consistently, until some time after temperatures do the same. I was expecting July’s value to have decreased over last year’s July value, but the August 3rd reporting was quite a surprise. The next day’s reporting was more in line with what I had expected (1.25), which is still the smallest increase across the same month in adjacent years since 2001.
I’m anxious to see the July values for RSS and UAH.
No one has taken a shot at these questions yet.
How many times in the past have these adjustment algorithms been run?
How long has Dr. Pieter Tan been responsible for adjustments?
Did his predecessor instruct in the necessity of these wholesale adjustments?
Are there any records of previous adjustments?
Did Dr. Keeling initiate a protocol that required regular adjustments?
Would it be worthwhile to use the way way back machine to set up more blink comparators?
This may be slightly paranoiac, but still would be good to know the truth.
Thanks all have enjoyed every comment,
Mike Bryant
WOW … Dee don’t be suckered by a nice scientist.
Scientists should publish the raw DATA. We don’t need the good Dr. to make an interpretive dance of DATA. I had assumed we were looking at RAW data, but now we know we are not. Please ask him to publish the raw data with the missing days. In my industry and most industries we do an 8D report when a mistake happens – no one is allowed to simply make a change in the dead of night and expect all others to accept it.
8D Process
1. Someone OTHER than the primary person responsible should form the correction team. (NOT DONE)
2. The problem must be completely described – Why was the equipment down, how long had the equipment been broken, why was the equipment broke, has their brokeness occurred before, has the brokeness contaminated other data, what is the back up system. (NOT DONE)
3. How is the problem going to be contained – Publish the options for eliminating the problem BEFORE implimenting the change. (NOT DONE)
4. Indentify root cause – Ask why the problem happened, then why your reason happen, again, and again, until their is not other reason — the last answer is probably the root cause. (NOT DONE)
5. Ensure that your solution is standardize such that the problem will not occur again. (NOT DONE)
6. Impliment change (DONE) and verify the change has not created other problems (DONE, in that the problem was AGW needed to be supported)
7. Impliment preventive measures (NOT DONE)
8. Credit the correction team. (NOT DONE)
So, the event was political, not science. Don’t be taken in Dee.
REPLY: I’m working on a post mortem, please hold for that. -Anthony
Dee,
Just before the correction was made, I suspected that the Mauna Loa Data was too clean.
Could you ask the good Dr.
“What is the fixed standard in CO2 measurement?
What is the material of the CO2 sensor?
How often is the CO2 sensor calibrated against the standard?
What is the drift coefficient of the CO2 sensor?
What is the linearity of the CO2 sensor with electronics?
What is the drift of the linearity of the CO2 sensor with electronics – is this calibrated as well?
How old is the electronics used to measurement the CO2 sensor?
How often is the electronics calibrated against known electronic standards?
How much random noise is in their measurement?
How specific is the sensor to only CO2?
Does the sensor react with any other gas?
Is the sensor calibrated over temperature?
Is the data adjusted for temperature?
What is the sensor accuracy vs. temperature?
How long does the sensor temperature soak before measurement?
Is the sensor calibrated over atmospheric pressure?
What is the sensor accuracy vs. atmospheric pressure?”
And what specifically broke on the instrument, why down for 20 days, etc.
McGrats (19:05:26) agreed. The credibility factor has been driven well below 1.0 by the shenanigans of Hansen et al. Un-announced, un-discussed adjustment of graphs is not the way to get it back. It’s like they’ve made their peace with “shoddy”.
Time for a Press Release
“Dr. Pieter Tans of NOAA caught altering Manua Loa CO2 data by AGW skeptics”
Altering data with no peer review, public comment, no before and after, no root cause identification, no plan to prevent this problem again, changing when the data is against your bias is very very wrong and Dee and Anthony are wrong to give this guy a pass. In life if a guy like Dr. Tans is comfortable enough to make a change like this in 24 hours, any detective will tell you it was not his first time.
[snip – potentially libelous label]
Wel, this has triggered a lot of comment!
I have followed the Mauna Loa data for a long while now and had a lot of discussion about their reliability with Richard and Ernst (Beck). Some background may be of interest here:
Air intake at Mauna Loa is continuous, with 4 samplings and 2 calibrations per hour. The average of the 4 samplings is noted as hourly average and these are available as raw data at:
ftp://ftp.cmdl.noaa.gov/ccg/co2/in-situ/
up to 2006.
If there is a large variation between the 4 samples, the one-hour average is flagged (NOT discarded or adjusted). If there are large variations between hourly averages, these are flagged too. The same for abnormal high readings, due to volcanic outgassing, and more frequently abnormal low readings due to upwind conditions (which brings CO2-depleted air from vegetation in the valleys upwards the mountain). Except for the large variations within an hour, the other flagged data are not used for daily and monthly averages. Daily averages need several hours of continuous reliable measurements, monthly values at least several days to be valid.
The explanation of the flags used are here:
ftp://ftp.cmdl.noaa.gov/ccg/co2/in-situ/README_insitu_co2.html
Daily, monthly and yearly averages are always smoothed, where clear outliers are omitted. That does not influence the overall year-by-year trend more than a tenth of a ppmv if you should include or exclude the outliers, only the variability around the trend is less.
Thus in my opinion, Pieter Tans was right to adjust the July data and the rest of the data back to the origin. These are no adjustments of the raw data, this only smoothes the presentation of the curve and doesn’t change the trend, which is now about +60 ppmv over the past 50 years. If you compare the Mauna Loa data with the SH stations like the South Pole, you see exactly the same trend, but with some delay and far less seasonal variability (2 ppmv vs. 6-10 ppmv for MLO).
That the positive trend of CO2 is human made is quite clear, only the year-by-year variation around the trend is influenced mainly by temperature variations and seasonal by temperature and mainly vegetation. I have made a web page which explains that (including comments on Mauna Loa) in full:
http://www.ferdinand-engelbeen.be/klimaat/co2_measurements.html
The year-by-year trend of CO2 can be formulated as follows:
dCO2 = 0.55 x CO2emissions + 3 x dT
The short-term influence of (ocean) temperatures on CO2 levels is about 3 ppmv/°C, the long-term influence (over an ice age – interglacial transition) about 10 ppmv/°C and about 55% of the emissions (as mass) stay in the atmosphere.
Pieter Tans extended the above formula further by including precipitation, which influences vegetation growth and thus CO2 levels…
There was only one major adjustment of the raw data when was discovered that the calibration gas used at that time (CO2 in nitrogen) did give a bias in the NIR measurements, compared to CO2 in air. All calibration gases used in some 500+ CO2 monitoring sites are calibrated themselves against a central standard.
Thus all together: there is no conspiracy, there may be some bias (but I am pretty sure, that if there was a similar sudden increase of CO2, that Pieter Tans should have adjusted it in the same way), the raw data are available (with some delay…), thus one can calculate the trends with any smoothing algorithm one likes, as that is completely unimportant for the general trend.
“The reason was simply that we had a problem with the equipment for the
first half of July, with the result that the earlier monthly average
consisted of only the last 10 days. Since CO2 always goes down fast
during July the monthly average came out low. I have now changed the
program to take this effect into account, and adjusting back to the
middle of the month using the multi-year average seasonal cycle. This
change also affected the entire record because there are missing days
here and there. The other adjustments were minor, typically less than
0.1 ppm. Too bad for the self-described “skeptics”.”
Am I wrong or does this sound like a self-fullfilling model instead of representation of a data set? Have missing days? Just smooth them all out, since CO2 is “well mixed”. Have data that doesn’t match the algorythm patterned to match expected seasonal swings? Just adjust data.
Missing days…why not just take a measurement a few times a year, and plot the increase as a straight line that is the expectation of the rate of increase. An algorythm could be made to model that. Then if the data from the few measurements don’t match the expected rate increase, just fudge that data.
niteowl, interesting graphs. If you use a wider line type, separate the graphs vertically and put it in PowerPoint, you may be able to sell it to Algore.
Personally, I would just like to know how many days the data has been thrown out based on the parameters or malfunctioning equipment. After a certain point you are not logging enough data to be meaningful.
As far as adjusting the current July using Deltas from previous years, it SOUNDS just ducky. There is a slight issue though. The previous years did NOT have a sun as quiet, ocean temps and air temps flat for as long or dropping… In other words, it may be the best they can do for their data analysis, BUT, it guarantees the data will look more like the past rather than whatever the heck is happening now!!!!
What is the point of monitoring something if you end up trying to make the data fit into your preconceptions??? Their quality control sounds more like they are guaranteeing their expectations!!