UPDATE: 08/07 2:PM PST MLO responds with improvements to the CO2 data reporting
Approximately 24 hours after I published my story on the January to July trend reversal of CO2 at Mauna Loa, the monthly mean graph that is displayed on the NOAA web page for Mauna Loa Observatory has changed. I’ve setup a blink comparator to show what has occurred:
For those who don’t know, a blink comparator is an animated GIF image with a 1 second delay consisting only of the two original images from NOAA MLO. Individual image URLS for: August 3rd ML CO2 graph | August 4th CO2 Graph
Now the there is no longer the dip I saw yesterday. Oddly the MLO CO2 dataset available by FTP still shows the timestamp from yesterday: File Creation: Sun Aug 3 02:55:42 2008, and the July monthly mean value is unchanged in it to reflect the change on the graph.
[UPDATE: a few minutes after I posted this entry, the data changed at the FTP site] here is the new data for 2008:
# decimal mean interpolated trend
# date (season corr)
2008 1 2008.042 385.37 385.37 385.18
2008 2 2008.125 385.69 385.69 384.77
2008 3 2008.208 385.94 385.94 384.50
2008 4 2008.292 387.21 387.21 384.46
2008 5 2008.375 388.47 388.47 385.46
2008 6 2008.458 387.87 387.87 385.51
2008 7 2008.542 385.60 385.60 385.25
and here is the 2008 data from Sunday, August 3rd:
2008 1 2008.042 385.35 385.35 385.11
2008 2 2008.125 385.70 385.70 384.85
2008 3 2008.208 385.92 385.92 384.38
2008 4 2008.292 387.21 387.21 384.59
2008 5 2008.375 388.48 388.48 385.33
2008 6 2008.458 387.99 387.99 385.76
2008 7 2008.542 384.93 384.93 384.54
Here is the MLO data file I saved yesterday (text converted to PDF) from their FTP site.
Here is the URL for the current data FTP:
ftp://ftp.cmdl.noaa.gov/ccg/co2/trends/co2_mm_mlo.txt
I have put in a query to Pieter Tans, the contact listed in the data file, asking for an explanation and change log if one exists.
UPDATE 08/05 8:55AM PST I have received a response from MLO:
Anthony,
We appreciate your interest in the CO2 data. The reason was simply that
we had a problem with the equipment for the first half of July, with the
result that the earlier monthly average consisted of only the last 10
days. Since CO2 always goes down fast during July the monthly average
came out low. I have now changed the program to take this effect into
account, and adjusting back to the middle of the month using the
multi-year average seasonal cycle. This change also affected the entire
record because there are missing days here and there. The other
adjustments were minor, typically less than 0.1 ppm.
Best regards,
Pieter Tans
UPDATE 08/05 4:03PM PST
I have been in dialog with Dr. Tans at MLO through the day and I’m now satisfied as to what has occurred and why. Look for a follow-up post on the subject. – Anthony
UPDATE 08/06 3:00PM PST
A post-mortem of the Mauna Loa issue has been posted here:
http://wattsupwiththat.wordpress.com/2008/08/06/post-mortem-on-the-mauna-loa-co2-data-eruption/
– Anthony
Ferdinand Engelbeen,
Is the raw data that is rejected based on the QC, as opposed to obvious equipment malfunction, available??
Ferdinand,
they need 6 hours of data within their guidelines before it is acceptable. We now know they need more than 10 days data to compute a month without infilling AND ARE INFILLING 20 DAYS DATA!!!!!! THIS IS JUNK!!!!!
Apparently you are wrong about the data not being changed as it HAS. See the previous posts by others where the data was downloaded.
Another technical issue is that the diurnal variation in CO2 is fairly large also. If the data they get is not from the same time of day due to QC rejections they are massaging the daily data to get normalised readings also. If they are actually rejecting the data because they can not get it during the right time period I can see a high rejection rate!!
Now, be serious. Why was this change necessary. This process has been ongoing for over 50 YEARS!!!!!!! They just realised that they did not have the correct procedure to deal with this situation?? This was the FIRST TIME IN OVER 50 YEARS THIS HAPPENED?!?!?!?!?!?
Are you seriously telling us that this has never happened before and they had to scramble to fix it?? For at least a WEEK no one paid any attention to the measurements and didn’t notice a thing until the incorrect results were posted?? How did they know they were incorrect?? When did someone actually look at them?? They rejected 2/3 of the data for the month and they didn’t realize they had an issue until someone e-mailed them about the large drop?!?!?!?!
At the least you have a group that is underperforming on a highly paid Hawaiian vacation and should be fired for their negligence!!! It is more likely that their is intentional fraud. I simply can’t believe they are THAT INCOMPETENT!!!
If I sound incoherent I AM!!!
Ferdinand Engelbeen:
“That the positive trend of CO2 is human made is quite clear”
That is the part that I believe is pulled out of thin air. Just exactly what makes it clear that the positive trend is human caused? It seems to me that you jump to the conclusion that since there is a positive trend, it must be human caused. I disagree. If it were human caused, I would expect to see the slope of the increase change with the slope of changes in global CO2 emissions but it doesn’t. The slope seems rather steady since 1960 while human generated CO2 was estimated to be 700% greater in 2001 than it was in 1950. Annual human emissions in 1970 were about half what they are now. But the rate of increase in atmospheric CO2 has been relatively unchanged. That tells me that the amount of human generated CO2 must be a negligible amount of the total CO2 being added to the atmosphere each year. The bulk of it must be coming from other sources. The amount increase in the amount generated by humans doesn’t even show up in the data.
These are no adjustments of the raw data, this only smoothes the presentation of the curve
I don’t understand this comment. By this standard, there is never an adjustment to any raw data, no?
and doesn’t change the trend, which is now about +60 ppmv over the past 50 years. If you compare the Mauna Loa data with the SH stations like the South Pole, you see exactly the same trend, but with some delay and far less seasonal variability (2 ppmv vs. 6-10 ppmv for MLO).
It seems to me that this is essentially the same argument which has been used to defend the hockey stick. “Ok, so Mann cut a few corners but his results were essentially correct.”
In this case, it’s very likely that the results are essentially correct but the incident is still troubling.
So if the data from the last 30 years has now been changed, how does that affect the climate models?
These climate models have [i]accurately[/i] mapped the correlation between CO2 and temperature. Apparently the [i]science is settled[/i].
Surely now those models are inaccurate, so the settled science of the calculations on CO2’s effect on temperature are wrong and will have to be readjusted despite the science being settled?
Niteowl,
Amazing graphs! If I’m interpreting them correctly it looks like very clear evidence that CO2 change is an effect of temperature change and not the other way around. Good stuff, and I hope they don’t get overlooked down here at the tail end of the Mauna Loa discussion, they deserve more scrutiny.
niteowl, yr Aug 05th, 19:49
yeah, did same some months ago,
agree with yr conclusions.
By the way, there is a way for a 12 months average:
Tavrg = (T-6/2 + T-5 + T-4 + … + T-1 + T0 + T1 + … + T5 + T6/2)/12
@Dee Noris
you write:
“Suppose the equipment failed on July 20th and the failure caused the loss of the monthly data to date the issue becomes one of sloppy data safeguarding.”
Yes it is a possibility.
So the failure made the machine delete all previous july data and thus we did not have a Mauno Loa “doing nothing” in 20 days of no data. It was only a sinlge incidence the 20´th july that erased 20days of july data.
Lucky that the machine did not erase june data? Or all other data?
That the machine should exactly erase july data seems a little hypothetical to me.
But anyway, i work at NovoNordic, medicine. i imagine if we had a machine that could possibly either
1) break down without anyone able to fix it faster than 20 days..
or
2) a machine that when breaking down erases all data in present month AND having no backup. Even my personal PC has a backup system..
In either case if this where NovoNordic we would have been closed down by the FDA in no time.
And yes I find it strange that an obvious problem of missing data has never been corrected for before alle the way back to 1974?
These things are not impossible, and to me it means a lot that Anthony Watts impression is that “they´re ok”.
But of course one cannot neclect, that this is one of 1000 corrections done by Alarmists that much much too often ends up supporting their views. Its just not reliable. Everytime theres “something with the equipoment” it ends up with a change who supports alarmists. Its simply not looking good even though we want to be understanding sensible and tollerant.
Stef (01:11:25) :
“So if the data from the last 30 years has now been changed, how does that affect the climate models?”
I doubt this change will affect the models in any way since the adjustment does not change the growth rate of the MLO CO2.
It will also only have a minor impact on the yearly CO2 growth rate.
In my comment in the beginning I said it looked like the values have been smoothed towards a straight line. I decided to check if that was true and made som calculations of the CO2 seasonal adjusted data to see if it was smothed or not.
I did like this:
A linear trendline was made for each year, Jan – Dec.
Then I compared if the new values where adjusted closer to or further away from this trendline compared to the old values.
When I put all those adjustments in a diagram it looked like there was an even distribution of adjustments closer and further away from the trendline.
For the last year it looks like, if you watch the blink gif in this thread, that the values have been smoothed and that is also true. But it is just a short cluster of points over a few years that where adjusted closer to the trendline. Overall I could not see a significant bias toward a smoothing of the values.
I am still trying to understand why the original data set was published if the problems with the data collection were known.
Is it common practice to publish information when there have been problems with equipment then simply change it later?
Why publish findings in this way – which are inconvenient to those who claim we are forcing up CO2 levels – then revise them to a more convenient narrative one day later?
And some people wonder why I have doubts about what we are told about CO2 and AGW…
I can understand the July adjustment, but the smoothing trend line back to 1974?
I believe the smoothing has nothing to do with the past, but may have a impact on the future readings. ‘As in cooler oceans not having much of an impact on the CO2 uptake..’
Maybe they have some insight to the satellite CO2 data of “will mixed”.
niteowl,
Wonderful graphics and nice reasoning.
What software do you use to generate those graphs?
Jack
Thanks for the information Jerker Andersson
Raw Data:
I found the raw data until Dec 2006. Why it stopped in 2006, I don’t know nor am I inclined to see something nefarious at this point.
ftp://ftp.cmdl.noaa.gov/ccg/co2/in-situ/mlo/
ftp://ftp.cmdl.noaa.gov/ccg/co2/in-situ/README_insitu_co2.html
@McGrats:
Dr. Tans is with NOAA, not NASA. I know that both are N-words (but not THE N-WORD) which strongly incite anger and frustration in some segments of the population, but neither organization is a monolithic block.
I have many friends at NASA who cringe at Hansen’s name and wish he would pack his desk and be gone forthwith. Unfortunately, Hansen built himself a small empire and would be near impossible to dislodge at this point. Witness the reaction to the White House restrictions his pronouncements to sound science. The Hansen empire will have to fall from within (and fall it will eventually).
@Mike Bryant:
I saw your questions but have not had the time to address them with Dr Tans, if I do at all. If I (or others who may be in contact with Dr. Tans) do get answers to them, rest assured the answers will get posted.
@John McDonald:
Having obtained ripe old age of mid-forties without any major mishap, I seldom get sucked by anyone unwillingly.
See my reference to raw data above. I am perplexed by the lack of follow through for the 2007 data and if possible, I will inquire about it. If you read the HTML file, I think that some of your answers will be there. Some of your answers are already known, but out of courtesy, I am waiting on Anthony’s update post (this is his blog after all, no?).
I am a great advocate of transparency in science and the apparent lack is bothersome to me (which is why I am taking the time to investigate this) but I hesitate to jump to conclusions as to the underpinnings of the translucence we are experiencing.
@ferdinand meeus Engelbeen:
Thank you for filling in some of the missing details on the CO2 collection process at Mauna Loa. Other than the cause of the increase, I believe we are mostly in agreement.
The total human liberation of C from natural sequestration minus the amount of C re-sequestered from the atmosphere does not equal to the change in atmospheric C.
I have a graph someplace that shows this and will try to dig it out later.
@Glenn Skankey:
The rate of decrease at the end of July is greater than the rate of decrease at the start of July, so using only the last 10 days as the mid-month mean would generate an incorrect value that is lower then the actual mean would have been. Clearly the July value was incorrect and good science required an attempt to correct it. The approach taken by Tans is not inappropriate in this situation.
It is unfortunate that it was the first half of the month that was lost and not the second. Had that occurred, the July data would have been erroneously skewed higher and we would be cheering NOAA for the correction with the AGWers would be having this discussion!
@Stef:
Not by anything worth all this fuss.
Other than the July correction, the historical adjustments go both ways and the net adjustment (excluding the July correction) is -0.48. So, Dr. Tans’ change LOWERED the historical CO2 Trend.
@frank:
I would imagine that by July 20th, the June data had already been removed and processed. This would explain why only the July data had been lost.
It’s likely a statistical artifact in the calculation method derived from missing data in calculating the moving average. Anyway the trend is still the same, and that’s the thing that’s most important when assessing global climate change. Global climate change assessments are generated on OVERALL TRENDS in datasets, not absolute change between individual measurements. Datasets are being refined almost all the time, that’s what research is all about..improving, refining, expanding. NOAA know what they’re doing, they’re all highly trained professionals. It was in fact diligent of them to correct for the errors in their dataset, rather than just leave the errors in there.
Hello, I’m slightly off topic here but I’m seeking guidance from you good people who’ve been looking at this longer than I.
I need some help please in understanding more about several areas regarding Anthropogenic CO2 vs global warming… the very basics I’ve got.
I need to learn a lot more about c12 and c13. This is often cited as a means of detecting the anthropogenic portion of the co2 part of the greenhouse effect, but I know almost nothing about this and haven’t found anything useful by ‘googleing’ it (I’ve no doubt it will be a word soon!). Can you give me some direction please?
I’m also very interested in anything you can direct me to regarding oceanic outgassing and the longer oceanic cycles.
Lastly, and closer to this particular thread, I hope you able to help me with this, which I can’t quite get straight.
I’ve just graphed the Scripps Mauna Loa CO2 and Alaska CO2 together. What I see is that the two go in lock step together seasonally but about a month apart. Loosely speaking end of summer gives a CO2 low, and end of winter gives a co2 high. That’s not unexpected. The bit I don‘t understand is why the co2 low in Alaska always precedes the co2 low in Mauna Loa. If the ‘leaf on, leaf off’ argument were true wouldn’t I see the opposite? The tropics should be sinking the CO2 in the summer and this deficit should show up in the more polar regions later? The opposite appears to be the case. Mauna Loa always follows Alaska by about a month and also the annual range for Alaska is much larger than Mauna Loa, which I don’t understand. I’m sure I’ve missed something here that you’re all very aware of and hope for a bit of guidance.
REPLY: I’m working on a post mortem, please hold for that. -Anthony
If Dr. Tan wants to get back in my good graces he could publish the dataset(s) and issue a paper concerning intramonth/seasonal variability.
My word, John! What a journey you have made! (Clearly, being a scientist yourself, your scientific sensibilities have been offended.)
I don’t know if the infraction here is sufficient to tip the argument, but it is symptomatic of the somewhat loose attitude towards proper procedure.
Actually I am one of those skeptics who actually believes (with reason) that the 3% of CO2 emitted by man is causing an “overflow” which results in a 0.4% increase in the atmospheric sink. And I know that sometimes data must needs be adjusted.
But it’s like a patient’s chart–if you alter it you need to say so beforehand, explain what you’re doing, and archive the old records just in case. There’s a procedural ethic involved. If a doctor doesn’t do that, his liability insurance will suffer accordingly. (Plus various other professional ramifications.)
Climatology does not have the immediacy of the medical profession. Or it didn’t used to, anyway. And I do not regard Dr. Tan as a particularly bad example–at least he answered, which is more than you can say of many.
But, OTOH, since what is at issue is whether or not to put the planet on a drastic course of socioeconomic chemotherapy (which will directly result in significant morbidity), it would behoove those involved in the diagnosis to begin acting according with a tighter procedure than they have in the past.
Wait, I may have gotten my names confused . . .
KuhnKat,
The raw hourly averages are available up to 2006 at:
ftp://ftp.cmdl.noaa.gov/ccg/co2/in-situ/
where MLO is Mauna Loa, SPO the South Pole, BRW is Barrow (North Alaska) and SMO is Samoa (island in the SH Pacific).
All raw data, except for instrument malfunction, are available. With instrument malfunction, of course the data are blank values. Extreme values, or values with known local influence are flagged and not used for daily and monthly averages. I have looked at the 2004 raw data: over 10% were missing due to instrument malfunction (including several weeks in June), over 10% were rejected due to upslope winds. Instrument malfunction can need extensive repair and/or replacement of equipment, which may need several weeks for inflight of equipment and calibration. Despite the high rate of rejections, there was (for 2004) no difference in trend by including or excluding the flagged data…
The problem they encountered was obvious now and specific for the NH, because July is typical a month with a sharp reduction in CO2. If they had this in October, it would have been the other way out. Something similar (a 20 days malfunction, 10 days good data at the end of the month) at the South Pole wouldn’t have had much impact, simply because there is far less seasonal variation in the SH (and seasonal opposite to the NH). I suppose that the 2007 raw data will be available soon, but for the 2008 data you will have to wait another year…
Thus we are talking now about a change in smoothing algorithm, which does affect the seasonal variation around a trend, but doesn’t influence the trend itself at all. If they had the July monthly average as first calculated, this would only represent the last 10 days, which are certainly lower than the “normal” average for July, even for the cold July 2008, and next month with full data would recover back to (near) normal… I don’t know if this type of problem ever occured before. It is possible not, or this would have been solved long ago.
Thus all together, a lot of fuss for nothing. The smoothing is not necessary at all, but gives a nicer graph. The only interest of the real variation around the trend may be for people interested in the seasonal uptake/release of CO2 by vegetation, which has changed in recent decades.
Ferdinand:
Thank you for the explanation which confirmed my assumption re data erroneously spiking either way. I only hope that those who decide what is erroneous have not missed something important. Hence the reason why it would be good to see a graph including all the days, even the ones when the volcano does it’s biggest burps.
I am still in the dark as to why the trend line shows a clear anthropogenic signal. I do sincerely want to understand this so would be grateful for a dummy’s guide as to why the anthropogenic signal is clear? Surely the line, if it is to be taken as an accurate depiction of CO2 levels, shows an increase that could very well be of some larger source as the rate of rise is so steadily above the seasonal signal. Why can we not see the contribution from the outgassing? why is that signal removed?
My own hunch that I would not expect you to comment on, would be regarding the change in the graph is that it probably wasn’t the equipment but simply that the data was not considered fair and that this July was a particularly poor crop. Rather than attract attention, the equipment was blamed.
Thank you again for the previous explaination.
In other words, how big are the burps?
Crosspatch,
There are a lot of arguments which point to humans as the source of increasing CO2 in the atmosphere (and the upper ocean layer), see:
http://www.ferdinand-engelbeen.be/klimaat/co2_measurements.html
The main points are:
– the mass balance: more CO2 is emitted by humans than is accumulating in the atmosphere. That means that natural sinks (over a year) are larger than natural sources. The net amount (as mass, not as individual molecules) accumulated from nature thus is zero.
– temperature in/decrease has a limited influence on CO2 levels (max. 10 ppmv/°C), can’t be the cause of 60/100 ppmv increase of CO2 in 50/150 years.
– the d13C level of the atmosphere and the upper oceans declined since the start of the industrial revolution. Fossil fuels and vegetation decay are the only sources of low d13C.
– oxygen levels decline in line with fossil fuel burning, but with a small deficit. This is caused by increased vegetation growth (which produces oxygen). That means that vegetation growth exceeds vegetation decay (by about 2 GtC/yr), thus vegetation is a source of d13C, not a sink.
– there is an extremely good correlation between accumulated emissions and increase in the atmosphere for the full period of Mauna Loa data and even a very good one for the previous period (1900-1960), where the accumulation in the atmosphere is about 55% of the emissions. This points to a simple first order process of absorption of extra CO2. The correlation between temperature and CO2 accumulation is short-lived (better for dCO2 over months) and far worse.
The most concerning part about this sordid affair is the lack of a quality process.
A quality process is NOT a single researcher changing the presentation of data, discovering “broken equipment” data one day after the data set is shown to not support AGW. Every major corporation (medical, engineering, industrial) worldwide understands the ISO9000 process and how to make engineering changes (PCNs). It would be nice to see Manua Loa at least make a small attempt to follow a quality process. Dr. Tans data is being used to support a massive attempt to regulate our lives and potentially cost us all a huge drop in our standard of living – so his data manipulation with no peer review, within 24 hours, no documentation, no impact study, etc. tells me they are NOT a serious science organization. As one poster said, after 50 years of doing it one way, within 24 hours they discover the need to change in 24 hours LOL. What was the rush, ask him that Dee!
While some of you may say “it is not a big change, did not change the slope”, this change goes to the credibility of that location, the whole data set. I, for example, believed the Mauna Loa data was raw and was unassailable only 48 hours ago. I never challenged it in any post, any discussion, etc. Now, I wonder.
Now we know, they have problems with their data despite “back up systems”, that the nearby volcano causes issues, that data outside of certain ranges is flagged and not included. Now we know why their data does not conform to the initial AIRS data, Now we know why Mauna Loa data looks way to smooth. That’s the inconvenient truth.
Would a smoothing algorithm affect the type of changes that niteowl is tracking in his graphs?
Mike Bryant