From FECYT – Spanish Foundation for Science and Technology
In order to gain a better knowledge of climate variations, such as those caused by global warming, and be able to tackle them, we need to understand what happened in the recent past. This is the conclusion of a research study led by the Rovira i Virgili University (URV), which shows that the scientific community today is only able to access and analyse 20% of the recorded climate information held. The remaining data are not accessible in digital format.
Some climate data in Europe go back to the 17th Century, but “not even 20% of the information recorded in the past is available to the scientific community”, Manola Brunet, lead author of the study and a researcher at the URV’s Centre for Climate Change, tells SINC.
This situation is even worse in continents such as Africa and South America, where weather observations did not begin until the middle of the 19th Century. These are the results of a study published in Climate Research, which highlights the need to urgently recover all the information recorded in perishable formats.
“Failure to decipher the messages in the climate records of the past will result in socioeconomic problems, because we will be unable to deal with the current and future impacts of climate change and a hotter world”, says Brunet.
Spain, along with the USA, Canada, Holland and Norway, is one of a small number of countries which allows partial access to its historic climate data. The rest of the world does not make these data available to the scientific community or the general public, despite recommendations to this effect by the World Meteorological Organization (WMO).
In order to overcome the political and legal hurdles posed by this currently poor access, “governments should adopt a resolution within the United Nations on opening up their historical climate data”, the researcher suggests.
Predicting heat waves
Weather services in all countries are faced with the overwhelming job of converting all their paper-based historical climate information, which is stored in archives, libraries and research centres, into digital format. The wide range of forms in which the information is held makes access harder, as do the purposes for which the meteorological service itself was actually created.
“The main objective is to provide a weather service to public, who want to know what the weather will be like the next day”, explains Brunet. This has led to climate science (which studies the range of atmospheric conditions characterising a region rather than focusing on weather forecasting) becoming the great ‘victim’, receiving fewer funds with which to digitise, develop and standardise data.
However, climate services do play a significant role in some European countries, the United States and Canada. It was these services that were able to explain last summer’s heat wave in Eastern Europe and put it into context, as well as the high temperatures recorded on the Old Continent in 2003.
“If we had access to all the historical data recorded, we would be able to evaluate the frequency with which these phenomena are likely to occur in the future with a higher degree of certainty”, the expert explains.
This kind of information is of scientific, social and economic interest, with insurance companies setting their premiums according to expected climate changes, for example. City councils and governments also “want to understand climate conditions and how these will change in future in order to improve land zoning and prevent urban development from taking place in areas likely to be affected by flooding”, concludes Brunet.
References:
Manola Brunet, Phil Jones. “Data rescue initiatives: bringing historical climate data into the 21st century”, Climate Research 47, 29-40, 2011. DOI: 10.3354/cr00960.
So, here are the numbers voor Holland: http://www.knmi.nl/klimatologie/daggegevens/antieke_wrn/index.html . I guess they represent those 20% of world climate data that are ‘disclosed’.
http://clima-virtual-vs-real.blogspot.com/2011/07/ny-times-finds-truth-abusive-for-its.html
Perhaps these data are not available in digital format, but they have persisted for hundreds of years, and they are the original data, and can be verified to be original. What will be the state of data, collected today, 300 years from now? How much will persist? How can it be verified as original and unmodified? We seem to have trouble with that issue already as part of Climategate. Where are the original data?
Really, getting to the core of it aren’t the surface records highly incomplete regardless? Climate is complex and dependency of surface temp records is the first clue it is largely junk science as a form.
We would have trouble today getting close to measuring an “average temperture” and then that being the only measure of “climate” is another stretch. Temps are the first of many narratives that warmists grabbed early on, climate “equilibrium” is another as if it were constant without co2 input. Truely laughable but consider the consequences we have suffered.
The desire to digitize all the records appears biased from the beginning, in that the assumption is being made, based on how I am reading the above post, that it will be anthropogenically warmer and wetter in the future. The desire to digitize world records should be blind to the premise any one entity wishes to demonstrate. However, our current crop of published climate scientists are by and large, not blind at all and seek only to prove their preconceived premise. It is any wonder many countries are so reluctant?
Snip . . OT indeed, but interesting. Please repost to Tips & Notes . . Thanks kb
So is that why the BEST project is taking so long??????
Seems to me some people who’ve done a lot of genealogical work and are looking for a new challenge could do a very useful citizens science project. From scans of the source data sheets to databases of weather data, volunteers could do a better job quicker than waiting for some gov’t funded project.
No, I don’t have time….
“If we had access to all the historical data recorded, we would be able to evaluate the frequency with which these phenomena are likely to occur in the future with a higher degree of certainty”, the expert explains.”
We don’t need hositoric data for that. In fact, we don’t need any data for that. We have models.
/sarc
My understanding was the BEST project would not be ready until Aug or Sept 2011
“governments should adopt a resolution within the United Nations on opening up their historical climate data”
While I have nothing against open access to climate data, the call for action through the United Nations smacks of world government to me. :/
Well, if the 80% are not yet “computer buggered” there is some hope we can recover some un-tainted un-adjusted un-homogenized un-manipulated raw data…
Co-author Phil Jones? Surely that’s not our Phil from UEA? Our Phil wouldn’t be admitting that the historical data is junk, and so we can have no long term view of global temperature, so any assertions that modern times are unduly warm, all because of the beastliness of mankind, are non-sensical, would he? So who is this Phil Jones?
Anyway I think the authors are being optimistic. In the 1850s, how many educated, leisured Columbian gentlemen were able to import expensive and sophisticated equipment, operate it properly, regularly taking readings; and did the civil service exist, to whom, through the world class Columbian Post Office, they could regularly send their data, and did that civil service have the archiving capabilities such that that perfect data is seamlessly preserved to this very day? And as for Africa, well, when since 1960 has the Congo not been at war? How might the records have survived in that environment, and regular updates be made? Oh, OK, we can have a long term global temperature and its perfectly valid without anything much worthwhile from South America and pretty much all Africa; oh and all Soviet Union records 1920-1990 can’t be trusted; nor anything from China since 1949; nor Russia and East Europe post 1990……but we can still make policy, yes?
data such as this should be gathered and collected by an independent group that does no modification of or research with the raw data … then and only then when it is available to anyone who wishes to study it can any real science be done …
this could be the one thing the UN would be best positioned to handle but I would have concerns about their ability to resist fudging …
E.M.Smith says:
July 20, 2011 at 8:46 am
Did you mean ‘buggered’ like this:
Notice: NCDC is working toward restoring access to the full period of record for all IPS documents as quickly as possible. Full access is not expected until mid-summer (July). We apologize for any inconvenience.
Which has been going on for months. They can’t be working hard on it, that would require a tech or two to go get the data from the backup server. Oh, the pain of it all. Oh, bother!
Ric Werme says:
July 20, 2011 at 8:21 am
Seems to me some people who’ve done a lot of genealogical work and are looking for a new challenge could do a very useful citizens science project. From scans of the source data sheets to databases of weather data, volunteers could do a better job quicker than waiting for some gov’t funded project.
The Old Weather project is a start at this, I think: crowdsourced digitisation of Royal Navy WW1 temperature records. Agreed it would be nice if it could be extended more widely, I’m not sure of any other similar initiatives.
http://www.oldweather.org/
“Well, if the 80% are not yet “computer buggered” there is some hope we can recover some un-tainted un-adjusted un-homogenized un-manipulated raw data…”
Agreed, because the other 20% is U.N.-tainted U.N.-adjusted U.N.-homogenized U.N.-manipulated raw data…
A fool’s errand to be sure. If we had all of the data in proper format, one would still not know the quality of the data from hundreds of years ago or even from 50 years ago. So ASSUME it is good data. Then what? Multiple regression analysis? What is the dependent variable? How does one project into the future the independent variables in order to predict the dependent variable? Time series? That assumes everything that happened in the past will repeat itself in a similar fashion.
The climate is chaotic with innumerable unpredictable causal variables which are not predictable even for tomorrow let alone for hundreds of years. Many of these variables are significantly inter-correlated making the task even more ponderous.
As a “climate skeptic” one would be wise to be skeptical of any climate predictions as they are doomed to failure. The climate is always changing and will continue to do so, maybe. The degree of change is what is most unpredictable as well as the timing.
Wonder if the hockey stick came about because mann used the “heat index” for the upward swing and real temps for earlier Values?
I think it is indeed a worthwhile effort to get more of the old weather observations keyed into online databases. There is a project here in the US that is quite interesting, to look at data from US forts and Smithsonian records from the 19th century. Lots of weather and climate information to be had. Of course the difficulty with many of these records is how homogenous and reliable they are in the past. There is quite a lot of information, though, which we must make use of to advance our understanding of weather and climate. One of the preliminary findings of these efforts is that the US experienced more hot and cold extremes and more precipitation extremes at various times in the 19th century than observed in the twentieth, suggesting perhaps that our climate has become milder. If that’s humanity’s “fault” then we should give ourselves a big pat on the back!
My site here
http://climatereason.com/LittleIceAgeThermometers/
contains scores of instrumental temperature records and also diary observations.
tonyb
“Failure to decipher the messages in the climate records of the past will result in socioeconomic problems, because we will be unable to deal with the current and future impacts of climate change and a hotter world”, says Brunet.
Yes, of course! How can our Global Climate Model computers give accurate predictions of the future if we only have access to 20% of past data?
Translation: we need more money. /sarc
In general there is no particular reason older reading are poorer than current readings. You need to look at the meta data – to see who did what where. Given a choice between a hobbiest who mesured in his garden every day for 30 years using simple a max min thermometer or a high tech equipment next to an expanding airport, I would tend to favour the old method. When you see “raw data” online, it is monthly. Temp is not measured monthly. Climate folks just do not seem to get the idea, we want to see the data as it was measured. You can adjust afterwards, but product is only valid if you can trace it back to what you saw first and can justify how you bend it afterwards. So going after written records and putting onlne them under a freeware license outside University or climate scientists “IP ownership”, is highly desirable.
“not even 20% of the information recorded in the past is available to the scientific community”
Good, that way they can’t go back and change the historical data.
Hmm. Its summer in Chicago. and I cannot tell you what the weather will be like next week. I can however tell you that December in Chicago will be colder than July.
“climate” is what we can say about locations over long periods of time. Those predictions are never perfect but they are useful. Climate is predictable. Weather may very well be chaotic. But, you plan for winters in Chicago. Why?
steven mosher says: “Hmm. Its summer in Chicago. and I cannot tell you what the weather will be like next week. I can however tell you that December in Chicago will be colder than July. “climate” is what we can say about locations over long periods of time. Those predictions are never perfect but they are useful. Climate is predictable. Weather may very well be chaotic. But, you plan for winters in Chicago. Why?”
Steven, while it is true that predicting climate is not the same as predicting weather, you strangely stop at a very qualitative, short term point to show that one can “predict” climate. Although there are different reasons why they are difficult, predicting the average temperature of a region one hundred years from now to within a degree is not necessarily easier than doing the same for a week from now. Predicting long term change is much more difficult than get a vague, qualitative description of the next winter correct. Models have no demonstrated skill in this regard.
http://www.met.reading.ac.uk/~ed/blog/2011/learning-about-past-climate-from-ships-logs/#more-415
have a look at this guys
http://blogs.zooniverse.org/oldweather/2011/03/better-than-the-defence/
The British Royal Navy daily logs surely contain weather reports complete with position on the planet, am not sure when this practise was begun but would love to see this data in a digital form. Does it already exist? And what of the US Navy polar records?
steven mosher says:
July 20, 2011 at 1:53 pm
Responding to this earlier quote:
The climate is chaotic with innumerable unpredictable causal variables which are not predictable even for tomorrow let alone for hundreds of years.
Hmm. Its summer in Chicago. and I cannot tell you what the weather will be like next week. I can however tell you that December in Chicago will be colder than July.
“climate” is what we can say about locations over long periods of time. Those predictions are never perfect but they are useful. Climate is predictable. Weather may very well be chaotic. But, you plan for winters in Chicago. Why?
You apparently believe the vaunted GCM models are accurate enough to predict temperatures 100 yearly cycles from now – in 2110 that is.
Further, apparently you believe that these predictions are dire enough that we must sacrifice trillions of dollars now in false prices, inflated energy costs, wasted material, and tens of trillions in lost economic ability worldwide to billions of people who will be killed and damaged through disease, hunger, malnutrition, bad water, no heating, bad food, and no food storage, shipping, or preparation solely due to your “concerns” about these supposedly dire predictions one hundred yearly cycles from now. In other words, you are demanding the deliberate harm to billions for 100 years, just to avoid the known and definite benefits of a temperature gain and more food from more CO2 in the atmosphere, right?
Based on your much-praised models, what are the specific probabilities of a beneficial temperature gain of 1 to 4 degrees in the next 100 years? Of the uncertain benefits of a 4 to 7 degree gain? Of the potential harm of a 7 to 10 degree gain – and exactly what will be the harm? By the way, since temps have been cooling or steady for 13 years, how many of your models have predicted a 13 year flat cycle – and what evidence do you actually have that cycle will not continue? The earth has been warming since the mid-1650 period: When will the Modern Warming Period reach a max: at this point in 2000-2010? 60 years from now in 2060-2070?
By the way, can you tell me what the temperature in Chicago will be (to 1/4 of one degree) one hundred days from now – after only 100 day and night temperature cycles?
Quick, notify Michael Mann, author of the Hockey Stick! No doubt he will give millions from his grants to gain access to all this wonderful historical data. And don’t forget Phil Jones! /sarc
I have always thought that it never made any sense to use a system that was set up to measure local weather and only needed to be accurate to a few degrees (because who cares if the high was 90 or 92) and try to use it to measure global climate to within a tenth of a degree.
Phil Jones? You just can’t make this stuff up. He is worried about preserving climate data? With his track record of preserving climate data?
There’s probably less than that, even though climate data (especially temperature data) is hyped and are seemingly parroted down the street doesn’t mean even half of it is digitized yet. Even less is accessible online, going by my own country’s standards (my country’s government want to get paid to deliver what the rest of us already paid for after all.)
About 70-80% of the population on this planet has yet to come online. I wonder if that has anything to do with the lack of data.
racookpe1978 says:
July 20, 2011 at 4:17 pm
Nice work! The person you address needs to be questioned on the meaning of “scientific prediction” and “evidence for physical hypotheses.” He has not a clue what either means.
For example, if I “predicted” today that our president in 2012 will be a female, the next in 2016 a gay, and the one in 2020 a transgender and all this turned out just as I “predicted,” then in 2020 he would declare me the greatest “predictor” of all times. Those who understand science know that my statements of “prediction” have nothing to do with scientific prediction whatsoever. They fail on several criteria. I will leave the matter of stating the criteria as a game for those who are interested. When he talks about Gaia Models “predicting” something or there being empirical evidence for Gaia Models, he is speaking the language of the village idiot.
racookpe1978
July 20, 2011 at 4:17 pm
###
I think you are reading a little to much in what steven mosher is saying. From reading his comments over the past few years, I get the impression that Steven Mosher is a big fan of easily accessible raw historic measurement data and sees modern information technology as a powerful tool in providing that. He has a good point, after all, it is real raw measurement data that will free us from the tyranny of fault politically biased models. OTH, he is a bit naive in not seeing how a central authority in charge of providing such data will eventually succumb to pressure to manipulate the record to serve the green agenda.
I think all Steven was doing was to provide a context for understanding the difference in the concepts weather vs. climate as it relates to data.
Since digital data technology becomes obsolete very quickly, some thought should be done about some more permanent storage of data for century long storage. Except for difficulty of access to large blocks of data, paper is one of the best media for century long storage of data.
How far back can you have meaningful records before you have to rely on proxies of some sort?
The history of the thermometer may be a good place to start:
http://inventors.about.com/od/tstartinventions/a/History-Of-The-Thermometer.htm
Your predictions about my opinions are worse than a climate model.
You’re predictions about my positions are getting worse.
How many models predicted a flat 13 years? The models fell into roughly two to 3 groups.
some low, some medium. some high.
Looking at individual runs( there were I think around 55 total), I’d say around 5% of them were zero trend to negative trends. That’s right. If you look at individual realizations you’ll find a fair percent that have negative trends for short periods. That’s utterly normal. the Majority of model runs show more warming than we have seen. But models with lower sensitivity have ensemble members that are as low or lower than what we have seen. That’s one reason why it’s hard to evaluate models on short time scales
Are there no scientists left who can analyse and use data without a computer model?
Jesse
You can go back about to 1772 with Central England Temparture series. It claims to go further back, but this early stuff is so poor quality it is generally thought to be usable. The max min thermometer came out around this time, so before this they really did not have the technology.
There are contemporary historical documents from earlier times, of how people lived – eg Dooms Day Book, of frost fairs and mountain snow lines, but is hard to convert into numbers.. Which has allowed the Little Ice Age & Medieveil Warm period to appear and disapear and re-appear in the mainstream view.
Ironic that one camp thinks that climate is important (the averaging of individual data, or better said, the sandpapering of the historical record), but it is not.
Imagine when Steve Mosher gets a call from his son’s guidance counselor because he might be in danger of failing 9th grade (just for example) …
🙂
You forgot dimension and sign.
steven mosher says:
July 20, 2011 at 1:53 pm
“”The climate is chaotic with innumerable unpredictable causal variables which are not predictable even for tomorrow let alone for hundreds of years.””
“Hmm. Its summer in Chicago. and I cannot tell you what the weather will be like next week. I can however tell you that December in Chicago will be colder than July.
“climate” is what we can say about locations over long periods of time. Those predictions are never perfect but they are useful. Climate is predictable. Weather may very well be chaotic. But, you plan for winters in Chicago. Why?”
At such a crude level as saying winter will be colder than summer, I will grant you that such is predictable. As long as we have an axial tilt to our planet that will be the case, however, I would call this predicting the climate, perhaps cyclical weather, not planetary climate in the sense of those who are attempting to predict climate change; degree, direction and timing and are failing miserably in terms of any predictability from their models. If we enter a true cold period of the type that ruins crops the difference will be notable.
Sean says:
“Given a choice between a hobbiest who mesured in his garden every day for 30 years using simple a max min thermometer or a high tech equipment next to an expanding airport, I would tend to favour the old method.”
I ‘d like to see the method that is the most reliable and long-running.
A max-min thermometer requires a reset each day, which means handling the thermometer, and maybe a poorly done reset, and maybe damage to the thermometer. How often is a max-min thermometer replaced, and with another that may read different?
I’d prefer to see 9am temperatures plotted for 100 or more sites around the world, for at least 100 years with NO alteration of the raw data, and NO averaging between sites.
My small country N.Z. would need four sites plotted, as the far north has no winter and the far south has no summer. An average temperature over NZ is nonsense, and so is an average over the whole world. Beside, a mistaken reading at one site is obvious, while it is not obvious when averaged with the rest of the world.
… and we have been paying how many billions in taxes dollars for climate research ?