Has the CRUTEM4 Data been fiddled with?

Guest post by Ed Thurstan

Abstract

It is apparent from the data that CRUTEM4 temperatures adjustments have, in part, been made with reference only to the earlier CRUTEM3 data, rather than raw temperature data. Further, the adjustments depend on the month for the data, and these adjustments are made for 20 or 30 consecutive years.

In the case of Adelaide (946720), for 30 years from 1857, CRUTEM4

  • Lowers all January temps by 1.4oC
  • Lowers all Feb temps by 0.9oC
  • Lowers all March temps by 1.7oC
  • With April to December all lowered by 0.5 to 1.1oC.

Thereafter, there are no adjustments until 2000, when a smattering of adjustments appear, mostly raising the temperature.

There are many examples of this practice. The total effect of all the differences between CRUTEM4 and CRUTEM3 where there is corresponding data is to accentuate warming trend by lowering pre 1995 temps by 0.1 to 0.2oC, and raising post 1995 temps by a similar amount.

This does not mean that the overall effect of all CRUTEM4 updates will induce the same magnitude change in HADCRUT4 anomaly, as no account has been taken of deleted and added stations, or relative numbers of stations in the database, vs the number which display differences as analysed here.

I believe that CRUTEM4 is seriously flawed, due to the apparent selective mechanical adjustments to blocks of station temperatures where the criteria for adjustment are a function of the month name. Until this is adequately explained, CRUTEM4 should be withdrawn.

Background

In March 2012 the Hadley Climate Research Unit released the land temperature dataset CRUTEM4, along with the station data from which it was constructed. A cursory inspection of the new dataset revealed some irregularities in Australian data. In particular, there were puzzling differences between CRUTEM3 and CRUTEM4 data.

A program was written to compare the two complete sets CRUTEM data, and highlight the differences. It compares the two sets to report:

· Stations in CRUTEM3 which do not appear in CRUTEM4. That is, they have been dropped in the construction of CRUTEM4.

· Stations in CRUTEM4 which do not appear in CRUTEM3. That is, new stations.

· Stations which appear in both sets and which have an arithmetical difference in any month of any year. The whole 12 months are reported for these. All missing data (reported as -99 in CRUTEM data) was converted to zero. This means that some data is lost if a valid temperature appears in one set with a matching -99 in the other dataset.

To exclude minor differences due to issues (like roundoff, precision) which could arise in comparing two datasets derived in different ways, a threshold of 0.22 was applied. That is, years are selected only if they contain differences whose absolute value exceeds 0.22oC.

Some simple statistics about the two databases include:

1. Station/Years of data: In CRUTEM3 set – 399,303; In CRUTEM4 set – 466,246.

2. Number of -99 months: In CRUTEM3 set – 674,993; In CRUTEM4 set – 568,606.

3. Number of CRUTEM3 stations dropped from CRUTEM4 286

4. Number of stations added to CRUTEM4 738

5. Matching station years where there is at least 1 month difference 104,296

6. Total stations in CRUTEM3 5,113

7. Total stations in CRUTEM4 5,565

Results

Old temperature data has been adjusted

In the following examples, only a snapshot of small parts of the data is presented to illustrate the point. Positive differences imply that CRUTEM4 is higher than CRUTEM3.

30050 is LERWICK UK; 42160 is ILULISSAT-JAKOBSHAVN Greenland;66450 is BASEL-BINNINGEN Switzerland; 67000 is GENEVE-COINTRIN Switzerland.

image

Adjustments of this magnitude can be seen through most of the CRUTEM4 database, but especially in Europe and adjacent areas. With data from such early years being so sparse, it is difficult to see both why such adjustments have been made, and the basis on which they were made. They were clearly made with reference solely to CRUTEM3 data, and not from original data. There seems little value in adjusting such early data, unless the purpose is to lower early temperatures.

There are strange repeating adjustments

Strange adjustments have been systematically applied to CRUTEM3 data to create CRUTEM4 data. For example, for a period of 22 years, from 1951 to 1972 Station 915540 (Vanuatu) has the following set of adjustments applied:

image

With the original CRUTEM3 data showing

1951 26.5 26.3 26.5 26.0 25.2 24.4 24.2 24.1 24.2 25.1 25.6 26.1

1952 26.8 26.6 26.3 25.6 24.8 23.9 23.0 22.6 22.8 23.8 24.5 24.8

1953 25.3 25.4 25.1 24.3 23.7 22.6 22.5 23.1 23.1 24.2 25.0 24.4

1954 25.2 25.1 24.8 24.2 23.6 23.3 23.3 22.2 23.5 23.7 23.9 24.8

1955 24.6 25.2 24.1 24.1 23.9 22.6 23.1 22.3 23.7 24.0 24.5 24.7

1956 24.9 25.6 24.8 24.3 23.9 23.7 22.4 22.5 24.1 24.4 24.4 25.7

1957 25.7 25.7 25.1 25.0 23.1 21.6 21.7 23.2 23.1 23.5 24.3 25.0

1958 25.6 26.3 25.7 24.9 23.9 23.6 21.5 22.0 23.9 23.8 24.1 25.4

1959 25.3 25.7 25.4 24.6 23.7 23.1 23.4 22.8 23.3 24.2 25.3 25.5

1960 24.9 24.8 24.9 24.4 24.1 23.6 22.6 23.3 24.0 23.8 24.7 24.4

1961 26.0 26.6 26.0 25.0 24.9 24.3 23.9 23.7 24.3 24.5 25.1 25.7

1962 26.1 25.8 25.2 24.9 24.7 23.8 23.3 24.1 23.9 24.4 24.7 25.3

1963 25.6 25.9 25.6 25.0 23.7 24.4 23.0 23.8 23.7 23.6 24.4 25.3

1964 25.8 26.5 26.2 25.3 24.5 24.4 23.2 24.0 23.9 24.5 25.5 25.4

1965 25.5 25.8 25.5 24.6 23.5 23.1 22.5 21.9 22.8 23.1 24.1 25.4

1966 25.8 25.9 25.8 25.1 23.5 23.4 22.4 23.0 23.3 23.8 24.3 24.5

1967 25.4 26.0 25.6 24.6 24.7 24.0 22.9 23.5 23.8 24.1 24.2 25.5

1968 25.5 26.0 25.4 24.5 24.0 23.5 22.9 22.6 23.2 24.1 24.5 25.3

1969 25.6 25.9 26.1 25.6 25.0 24.4 23.3 23.6 23.2 24.1 24.7 25.3

1970 25.9 26.0 26.5 25.1 24.4 23.8 23.6 24.2 24.0 24.5 24.5 25.5

1971 25.3 25.7 25.2 24.8 24.1 24.1 23.2 23.9 23.9 24.1 24.5 24.8

1972 25.3 25.6 25.2 24.8 25.1 23.7 22.4 21.7 23.1 24.0 25.2 25.5

And the original CRUTEM4 data showing

1951 26.9 26.7 26.9 27.0 26.0 25.0 25.0 25.3 25.2 25.9 26.4 26.7

1952 27.2 27.0 26.7 26.6 25.6 24.5 23.8 23.8 23.8 24.6 25.3 25.4

1953 25.7 25.8 25.5 25.3 24.5 23.2 23.3 24.3 24.1 25.0 25.8 25.0

1954 25.6 25.5 25.2 25.2 24.4 23.9 24.1 23.4 24.5 24.5 24.7 25.4

1955 25.0 25.6 24.5 25.1 24.7 23.2 23.9 23.5 24.7 24.8 25.3 25.3

1956 25.3 26.0 25.2 25.3 24.7 24.3 23.2 23.7 25.1 25.2 25.2 26.3

1957 26.1 26.1 25.5 26.0 23.9 22.2 22.5 24.4 24.1 24.3 25.1 25.6

1958 26.0 26.7 26.1 25.9 24.7 24.2 22.3 23.2 24.9 24.6 24.9 26.0

1959 25.7 26.1 25.8 25.6 24.5 23.7 24.2 24.0 24.3 25.0 26.1 26.1

1960 25.3 25.2 25.3 25.4 24.9 24.2 23.4 24.5 25.0 24.6 25.5 25.0

1961 26.4 27.0 26.4 26.0 25.7 24.9 24.7 24.9 25.3 25.3 25.9 26.3

1962 26.5 26.2 25.6 25.9 25.5 24.4 24.1 25.3 24.9 25.2 25.5 25.9

1963 26.0 26.3 26.0 26.0 24.5 25.0 23.8 25.0 24.7 24.4 25.2 25.9

1964 26.2 26.9 26.6 26.3 25.3 25.0 24.0 25.2 24.9 25.3 26.3 26.0

1965 25.9 26.2 25.9 25.6 24.3 23.7 23.3 23.1 23.8 23.9 24.9 26.0

1966 26.2 26.3 26.2 26.1 24.3 24.0 23.2 24.2 24.3 24.6 25.1 25.1

1967 25.8 26.4 26.0 25.6 25.5 24.6 23.7 24.7 24.8 24.9 25.0 26.1

1968 25.9 26.4 25.8 25.5 24.8 24.1 23.7 23.8 24.2 24.9 25.3 25.9

1969 26.0 26.3 26.5 26.6 25.8 25.0 24.1 24.8 24.2 24.9 25.5 25.9

1970 26.3 26.4 26.9 26.1 25.2 24.4 24.4 25.4 25.0 25.3 25.3 26.1

1971 25.7 26.1 25.6 25.8 24.9 24.7 24.0 25.1 24.9 24.9 25.3 25.4

1972 25.7 26.0 25.6 25.8 25.9 24.3 23.2 22.9 24.1 24.8 26.0 26.1

Similar adjustments can be seen in many other stations. The above Vanuatu data comes from a tropical area where temperature varies only 2 to 4oC over the year. In this situation, an adjustment of 0.4 to 1.2oC degree seems extreme. Furthermore, there are no adjustments after 1972. But the same type of adjustment appears in more temperate 946720 (Adelaide Australia). For the 30 year period 1857 to 1886 the following adjustment has been applied to CRUTEM3 to create CRUTEM4.

image

when the two Datasets show

CRUTEM3

1857 22.6 28.3 19.7 17.5 13.0 11.6 11.8 12.2 14.2 15.4 18.4 23.4

1858 26.4 24.0 22.3 18.3 13.5 12.0 10.3 11.7 12.6 15.8 21.1 22.5

1859 23.8 22.4 20.3 17.1 13.2 10.9 10.7 12.4 13.3 17.7 19.7 22.9

1860 26.1 23.8 22.4 17.0 14.4 12.5 11.7 13.4 15.5 17.0 19.9 22.5

1861 23.2 22.3 23.2 19.1 14.2 13.7 10.6 11.4 14.3 17.7 20.0 19.8

1862 25.4 22.4 23.0 17.0 14.9 11.9 13.2 13.1 15.3 18.6 21.4 23.6

1863 23.5 24.0 22.3 20.7 15.8 13.3 11.7 12.2 13.5 16.3 19.0 21.9

1864 23.0 21.9 21.1 18.1 15.3 11.7 11.5 11.7 15.4 16.1 20.6 21.0

1865 21.6 21.7 21.3 19.5 13.6 11.8 10.6 12.7 14.6 17.1 21.7 20.8

1866 23.9 25.2 21.2 19.0 15.8 12.6 11.6 12.6 13.8 16.6 17.8 22.1

1867 24.1 24.3 20.7 18.2 15.6 14.0 11.5 12.6 13.2 16.2 18.9 20.1

1868 20.6 22.9 23.4 17.8 15.8 12.1 10.4 12.1 14.6 17.9 20.2 22.0

1869 22.1 22.6 21.3 17.4 13.4 12.4 11.3 12.8 11.8 16.4 20.4 21.7

1870 23.4 25.3 21.7 18.7 13.9 12.7 10.8 11.5 12.9 17.2 18.0 21.9

1871 23.0 23.8 20.3 18.6 15.4 13.4 11.6 13.4 14.8 16.6 18.6 24.0

1872 25.9 23.3 22.4 17.1 13.5 12.6 11.1 9.9 14.1 16.2 21.0 20.0

1873 23.9 22.6 19.9 16.7 14.9 11.8 10.7 12.6 14.2 18.3 17.3 23.6

1874 24.0 21.7 19.8 19.4 14.0 11.7 9.8 11.5 12.1 17.6 17.9 21.9

1875 23.6 22.9 20.9 18.1 13.2 11.9 10.6 12.2 13.8 16.6 18.2 19.3

1876 22.8 22.1 23.8 16.5 13.3 10.8 10.1 11.5 13.5 16.1 18.6 23.4

1877 23.1 24.7 20.3 17.7 14.2 11.7 11.2 13.6 12.4 16.1 17.2 20.6

1878 25.6 23.0 21.6 18.3 14.1 9.9 11.7 12.9 14.3 17.5 20.1 21.4

1879 24.4 23.9 20.9 18.6 12.4 11.6 10.1 12.1 13.2 16.2 18.6 21.1

1880 25.6 26.4 21.5 17.5 13.7 12.1 10.4 12.9 13.7 15.4 17.8 21.9

1881 23.2 21.7 21.4 17.4 14.9 10.6 10.6 11.8 13.9 15.3 18.6 21.3

1882 22.9 24.0 22.6 17.4 15.1 10.6 9.6 11.2 14.0 16.9 20.8 21.6

1883 23.6 21.9 20.9 18.4 13.2 13.1 10.8 11.3 12.4 15.3 19.6 20.9

1884 21.2 23.6 22.1 17.1 13.9 12.2 9.8 13.3 14.1 15.5 18.9 19.5

1885 21.6 21.6 18.9 17.2 15.9 10.8 10.7 12.8 14.0 18.0 19.2 23.2

1886 24.4 20.6 20.4 17.3 14.0 11.6 11.6 12.4 16.1 14.7 19.9 21.9

CRUTEM4

1857 21.2 27.4 18.0 16.4 12.1 11.0 11.4 11.4 13.7 14.7 17.5 22.6

1858 25.0 23.1 20.6 17.2 12.6 11.4 9.9 10.9 12.1 15.1 20.2 21.7

1859 22.4 21.5 18.6 16.0 12.3 10.3 10.3 11.6 12.8 17.0 18.8 22.1

1860 24.7 22.9 20.7 15.9 13.5 11.9 11.3 12.6 15.0 16.3 19.0 21.7

1861 21.8 21.4 21.5 18.0 13.3 13.1 10.2 10.6 13.8 17.0 19.1 19.0

1862 24.0 21.5 21.3 15.9 14.0 11.3 12.8 12.3 14.8 17.9 20.5 22.8

1863 22.1 23.1 20.6 19.6 14.9 12.7 11.3 11.4 13.0 15.6 18.1 21.1

1864 21.6 21.0 19.4 17.0 14.4 11.1 11.1 10.9 14.9 15.4 19.7 20.2

1865 20.2 20.8 19.6 18.4 12.7 11.2 10.2 11.9 14.1 16.4 20.8 20.0

1866 22.5 24.3 19.5 17.9 14.9 12.0 11.2 11.8 13.3 15.9 16.9 21.3

1867 22.7 23.4 19.0 17.1 14.7 13.4 11.1 11.8 12.7 15.5 18.0 19.3

1868 19.2 22.0 21.7 16.7 14.9 11.5 10.0 11.3 14.1 17.2 19.3 21.2

1869 20.7 21.7 19.6 16.3 12.5 11.8 10.9 12.0 11.3 15.7 19.5 20.9

1870 22.0 24.4 20.0 17.6 13.0 12.1 10.4 10.7 12.4 16.5 17.1 21.1

1871 21.6 22.9 18.6 17.5 14.5 12.8 11.2 12.6 14.3 15.9 17.7 23.2

1872 24.5 22.4 20.7 16.0 12.6 12.0 10.7 9.1 13.6 15.5 20.1 19.2

1873 22.5 21.7 18.2 15.6 14.0 11.2 10.3 11.8 13.7 17.6 16.4 22.8

1874 22.6 20.8 18.1 18.3 13.1 11.1 9.4 10.7 11.6 16.9 17.0 21.1

1875 22.2 22.0 19.2 17.0 12.3 11.3 10.2 11.4 13.3 15.9 17.3 18.5

1876 21.4 21.2 22.1 15.4 12.4 10.2 9.7 10.7 13.0 15.4 17.7 22.6

1877 21.7 23.8 18.6 16.6 13.3 11.1 10.8 12.8 11.9 15.4 16.3 19.8

1878 24.2 22.1 19.9 17.2 13.2 9.3 11.3 12.1 13.8 16.8 19.2 20.6

1879 23.0 23.0 19.2 17.5 11.5 11.0 9.7 11.3 12.7 15.5 17.7 20.3

1880 24.2 25.5 19.8 16.4 12.8 11.5 10.0 12.1 13.2 14.7 16.9 21.1

1881 21.8 20.8 19.7 16.3 14.0 10.0 10.2 11.0 13.4 14.6 17.7 20.5

1882 21.5 23.1 20.9 16.3 14.2 10.0 9.2 10.4 13.5 16.2 19.9 20.8

1883 22.2 21.0 19.2 17.3 12.3 12.5 10.4 10.5 11.9 14.6 18.7 20.1

1884 19.8 22.7 20.4 16.0 13.0 11.6 9.4 12.5 13.6 14.8 18.0 18.7

1885 20.2 20.7 17.2 16.1 15.0 10.2 10.3 12.0 13.5 17.3 18.3 22.4

1886 23.0 19.7 18.7 16.2 13.1 11.0 11.2 11.6 15.6 14.0 19.0 21.1

The updates accentuate the global warming argument

There is pronounced tendency to flex the graph of global temperature vs time in a way which accentuates warming in recent years. However, the dominant effect is to lower temperatures prior to 1995.

The differences between CRUTEM3 and CRUTEM4, with zero tolerance, were consolidated to annual temperature differences, and plotted against time along with the count of the stations which contributed to the graph. This gives

clip_image002

The adjustments indicate that in CRUTEM4:

· A comparatively small number of stations have been cooled between about 1820 and 1900 anywhere up to 0.4oC.

· A substantial number of stations have been cooled about 0.1oC between about 1910 and 1995.

· Between about 100 and 400 stations have been warmed by up to about 0.2oC since about 1995.

This does not translate into the same change in the anomaly vs time graph, because it is not the whole dataset, but simply the changed data between CRUTEM3 and CRUTEM4. Nor does it take into account the effect of dropped and added stations. But it does indicate the direction of the change.

Nitpicking Differences

The nature of the CRUTEM database makes it obviously difficult to manage. Data comes from many sources. It is unreasonable to expect Hadley or UEA personnel to understand the geography of the data they receive. So errors will appear in the database. Some positional errors can be dismissed on the grounds that CRUTEM is directed at anomalies. This means that if a station’s position data is wrongly represented, then provided the error is contained within the same 5o v 5o gridcell, the anomaly is unaffected.

However, it is difficult to see how Station number 237070, name listed as “Unknown, Russia”, with a Lat/Long of -99.9/-999.9 (ie unknown), could escape being found by Quality Control, while still feeding temperatures to the gridding/anomaly calculation. Station 288020 is similarly identified, and also supplies data.

“Normals” are, by convention, calculated over 30 years from 1961 to 1990. While this appears to be observed, it is quite common to calculate Standard Deviations over a different period, commonly 1941 to 1990.

Extreme differences occur when the same station Number is used for two different locations. For example, In CRUTEM3, Station 840270 is the high altitude Tulcan El Rosal in Ecuador. But in CRUTEM4 station 840270 is Esmeraldas Tachina, also in Ecuador, but several hundred km distant from its CRUTEM3 namesake, and at almost sea level. This appears to be a CRUTEM3 error, probably difficult to find once the error is made.

Sydney Airport (947670) is still badly identified. The data runs from 1859 to the present, but only data after 1990 comes from Sydney Airport. The earlier data comes from Sydney Observatory, 10km away.

Effect of added and deleted stations

No attempt is made in this report to assess the effect of the 286 station deletions and the 738 additional stations, except to observe that most of the additions are high latitude NH stations.

Discussion

When updating a database such as CRUTEM, I would expect the steps to be roughly

· Tidy up the precursor database, which should be mostly stable data that has had years of scrutiny, and which would require limited correction or additions.

· Delete data which is considered poor quality.

· Add new data, edited and homogenised.

· Run a check as has been done in this report, looking at differences which might suggest irregularities.

But Hadley/CRU do not have appears to have done this. They have added and deleted stations, but it seems strange that very early data – 18th and 19th Century data, should be added, especially when much of it is sparse, of perhaps questionable quality and not germane to the current temperature/time/CO2 discussion.

But the biggest problem with the new HADCRUT4 database is the frequent practice of systematic, repetitive temperature modifications on blocks of station data. What reason could there be for making the following set of adjustments on Adelaide data for every year from 1857 to 1886 ?

image

The perpetrators of this change have clearly not gone back to original raw data and re-appraised their original homogenisation processes. They have simply taken every year of CRUTEM3 data (with it’s possible homogenisation adjustments), from 1857 to 1886, and

· Deducted 1.4oC if the month name was January,

· Deducted 0.9oC if February,

· Deducted 1.7oC if March

· And so on.

It is difficult to conceive w­­­­­­hat the justification could be for these changes. There are many instances of such changes in HADCRUT4, covering periods of 5 to 30 years, in both ancient and recent data. The adjustment vector differs in each, but the magnitudes of the individual elements are equally large. Their signs may vary.

The temperature differences, where there are corresponding data in each of the datasets works to promote the impression of a temperature surge in the 1990s.

clip_image004

It has done this by lowering temperatures before about 1995, and raising them thereafter.

The effect of this on the final gridded anomaly data has yet to be calculated. But it will be less than depicted above. The above graph shows the difference in the data induced by changes in data which appear in both CRUTEM3 and CRUTEM4. It does not include the effect of

· Dropped stations

· Added stations

· Stations which record a valid temperature in one set, but a Null (-99) in the other.

There are other relatively minor issues regarding data quality. I am surprised that neither CRU quality control nor users of the data have picked up these errors.

The CRUTEM4 database appears even more like one which has been assembled by amateurs with little concept of accuracy or integrity. If you were of a suspicious nature, you might get the impression that the changes are targeted at accentuating the Warmist message.

Until the matters raised in this report have been adequately explained, CRUTEM4, and therefore HADCRUT4 are seriously flawed.

Acknowledgements

Thank you Warwick Hughes (http://www.warwickhughes.com/blog/) for encouraging me to write this report, and for helpful suggestions on the construction of the report.

Data available

The original data is available at: http://www.cru.uea.ac.uk/cru/data/temperature/.

The data and processing programs used in preparation of this report are available from the author at thurstan@bigpond.net.au., or from Warwick Hughes blog http://www.warwickhughes.com/blog/ They are in two zipped packages.

Data

This comprises

· The CRUTEM3 and CRUTEM4 station data for all examples mentioned in this report.

· The output of a full CRUTEM run, with some analysis of the output. It will not run under XL97, which is limited to 64K rows, when this workbook holds over 400,000 rows.

This is about 26mb compressed.

Programs

· An Excel 2010 Workbook. The one that produces the output used in this report. It is a Visual Basic driven Excel workbook. It extracts stations missing from each dataset, and the differences. An untested XL97 version is included. It will not run the full CRUTEM database, as XL97 is restricted to 64K rows, and CRUTEM generates over 100,000. But it is possible to split the two sets of input into smaller parcels. Australia/New Zealand generated only about 650 rows of output.

· Instructions for use are on Worksheet “MAIN”.

Advertisements

  Subscribe  
newest oldest most voted
Notify of
crosspatch

Stupid data tricks.

Surely this is a clear case of Criminal mis-representation and distortion? The next step is to throw out all the data and just make up whatever you want to fit in with your crippled Theory.

pat

will anyone understand it if it was??
3 May: Australian: Fiona Gruber: Fighting ‘catastrophilia’ with wit
RICHARD Bean greets me in the foyer of London’s National Theatre. He’s a strapping man with the air of a pugilist. He has tight grey curls on a battering-ram head and a bluff northern manner to go with it.
He’s friendly, but you wouldn’t want to pick a fight with him. I discover this when I start on the subject of global warming. It’s apposite because we’re here to discuss his play The Heretic, opening at the Melbourne Theatre Company this month…
Every climate model has “failed laughably”, he says, and these are the models that are the whole basis for global warming alarmism. The scientists who push their gloomy predictions are politically motivated, he claims, and the politicians are too ignorant to understand the arguments.
“There’s one single bachelor of science in the House of Commons. They don’t understand a word of it and I bet your government is much the same.”…
http://www.theaustralian.com.au/arts/fighting-catastrophilia-with-wit/story-e6frg8n6-1226345167184

Keith Battye

What a bloody mess.
Who and how will this record be “adjusted” to show consistent reality?

Peter Miller

All climate data sets will be routinely adjusted to lower historic temperatures and raise current ones. There will be no exceptions.
This is basic ‘Climate Science’ 101.
I believe your study is just the tip of the iceberg in exposing deliberate scientific malpractice in the creation of CRUTEM4.

Andrew

What is so head-shakingly galling is the utter shamelessness of these data-fiddling cultists. They know they are being scrutinised. They clearly couldn’t care less. When challenged they will, as usual, just deny it. Post Normal Science. Shameless charlatans.

KnR

Adjustments own their own are not a problem . But there needs to be clarity on what was done and good scientific support for why it was done and top of that the ability to recall the raw unadjusted data , now with that being missing there is a very real problem. And this does seem to be a ‘problem’ of ‘climate science’

crosspatch

Looks like a cold weekend in the UK, will be interesting to see how HADCRUT4 reports it. Calling for the possibility of the coldest May temperatures ever recorded with -9C forecast along with snow in Scotland and N. England.
http://www.dailymail.co.uk/news/article-2138728/The-big-May-Day-Bank-Holiday-weekend-freeze-Forecasters-predict-snow-sub-zero-temperatures-biting-winds-rain.html

Brendan

I cant seem to get my head around why they would do this other than to mislead and perpetuate their madness. It disappoints and maddens me simultaneously.
On the plus side. Polar bears can now swim. Who knew!
http://www.guardian.co.uk/environment/2012/may/02/poles-scienceofclimatechange

The changes in CRUTEM4 in comparison with CRUTEM3 is that we have more Arctic represented and thus more Arctic amplification of warming.
However, this Arctic amplification should then also be seen around 1930-45, but its just not really there to be seen:
http://hidethedecline.eu/media/ADJ/11.13.gif
So yes CRUTEM4 is not surprisingly yet another freely painted dataset, and thank you for bringing it up!

Well done in tackling this. I daresay we shall need a lot more such investigation before these temperature compilations may be trusted. The Climategate 1 documents revealed people who had neither the moral nor the technical competence to be entrusted with such compilations. In the absence of any prospect of a major audit by competent and uncompromised authorities, work such as yours is invaluable.

braddles

Forget these massive databases. We need a “few good men”, maybe 50-100 unimpeachable rural locations with long records but still operating, spread around the globe. This would produce a much more reliable and trustworthy record, and it would be much harder for CRU, Hansen etc to apply their “tricks” to the data without being caught.
Adding vast quantities of unreliable data to good data and then trying to correct the errors is scientific idiocy. Sydney Airport is included? What a joke!

crosspatch

Considering the whole polar amplification thing … why are not the satellite-based measurements showing it? And why does HADCRUT rely on “adjustments” to show it.

Galvanize

What, exactly, is the justification for adjusting historic data? Do they have the original instrumentation and methodolgy to hand to know that they were over reading during Victorian times?
The “fact” that adjustment directions are virtually 100% predicatable is something of a smoking gun.

wshofact

The Ed Thurstan files are zipped and can be downloaded;
http://www.warwickhughes.com/papers/etdata.zip – approx 27Mb
http://www.warwickhughes.com/papers/etodds.zip – other 2 under 500Kb each
http://www.warwickhughes.com/papers/etprogrampack.zip

NZ Willy

Hey, by the time CRUTEM20 comes out, the 19th century should be featuring subarctic temperatures for the whole temperate world. They’ll need to rewrite history that the Civil War was fought in the snow all year long. And Washington did not cross the Delaware in boats, they walked across the ice! It’s no longer good enough to “flatline the whole Holocene”, it needs to be driven 6 degrees under (zero) as well. Also, thermometers will need to be rezeroed to show more degrees.

I know Lerwick is down, but even so … why the **** are they adjusting it at all?
Lerwick is on Shetland. For those who may not know, it is an island some 100miles away from the mainland. So, I doubt there was another station within 100miles. On what possible basis can they change historic data?
Why would I change this? What legitimate reason could I have to do this?
Answer: unless I had a thermometer which I knew to be more accurate and which I could compare directly with this station (which seems incredibly unlikely on Shetland) I can’t think of any reason on earth to change this.
Could any kind of averaging come into play? One might be able to argue (after extensive field tests to confirm the thesis) that historic data was biased in some way and that an average were more accurate. This however would be something added to the global average figure and not individual stations.
The only possible even vaguely credible explanation (and it still stinks to high heaven unless it is openly discussed) is that stations were adjusted in different classes depending on the exact instrumentation being used. E.g. one might possibly argue that mercury thermometers … I can’t think of a reason why … but if one had a plausible rationale that they wouldn’t read the right temperature. OK, perhaps But each station would need to be documented, and my impression is that there just isn’t the historic evidence to go about such a wholesale modification.
…. this isn’t science, its not engineering is quackery!

braddles says:
” We need a “few good men”, maybe 50-100 unimpeachable rural locations with long records but still operating, spread around the globe. ”
I central and western Europe there are just a handful such rural stations not placed on a coast and not placed on a mountains (both shows ocean trends).
Heres what happens for Europe temperatures when examining the few.. :
http://hidethedecline.eu/pages/ruti/europe/western-europe-rural-temperature-trend.php
In general we need more data and thats why my “RUTI” has to compromise on “Rural” data by using more urban data, although not the metropols:
RUTI is world wide, but here for europe, (I use RAW GCHN V2, RAW Nordklim and RAW NACD version1..!) + data from original writings, which are presently coming in fast so I have to update soon.
Overview RUTI results europe:
http://hidethedecline.eu/pages/ruti/europe.php
Im awware of all the arguments against this project, but I would appreciate more help and backup in stead since there is no alternative I we should just be honest.

Dodgy Geezer

Is anyone going to start asking the perpetrators what the justification for this is?

John Doran

This is what I believe marketeers call post-hoc justification. Their deception is not aimed at skeptics but at those evangelists who already believe and thus whose vigour will be renewed………..

Graeme M

This may not be the best place for this question, but it is sort of relevant. I don’t really understand why all the adjustments to temperature series, but presumably there are good reasons.
Something I have amused myself with is to look at the greater than 10 year trends in temp data using Wolfram Alpha. I don’t know what data this uses, or whether it’s raw or adjusted. However, most places I do this for show next to no trend in either direction.
That makes me wonder what would happen if we looked at raw unadjusted data over time. Has that been done? Is that what BEST did? Without adjustment for various effects we may not get a true representation of actual temps, but any trend would surely be obvious and relevant. Wouldn’t it?

Stephen Wilde

It looks like the most reliable historical temperature information we now have is the anecdotal evidence from personal diaries and records, ships logs and media reports.
Everything else would appear to be worthless.
If the evidence is about to destroy one’s case then corrupt the evidence.

Chuckles

As I understand it, the creators of the above ‘know’ what the right answer is, with absolute certainty.
Since the data doesn’t show this right answer, it is clearly wrong, and must therefore be adjusted so that it does.
As the ‘global mean temp’ is a concept, and has no physical meaning or relevance to anything, I suppose they can do whatever they like to construct it?

Tom

Block adjustments only make sense to me if there is some known change to the measurement method – if, say, Adelaide had a poorly set up station from 1857 to 1886, then we might be able to estimate a set of offsets to apply to a block of data. And it’s not so unreasonable to suppose that those offsets might be calculated month by month, based on averages for those months.
One thing that springs to mind is that Adelaide has unusually low humidity compared both to the rest of Australia’s populated areas and Northern / Western Europe, so there is conceivably some humidity-related adjustment that has been calculated.
But of course this is groping for explanations. I don’t see how you can get away with publishing this sort of data set without also giving at least some specific explanation for each adjustment made (something at the level of, “Compensation for factor X, calculated monthly according to x = y * z). And when all those unexplained adjustments just happen to nudge the line more towards a stick used for playing hockey, it all smells a bit.

Johan i Kanada

Ok, so adjustments have been made.
But what is the stated rationale for such adjustments? Presumably not even the team will make all kinds of adjustments without some sort of reason?
If anyone knows, or can point to some source, that would be appreciated.

Ryan

All adjustments in the same direction that just happens to fit the original theory?
Absolutely screams “confirmational bias” to anyone that knows anything about science, measurement and statistics.

Dutch Erik

The CRU doesn’t even need to be hacked to conclude that data manipulation without sound scientific and/or statistical arguments is one of their main trademarks! This is just more of the same that came out during the climategate hack and it’s sad that nothing has changed and the responsible people are still running the “catastrophic climate show”….

Bloke down the pub

As Phil Jones might say- trust me, I’m a doctor.

cartoonasaur

There can be no rational reason to adjust very old historical temperatures.
If we cannot time travel to calibrate the OLD against the NEW, then we CANNOT adjust the old data at all.
Yet clearly, the old data WAS adjusted, on a massive scale.
Won’t someone arrest these lying sacks of manure already, and have them time-travel their way out of jail…

Michael in Sydney

Gotcha!

I understand that CRUTEM4 still uses GHCN V2 data. When they move to V3.1, even more unexplained adjustments will appear.
What is often forgotten is that much of the “raw” data CRU (and GHCN and GISS) use is in fact already adjusted at source, for non climatic influences, by the national met offices. (And where they are not, I would question whether their quality is robust enough to warrant inclusion in a global database)

tango

its all right to fuge temp readings as long as your not found out

Robert of Ottawa

The block adjustments of data over years is unfathomable and unjustifiable.

michaeljmcfadden

[snip . . OT . . kbmod]

michaeljmcfadden

[snip . . OT . . kbmod]

Nick Stokes

“The perpetrators of this change have clearly not gone back to original raw data and re-appraised their original homogenisation processes. “
I don’t think the CRUTEM3 data for Adelaide was homogenized, in the period you have highlighted. It seems to be identical, to GHCN unadjusted. It’s possible that it should have been, and that is the reason for the change. That would probably account for the monthly pattern.

Urederra

If you adjust data you are not going to be able to predict/project future weather/climate.
Good data does not necessarily mean good predictions, but surely bad data will render faulty predictions/projections/whatever-you-want-to-call-them.

Kev-in-UK

are the warmista going to appear and tell us the logical reasons for these obvious adjustements?
somehow, I doubt it……
as I’ve mentioned many times, does anyone actually have the ‘raw’ data? (from Jones testimony, it seems not?) does anyone have a record of the adjustments since the first prepared data sets? (again, IIRC, from Jones testimony, I think not!). So, in a nutshell, this data is not ‘as recorded’ data anymore, it is essentially worthless to use for any meaningful interpretation, IMHO.

Dr Burns

Phil Jones was obviously determined to reset the record straight after admitting there had been no warming for 15 years.

kim

The moving thermometer writes, and having writ, adjusts itself.
=========================

Retired Dave

Thank you for your hard and professional work Ed.
As a meteorologist I was always a bit distrustful of climate data adjustments, even in the days before AGW raised its ugly head – datasets can need adjustment, but it can’t always be in the one time related direction, but it is. Amazing stuff.
Thanks again

wayne

After reading enough comments I’m kindof wondering if there exists a complete dataset of just the raw temperatures anywhere in this world, or has it basically been destroyed through these multilayered, multisite adjustments to the data itself?
If it does, who holds it? Is it public and available?
For instance, I have download and processed both of the BEST datasets, about 13-17 gigs each unpacked, that is supposed to be “RAW” just to find out that even their “RAW” data has already gone through a whole series of manipulations and the worst, detrended. Where’s the beef?

R James

Can someone else take all this raw data, and just compile it without all the adjustments, and see what it shows?

Ian W

“It is difficult to conceive w­­­­­­hat the justification could be for these changes.”
Not at all. It would be charitable to call it inept programing and abysmal quality control. However, the very purpose of these people is to produce a dataset to be used by the world. It cannot be that they are so useless at their primary task. Therefore, it is no accident this is deliberate.
It is possible that the intent is to provide useful figures to Rio+20 supporting a hockey stick in the hope that the morass of figures with changed station numbers would hold reviewers at bay for a few months.
How very sad that an accredited academic institution should be involved.

KenB

In the book published in 1913 by the Commonwealth Bureau of Meteorology “Climate and weather of Australia by H.A.hunt and Griffith Taylor and E.T.Quayle” There is at page 11 a mean monthly Temperature and rainfall of all the Australian Capitals (cities) including Adelaide – I assume this is a 1913 snapshot across Australia. There was a printable copy on line (from University of California) but the original link
(http://www.archive.org/bookreader/print..php?id+climateweatherof00huntrich&server=1a331414) no longer works.
It contains a wealth of contemporary weather information including hottest and coldest recorded temperatures, plus details of droughts and floods up to 1913 and 59 weather maps on 75 printed pages. There are copies held in libraries including at the BOM Melbourne, Interesting synopsis of Australian weather history. sorry the link wont work.. Maybe someone else might be able to post a copy of the relevant page from an electronic image.

David

Surely a matter for investigative journalists in the mainstream press to get their teeth into..?

Keith Pearson, formerly bikermailman, Anonymous no longer

Come on now, if a “Living Breathing” way is good enough for the US Constitution….

ozspeaksup

kens kingdom @wordpress did aus remps a while back
the BoM should be taken to task.
I also noted old info that was easy to find vanished 2 yrs back, filed in some obscure program and hidden away

Frank Lansner:: Overview RUTI results europe:
http://hidethedecline.eu/pages/ruti/europe.php
Im aware of all the arguments against this project, but I would appreciate more help and backup.
Frank I’m interested, but I don’t see the point of another index which shouts at me “don’t trust me I’m written by someone who is biased”. I know it sounds pedantic, because no warmist will ever refer to it, but a long time ago … I had this dream of finding a temperature index I could trust. Not upjusted, nor down-justed, just the best and honest appraisal of the temperature record.
At the very least you have to draw a very clear distinction between what is and what is not your own views. That means a neutral web name otherwise it screams: “bias”.

Steve in SC

I’m with Rooster Cogburn on this.
“Give em a fair trial and a nice hanging.”