I mentioned the poster earlier here. Now I have the video interview and the poster in high detail. See below.
Here is the poster:
And in PDF form here, where you can read everything in great detail:
I mentioned the poster earlier here. Now I have the video interview and the poster in high detail. See below.
Here is the poster:
And in PDF form here, where you can read everything in great detail:
One of Zeke’s comment that struck me as curious was his saying that their new time series agrees with ‘reanalysis’ model output more than it does with other datasets.
He concludes that this gives them more confidence that they are finding physically real phenomena.
Now , to my ear, that says he has a basic belief that models are more accurate reflection of physical climate than the data they are supposed to be modelling and that if BEST is nearer to model output it is better than existing datasets.
So now datasets are being created and the criterion of their merit is whether they get close to reflecting what models produce.
ie More of make the data fit the models , not make the models fit the data.
Problem !!
Sorry, I do not believe this data. The Historical REAL data used up to the 1990s definitely showed the 1930/40s as being much warmer than they are shown in the current BEST (and GISS) data.
I can see no justification for cooling history, it is an insult to the people who took those readings.
Ryan Scott Welch says:
Why did they start at 1850? Is it because it was the best year to start the record due to temperature measuring sites, quality of data, or because that was the end of the Little Ice Age?
===
It is also the part of the climate record which nicely fits a quadratic (or exponential) rise:
http://climategrog.wordpress.com/?attachment_id=746
In fact the earlier part could also fit the same quadratic but not one that can be linked to AGW.
This is why all the “bias corrections” reduce earlier variability and cool the 19th century.
The BEST data is in fairly good agreement with other sources since 1910.
http://www.woodfortrees.org/plot/hadcrut4gl/mean:180/mean:149/mean:123/from:1910/plot/crutem4vgl/mean:180/mean:149/mean:123/from:1910/plot/best/mean:180/mean:149/mean:123/from:1910/plot/gistemp/mean:180/mean:149/mean:123/from:1910
Fig 1. HadCrut4, CruTem4,, BEST, GisTemp Gaussian low pass filtered (since 1910)
There is very poor agreement between any of the data sets prior to this time however.
http://www.woodfortrees.org/plot/hadcrut4gl/mean:180/mean:149/mean:123/plot/crutem4vgl/mean:180/mean:149/mean:123/plot/best/mean:180/mean:149/mean:123/plot/gistemp/mean:180/mean:149/mean:123
Fig2. HadCrut4, CruTem4,, BEST, GisTemp Gaussian low pass filtered (whole record).
Unless there is some good explanation as to why this relationship breaks down at that point in time then the data and results from that data must be in question.
The question as to why the well observed ~60 year cycle disappears in the early part of the BEST record is worth answering I think.
Few days ago Steve Mosher said somthing along the lines ” …we do not create gridded data , there is raw data that is crap , we slice it an locate brace and then calculate a field ….. ” in an answear to someone ( not sure if it was here or at on another site ) who was critical of the BEST project methodology.
I had been fooling a round with older data from the weather station in a place named Stykkishólmur in Iceland , monthly data back to 1830 are available for that station on the Iclandic Meterologigal Offiice, and had the portion covering 1830-1948 (incl) in a file on my desktop, so I went to the BEST website and pulld down their data for that station for a quick comparision. Browsing through both timeseris side by side showed that the the monthly averages were in good agreement most of the time and in fact had the same value and with the same single decimal precision. However i also noticed that there were few divergences and all of the seemed to be caused by there being a different sign on the BEST and IMO values ( absolute values for both agreed ) for the months in question, so I filtered those instances out. Here below is a table of the the result ( cross my fingers and hope wordpress does not mangle the formatting )
Comaparision of Iceland Met Office Data for Stykkishólmur
to BEST project raw input data Data for same station.
period 1830 to 1948. ( all temperature values are in °C)
Raw Raw Data
BEST IMO difference Bias Divergence
Year Month Temp Temp IMO-BEST direction Num
1884 3 -1.6 1.6 3.2 Lower BEST 1
1905 3 -1.0 1.0 2.0 Lower BEST 2
1908 3 -0.3 0.3 0.6 Lower BEST 3
1911 3 -0.3 0.3 0.6 Lower BEST 4
1918 3 -0.3 0.3 0.6 Lower BEST 5
1922 1 -0.1 0.1 0.2 Lower BEST 6
1922 3 -0.9 0.9 1.8 Lower BEST 7
1923 3 -3.7 3.7 7.4 Lower BEST 8
1924 1 -0.3 0.3 0.6 Lower BEST 9
1926 1 -0.2 0.2 0.4 Lower BEST 10
1927 3 -2.0 2.0 4.0 Lower BEST 11
1928 3 -1.3 1.3 2.6 Lower BEST 12
1929 1 -1.8 1.8 3.6 Lower BEST 13
1929 3 -5.4 5.4 10.8 Lower BEST 14
1932 3 -1.7 1.7 3.4 Lower BEST 15
1933 1 -0.9 0.9 1.8 Lower BEST 16
1933 3 -0.4 0.4 0.8 Lower BEST 17
1935 1 -1.6 1.6 3.2 Lower BEST 18
1935 3 -1.8 1.8 3.6 Lower BEST 19
1939 3 -1.9 1.9 3.8 Lower BEST 20
1940 1 -0.6 0.6 1.2 Lower BEST 21
1941 3 -0.3 0.3 0.6 Lower BEST 22
1942 1 -1.5 1.5 3.0 Lower BEST 23
1942 3 -1.3 1.3 2.6 Lower BEST 24
1944 3 -0.6 0.6 1.2 Lower BEST 25
1945 3 -2.9 2.9 5.8 Lower BEST 26
1946 1 -2.5 2.5 5.0 Lower BEST 27
1946 3 -1.5 1.5 3.0 Lower BEST 28
1947 1 -3.0 3.0 6.0 Lower BEST 29
1948 3 -2.8 2.8 5.6 Lower BEST 30
So a total of 30 monthly values were diffrent 29 of the in the period from 1905 to 1948, diffrences being as low as 0.2 to 10.8 degrees and all either occurring in either the 1. or 3.
month of the year and the bias beeing unidirectional and so that BEST is always has the negative value. Rather strange is it not ?.
The BEST data table has columns for failde quality checks (QC fail) and continuity brakes
were I belive a value diffrent from zero indicates that somthing is awray or somthingh and
in all those instances above it had those columns had zero values for all the lines above, which I to as a sign that the the BEST codes had accepted their raw input as god and valid.
Followed by those two QC and Continuity breaks columns in the BEST data table there are columns for adjusted temperatures and the corresonding calculated anomaly , and also something they call regional expectations also giving both a themperature value and a correponding anomaly.
I did not go much farther with this, just tok the sums of the divergence of the temperature the columns, an verified that the BEST routines carry most of it through to their final anomaly result.
The sum of the above diffrences for those 30 months is 89 and when that is diveded and rounded shows a that BEST’s monthly is 3°C cooler than IMO. It of course does not have any big effect on the average for the total of 1428 months coverd by the whole period. But if we discard prior to 1905 and look only at that period we have 528 months against 29 each of the 29 off also by ~3°C and adding the fact that the annual average is for the place in that periodis 3.3 degrees it gives us around 0.16°C lowering of the average by using the BEST data instead of IMO for the first half of the 20th century. That’s surely large enough to affect the localal trend at least.
Now I also did a similar quick check for latter half of the 20th century data an this time the BEST was running hotter than IMO and the sum of the diffrences was twice as almost twice as great around 160 °C higher for and in all but 4 instances BEST was the carrying the higer valued one , but not so much because of a oppsite signs on the data. starting in 1949 BEST’s monthly data is given with 3 decimals precision , while IMO only gives one decimal and nearly every month has some difference , but each time it’s a small one, first decimal is mostly zeros and more uncommonly a 1 , never higher, so I suspect that BEST is simply using daily high-low temp data which is available for calculating daily averages and then monhtlies from that while IMO uses perhaps 3 or 6 evenly spaced in time values or perhaps the swedish type 6-12-6 + plus high and low for their average calculations , I not privy to whats in use but I think the high-low mean has not been the their base method for calculating the daily mean since long way back.
But anyway this is but a single station and as such does not affect the whole picture much, it but it might also be said that if as we have here for the first half of the the 20th century, where possibly one out of every 19 monthly values is 3°C out of whack then in the worst case we might say that in the BEST database there could be (using Zekes quted 40000+ numer of stations , and say 250 year ( 1750-2000) continous history ) somthing like 120 million seriously crappy monthly means in their final values. ( I do not really think it is this serious, I am just pointing out that the possibility exist, and could not resist the poke {:-)} ).
And also , I both know something of the history of this particular station f.x that is was started by the local merchant who it seem, had become intrested in meterology , and when it later became an offical weather station in the danish weather service register , they contracted the the job of keeping and maintaining it to the merchant and incorporated his older records into their register, and I have been told that he was a very meticolous recorder, and among other things he had not one but three then state of art thermometers ( one with a Reamur scale, another with Farenheit scale and the third one with a Celsius ) , and his records had a colum for each one. And I have the impression that the the old book have been thouroughly verified by some of the meterologists at the IMO who are intrested in the local weather history, so I while I agree with Mosher that raw data is in many ways a crap , I suspect that in this particular case at least the BEST raw data is maybe more of a crap than the offering from IMO, and perhaps the quality check procedures of his pet prjoect needs some revision and extensions , but that of course is a something of gut feeling , rather than a 100% verified fact. And I think when you have crappy input data ,that it is only of limitied value to use mechanical methods only to sort the out.
Stephan Rasey says: ” Now, you take the scalpel and cut it somewhere in half with an unspecified breakpoint shift. You want to tell me that you can still estimate an 80 year cycle from two uncorrelated 20 year signals? You’d be lucky to get back to the 40 year signal you left on the cutting room floor.”
I have the same reservations about the BEST method.
I don’t think it is a coincidence that it shows less variability that other datasets. For example in their global dataset 1998 El Nino it barely noticeable.
I wanted to investigate the BEST method for this effect when it first came out but they only provided massive files that could not even be loaded on a PC. Despite the claims of total openness, this effectively meant it was not available to be checked.
The central panel of the posted (pdf version) is quite informative.
NCDC map looks blurred, PRISM looks sharp and BEST looks flat.
It’s interesting that with high temporal and spacial resolution they manage to remove so much detail.
Where does UHI fit into all of this? The UHI effect is gradual and does not create a breakpoint.
Is there a danger that the relatively few, genuinely rural sites are actually adjusted up to urban trends?
The fact is global temperatures have been flat for the past 17 years. There is no warminge even using all the fiddled data from hadcrut giss etc.. The BEST project was just an exercize in futility trying to flog a dead horse just like all the adjusments done by GISS etc just look at Steven Goddard endless graphs of manipulated USA data. LOL RSS and Uha show no warming at all for tropics and SH ie no global warming since 1979 either, LOL
Ryan Welch: I really don’t know that much about west Texas and the plains there. I’m not arguing that the climate hasn’t changed. However, it seems to me that the time period that you are referring to also coincides with the removal of large herds of large grazing animals. Is it possible that fencing, farming, and reduced numbers of hoofed animals caused a lot of what the farmers and their ancestors saw?
@ur momisugly Bjorn A picture = 1,000 words.
“The fact is global temperatures have been flat for the past 17 years. There is no warminge even using all the fiddled data from hadcrut giss etc.. The BEST project was just an exercize in futility trying to flog a dead horse just like all the adjusments done by GISS etc just look at Steven Goddard endless graphs of manipulated USA data. LOL RSS and Uha show no warming at all for tropics and SH ie no global warming since 1979 either, LOL”
1. yes the temperatures have been flatish for 17 years. people on relying on the accuracy of our method to make the ‘flat’ claim. get it? the whole claim that there is a pause DEPENDS on us doing things correctly.
2. Goddard is wrong. He neglects to mention that
A) the data in his comparisons are two entirely different datasets
B) the results he compares use two differrent algorithms
C) hansens results can be replicated using entirely different data and different methods
3. We agree with UAH . thats the point of this poster. When you trust them you vindicate us
“Paul Homewood says:
December 19, 2013 at 3:17 am
Where does UHI fit into all of this? The UHI effect is gradual and does not create a breakpoint.
Is there a danger that the relatively few, genuinely rural sites are actually adjusted up to urban trends?
######################################
1. You assume that UHI is gradual. That has never been established. The biggest ef
2. In previous work we show that UHI in the US was something on the order of .04C
per decade. That is not the point of this dataset. after minimizing the error in the weather
field this bias is taken toward zero
3. Since we agree with UAH and UAH has no UHI, you can draw a logical conclusion
Mosher, your 40,000 number doesn’t make sense if “about two-thirds” of 75000 are precipitation only. You are also asking me to believe that GHCN Monthly has significantly fewer stations than GHCN Daily temperature stations. I need to see a link on that.
##############################################
MORON.
For the world after deduplication there are roughly 40,000 stations.
1. GHCN Daily has about 75K stations, 2/3 of which are precipitation only
of the 50K stations with temperature data 20K are in the US. ly Not all can be used
because some have only a few months of data.
2. GHCN Monthly has only 7k stations.
http://cdiac.ornl.gov/epubs/ndp/ndp041/ndp041.html
Or you could just use the free software I’ve provided to download both daily and monthly data.
GHCN Monthly has fewer stations for historical reasons. That dataset will go away over time
and I suspect a new monthly datset will be built from daily.
##################################################
I would also very much like to see a distribution of the length of station records in the database and a distribution of the lenght of records after they have been through the scalpel.
Download the datasets and compute. get off your ass
“I wanted to investigate the BEST method for this effect when it first came out but they only provided massive files that could not even be loaded on a PC. Despite the claims of total openness, this effectively meant it was not available to be checked.”
Before I actually Joined the best team I wrote software to download and use it on a PC. takes up about 2GB of memory. maybe you dont know what you are doing
Ryan Scott Welch says:
Why did they start at 1850? Is it because it was the best year to start the record due to temperature measuring sites, quality of data, or because that was the end of the Little Ice Age?
###########################
These are CONUS sites. We took the record back as far as we could go given the method.
“A C Osborn says:
December 19, 2013 at 2:04 am
Sorry, I do not believe this data. The Historical REAL data used up to the 1990s definitely showed the 1930/40s as being much warmer than they are shown in the current BEST (and GISS) data.
I can see no justification for cooling history, it is an insult to the people who took those readings.
##############################
1. The notion that the 30-40s were warmer comes from early work done by hansen
2. That work is known to be wrong.
A. Hansen Stitched together stations that were NOT the same station. This is called
the ‘reference station method’ Stitching together multiple stations simply because they
are with 20 km of each other is a mistake.
B. The data use was corrupted by changes in methods of observations.
Now of course we can go back to what the actual records were. We avoid using the contaminated data that hansen used. We avoid mashing together stations simply because they are close to each other. We avoiding using monthly data and go back to daily daily.
Hi Bjorn,
Here is the detailed Berkeley analysis of that station in Iceland: http://berkeleyearth.lbl.gov/stations/155466
You can see that two breakpoints are detected; one empirically detected one ~1860 and a second associated with a documented station move around 1940. The net effect of these two breakpoint adjustments is to slightly decrease the trend relative to the raw data.
Stephen Rasey,
The answer is (A), we use 40,747 independent temperature sensors (though not all have long periods of coverage). You can find all individual records used on our website, and download the raw data: http://berkeleyearth.org/source-files
Paul Homewood,
UHI is important, and as far as we can tell seems to show up as breakpoints that are mostly removed by homogenization. In our recent JGR paper we looked in depth if it were possible for urban stations to be “spreading” warmth to rural stations during homogenization. We did a test where we only used rural stations (and ignored urban ones) when detecting and correcting for breakpoints. The results were by-and-large identical, indicating that this is not occurring in practice (though, interestingly enough, using only urban stations to homogenize all stations did introduce some spurious warming). You can find our paper here: ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/papers/hausfather-etal2013.pdf
Greg,
Berkeley has a lot of high-resolution spatial features for absolute temperatures and single month anomalies. Our trend fields are much smoother, but we argue that they reflect underlying climate changes, which are more regional than local. The idea that two locations within 50 kilometers of eachother might have warmed 1.5 degrees and cooled 1.5 degrees, respectively, over a 30 years period is rather hard to believe to be physically plausible. Its much more likely that biases due to station moves, TOBs changes, and instrument changes over the last 30 years added a large amount of noise relative to the (smaller) trend signal.
Zeke/Mosher I think your views of UHI are just as stunted as David Parker.
From:
Urban Heat Island Modeling in Conjunction with Satellite-Derived
Surface/Soil Parameters
JAN HAFNER AND STANLEY Q. KIDDER
1998 Journal of Applied Meteorology
They write:
“UHI is important, and as far as we can tell seems to show up as breakpoints that are mostly removed by homogenization. ”
UHI isn’t a step function as Zeke asserts, that’s a siting or equipment change issue.
UHI is a slow signal, separate from siting and equipment. It doesn’t show up in fixing breakpoints, the temporal resolution of that method is all wrong.
I still think your paper on homengenization is crap. My findings suggest homogenization is nothing more than a data blender that really doesn’t fix all of the problems it purports to.
Why try to salvage compromised data by mixing it with good data? In forensics, that would get thrown out of court. In climate science, entire careers are made trying to make use of compromised data. Gavin once said we could monitor 50 stations and get a useful global temperature. Why not simply focus on finding the BEST stations (Yes, pun intended) and using those instead of playing mud pie maker with mixed provenance data?
BEST really isn’t doing anything truly new here, its just another take on the methods and mixed data already in use.
Mario Lento,
Reanalysis products use weather models (not climate ones) fed by observations to estimate changes in temperature, precipitation, etc. at both the surface and various levels of the atmosphere. You can find more about them here: https://climatedataguide.ucar.edu/climate-data/atmospheric-reanalysis-overview-comparison-tables
I would like to seize this opportunity and pose some questions to Steven Mosher:
Here is the record #1 and here is the record #2 from the BE surfacestation pool.
I would like to ask Steven Mosher:
Why the take on the records before 1950 is different for the two records?
Why there are only 6 break point identified in the No. 1. record while there are 9 for the record No. 2?
What is the justification for the 1 (record No. 1) and 3 (record No. 2) breakpoints identification early in the record before 1810.
What stations exactly were used to define the regional expectation for this period (1775-1810) and to identify the purported bias in the two records for the period?
How the method used justifies the two different results?
Is it the “raw data” which is (to use your own word) “crap” here or is it the method?
Why the BE uses the double-merged record from the Prague-Klementinum+Ruzyne+Libus (btw. the stations differ in altitude almost 200 meters!) instead of the original Klementinum record kept since 1775 uninterrupted until very present?
To explain: because I’ve researched this before* and I have found discrepancies already with the GISS station pool, I know positively that IN REALITY the temperature time series in both the two particular records No 1 and No 2 before 1950 SHOULD BE IDENTICAL.
That’s because in fact there doesn’t exist any record neither for the Prague-Ruzyne nor for the Prague-Libus stations before 1950 and in fact the part of the record No. 2 before 1950 is and must be the record from the Prague-Klementinum station – one of the rarest and most unique temperature time series in the history of the instrumental temperature measurements which goes back to 1700s (kept since 1771, uninterrupted since 1775) and is kept until today. (BTW, the BE erased warm decade 1790-1799 is established not only by the Klementinum temperature record.)
——————————–
*I note just BTW that my 200+ years UHI bias estimation for Klementinum record linked above is extremely conservative and in magnitude very simmilar to that estimated in the related literature just for the period 1922-95.
Because No 2 record link somehow didn’t go through here is the link again.
Paul Homewood says:
December 19, 2013 at 3:17 am
Where does UHI fit into all of this? The UHI effect is gradual and does not create a breakpoint.
Is there a danger that the relatively few, genuinely rural sites are actually adjusted up to urban trends?
######################################
1. You assume that UHI is gradual. That has never been established. The biggest ef
2. In previous work we show that UHI in the US was something on the order of .04C
per decade. That is not the point of this dataset. after minimizing the error in the weather
field this bias is taken toward zero
3. Since we agree with UAH and UAH has no UHI, you can draw a logical conclusion
Steve
As UAH starts in 1979, it does not automatically follow that we can ignore UHI prior to 1979.
You make an interesting comment about the 0.04C/decade. According to NCDC, the US warming trend since 1895 is 0.07C/decade, so you are suggesting over half is down to UHI?
Paul
Here is the Hafner and Kidder paper I cited in my previous comment:
http://amsu.cira.colostate.edu/Kidder/UHI.pdf
Zeke
Here is the detailed Berkeley analysis of that station in Iceland: http://berkeleyearth.lbl.gov/stations/155466
You can see that two breakpoints are detected; one empirically detected one ~1860 and a second associated with a documented station move around 1940. The net effect of these two breakpoint adjustments is to slightly decrease the trend relative to the raw data
That’s interesting because the GHCN set adds about half a degree of warming on from 1965 for Stykkisholmur
ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v3/products/stnplots/6/62004013000.gif
They seem to have been confused by the well documented sharp drop in Icelandic temperatures then, a time known in Iceland as the “sea ice years”. The GHCN algorithm seems to think this was a measurement error.
Anthony Watts says:
December 19, 2013 at 9:15 am
“… UHI isn’t a step function as Zeke asserts, that’s a siting or equipment change issue.
UHI is a slow signal, separate from siting and equipment. It doesn’t show up in fixing breakpoints, the temporal resolution of that method is all wrong.
I still think your paper on homengenization is crap. My findings suggest homogenization is nothing more than a data blender that really doesn’t fix all of the problems it purports to.”
Why try to salvage compromised data by mixing it with good data? In forensics, that would get thrown out of court. In climate science, entire careers are made trying to make use of compromised data. Gavin once said we could monitor 50 stations and get a useful global temperature. Why not simply focus on finding the BEST stations (Yes, pun intended) and using those instead of playing mud pie maker with mixed provenance data?
BEST really isn’t doing anything truly new here, its just another take on the methods and mixed data already in use.”
++++++++++++++++
Thank you for posting this. I’d been trying to get Mosher to fess up, but he will not collaborate on seeking truth. The test that should give any rational person pause is that.
1) Urban areas show more warming than average
2) Rural areas show less warming than average
Steven Mosher says:
December 19, 2013 at 8:32 am (Edit)
“A C Osborn says:
December 19, 2013 at 2:04 am
Sorry, I do not believe this data. The Historical REAL data used up to the 1990s definitely showed the 1930/40s as being much warmer than they are shown in the current BEST (and GISS) data.
I can see no justification for cooling history, it is an insult to the people who took those readings.
##############################
1. The notion that the 30-40s were warmer comes from early work done by hansen
2. That work is known to be wrong.
A. Hansen Stitched together stations that were NOT the same station. This is called
the ‘reference station method’ Stitching together multiple stations simply because they
are with 20 km of each other is a mistake.
Steve
Is there any reason, or logic, why all these “errors of Hansen” all seem to have gone the same way, and significantly overestimated past temperatures? Surely, statistically, they would be likely to cancel out?