ERRATA: I made a mistake regarding the 2% figure, I misheard what was being presented during my visit with the BEST team at Berkeley. As many of you may know I’m about 80% hearing impaired and the presentation made to me was entirely verbal with some printed graphs. Based on the confidentiality I agreed to, I did not get to come back with any of those graphs, notes, or data so I had to rely on what I heard. I simply misheard and thought the 2% were the Japan station analysis graphs that they showed me.
I was in touch with Dr. Richard Muller on 3/28/2011 who graciously pointed out my misinterpretation. I regret the error, and thus issue this correction about the 2% figure being truly a random sample, and not just stations in the Japan test presentation shown to me.
I am told of another correction, and that is that Dr. Caldeira was shown a paper they are working on related paper on oceans, and that contained the preliminary 2% graph from the surface analysis, but was not the full surface analysis paper, which hasn’t yet been written.
According to Dr. Muller, that 2% test run does not contain all the bias corrections they plan to apply for station moves, discontinuous records, UHI and other station effects. I look forward to seeing the data when those are applied to the full dataset.
This episode where Mr. Romm gets an email from Dr. Caldeira and creates a “finding” illustrates the danger in rushing to judgment on snippets of preliminary results, and as BEST says: “The Berkeley Earth team feels very strongly that no conclusions can yet be drawn from this preliminary analysis.”
I still believe that BEST represents a very good effort, and that all parties on both sides of the debate should look at it carefully when it is finally released, and avail themselves to the data and code that is promised to allow for replication.
– Anthony Watts
Last week Willis told you about how Joe Romm at Climate Progress botched a blog post so bad, Joe had not only to fix his own post by removing false claims about population trends, so did the paper’s authors. Then, rather than simply admit a mistake and move on, he spun it into some sort of twirling victory dance, bizarrely claiming that because Willis put up a chart of CO2 rates, “he” got us to admit that CO2 rates were increasing because Willis chose it as a reference. Heh, well if that floats your boat, you go Joe. WUWT has quite a history in discussing CO2 with graphs, rates, and guest essays, no news there.
This week, it’s the old pea and thimble trick combined with desperation and some silly claim of “exclusive”, like some cheap MSM news labeling graphic where they’ve caught some sex poodle on tape. After earlier writing a piece condemning the Berkeley Earth Surface Temperature Project (BEST) he’s now put up his interpretation of an email from scientist Ken Caldiera who said:
I have seen a copy of the Berkeley group’s draft paper, which of course would be expected to be revised before submission.
Their preliminary results sit right within the results of NOAA, NASA, and HadCRU, confirming that prior analyses were correct in every way that matters. Their results confirm the reality of global warming and support in all essential respects the historical temperature analyses of the NOAA, NASA, and HadCRU.
Their analysis supports the view that there is no fire behind the smokescreen put up by climate science deniers.
The only problem is this: there’s no “draft paper” yet, there’s nothing that is “submission ready”, not even close. It hasn’t been written. In fact, BEST hasn’t even done a full global analysis yet. How do I know this? It’s simple; I’ve visited the team and asked them directly, something Romm has not done.
Caldeira was simply looking at the same set of data (some preliminary charts and graphs that Richard Muller carries around with him in a file folder), that BEST has been showing to several people, including me. The only difference is that people like myself, Steven Mosher, and his friend Zeke Hausfather who visited BEST with him, haven’t run off the rails to make early and unsubstantiated claims about it “confirming” anything yet. And now, Romm’s adding to his original blogpost, is backpedaling, while at the same time picking a fight with Steven Mosher for notifying him on the issue in comments. It is sad, comical, and oh-so-typical of the sort of thing we’ve come to expect at Climate Progress. Romm simply got excited and jumped the shark. He’s not doing himself any favors with this sort of thing.
Here’s the Initial Findings statement from BEST, written by lead scientist Robert Rhode (of globalwarmingart.com) which pretty much mirrors what Caldeira is saying:
The Berkeley Earth Surface Temperature project has not yet done the analysis of the full data set with the corrections to produce a global surface temperature trend. We are first analyzing a small subset of data (2%) to check our programs and statistical methods and make sure that they are functioning effectively. We are correcting our programs and methods while still “blind” to the results so that there is less chance of inadvertently introducing a bias.
A preliminary analysis of 2% of the Berkeley Earth dataset shows a global temperature trend that goes up and down with global cycles, and does so broadly in sync with the temperature records from other groups such as NOAA, NASA, and Hadley CRU. However, the preliminary analysis includes only a very small subset (2%) of randomly chosen data, and does not include any method for correcting for biases such as the urban heat island effect, the time of observation bias, etc.
The Berkeley Earth team feels very strongly that no conclusions can yet be drawn from this preliminary analysis.
That 2% subset they refer to is some weather stations in Japan. They chose Japan because it made for a compact insular test case for the code, combining rural, urban, and airport stations under one organization’s output to keep it simple. Like Ken Caldeira, I’ve seen that preliminary 2% output. I’ve also seen a lot of other things, some things Caldieira hasn’t seen that the BEST team has shared with me. So has Zeke and Mosher, but neither they nor I are screaming “exclusive” and jumping to conclusions like Romm is doing over Caldeira’s general statement on that 2% sample run to test the code.
What’s even funnier is that whenever we mention USHCN trends for USA stations, AGW proponents are quick to point out that the USA has only about 6% of the land surface area of the Earth (USA: 9,629,091 km2, Earth: 148,940,000 km2 source), but they are now willing to go with the weather station data from 377,930 square kilometers of Japan’s land area which is 0.25% of the Earth’s surface area, as enough for “confirmation” of a global trend.
In response to this latest yapping from Romm, BEST has also now updated their FAQs page here, and says this:
NEW – What do your results show?
The Berkeley Earth Surface Temperature project has not yet done the analysis of the full data set with the corrections to produce a global surface temperature trend. We are first analyzing a small subset of data (2%) to check our programs and statistical methods and make sure that they are functioning effectively. We are correcting our programs and methods while still “blind” to the results so that there is less chance of inadvertently introducing a bias.
A preliminary analysis of 2% of the Berkeley Earth dataset shows a global temperature trend that goes up and down with global cycles, and does so broadly in sync with the temperature records from other groups such as NOAA, NASA, and Hadley CRU. However, the preliminary analysis includes only a very small subset (2%) of randomly chosen data, and does not include any method for correcting for biases such as the urban heat island effect, the time of observation bias, etc. The Berkeley Earth team feels very strongly that no conclusions can yet be drawn from this preliminary analysis.
Compare Romm’s “exclusive” to Zeke’s writeup over at Lucia’s last week which preceded Romm’s. Zeke’s essay has firsthand accounts, is a lot easier to read, and doesn’t need gratuitous exclamation points. For those who don’t know him, Zeke Hausfather is very much in the warming camp, but he’s also a reasonable person. Zeke wrote about a technique that I agreed to keep in confidence until they had a paper accepted for publication or chose to announce it on their own, but it may have been just a slip or communications misunderstanding:
Their major innovation, in addition to those that overlap the work of other bloggers, is to treat inhomogenities as the start of separate records. The least squares method of record combination has the major benefit of allowing relatively short records to be combined together without introducing biases. This means that instead of trying to artificially correct inhomogenities detected by comparing individual stations to their neighbors, they can simply treat these as break points, where subsequent measurements from the same site are treated as a separate record and are optimally fit to the larger series using the LSM approach.
The issue hasn’t been the slight warming over the past century, we’ve always conceded that there is some. The issue has always been magnitude, uncertainties, and cause. With the BEST project, we’ll get closer to the ground truth of magnitude and uncertainties, but it will say nothing about the cause, except perhaps to help define the contributions of UHI and station siting.
I’ll repeat what I said earlier about BEST:
And, I’m prepared to accept whatever result they produce, even if it proves my premise wrong. I’m taking this bold step [providing my surfacestations data to them] because the method has promise. So let’s not pay attention to the little yippers who want to tear it down before they even see the results. I haven’t seen the global result, nobody has, not even the home team…
…
My gut feeling? The possibility that we may get the elusive “grand unified temperature” for the planet is higher than ever before. Let’s give it a chance.
More science, less barking.
I have seen a copy of the Berkeley group’s draft paper, which of course would be expected to be revised before submission.
Their preliminary results sit right within the results of NOAA, NASA, and HadCRU, confirming that prior analyses were correct in every way that matters. Their results confirm the reality of global warming and support in all essential respects the historical temperature analyses of the NOAA, NASA, and HadCRU.
Their analysis supports the view that there is no fire behind the smokescreen put up by climate science deniers.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Anthony,
Their technique of treating inhomogenities as break points isn’t really a secret (and Muller never to my knowledge requested we keep it as such), as it features rather prominently in their methodology document: http://www.berkeleyearth.org/Resources/Berkeley_Earth_Summary.pdf
“In addition to the uncertainty adjustments detailed above, we might also search for statistically significant discontinuities in the record that could indicate undocumented station moves and similar problems. The impact of discontinuities can be resolved by partitioning such data into two time series with independent baseline estimators. Hence the corrections for such discontinuities can become part of the simultaneous solution to the larger averaging problem rather than a series of local adjustments.”
Regardless, their results should be out soon for us all to look at. Lucia is considering setting up a betting pool so folks can wager how close/far it will be from NCDC’s record 😛
REPLY: He did ask me to keep it confidential until publication, perhaps he just forgot when you visited. Probably related to my blog reach, as Romm has demonstrated too much noise can be distracting. I wasn’t aware it was in the PDF, thanks for pointing that out. – Anthony
“The least squares method of record combination has the major benefit of allowing relatively short records to be combined together without introducing biases.”
Be advised that this statement is in itself, woefully incorrect. As one who taught graduate level statistics in a number of Univerisities, including Oxford, “least squares” would only be somewhat appropriate depending on the purity of the sample, i.e., the sample must be scrubbed clean and clear of any and all outliers, and more importantly be truly random, the latter point being critical.
Temperature measurements are hardly random, especially when considering the failed techniques that everyone, including Anthony, has documented. Moreover, if the majority of measurements come from only a few places on earth, and are not randomly placed around the entire globe, the entirety of “least squares” methodology blows up in your face, and it cannot be corrected for bias, since the sample itself is replete with what can only be described as statistical confabulations!
Why is it the warming climate has never proven even adequate in understanding that their experimental design has remained completely misframed.
Honestly, these are simple straightforward rules and regs belonging to the arena of statistical design, understandings that are obviously not embraced by many in the warming camp. They appear incapable of designing a scientific experiment with the right tools and methods, which most of the rest of us have respected our entire professional and academic careers. What’s up with that?
Hence, from just a statistical perspective, the idea that “we will finally get some sound data” from the “warming crowd” is pure fiction. Garbage in — Garbage out!
“Why is it the warming climate has never proven even adequate in understanding that their experimental design has remained completely misframed.”
Correction: the above should read:
Why is it the warming crowd has never proven even adequate in understanding that their experimental design has remained completely misframed?
Baghdad Romm
I wonder what percentage of the weather stations from which those data are derived still even exist in Japan?
“Who is Joe Romm and why is he important?” Most here know the history, but for those who don’t it’s quite interesting, because this story more than anything else illustrates how this is overwhelmingly a political, rather than a scientific battle.
(btw, everything I’ll put here can easily be confirmed with a few quick checks)
Joe Romm was a mid-level Energy Department official during the Clinton Administration. From the start he established himself as one of Hansen’s allies and became one of that administrations primary proponents of Global Warming alarmism. It’s possible that he was the man who actually convinced Al Gore to sign on to the project, but that’s unclear. They certainly knew each other.
When that administration left office and Romm lost his political job, he set himself to profiting off the coming Global Warming legislation by setting up a number of shell companies which would profit greatly by leveraging his political connections and administrative expertise for companies that fell under the new Cap and Trade regulations.
Took longer to pass than he thought, so he set up a blog to advocate for the positions that he was set to personally and politically profit from. Then he quickly made an arrangement with John Podesta, ex-chief of staff for Bill Clinton and a personal friend from old Clinton administration days. John Podesta is now President for the Center for American Progress, which provides all the funding for Climate Progress, Think Progress, and other operations. It is fair to say that it is a privately funded arm of the Democratic Party, since many of its most influential associates are members of the current administration or were members of the Clinton administration.
The Center for American Progress is a 501(c)(3) organization which does not have to reveal its donors or the amounts they give, but from it’s wiki page:
“The institute receives approximately $25 million per year in funding from a variety of sources, including individuals, foundations, and corporations, but it declines to release any information on the sources of its funding. No funders are listed on its website or in its Annual Report. From 2003 to 2007, the center received about $15 million in grants from 58 foundations. Major individual donors include George Soros, Peter Lewis, Steve Bing, and Herb and Marion Sandler. The Center receives undisclosed sums from corporate donors.”
http://en.wikipedia.org/wiki/Center_for_American_Progress
So, Joe is not just some average blogger. He is the salaried representative of the political forces seeking to impose Global Warming legislation on the world for their own political and financial reasons.
And that’s why, at it’s core, this issue has nothing at all to do with Science. This is about political power, plain and simple, and it always has been.
p.s. – the funny part is that if Joe is the best they can do, they are all in big, big trouble!
Dave Springer says:
March 22, 2011 at 6:15 am
“REPLY:WUWT will have coverage, no matter what the result. We are watching science being done – Anthony”
Perhaps in the same manner that watching a superbowl commercial is seeing football being done…
=============================================
Dave, at first I didn’t agree with you..
..then thought about it, and now I do.
In the sense that it’s still weathermen/climatologists trying to predict the weather.
Now someone is going to say that climate is not weather…..
Then explain why climatologists describe climate as weather…
More snow, less snow, more rain, less rain, more hurricanes, less hurricanes, more droughts, less droughts, more cold, less cold, and on and on….
…warmcold, wetdry, droughtflood………………
Many major breakthroughs in science were initially sparked by the observations of just plain folks. Autism being a prime example. And many major downfalls of science were initially sparked by the observations of scientists. Autism being a prime example. Humble pie all around Mr. Springer.
Why are we so blind to the color we paint on others?
I think what we are seeing is Romm-dom chaos theory applied to social advocacy science. How does the question go, “How much wood could a woodchuck chuck…”
In the comments, discredited climate science disinformer Steven Mosher…
ROFLMAO
Man, Joe Romm is such a comic.
Romm is currently going nuts / bananas over the rise in food prices. I see no mention of biofuels on his post. Maybe I missed it but I pointed it out anyway.
http://climateprogress.org/2011/03/22/food-prices-stamps-subsidies/
It appears BEST will NOT be correcting for changes in bright sunshine hours.
Thats makes it a failure already.
@Barry Woods
“I’m thinking about writing an article about an influential blog that writes critically about WUWT (and others) and in my opinion misrepresents WUWT, yet the writer remains anonymous and the articles are tweeted to the worlds media and followed by those with influence.
How to judge an article, is the author, impartial, activist, axe to grind, scientifically litterate? who knows if the author is anonymous.”
Funny, but there’s a whole paradoxical article on WUWT that points out the fact of what’s important and what is not, that points out that if someone is pointing towards the moon it’s probably the moon you should look at and not the ugly finger that’s happen to be rammed up in the man in the moon’s nose.
Since when has news media and “those with influence” ever concerned themselves with following the words of the wise men only? And besides, what some people take to be wise, named, men are just not very mentally stable that points on everything and calls it catastrophic proportions. And their are mentally health, however much of a simpletons, that manage to point to the moon even, without wiggling their fingers. Now which finger? All the fingers of the well educated, but phd crazy one, or the healthy, but ordinary simpleton’s single point?
And if you’re more interested about the person than what the person has to say, then you make the same mistake as the good “finger-to-the-moon” author did.
I wonder if the open source movement had happened had everyone been focused on people’s names?
(Do not stare at the finger for to long, especially if it is moving in a tantalizing way and looks like a paw, you might just wake up a bagel.) :p
Travelled over to Joe’s blog and read the comments section to the “We made them admit CO2 is rising” story. Twirling victory dance indeed. I can’t recall at anytime in all my reading of WUWT, has there ever been any “denial” that CO2 is rising. Of course it is, the questionsat WUWT are about why and what impact is it having.
Such a depressing bunch of academics, so fearful the sky is falling. Joe definitely has a crowd his rants appeal to. Personally, I don’t know how they all get out of bed in the morning.
Off topic, but instapundit has a link to a Monbiot Guardian posting in which, on the basis of the Fukushima situation, he has accepted the relative safety of nuclear energy and allows as how it might be the better solution to the ending dependence on fossil fuels.
Does anyone know if BEST or any other researchers are comparing the NASA satellite UHI measurements released last year with the UHI adjustments in the datasets?
wws says:
March 22, 2011 at 7:43 am
“Who is Joe Romm and why is he important?”
Thanks wws for that summary. Fascinating that a paid shill (Romm) continually refers to skeptics as being paid shills. Seems to be a lefty ‘thing.’
bigcitylib says:
March 22, 2011 at 3:18 am
Muller has already confirmed the draft conclusions. Are you saying that the project head is misinterpreting his own data. Come to think of it, you must think project funder Caldiera is misinterpreting the data too. Or do you have any idea what you are trying to say?
############
BCL, you’re missing the point. I’m well aware that a small sample has been run.
I’m also well aware that writing has BEGUN. But what was news to me was that
draft was circulating. Circulating a draft would mean that you had some conclusions,
that you had run the full dataset. Now that would have been news to me. Without running the full dataset you dont have conclusions.
I think BEST will come up with roughly the same answers. I think that is a GOOD THING for skeptics. Why? because then they will FOCUS on the issue of sensitivity.
observa quotes:
March 22, 2011 at 6:35 am
‘BREAKING UPDATE: The head of the Berkeley team, Richard Muller, confirmed at a public talk on Saturday that they have started writing a draft report and based on their preliminary analysis, “We are seeing substantial global warming” and “None of the effects raised by the [skeptics] is going to have anything more than a marginal effect on the amount of global warming.”’
The question is whether Muller and team are going to be just more Warmista. If they are going to produce new temperature charts and nothing else then they are just more Warmista. If they have no scientific explanation of how it is that CO2 is causing a dangerous rise in temperature, something beyond arrhenius’ work, then they are just more Warmista. If so, then we must focus the blogosphere on this fact. If they are just more Warmista, then, as they say, the bigger they come the harder they fall.
Romm…
What a complete and utter prat!
Mere words are entirely inadequate to describe this asinine and banal behaviour.
Can’t wait to get my grubby little paws on the entire BEST data set and see what is what.
While part of Professor Muller’s video takes the Team (Mann, Briffa, Jones, Wahl, et. al.) to task for stuff you can’t do in science, the longer version makes it clear that the Professor is biased towards the Catastrophic AGW hypothesis claims. Unfortunately the Professor doesn’t explain the reasoning behind his claims or his support for the CAGW claims.
The extract from the longer talk with Professor Muller taking the Team to task for what you can’t do in science and rebuking them by asserting that he now has a list of people whose papers he won’t read anymore. Ouch, cast them out of the science club. Three cheers for professor Muller for standing up for scientific integrity.
The full 52 minute talk where professor Muller makes clear his views and conclusions but doesn’t provide much if any supporting evidence for those views and conclusions (except for the portion where he takes the Team to task, that is explained very well, although he seems to let the Team off light).
I would really like to see professor Muller substantiate every Catastrophic AGW Hypothesis claim that he is supporting with direct hard evidence that can be openly verified by others along every step of the way from the assessment that the planet is warming (the BEST being a start) on through to every single conclusion that that is somehow “bad”. It’s not enough to simply assert that it’s the Green House Gases and we’re doomed, it needs to be substantiated with verifiable evidence – preferably experiments and direct observational data if possible – every step of the way on each and every doomsday claim. That would be something to see and review in depth.
“If your experiment needs statistics, you ought to have done a better experiment.”
~ Lord Ernest Rutherford
“You can’t fix by [statistical] analysis what you bungled by design.”
~ Light, Singer and Willett, page v
_____
1. The unknown systematic errors associated with many land stations probably are at least an order of magnitude larger than the effect [temperature change] that is the output of the analysis.
2. Does this analysis include the oceans which cover 71% of the planet? If so, then well- sampled reliable data has been only available since the completion of the ARGO system in 2003. If not, then why bother?
3. It is the heat content of the atmosphere [energy in Joules] rather than the temperature [in Celsius] that matters: http://goo.gl/Jz0N9
_____
“Something not worth doing is not worth doing well”
~ Harry Johnson
this “project” is nothing more than a revenue and prestige program …
they know that the station data is useless on a global scale due to sighting, time, possible UHI and observation differences … they also know that only a tiny percentage of the planets surface has more than 40 years of records …
waste of time and money that would be better spent on establishing a global temperature sighting initiative with consistant equipment and timing and locations …
the raw data is a mess and until that changes it is GIGO …
I think ‘Romm’ needs to be transformed from a mere name to a verb, noun or adjective which connotes a variation on spinning with an emphasis on deceptive tricks.
Romm should become as firmly entrenched in the language as Ponzi.