Doug Keenan, who readers may remember doggedly pursued and won some tree ring data that Queens University held back, was asked to comment of the BEST papers by the Economist. He posted up the full correspondence, including his critiques. There’s some interesting things in there. Since Dr. Muller and BEST want full transparency, in that interest, I’m making this available here. Start from the bottom up to maintain the timeline. h/t to Bishop Hill
He writes:
The Economist asked me to comment on four research papers from the Berkeley Earth Surface Temperature (BEST) project. The four papers, which have not been published, are as follows.
- Decadal variations in the global atmospheric land temperatures
- Influence of urban heating on the global temperature land average using rural sites identified from MODIS classifications
- Berkeley Earth temperature averaging process
- Earth atmospheric land surface temperature and station quality
Below is some of the correspondence that we had. (Note: my comments were written under time pressure, and are unpolished.)
From: D.J. Keenan
To: Richard Muller [BEST Scientific Director]; Charlotte Wickham [BEST Statistical Scientist]
Cc: James Astill; Elizabeth Muller
Sent: 17 October 2011, 17:16
Subject: BEST papers
Attach: Roe_FeedbacksRev_08.pdf; Cowpertwait & Metcalfe, 2009, sect 2-6-3.pdf; EmailtoDKeenan12Aug2011.pdf
Charlotte and Richard,
James Astill, Energy & Environment Editor of The Economist, asked Liz Muller if it would be okay to show me your BEST papers, and Liz agreed. Thus far, I have looked at two of the papers.
- Decadal Variations in the Global Atmospheric Land Temperatures
- Influence of Urban Heating on the Global Temperature Land Average Using Rural Sites Identified from MODIS Classifications
Following are some comments on those.
In the first paper, various series are compared and analyzed. The series, however, have sometimes been smoothed via a moving average. Smoothed time series cannot be used in most statistical analyses. For some comments on this, which require only a little statistical background, see these blog posts by Matt Briggs (who is a statistician):
Do not smooth times series, you hockey puck!
Do NOT smooth time series before computing forecast skill
Here is a quote from those (formatting in original).
Unless the data is measured with error, you never, ever, for no reason, under no threat, SMOOTH the series! And if for some bizarre reason you do smooth it, you absolutely on pain of death do NOT use the smoothed series as input for other analyses! If the data is measured with error, you might attempt to model it (which means smooth it) in an attempt to estimate the measurement error, but even in these rare cases you have to have an outside (the learned word is “exogenous”) estimate of that error, that is, one not based on your current data.
If, in a moment of insanity, you do smooth time series data and you do use it as input to other analyses, you dramatically increase the probability of fooling yourself! This is because smoothing induces spurious signals—signals that look real to other analytical methods.
This problem seems to invalidate much of the statistical analysis in your paper.
There is another, larger, problem with your papers. In statistical analyses, an inference is not drawn directly from data. Rather, a statistical model is fit to the data, and inferences are drawn from the model. We sometimes see statements such as “the data are significantly increasing”, but this is loose phrasing. Strictly, data cannot be significantly increasing, only the trend in a statistical model can be.
A statistical model should be plausible on both statistical and scientific grounds. Statistical grounds typically involve comparing the model with other plausible models or comparing the observed values with the corresponding values that are predicted from the model. Discussion of scientific grounds is largely omitted from texts in statistics (because the texts are instructing in statistics), but it is nonetheless crucial that a model be scientifically plausible. If statistical and scientific grounds for a model are not given in an analysis and are not clear from the context, then inferences drawn from the model should be regarded as unfounded.
The statistical model adopted in most analyses of climatic time series is a straight line (usually trending upward) with noise (i.e. residuals) that are AR(1). AR(1) is short for “first-order autoregressive”, which means, roughly, that this year (only) has a direct effect on next year; for example, if this year is extremely cold, then next year will have a tendency to be cooler than average.
That model—a straight line with AR(1) noise—is the model adopted by the IPCC (see AR4: §I.3.A). It is also the model that was adopted by the U.S. Climate Change Science Program (which reports to Congress) in its analysis of “Statistical Issues Regarding Trends”. Etc. An AR(1)-based model has additionally been adopted for several climatic time series other than global surface temperatures. For instance, it has been adopted for the Pacific Decadal Oscillation, studied in your work: see the review paper by Roe [2008], attached.
Although an AR(1)-based model has been widely adopted, it nonetheless has serious problems. The problems are actually so basic that they are discussed in some recent introductory (undergraduate) texts on time series—for example, in Time Series Analysis and Its Applications (third edition, 2011) by R.H. Shumway & D.S. Stoffer (see Example 2.5; set exercises 3.33 and 5.3 elaborate).
In Australia, the government commissioned the Garnaut Review to report on climate change. The Garnaut Review asked specialists in the analysis of time series to analyze the global temperature series. The report from those specialists considered and, like Shumway & Stoffer, effectively rejected the AR(1)-based statistical model. Statistical analysis shows that the model is too simplistic to cope with the complexity in the series of global temperatures.
Additionally, some leading climatologists have strongly argued on scientific grounds that the AR(1)-based model is unrealistic and too simplistic [Foster et al., GRL, 2008].
To summarize, most research on global warming relies on a statistical model that should not be used. This invalidates much of the analysis done on global warming. I published an op-ed piece in the Wall Street Journal to explain these issues, in plain English, this year.
The largest center for global-warming research in the UK is the Hadley Centre. The Hadley Centre employs a statistician, Doug McNeall. After my op-ed piece appeared, Doug McNeall and I had an e-mail discussion about it. A copy of one of his messages is attached. In the message, he states that the statistical model—a straight line with AR(1) noise—is “simply inadequate”. (He still believes that the world is warming, primarily due to computer simulations of the global climate system.)
Although the AR(1)-based model is known to be inadequate, no one knows what statistical model should be used. There have been various papers in the peer-reviewed literature that suggest possible resolutions, but so far no alternative model has found much acceptance.
When I heard about the Berkeley Earth Surface Temperature project, I got the impression that it was going to address the statistical issues. So I was extremely curious to see what statistical model would be adopted. I assumed that strong statistical expertise would be brought to the project, and I was trusting that, at a minimum, there would be a big improvement on the AR(1)-based model. Indeed, I said this in an interview with The Register last June.
BEST did not adopt the AR(1)-based model; nor, however, did it adopt a model that deals with some of the complexity that AR(1) fails to capture. Instead, BEST chose a model that is much more simplistic than even AR(1), a model which allows essentially no structure in the time series. In particular, the model that BEST adopted assumes that this year has no effect on next year. That assumption is clearly invalid on climatological grounds. It is also easily seen to be invalid on statistical grounds. Hence the conclusions of the statistical analysis done by BEST are unfounded.
All this occurred even though understanding the crucial question—what statistical model should be used?—requires only an introductory level of understanding in time series. The question is so basic that it is discussed by the introductory text of Shumway & Stoffer, cited above. Another text that does similarly is Introductory Time Series with R by P.S.P. Cowpertwait & A.V. Metcalfe (2009); a section from that text is attached. (The section argues that, from a statistical perspective, a pure AR(4) model is appropriate for global temperatures.) Neither Shumway & Stoffer nor Cowpertwait & Metcalfe have an agenda on global warming, to my knowledge. Rather, they are just writing introductory texts on time series and giving students practical examples; each text includes the series of global temperatures as one of those examples.
There are also textbooks that are devoted to the statistical analysis of climatic data and that discuss time-series modeling in detail. My bookshelf includes the following.
Climate Time Series Analysis (Mudelsee, 2010)
Statistical Analysis in Climate Research (von Storch & Zwiers, 2003)
Statistical Methods in the Atmospheric Sciences (Wilks, 2005)
Univariate Time Series in Geosciences (Gilgen, 2006)
Considering the second paper, on Urban Heat Islands, the conclusion there is that there has been some urban cooling. That conclusion contradicts over a century of research as well as common experience. It is almost certainly incorrect. And if such an unexpected conclusion is correct, then every feasible effort should be made to show the reader that it must be correct.
I suggest an alternative explanation. First note that the stations that your analysis describes as “very rural” are in fact simply “places that are not dominated by the built environment”. In other words, there might well be, and probably is, substantial urbanization at those stations. Second, note that Roy Spencer has presented evidence that the effects of urbanization on temperature grow logarithmically with population size.
Putting those two notes together, we might expect that the UHI effect will be larger at the sites classified as “very rural” than at the sites classified as urban. And that is indeed what your analysis shows. Of course, if this alternative explanation is correct, then we cannot draw any inferences about the size of UHI effects on the average temperature measurements, using the approach taken in your paper.
There are other, smaller, problems with your paper. In particular, the Discussion section states the following.
We observe the opposite of an urban heating effect over the period 1950 to 2010, with a slope of -0.19 ± 0.19 °C/100yr. This is not statistically consistent with prior estimates, but it does verify that the effect is very small….
If the two estimates are not consistent, then they contradict each other. In other words, at least one of them must be wrong. Hence one estimate cannot be used “verify” an inference drawn from the other. This has nothing to do with statistics. It is logic.
Sincerely, Doug
* * * * * * * * * * * *
Douglas J. Keenan
From: Richard Muller
To: James Astill
Cc: Elizabeth Muller
Sent: 17 October 2011, 23:33
Subject: Re: BEST papers
Dear James,
You’ve received a copy of an email that DJ Keenan wrote to me and Charlotte. He raises lots of issues that require addressing, some that reflect misunderstanding, and some of which just reflect disagreements among experts in the field of statistics. Since these issues are bound to arise again and again, we are preparing an FAQ that we will put on our web site.
Keenan states that he had not yet read our long paper on statistical methods. I think if he reads this he is more likely to appreciate the sophistication and care that we took in the analysis. David Brillinger, our chief advisor on statistics, warned us that by avoiding the jargon of statistics, we would mislead statisticians to think we had a naive approach. But we decided to write in a more casual style, specifically to be able to reach the wider world of geophysicists and climate scientists who don’t understand the jargon. Again, if Keenan reads the methods paper, he will have a deeper appreciation of what we have done.
It is also important to recognize that we are not creating a new field of science, but are adding to one that has a long history. In the past I’ve discovered that if you avoid using the methods of the past, the key scientists in the field don’t understand what you have done. As my favorite example, I cite a paper I wrote in which I did data were unevenly spaced in time, so I did a Lomb periodogram; the paper was rejected by referees who argued that I was using an “obscure” approach and should have simply done the traditional interpolation followed by Blackman-Tukey analysis. In the future I did it their way, always being careful however to also do a Lomb analysis to make sure there were no differences.
His initial comment is on the smoothing of data. There are certainly statisticians who vigorously oppose this approach, but there have been top statisticians who support it. Included in that list are David Brillinger, and his mentor, the great John Tukey. Tukey revolutionize the field of data analysis for science and his methods dominate many fields of physical science.
Tukey argued that smoothing was a version of “pre-whitening”, a valuable way to remove from the data behavior that was real but not of primary interest. Another of his methods was sequential analysis, in which the low frequency variations were identified, fit using a maximum likelihood method, and then subtracted from the data using a filter prior to the analysis of the frequencies of interest. He showed that this pre-whitening would lead to a more robust result. This is effectively what we did in the Decadal variations paper. The long time scale changes were not the focus of our study, so we did a maximum-likelihood fit, removed them, and examined the residuals.
Keenan quotes: “If, in a moment of insanity, you do smooth time series data and you do use it as input to other analyses, you dramatically increase the probability of fooling yourself! This is because smoothing induces spurious signals—signals that look real to other analytical methods.” Then he draws a conclusion that does not follow from this quote; he says: “This problem seems to invalidate much of the statistical analysis in your paper.”
He is, of course, being illogical. Just because smoothing can increase the probability of our fooling ourselves doesn’t mean that we did. There is real value to smoothing data, and yes, you have to beware of the traps, but if you are then there is a real advantage to doing that. I wrote about this in detail in my technical book on the subject, “Ice Ages and Astronomical Causes.” Much of this book is devoted to pointing out the traps and pitfalls that others in the field fell into.
Keenan goes on to say, “In statistical analyses, an inference is not drawn directly from data. Rather, a statistical model is fit to the data, and inferences are drawn from the model.” I agree wholeheartedly! He may be confused because we adopted the language of physics and geophysics rather than that of statistics. He goes on to say that “This invalidates much of the analysis done on global warming.” If we are to move ahead, it does no good simply to denigrate most of the previous work. So we do our work with more care, using valid statistical methods, but write our papers in such a way that the prior workers in the field will understand what we say. Our hope, in part, is to advance the methods of the field.
Unfortunately, Keenan’s conclusion is that there has been virtually no valid work in the climate field, that what is needed is a better model, and he does not know what that model should be. He says, “To summarize, most research on global warming relies on a statistical model that should not be used. This invalidates much of the analysis done on global warming. I published an op-ed piece in the Wall Street Journal to explain these issues, in plain English, this year.”
Here is his quote basically concluding that no analysis of global warming is valid under his statistical standards: “Although the AR(1)-based model is known to be inadequate, no one knows what statistical model should be used. There have been various papers in the peer-reviewed literature that suggest possible resolutions, but so far no alternative model has found much acceptance.”
What he is saying is that statistical methods are unable to be used to show that there is global warming or cooling or anything else. That is a very strong conclusion, and it reflects, in my mind, his exaggerated pedantry for statistical methods. He can and will criticize every paper published in the past and the future on the same grounds. We might as well give up in our attempts to evaluate global warming until we find a “model” that Keenan will approve — but he offers no help in doing that.
In fact, a quick survey of his website shows that his list of publications consists almost exclusively of analysis that shows other papers are wrong. I strongly suspect that Keenan would have rejected any model we had used.
He gives some specific complaints. He quotes our paper, where we say, “We observe the opposite of an urban heating effect over the period 1950 to 2010, with a slope of -0.19 ± 0.19 °C/100yr. This is not statistically consistent with prior estimates, but it does verify that the effect is very small….”
He then complains,
If the two estimates are not consistent, then they contradict each other. In other words, at least one of them must be wrong. Hence one estimate cannot be used “verify” an inference drawn from the other. This has nothing to do with statistics. It is logic.
He is misinterpreting our statement. Our conclusion is based on our analysis. We believe it is correct. The fact that it is inconsistent with prior estimates does imply that one is wrong. Of course, we think it is the prior estimates. We do not believe that the prior estimates were more than back-of-the-envelope “guestimates”, and so there is no “statistical” contradiction.
He complains,
Considering the second paper, on Urban Heat Islands, the conclusion there is that there has been some urban cooling. That conclusion contradicts over a century of research as well as common experience. It is almost certainly incorrect. And if such an unexpected conclusion is correct, then every feasible effort should be made to show the reader that it must be correct.
He is drawing a strong a conclusion for an effect that is only significant to one standard deviation! He never would have let us claim that -0.19 ± 0.19 °C/100yr indicates urban cooling. I am surprised that a statistician would argue that such a statistically insignificant effect indicates cooling.
Please be careful whom you share this email with. We are truly interested in winning over the other analysts in the field, and I worry that if they were to read portions of this email out of context that they might interpret it the wrong way.
Rich
From: D.J. Keenan
To: James Astill
Sent: 18 October, 2011 17:53
Subject: Re: BEST papers
James,
On the most crucial point, it seems that Rich and I are in agreement. Here is a quote from his reply.
Keenan goes on to say, “In statistical analyses, an inference is not drawn directly from data. Rather, a statistical model is fit to the data, and inferences are drawn from the model.” I agree wholeheartedly!
And so the question is this: was the statistical model that was adopted for their analysis a reasonable choice? If not, then–since their conclusions are based upon that model–their conclusions must be unfounded.
In fact, the statistical model that they adopted has been rejected by essentially everyone. In particular, it has been rejected by both the IPCC and the CCSP, as cited in my previous message. I know of no work that presents argumentation to support their choice of model: they have just adopted the model without any attempt at justification, which is clearly wrong.
(It has been known for decades that the statistical model that they adopted should not be used. Although the statistical problems with the model were clear, for a long time, no one knew the physical reason. Then in 1976, Klaus Hasselmann published a paper that explained the reason. The paper is famous and has since been cited more than 1000 times.)
We could have a discussion about what statistical model should be adopted. It is certain, though, that the model BEST adopted should be rejected. Ergo, their conclusions are unfounded.
Regarding smoothing, the situation here requires only little statistics to understand. Consider the example given by Matt Briggs at
Do NOT smooth time series before computing forecast skill
We take two series, each entirely random. We compute the correlation of the two series: that will tend to be around 0. Then we smooth each series, and we compute the correlation of the two smoothed series: that will tend to be greater than before. The more we smooth the two series, the greater the correlation. Yet we started out with purely random series. This is not a matter of opinion; it is factual. Yet the BEST work computes the correlation of smoothed series.
The reply uses rhetorical techniques to avoid that, stating “Just because smoothing can increase the probability of our fooling ourselves doesn’t mean that we did”. The statement is true, but it does not rebut the above point.
Considering the UHI paper, my message included the following.
There are other, smaller, problems with your paper. In particular, the Discussion section states the following.
We observe the opposite of an urban heating effect over the period 1950 to 2010, with a slope of -0.19 ± 0.19 °C/100yr. This is not statistically consistent with prior estimates, but it does verify that the effect is very small….
If the two estimates are not consistent, then they contradict each other. In other words, at least one of them must be wrong. Hence one estimate cannot be used “verify” an inference drawn from the other. This has nothing to do with statistics. It is logic.
The reply claims “The fact that [their paper’s conclusion] is inconsistent with prior estimates does imply that one is wrong”. The claim is obviously absurd.
The reply also criticizes me for “drawing a strong a conclusion for an effect that is only significant to one standard deviation”. I did not draw that conclusion, their paper suggested it: saying that the effect was “opposite in sign to that expected if the urban heat island effect was adding anomalous warming” and that “natural explanations might require some recent form of “urban cooling””, and then describing possible causes, such as “For example, if an asphalt surface is replaced by concrete, we might expect the solar absorption to decrease, leading to a net cooling effect”.
Note that the reply does not address the alternative explanation that my message proposed for their UHI results. That explanation, which is based on the analysis of Roy Spencer (cited in my message), implies that we cannot draw any inferences about the size of UHI effects on the average temperature measurements, using the approach taken in their paper.
I has a quick look at their Methods paper. It affects none of my criticisms.
Rich also cites his book on the causes of the ice ages. Kindly read my op-ed piece in the Wall Street Journal, and especially consider the discussion of Figures 6 and 7. His book claims to analyze the data in Figure 6: the book’s purpose is to propose a mechanism to explain why the similarity of the two lines is so weak. In fact, to understand the mechanism, it is only necessary to do a simple subtraction–as my piece explains. In short, the analysis is his book is extraordinarily incompetent–and it takes only an understanding of subtraction to see this.
This person who did the data analysis in that book is the person in charge of data analysis at BEST. The data analysis in the BEST papers would not pass in a third-year undergraduate course in statistical time series.
Lastly, a general comment on the surface temperature records might be appropriate. We have satellite records for the last few decades, and they closely agree with the surface records. We also have good evidence that the world was cooler 100-150 years ago than it is today. Primarily for those reasons, I think that the surface temperature records–from NASA, NOAA, Hadley/CRU, and now BEST–are probably roughly right.
Cheers, Doug
From: James Astill
To: D.J. Keenan
Sent: 18 October 2011, 17:57
Subject: Re: BEST papers
Dear Doug
Many thanks. Are you saying that, though you mistrust the BEST methodology to a great degree, you agree with their most important conclusion, re the surface temperature record?
best
James
James Astill
Energy & Environment Editor
From: D.J. Keenan
To: James Astill
Sent: 18 October 2011, 18:41
Subject: Re: BEST papers
James,
Yes, I agree that the BEST surface temperature record is very probably roughly right, at least over the last 120 years or so. This is for the general shape of their curve, not their estimates of uncertainties.
Cheers, Doug
From: D.J. Keenan
To: James Astill
Sent: 20 October, 2011 13:11
Subject: Re: BEST papers
James,
Someone just sent me the BEST press release, and asked for my comments on it. The press release begins with the following statement.
Global warming is real, according to a major study released today. Despite issues raised by climate change skeptics, the Berkeley Earth Surface Temperature study finds reliable evidence of a rise in the average world land temperature of approximately 1°C since the mid-1950s.
The second sentence may be true. The first sentence, however, is not implied by the second sentence, nor does it follow from the analyses in the research papers.
Demonstrating that “global warming is real” requires much more than demonstrating that average world land temperature rose by 1°C since the mid-1950s. As an illustration, the temperature in 2010 was higher than the temperature in 2009, but that on its own does not provide evidence for global warming: the increase in temperatures could obviously be due to random fluctuations. Similarly, the increase in temperatures since the mid 1950s could be due to random fluctuations.
In order to demonstrate that the increase in temperatures since the mid 1950s is not due to random fluctuations, it is necessary to do valid statistical analysis of the temperatures. The BEST team has not done such.
I want to emphasize something. Suppose someone says “2+2=5”. Then it is not merely my opinion that what they have said is wrong; rather, what they have said is wrong. Similarly, it is not merely my opinion that the BEST statistical analysis is seriously invalid; rather, the BEST statistical analysis is seriously invalid.
Cheers, Doug
From: James Astill
To: D.J. Keenan
Sent: 20 October 2011, 13:19
Subject: Re: BEST papers
Dear Doug
Many thanks for all your thoughts on this. It’ll be interesting to see how the BEST papers fare in the review process. Please keep in touch.
best
james
James Astill
Energy & Environment Editor
A story about BEST was published in the October 22nd edition of The Economist. The story, authored by James Astill, makes no mention of the above points. It is subheaded “A new analysis of the temperature record leaves little room for the doubters. The world is warming”. Its opening sentence is “For those who question whether global warming is really happening, it is necessary to believe that the instrumental temperature record is wrong”.
The old chestnut “lies, damned lies and statistics” has been used for over a century. Clearly this term has lost much of its original meaning in our modern world.
I humbly suggest a completely new phrase more appropriate to the post-normal modern era:
“Lies, damned lies and man-made global warming”
If we are at a point where the science needs to be left in the hands of statisticians, we will never get a good scientific explanation of the climate system.
This exchange should put paid to the idea that there are any understanding or open minds in critical positions in the old line publishing industry. I’m very thankful that there are people like Anthony and others who take the time to put real information on the web.
If you go to this website:
http://notalotofpeopleknowthat.wordpress.com/2011/10/20/temperature-trends-in-kansas/#comment-35
There is a marvelous chart showing the change in “average temperature) between 1910-1920 and 2000-2010.
It takes < 5 minutes to graph these average differences versus the community population. (Excel)
The graph is not "linear" (something between a 3rd and 4th power relation), but it is MONOTONIC and CONTINUOUSLY INCREASING. (I.e., the higher the population, the higher the temperture difference.)
As the old saying goes, "It doesn't take a ROCKET SCIENTIST" to figure this one out.
I wish I could find it right now, but there is some Norweigen fellow (U of Oslo, retired I believe. Geography department.) who did, indeed, take the "Rural" stations (the <0.5 C biased) from the USA "officical" data sets. He compared them with urban results. (The number 4's of S.S.org) His results are stunning, because they show only NOISE, no "trend" in temperatures from the RURAL stations. But all the URBAN stations go up with population growth!
So the question is to you want to trust the stastistically manipulative "wonks" or your own "lying eyes". I've been watching the shells. And I say the bean is under NONE of them, having been "palmed" a long time ago.
Max
What the hell, the series of e-mails is inverted in time. 🙂 I thought the most recent ones would be at the top, and the oldest at the bottom, thus preserving the sequence if I read bottom up. Now my whole 4th dimension is disoriented.
The argument seems to primarily deal with the effects of smoothing on correlation. I don’t like the response of BEST on this one. It amounts to: ‘Just because we might have altered our results doesn’t mean we did,’ and ‘new methods at analyzing time series are always worth trying, we’re trying something new.’
If you’re trying to validate a statistical method, by all means, try something new, find a time series and stick to it until you find an original method of analyzing it. Then publish your new method when you’ve fleshed out how useful it is.
If you’re trying to validate a temperature record, stick to what we know doesn’t lie to us, and avoid “reaching” for tempting new methods that might confirm or lead you astray.
The BEST team fails to stay within the scope of their research by saying that it is OK for them to try new statistics in an attempt to validate physical effects in a time series using those new methods. They presume to validate methods in two different fields of science at the same time, but put their method up for review by climatologists while ignoring the objections of statisticians.
“Demonstrating that “global warming is real” requires much more than demonstrating that average world land temperature rose by 1°C since the mid-1950s. As an illustration, the temperature in 2010 was higher than the temperature in 2009, but that on its own does not provide evidence for global warming: the increase in temperatures could obviously be due to random fluctuations. Similarly, the increase in temperatures since the mid 1950s could be due to random fluctuations.” — D.J.Keenan
This may be definitional, but showing the average temperature has risen proves a real warming. What it does NOT prove is *anthropogenic* global warming, or *permanent, irreversable* global warming. Gotta watch those adjectives.
AW – Did you really mean to say ‘start from the bottom up to maintain the timeline’?
As a matter of interest I have posted the following email; to the BBC
—I read your big spread about the BEST global warming story as published (prior to peer review) by The Economist.
I would like to draw your attention to the following comments from mathematician Doug Keenan ( http://www.informath.org/ ) which puts the story in perspective and suggest, in the interest of balance, that you run another story which covers the points he made.
http://wattsupwiththat.com/2011/10/21/a-mathematicians-response-to-best/#more-49696
The fundamental point (although there are others) seems to me to be these quotes –
‘Although an AR(1)-based model has been widely adopted, it nonetheless has serious problems. The problems are actually so basic that they are discussed in some recent introductory (undergraduate) texts on time series—for example, in Time Series Analysis and Its Applications (third edition, 2011) by R.H. Shumway & D.S. Stoffer (see Example 2.5; set exercises 3.33 and 5.3 elaborate). … To summarize, most research on global warming relies on a statistical model that should not be used. This invalidates much of the analysis done on global warming. I published an op-ed piece in the Wall Street Journal to explain these issues, in plain English, this year’
‘BEST did not adopt the AR(1)-based model; nor, however, did it adopt a model that deals with some of the complexity that AR(1) fails to capture. Instead, BEST chose a model that is much more simplistic than even AR(1), a model which allows essentially no structure in the time series. In particular, the model that BEST adopted assumes that this year has no effect on next year. That assumption is clearly invalid on climatological grounds. It is also easily seen to be invalid on statistical grounds. Hence the conclusions of the statistical analysis done by BEST are unfounded.’
If you do not chose to run a piece on these refuting and warning comments I would be pleased if you could tell me why as this would assist me in my complaint to the BBC governors.
I should add that I have posted a copy of my request on the WUWT website as a courtesy in case you wish to contact Anthony Watt or Doug Keenan. —
Seems this is all Astill heard: “buzz buzz buzz world has warmed buzz buzz buzz”. So much for science reporting.
PS further to my last comment the place on the BBC website that I made my request was –
http://news.bbc.co.uk/newswatch/ukfs/hi/newsid_3950000/newsid_3955200/3955223.stm
You do not have to log in or register as a blog commentator to contact the BBC with any comments on their news stories or any knowledgeable info you have.
Myself I am just a bloke on a couch with no scientific knowledge at all. Others beside me might make better and more informed correspondents to the BBC.
Anthony – Ben Goldacre (about video games)
“Serious claims belong in a serious scientific paper
If you have a serious new claim to make, it should go through scientific publication and peer review before you present it to the media” http://www.guardian.co.uk/commentisfree/2011/oct/21/bad-science-publishing-claims
Wow – thank you for posting this. For years I’ve known the AGW “facts” didn’t bear close inspection, but it’s truly enlightening to sneak a peek behind the curtain and see how the review process (doesn’t) work. The explanations and hand waving have always smelled off, but to see the fundamental flaws and general incompetence laid bare in such a high profile case is breathtaking.
I’m no scientist or statistician, but his points were so concise and fundamental that the flaws were clear to see.
Enlightening, but ultimately a very small step in the right direction.
Keep it up.
Might smoothing explain why the graph seems to anticipate the 1815 eruption of Mt. Tambora that made 1816 the “Year without a Summer” ? Shouldn’t that be a plummet rather than a decade of decline? Maybe there were other volcanoes, I am no expert.
“Demonstrating that “global warming is real” requires much more than demonstrating that average world land temperature rose by 1°C since the mid-1950s”
I assume that what is being referred to is the conclusion that man-made global warming of an unusual or dangerous kind is real.
In that regard, it is impossible to make judgements without some control with which to compare the present warming (which is not really in doubt except as to degree and rate). That seems to have been well-recognised by people like Michael Mann who sought to compare recent warming with what has happened over a millenium or two. There are all sorts of problems with his analysis which may call his conclusions into question, BUT the reasoning behind the attempt is absolutely valid i.e. to show that recent warming is different (or not).
It follows that it is a leap of faith, not of logic, to draw conclusions as to cause (e.g. CO2) having regard only to the fact of recent warming. If one does not know that present warming is unusual and that it therefore requires an explanation, then proposing a ’cause’ is an exercise in futility.
Basic research on the heating effect of CO2 and other gases in the atmosphere, effect of clouds, soot, high energy particles etc etc is great. Who wouldn’t want to know as much about these things as possible. But to find out where the recent past sits in the scheme of things, we need a yardstick which extends well beyond the time scales of a couple of millenia.
In the absence of such a yardstick, the assertions of dangerous global warming due to human activity remain as speculation.
The problems are actually so basic that they are discussed in some recent introductory (undergraduate) texts on time series—for example, in Time Series Analysis and Its Applications (third edition, 2011) by R.H. Shumway & D.S. Stoffer (see Example 2.5; set exercises 3.33 and 5.3 elaborate).
Heheh…. Ooooouch.
What is it about Global Warming that so attracts the incompetent?
I would suggest that BEST’s as-yet unpublished papers have already been well and truly ‘fisked’ by this article!
I see that the UK’s Daily Wail seems to have grasped the basics of what has been looked at, i.e. that temperatures have gone up a little, but no-one knows why.
http://www.dailymail.co.uk/sciencetech/article-2051723/Climate-change-New-analysis-1-6bn-weather-records-concludes-globe-IS-warming.html
Some of the comments to the DM article are amusing!
Until what year was the BEST analysis done 2007 or 2011? Anyone???
dearieme says:
October 21, 2011 at 2:03 pm
What is it about Global Warming that so attracts the incompetent?
I reckon its just that the subject is so large, so complex and there are so many unknowns that it is easy to ‘put up a shingle’ and go into business on pronouncements. It then takes a long time to receive any contradictions!
Anthony,
Take a rest, see you Monday.
Let the reserves fight the battle this weekend ????
I wonder if Peter ( http://www.youtube.com/watch?v=F_G_-SdAN04 ) is “on the run” these days with a bunch of RABID AWG’s wonks chasing him. Poor boy, doesn’t he know that when you point out the Emporer has no new clothes, not only does he blush..but he puts you on the RACK.
Max 🙂
Sadly it doesn’t really matter now whether the papers are accepted by the peer reviewers or not. The message is out there in the MSM that AGW is real and has been proven independently, and now accepted even by those who were deniers. That’s a powerful message, no doubt carefully timed.
All the careful analysis of the papers by people who know what they’re talking about will have no impact because the publicists struck first. A neat job, clearly indicating that this is nothing to do with science and everything to do with winning (back?) the hearts and minds of politicians and the general public. Such is the scientific method now.
OK I see now it seems up to and including 2010. If they included 2011 I put it to you that its not significant anymore. WE shall see as the years progress… BTW from their graph (in PDF file), from 1998 there is no warming. On that basis alone you cannot conclude that the earth is now warming its NOT.
Ray says:
October 21, 2011 at 1:13 pm
Er, um, climate is a statistical sum of weather over time. How can it be in hands other than the statisticians’?
Peter makes a good point. That BEST went to the press before going to peer review is rather telling.
I think they are also playing a simple semantic trick on the world at this point given their insistence of using the term “Global Warming” rather than “Anthropogenic Global Warming”… I think they know the average layman doesn’t know the difference, but by simply stating that it has gotten warmer since the 1950s it must be man that is doing it.
When called to task on this misinformation they can simply retreat to claiming to make no inference with respect to cause of the warming.
In other words they are proving what nobody doubts in order to infer what they can not prove.
I was wondering the same thing, as some of the recent flattening of temperatures, seems to be missing. WUWT. GK
Seems like BEST is nothing more than a well coordinated, expensive, year-in-the-making PR stunt to get a message into the MSM that the actual paper doesn’t even fully endorse. This is nothing more than an attempt to jump-start the stalling alarmism of AGW and Muller seems to fully endorse the press running with the wrong/inaccurate message. Seems the Team has a new draft-pick.
I suspect when the code/data is further analysed over the next few weeks by people with proper backgrounds in stats their findings will be unsupported by the data.
I must say that I found Richard Muller’s email to be correct in every particular, which increases my confidence in BEST’s results, given the data.
“Demonstrating that “global warming is real” requires much more than demonstrating that average world land temperature rose by 1°C since the mid-1950s”
They haven’t even demonstrated that.
As I point out in another thread today, they use the same (methodological) assumption as GISS/HADCRU, namely that the mean of Tmin+Tmax = average daily temperature.
Australian temperature data recorded at fixed times (4 hourly intervals) clearly shows that increased early morning warming is increasing Tmin and to a lesser extent Tmax. While this increased morning/daytime warming is lost at night with no comparable increase in temperature at 4AM.
In particular, the idea of fitting data to a model is not unlike smoothing it. The preparation of data-driven tools requires smoothing, otherwise you can’t define the tools. You then use the tools onwards. I reckon the quoted prohibition on using smoothed data is a bit overgeneralized.
From: Berkeley_Earth_UHI.pdf http://berkeleyearth.org/Resources/Berkeley_Earth_UHI
On page 5 the authors state:
On page 6 the authors state:
“Far” and “very-rural” is defined then as one tenth of a degree in latitude or longitude. Given that there are approx. 60 nautical miles per degree of latitude, one tenth would correspond to about 6 nautical miles or close to 7 normal miles. The authors have used terms that may be misleading, as “far” and “very-rural” are not terms that I would submit most people would use to describe sites that are as close as 7 miles to built-up areas. I would submit that most people would probably use the term “suburban” to describe sites that close to built-up areas.
Looking at Figure 2 on page 7, the “very-rural” stations are plotted in black. The continental United States, Europe and the southeast of Australia have so many of these “very-rural” stations that the map is almost solidly black. In short, the “very-rural” stations are predominantly located in areas of the world that are highly developed. As such, stations located in highly developed areas may swamp the statistics.
It is well known that suburban areas have higher growth rates than urban areas. Urban areas are simply already developed. Accordingly, one would expect the higher growth areas in the suburbs to have higher warming trends due to UHI than urban areas, since the urban areas are already hot and are probably not getting much hotter due to UHI. In short, BEST may have simply compared suburban UHI trends to urban UHI trends and found that suburban UHI trends were a little higher overall. No surprise there.
Unfortunately, BEST did not disclose the data used for the UHI study. On page 6, the authors state:
Anthropological effects are not just limited to buildings and pavement. Irrigation, clearing of land and changes in vegetation can also have important effects on temperature trends and are most prevalent in suburban areas. Further evaluation will have to wait until BEST releases their data and methods, but it appears that this UHI study may have some serious flaws.
How much difference would the analysis look if done the way Dr Keenan would like? Perhaps he should do that himself, and publish it.
It is more to this:
http://www.informath.org/media/a42.htm
Here D Keenan discusses time series and the IPCC time-serie from early 1900 till 2005 in particular.
IPCC uses a method AR1 that even Dr Mann, Schmidt, Annan et al finds unreliable.
From the article: “(He still believes that the world is warming, primarily due to computer simulations of the global climate system.)”
Ouch!! Those computers must be running really hot!
I think this is what they call being “better at communicating”.
Oh yes, they did their BEST, they pulled a Muller on us all.
My guess is that Trenberth will pull a Muller on you too.
Sad to see.
Sadly, it is clear the BEST project isn’t interested in science, truth, or even plausible extrapolations. Have you guys seen the global anomaly video put out by BEST?
http://www.berkeleyearth.org/movies.php There’s absolutely no way they can legitimately invent coverage where it never took place. But, they do it anyway. Here’s a couple of screen shots.
http://suyts.wordpress.com/2011/10/21/is-that-the-best-they-can-do/
Guys, I’d not waste much time on these people and just go straight to laughing at them.
dearieme says:
October 21, 2011 at 2:03 pm
What is it about Global Warming that so attracts the incompetent?
Easy – the substantial amount of money going to the CAGW supporters.
Don’t bias your results and you probably won’t get published or received further grants.
Bias you results and you don’t even have to deal with “rigorous” peer review and your name goes on the list for further grant money.
Doesn’t get any simpler than that.
Muller and his ‘team’ are simply protecting their money grubbing scam, as shilled here:
http://www.mullerandassociates.com/
E.g. “GreenGov™ is a service offered by Muller & Associates for Governments, International Organizations, non profits, and other organizations that work with Government.”
(How many scientists do you know who have trademarked products for governments?
Remember the precedence order in climatology:
1) Policy changes
2) Publicity
3) Scientific publication (in climatology also known as copying and pasting)
….
Optional) Proof
Why bother? Governments need increased taxes, and thereby need crises, Muller et al are well funded by governments to keep the crisis alive.
Joe says:
October 21, 2011 at 2:30 pm
… they are proving what nobody doubts in order to infer what they can not prove.
Yea, that is the CAGW way, isn’t it?
Well, Science Daily is touting it pretty hard right now
Math fun, what is one plus one? :p
Let me guess, soon there’ll be a bunch of old, same old, of the same old hippies popping up without the goatee smeared across their faces what with those without seem to be so much more rational. :-()
@Kohl “I reckon its just that the subject is so large, so complex and there are so many unknowns that it is easy to ‘put up a shingle’ and go into business on pronouncements. It then takes a long time to receive any contradictions!”
Indeed. Mercer et al, Nature, 1978 “West Antarctic ice sheet and CO 2 greenhouse effect- A threat of disaster” (415 references!) long tirade about the danger of CO2 doubling during next 50 years or so is gradually approaching the date when it can be scrutinized. So here we are, 40+ years since publication, where is the alleged 10C antarctic warming?
If Keenan wasn’t a polite academic, he could summarize and conclude by saying, “The Best report isn’t worth the bits it’s written on.” How many ways did he try to tell them it’s all invalid?
The response? “Talk to the hand!”
Typical.
This problem seems to invalidate much of the statistical analysis in your paper.
To me, Muller adequately addressed that problem, namely the smoothing of time series. As to the citations of textbooks, Dr. David Brillinger also wrote one, and has given invited addresses on time series analysis to professional organizations such as the Institute of Mathematical Statistics and American Statistical Association — he has a long record of important publications and consultations with the oil industry (among others); Muller had advice directly from him. What you never do is “smooth wily-nily”, or without checking the validity of the results.
By admitting that the BEST results are roughly right, he implicitly affirms that they are better than what went before, and that the averaging did not increase any roughness unacceptably, if at all.
Citing Briggs is an “appeal to authority” when Muller already had the advice of a greater authority.
RE: JohnWho: (October 21, 2011 at 3:22 pm)
“dearieme says: October 21, 2011 at 2:03 pm What is it about Global Warming that so attracts the incompetent?”
“Easy – the substantial amount of money going to the CAGW supporters . . .”
Of course, there are also those who believe that the Earth is a fragile sacred object being polluted by modern industry so deeply that they resort using the most flimsy and at times dishonest evidence to make the public accept their ideology.
Doug,
“I suggest an alternative explanation. First note that the stations that your analysis describes as “very rural” are in fact simply “places that are not dominated by the built environment”. In other words, there might well be, and probably is, substantial urbanization at those stations.”
So true. The choice of the term ‘urbanization’ and ‘urban heat island’ are is unfortunate. It masks substantial microclimate effects that are not intuitively captured by the term ‘urbanization’ or actually captured by the coarse selection criteria for ‘urban’ vs ‘rural’ sites.
Consider that where ever a thermometer is, at an ‘urban’ or ‘rural’ location, it is undoubtedly in near proximity to a building. New buildings and other infrastructure tend to be located near existing buildings. The rural thermometer sitting in the side yard of a farm house might be near the only building in the county. Thirty years later, the farmer has added a barn, chicken coop, ‘mother in law’ house and shop, paved the vehicle access between them, and put central air conditioning in the dwellings. Those might still be the only five buildings in the county, so the site is rated ‘very rural ‘ by population or nighttime lights or other ‘urbanization’ detection metric, but the microclimate effects on that thermometer have changed dramatically.
“The reply claims “The fact that [their paper’s conclusion] is inconsistent with prior estimates does imply that one is wrong”. The claim is obviously absurd.”
No, that claim is correct. You appear to have read it wrong.
“Yes, I agree that the BEST surface temperature record is very probably roughly right, at least over the last 120 years or so. This is for the general shape of their curve, not their estimates of uncertainties.”
It would have been very much better if you had provided an explanation of the import of that qualification. Remember that you are dealing with a reporter. He most likely does not have any appreciation of the role of the uncertainties in this issue, and what they mean wrt how the results should be interpreted. All he hears from this is “Yes, I agree that the BEST surface temperature record is very probably roughly right, blah waa blah blah, waaa wa blah waa blah blah.”
“Demonstrating that “global warming is real” requires much more than demonstrating that average world land temperature rose by 1°C since the mid-1950s.”
Yeah, at minimum it would require demonstrating something about the 75% of the planet that is not land. I am astounded at how many have taken no notice whatsoever of this glaring inconsistency between the paper and what is being claimed about it.
I for one am greatly disappointed in BEST. One expects better from the researchers. Even this geologist, a notable mathematical underachiever, knows and knew for many years you can smooth a time series. The Kind of errors that are apparent make the thing look like a high school science fair project gone wrong. I read there press release at Science Daily. It should be immediately withdrawn as inaccurate and misleading. This people is not science it is sophistry shame, shame, shame.
“We take two series, each entirely random. We compute the correlation of the two series: that will tend to be around 0. Then we smooth each series, and we compute the correlation of the two smoothed series: that will tend to be greater than before. The more we smooth the two series, the greater the correlation. Yet we started out with purely random series. This is not a matter of opinion; it is factual. Yet the BEST work computes the correlation of smoothed series.”
As someone with no statistical expertise, understanding that was like a light switching on in my head. Brilliant illustration of how inappropriately
..oops.. applied techniques can be used to massage the message.
JB
Just a couple of questions –
a) How much did this study cost?
b) Who’s pockets did the funding find it’s way into?
I guess my reading comprehension is seriously lacking. This is what I gleaned from the above post:
Mathematician: “You can’t use statistical models A, B, C, or D on various statistical grounds. Even if you could, then they aren’t valid scientifically, because scientific grounds aren’t given in the analysis.”
Climatologist: “I like model B because of the pretty pictures it gives, and it agrees with my worldview.”
Mathematician: “Great Job! I agree. Let me know if I can be of any further assistance.”
Jim
Spector says:
October 21, 2011 at 4:07 pm
RE: JohnWho: (October 21, 2011 at 3:22 pm)
“dearieme says: October 21, 2011 at 2:03 pm What is it about Global Warming that so attracts the incompetent?”
“Easy – the substantial amount of money going to the CAGW supporters . . .”
Of course, there are also those who believe that the Earth is a fragile sacred object being polluted by modern industry so deeply that they resort using the most flimsy and at times dishonest evidence to make the public accept their ideology
Yeah, but it isn’t polite to discuss the lunatic fringe here.
🙂
The answer to this is really simple – if Dr Keenan, or anybody else, does not like the way the BEST group analysed the data, then they should just do it themselves. If they get a different result, then that is interesting. If they get the same result that is also interesting. But complaining about statistical techniques and not doing anything, is not interesting.
“Second, note that Roy Spencer has presented evidence that the effects of urbanization on temperature grow logarithmically with population size.
The Global Average Urban Heat Island Effect in 2000 Estimated from Station Temperatures and Population Density Data
Putting those two notes together, we might expect that the UHI effect will be larger at the sites classified as “very rural” than at the sites classified as urban. And that is indeed what your analysis shows. Of course, if this alternative explanation is correct, then we cannot draw any inferences about the size of UHI effects on the average temperature measurements, using the approach taken in your paper.”
I was glad to see this point finally addressed. However, I found it disturbing to see the response avoiding it.
The other point is, that latest Hadley sea surface data paper shows only about an increase of 0.2 deg from the top of the 1940s temperatures (and 0.3 deg applying a questionable raw data manipulation).
So globally, we are talking about a very tiny increase and another question asked by Frank Lasner is, why do trends of land and ocean data not converge ?
You make good points. People often overlook just how far you need to get from “urban centers” to get to truly rural areas. For instance, looking at just Southern California on that map, I don’t see much blue. I would certainly expect nearly 1-2 degrees square of lat/longitude to be blue in that area. The Los Angeles Basin is a *very* spread out urban center, and a significant portion of the buildings have air conditioning (this is a desert/swamp (yes I know that’s a contradiction)). Perhaps it’s map distortion, or the size of the map, but I don’t see that much blue there. Instead, that entire area looks black. Frankly, just looking at that map I’d bet I could explain why they saw no UHIE.
James Sexton says:
October 21, 2011 at 3:10 pm
Sadly, it is clear the BEST project isn’t interested in science, truth, or even plausible extrapolations. Have you guys seen the global anomaly video put out by BEST?
http://www.berkeleyearth.org/movies.php There’s absolutely no way they can legitimately invent coverage where it never took place. But, they do it anyway. Here’s a couple of screen shots.
http://suyts.wordpress.com/2011/10/21/is-that-the-best-they-can-do/
Guys, I’d not waste much time on these people and just go straight to laughing at them.
=================================================
Strange…..
You picked one of their temp maps – 1891 – and they show the SW cool….
1891 was a famous drought and heat wave in the SW
Is this another case of inventing the past cooler….to make the present warmer?
I just feel BEST is fishy. I don’t like its smell. mmm… Difficult to prove. mmm… Let me try to give the gist of what comes across:
First BEST woos Anthony who says “I’m prepared to accept whatever result they produce, even if it proves my premise wrong….”. Then it abandons proper time-honoured procedure, in a way that Anthony thought unthinkable. Then it says in effect “you lot, like us, had good cause to be skeptics… Anthony Watts showed the weather stations were really substandard and untrustworthy… We have now looked at all that and decided they are trustworthy… so skeptics can relax and give up skepticism like we did…”
Sorry. Something stinks. IMHO, much as Peter Ward says (October 21, 2011 at 2:15 pm)
I’m glad David here liked my UHI page. Comparisons of individual pairs of stations show a huge UHI effect.
I want to see a CA-standard audit on the BEST UHI methodology. I suspect BEST of fudging-in UHI by the back door. Phil (October 21, 2011 at 2:59 pm) may be describing that back door – noting significantly that the relevant UHI data are not available to check.
JJB MKI: “We take two series, each entirely random. We compute the correlation of the two series: that will tend to be around 0. Then we smooth each series, and we compute the correlation of the two smoothed series: that will tend to be greater than before. The more we smooth the two series, the greater the correlation. Yet we started out with purely random series. This is not a matter of opinion; it is factual. Yet the BEST work computes the correlation of smoothed series.”
The same thing happens if you have two trends each masked by lots of noise. Initially the correlation is near 0; as you smooth more, it increases; when you get to the optimal amount of smoothing, then you have the best estimate of the relationship between the two trends; over-smoothing can obliterate the relationship.
So in conclusion: (a) undisciplined smoothing can produce an apparent correlation where there is none and (b) undisciplined resistance to smoothing can produce no correlation where there is some. Muller addressed this point in different language. How much smoothing is optimal? One almost never knows a priori.
Joe says (October 21, 2011 at 2:02 pm): “Heheh…. Ooooouch.”
My reaction exactly. It reminded me of Spencer schooling Dessler…only MORE!
Sceptic Matthew,
“Citing Briggs is an “appeal to authority” when Muller already had the advice of a greater authority.”
uhh greater authority?? He appealed to God?? (snicker)
I wouldn’t call Muller’s work trash, but, since it is most likely based on the same adjusted NCDC data, or the raw NCDC data with similarly unjustified adjustments, it should be similar.
Until someone gets a time machine and goes back and sets up climate recording stations outside of anthro influence we simply will NOT know the real change until UHI effects go flat. This MAY be what is happening in the US right now with the large urban stations since our trend is generally lower than the rest of the Northern Hemisphere. Of course that could be because we have higher quality stations also!! Either way it would seem to show a problem with the rest of the world. I would also suggest some adjustments were designed and tested only on US stations and environments. Might not be appropriate for the rest of the world.
Statistics…..Rutherfords quote say’s it all….”if you need statistics you ought of done an better experiment”
Why do climate scientists use statistical models, you can make any data appear as you!!!…….oh hang on i’ve answe…….!!!!!
Latitude says:
October 21, 2011 at 4:48 pm
James Sexton says:
October 21, 2011 at 3:10 pm
…………
http://www.berkeleyearth.org/movies.php …………
http://suyts.wordpress.com/2011/10/21/is-that-the-best-they-can-do/
Guys, I’d not waste much time on these people and just go straight to laughing at them.
=================================================
Strange…..
You picked one of their temp maps – 1891 – and they show the SW cool….
1891 was a famous drought and heat wave in the SW
Is this another case of inventing the past cooler….to make the present warmer?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lol, Thanks Lat, I picked 1891 because that’s the year they show full temp coverage for Africa and South America……… I’m calling BS. The other one, 1956 is when it shows full coverage of the Antarctic. Beyond academic pursuit, there isn’t much point in arguing statistical approaches to invented numbers. I think when we do such things, it lends an aura of legitimacy to outfits such as BEST. .
Prof Richard Muller, President and Chief Scientist of http://www.mullerandassociates.com/ (h/t ZT, 3.24pm)
Is this the fishy smell I smelt just now?
This one sentence shows that this entire BEST project may merely be a bald faced lie. Think about it:
He is a mathamatics professor.
He sees a paper, one of two things happen.
*He sees the paper is mathematically correct, he need not comment on it in that case, and so does not.
*He sees a paper that is not mathematically correct, it is his job to point that out, and he does so.
*So, what they are saying is, he is doing exactly what his job description says he should do, and because of that, we should ignore his criticism.
Second, look above, ” a quick survey of his website”, now, that is an accurate, comprehencive, why, even a scientific way to determine whether one should listen to him, right? And further, many of these articles seem to be skeptical of AGW, so, are they saying “oh no, he’s one of them, we don’t listen to them”? This strongly suggests that the BEST team members are, in fact, not “skeptics” as they claim.
In other words, we should continue to keep doing what we have always done, we should not bother to try anything new. Besides, if we do, why, the other scientists will not accept us, we wouldn’t want that! Right, we should continue to use the old crystal sphere models to tell us how the planets revolve, those telescope thingies are just too newfangled for us, no one will ever accept them.
This strongly suggests one of two things. The most charitable is that at the very least, peer pressure means that the BEST team members wish to receive the approval of their “peers”. That means that they will not contradict the AGW idea, since it has been clearly shown what happens to those who do. This is not suprising, they are in Berkely, surrounded by left leaning types, and AGW is the best hope for socialisim and control of the means of production (and pretty much everything else). I see no way that anyone doing such a project in that city could excape constant peer pressure to do so. However, considering the “quick survey” they did above, and how it shows that they are attempting to use logical fallacies (clever lies) and rhetoric to discredit Doug Keenan, I am not inclined to be charitable. Instead I believe this to be a false flag operation.
FALSE FLAG
False flag (aka Black Flag) operations are covert operations designed to deceive the public in such a way that the operations appear as though they are being carried out by other entities. The name is derived from the military concept of flying false colors; that is flying the flag of a country other than one’s own.
In other words, it goes like this:
Allie
“We are your friends, fellow skeptics like you, you should accept us!”
Neutralize
“See, even skeptics agree now that the world is warming!”
Destroy
“So you should stop being skeptics, now that you see that warming was true all along!”
This tactic, allie, neutralise, destroy, is the standard tactic of communism. The most likely place to find die hard marxists in the USA is in Berkely. It looks like they are still up to their old tricks.
I see three possibilities here:
1)This started out as science, with people who have some integrity. However, over time, and since they are in Berkely after all, the constant peer pressure of everyone around them (and the grant money) has caused them to unconsiously cave in, and “go with the flow”. Thus they may be doing it unconsciously, but they are agreeing with the AGW crowd, and against the skeptics, because they wish to keep the approval of their peers (and keep the money flowing). They may even still think they are skeptics, or have “healthy skeptisism”, however, all that peer pressure over time means they tell themselves this only because otherwise their consciences will nag them. Thus, when they say “we are skeptics”, they are only saying so to feel better about themselves, they actually stopped being skeptics a while back without admitting to themselves that they caved in.
2)Some of them are sincere, or were, however, some were not, and planned to co-opt this for their own agenda. They need not be the actual scientists at all, they could be the administrators, the guys with the money, and especially, the publicisers who will publish and press release selectivly only what supports the AGW position. There only need be enough of the co-opters seeded in amoing the scientists to steer it and pressure it in the direction they wish it to go.
3) 3) This could be a false flag operation from the get go, they may have all been lying deliberatly when they said “we are skeptics”.
Personally, while I think option 3 above is possible, it comes out the same if it is option 1 or 2 (or mostl likely, a mixture). It is quite possible that the lying going on is people telling themselves “I am a skeptic”, so as not to admit “I caved in to peer pressure”, which would be admitting “I am a coward”. People have an amazing ability to lie to themselves. A lie is a lie.
However, when they use clever tactics to try and disprove the very well prove Urban Heat Island Effect, which has been tested over and over again and even quantified (you know, that “the scientific method” thing), I wonder, could anyone lie to themselves that</b.much? The bad news, yes, that is possible, the equelly bad news, they instead may be lying to you. It comes out the same either way in the end.
So whether this was a false flag operation from the word go, or has become one with a bunch of people who still call themselves skeptics, but are now surrounded by a pro AGW crowd who (conciously or unconsciously) are carrying them along and steering them in the direction they want them to go (or which unconciously feels better), does not matter, either way, it comes out exactly the same.
And why wuld you have been fooled? Simple, because it all sounded so good! This has all happened before (and will again), see Operation Trust and Curveball. This method works by the simple expedient of telling people what they most want to hear. The preceeding two examples show that even huge numbers of people can be fooled this way. If it sounds too good to be true, it probably is.
One thing AGW has taught me:
Warmist leaders, like all firstclass rogues, point to those whom they fear most, the skeptics most likely to reveal their BS… and proactively accuse them of the very sins of which those warmists are guilty. Of course, they have a lot of insider understanding.
The cynic in me says, politically-savvy BEST said “lets make WUWT an offer they cannot refuse. Then we can get the media to say that even the skeptics now agree with us” – thereby, once again, gagging the skeptics. A pre-Durban pre-emptive strike.
Showing “global warming” is not surprising, in that temperature has been rising since the Little Ice Age. The challenge is to quantify the portions of anthropogenic versus natural causes.
Ross McKitrick presents his findings that socioeconomic development influences the temperature data: The influence of anthropogenic surface processes and inhomogeneities on gridded global climate data
Atmospheric Circulations do not Explain the Temperature-Industrialization Correlation
Keenan notes “Roy Spencer has presented evidence that the effects of urbanization on temperature grow logarithmically with population size.”
Could declining rural population be a cause for such a reduction in UHI?
Tad Patzek reports variations in GDP and energy with time. The Hubbert Peaks
Exponential growth, energetic Hubbert cycles, and the advancement of technology
This suggests that the BEST temperature data need be similarly tested against GDP, population, and energy use. Else the BEST publicity are but an argument from ignorance.
Let me get this straight..
BEST takes the temps… averages them.. then compiles them into a program which then extrapolates what the real temps were…
Then they take those findings and places them into a paper… refused to properly vet/review the paper and refuses to correct even basic errors…..
The IPCC, MET, EAU, CRU all use this same method of falsification by model… then try and get people to act out of emotion and not reason… before thinking….
Nothing more than the Obama Hand of the EPA justifying the need for regulations now without thinking or science to back it up….
HAS ANYTHING CHANGED SINCE CLIMATE-GATE? looks like buisness as usual to me..
Bill
_____________________________________________________________
I posted this in the other thread and it just seems so relevant….
kuhnkat: uhh greater authority?? He appealed to God?? (snicker)
no.
I should also mention, with Judith Curry’s name on most of the papers, I find reason to doubt her motivations again.
Sorry Judith, you reap what you sow. Next time don’t put your name on such trash that is clearly intended to be used as media nonsense rather than incrementally good science.
Well, I suspected I smelled a false flag operation (see my previous post), then I come here and discover Lucy Skywalkers post, and now see that there is proof. Lets look at some:
Hmm, GreewnGov, sure sounds like he already believes in “green”, not usually associated with ‘skeptics”. And he combines “Green” and “Gov”, sure sounds like pro AGW to me.
Note the wording, “we know”, not even we believe, or we think, know, ‘the science is settled”.
He already believes that carbon dioxide is to be avoided, why, I thought he was a “skeptic”? He uses the words “climate change”, once again, proof he is not a skeptic.
He is “helping governments” (and private business) to avoid “climate change” and “carbon dioxide”, so he has a conflict of interest with the BEST project. Since he is helping these people avoid the evil carbon dioxide, he wants carbon dioxide to be evil, thus, the BEST project cannot be “skeptical”, or neutral, in this regard. If AGW is proved wrong, he loses money. It looks like the BEST project is merely made in support of this, to set him up as an ‘expert”, to “prove” AGW is true so that they want to hire GreenGov to “help” them, and, of course, he can get cash for the actual doing of BEST while he is at it. I call that a win win win solution for him.
My conclusion, in my previous post, I was far too “charitable” with my options. Option #3 is true, this was a deliberate false flag operation from the start. What I said then holds true:
Allie
“We are your friends, fellow skeptics like you, you should accept us!”
Neutralize
“See, even skeptics agree now that the world is warming!”
Destroy
“So you should stop being skeptics, now that you see that warming was true all along!”
It should be shouted from the housetops, the BEST project is just another pro AGW project, in this case, it is falsely masquerading as a “skeptical” one. As such, the whole thing is nothing more than a bald faced lie. The words BEST and LIE should now be the words everyone should associate together.
The important thing is to make sure that all skeptics know this, and are not sucked in. Otherwise, they will first be neutralized, and then destroyed. There is a sucker born every minute, lets at least make sure that they don’t get suckered by this one. Judith Curry may be one such who has been suckered in and fooled.
And I hate to say I told you so, but way back when you first mentioned BEST, well, I smelled something fishy back then, and said so. I saw how you so much wanted to believe it, yet how it was based in Berkeley, and I immediately thought of Operation Trust. I suggest you click on that link, and never get fooled again.
Legatus says:
October 21, 2011 at 5:35 pm
Allie
“We are your friends, fellow skeptics like you, you should accept us!”
Neutralize
“See, even skeptics agree now that the world is warming!”
Destroy
“So you should stop being skeptics, now that you see that warming was true all along!”
—————————-
Thanks for the detail on the black flag subject. lol, I have experienced this tactic in business for 25 years and we termed it as the skunk and the squirrel / striped squirrel. Didn’t have a clue there was a proper name.
And yes I agree with option #3.
Here’s something I just posted at the end of an earlier thread:
And airports account for only .001% of the land area. But 10% (per my SWAG) of climatologists’ thermometers.
Surely I shouldn’t have to point this out to them. Surely they’ve read enough contrarian comments to realize that “Urban” encompasses airports.
Or maybe they’ve read only the published literature, or believers’ sophistical dismissals, and figured there’s no need to delve deeper.
jimmi_the_dalek says:
October 21, 2011 at 4:39 pm
The answer to this is really simple – if Dr Keenan, or anybody else, does not like the way the BEST group analysed the data, then they should just do it themselves. …
It only seems simple. BEST had financial support. Contrary to the conspiracy theories current among the AGW faithful, few, if any, critics [sceptics] do. Most “big energy” money for research that isn’t spent chasing subsurface hydrocarbons, goes into looking for paying substitutes. Western geological theory assumes that petroleum is limited and regardless of the apparent antipathy between “Big Oil” and the “green movement,” both parties assume a priori that we MUST have a substitute. They – BO that is – don’t waste money defending the use of petroleum, they want to be ready with new sources of energy when petroleum is gone.
Having to make a living, critics can’t simply turn around and do things correctly. There’s no financial support for that. Also, if you pay attention, there has never been any serious question that things have warmed. The real questions, for so-called skeptics revolve around “how much,” “why,” and “for how long.” The fundamental disagreement over AGW falls in the “A” – nothing else.
Keenan’s critique of BEST, in effect, points out that using essentially the same mathematical tools as other studies that have been argued to support AGW, BEST received effectively the same results concerning planetary temperature history. They could hardly have done anything else. What he calls for is the use of different tools – e.g. AR(4)-based models as opposed to an AR(1) model. Nothing in BEST, at least according to Keenan, helps to better clarify those key issues of “how much,” “why,” and “how long.”
Muller’s response is not to Keenan – which seems rather chicken-livered. Instead it is directed to Astill and in fact, the remainder of the exchange seems to go through Astill. I suspect that one “hidden variable” there is that Muller might have worried that if he and Keenan actually discussed the mathematical issues substantively, then, if Keenan convinced him of the reality of the issues that concerned him, Muller might have found it necessary to return to the chalk board – or Matlab or whatever – blowing his deadlines.
About my post of October 21, 2011 at 5:35 pm, I ddin’t mean for the whole several last paragraphs to be in bold, just a few words of it really, I’m not shouting at anyone here…
It is puzzling that there is no mention of LTP(Long Term Persistence), Koutsoyannis and the Hurst coefficient in this discussion of climatic time series data. Maybe its there but i couldnt find it.
A point to consider when looking at temperature data beginning in the 1950s. Above, someone made the comment that truly urban areas have little increase in temperatures due to UHI, because they are already built up.
I disagree, for the reason that air conditioning consumes considerable electricity, and all that electricity eventually becomes heat due to Second Law of thermodynamics. The 1950s and onward was the period when air conditioning became more and more prevalent across the western world, especially in urban areas. Urban buildings and residences were converted to air conditioning in that time period (typically the late 1950s and early 1960s).
This was definitely the case in US cities such as Houston, Dallas, San Antonio, and many others. The effect of the air conditioners and the heat from their condensers may not have had much effect on daytime high temperatures, but was probably more noticeable in the evening minimum temperatures.
Doug, you are the first person I have read putting the correct emphasis on Dr. Spencer’s urban denstity study. Your whole article is right on the mark. Thanks.
We defined a site as “very-rural” if the MOD500 map showed no urban regions within one tenth of a degree in latitude or longitude of the site.
Chicago’s latitude and longitude are listed as 41-52-55 and 87-37-40. O’Hare International Airport’s latitude and longitude are 41-58-41 and 87-54-28. That would make O’Hare “very rural” by the definition stated.
Roger Sowell – IIRR the average power density of urban areas is about four or five times that claimed for CO2 forcing. Might have remembered that wrong – please check.
I looked at paper 2 about the positions of thermometers. They talk about being unable to find out exactly where many of the stations were because the positions were given only to a tenth of a degree of Lat. or Long..
When I was navigating deepsea using my sextant, we calculated to seconds of a degree, and all positions were given in degrees, minutes and seconds.
Does no-one even use minutes any more?
Or do they think that 59 degrees 50 minutes ( written as 59.50 perhaps ) means 59 and a half degrees?
Or am I looking for mistakes where none exist?
An excellent assessment of the BEST work. The more you read the more one thinks that peer review will fail.
I looked up the study that Doug Keenan said invalidated the use of AR(1). I didn’t find that stated – they mentioned some variants. But I did notice the abstract, which said:
“This may be definitional, but showing the average temperature has risen proves a real warming. What it does NOT prove is *anthropogenic* global warming, or *permanent, irreversable* global warming. Gotta watch those adjectives.”
Proves a real warming of that stations(not Earth) and only in this circunstances:
-Only with method currently in use to measure “average”
-That there were no changes in stations.
-If the rise of average temperature is above error.
I have yet to read the BEST papers (I want to give them proper time and consideration), but I’m not surprised by the headlines and news stories. The devil is in the detail.
This assessment on the other hand goes right to the heart of the whole issue, and I am very grateful to Doug for writing it. This is one non-mathematician, who now ‘gets’ it much better.
The email to James Astill suggests it was not meant for Doug Kennan’s eyes – especially:
“Please be careful whom you share this email with. And now it is on a blog for all to see.
I am with Richard Muller on the question of smoothing. Keenen’s assertion that smoothing is incompatible with subsequent modeling seems a bit exaggerated.
There have been several shrill proclamations here at WUWT that any kind of smoothing renders time series data completely unanalysable and devoid of any predictive value. This must be nonsense. Richard Muller has provided a valuable service to WUWT by explaining why statistically smoothing does not empty data of any meaning.
I work in micro-tomography where reconstructed images are sometimes noisy and require singificant smoothing to allow real structure to emerge from noise. It would make no sense to assert that such images were devoid of real information following smoothing.
This spurious assertion that smoothing invalidates timeseries data has obstructed the discussion of a number of interesting hypotheses here on WUWT. It should do so no longer.
Slightly amending Legatus (October 21, 2011 at 6:35 pm) – thanks!
False Flag –
Allie “We are fellow skeptics like you! Watts’ concern is important!!”
Neutralize “Our results show that Watts’ work, though a salutory check, is actually nothing to worry about!”
Destroy “MEDIA MEDIA MEDIA!!! Even skeptics now see that warming has been true and records are trustworthy!”
So, standard Communist tactics all along, eh? Berkeley a Marxist bastion, eh?
This tactic was also used by the Inquisition. Inquisitors worked in pairs, one had the brutal touch, the other had the soft appealing touch. Evidently it’s thought to work.
phlogiston says:
October 22, 2011 at 6:37 am
“There have been several shrill proclamations here at WUWT that any kind of smoothing renders time series data completely unanalysable and devoid of any predictive value. This must be nonsense.”
On a perfectly normal day the temperature can easily vary between 0 deg C and 20 deg C, right now, here in Germany. That makes the IR backradiation vary by about 25% as it varies with the 4th power of the actual temperature, not the smoothed one.
Do you still think a model that manages to reproduce the *smoothed* time series has any predictive value, or any resemblance to reality? 😉
Since this post is about statistics and climate, can anybody help me make sense of the NCDC disparity? (see link below) NCDC reports the temperature trend for the 48 contiguous states is 1.2 degrees F per century. However, the mean of the individual states’ trends is 0.78 degrees F per century, and the area-weighted average for the 48 states is 0.74 degrees F per century.
Should not the trend of the entire 48 states should be very close to the average of each state’s trend?
http://sowellslawblog.blogspot.com/2011/09/us-long-term-temperature-trend-from.html
More “lying by omission,” this time by James Astill. I wonder if he is proud of himself for that.
What exactly makes Keenan a Mathematician?
A Google search and a visit to his website do not point to him having a degree in Mathematics; he certainly is no Professor as someone has falsely claimed here. Second, he has posted links to seven papers on his website, none of which have anything to do with Mathematics.
phlogiston says:
October 22, 2011 at 6:37 am
This spurious assertion that smoothing invalidates timeseries data has obstructed the discussion of a number of interesting hypotheses here on WUWT. It should do so no longer.
Then get a load of what Best’s “smoothing” seems to have done according to the screen shots linked below and the time-space course presented showing the existence of the surface stations, also alleged to cover the whole world. I’m thinking that the changing reality and just making things up problems shown below are partly what Keenan is talking about here and in his WSJ op ed. piece [AR 1 assumption, temporally and spatially?], but I could be wrong!
Latitude says:
October 21, 2011 at 4:48 pm
James Sexton says:
October 21, 2011 at 3:10 pm
Sadly, it is clear the BEST project isn’t interested in science, truth, or even plausible extrapolations. Have you guys seen the global anomaly video put out by BEST?
http://www.berkeleyearth.org/movies.php There’s absolutely no way they can legitimately invent coverage where it never took place. But, they do it anyway. Here’s a couple of screen shots.
http://suyts.wordpress.com/2011/10/21/is-that-the-best-they-can-do/
Guys, I’d not waste much time on these people and just go straight to laughing at them.
=================================================
Strange…..
You picked one of their temp maps – 1891 – and they show the SW cool….
1891 was a famous drought and heat wave in the SW
Is this another case of inventing the past cooler….to make the present warmer?
DirkH says:
October 22, 2011 at 8:34 am
On a perfectly normal day the temperature can easily vary between 0 deg C and 20 deg C, right now, here in Germany. That makes the IR backradiation vary by about 25% as it varies with the 4th power of the actual temperature, not the smoothed one.
Do you still think a model that manages to reproduce the *smoothed* time series has any predictive value, or any resemblance to reality? 😉
I guess it depends what you are trying to predict. The discussions I had in mind were of multidecadal trends of global and ocean temperature for instance.
This whole BEST thing seems a straw man. Not that many here at WUWT ever asserted that the whole of recorded global tmperature increase in the 20th century was an artefact of UHI and other fabrications. The rise is generally accepted. The substance of the debate and research is – what is the reason for the rise – is it anthropogenic or is it cyclical? Or in other words, is it more plausible and intelligent to propose temperature stasis or oscillation as the normal state or null hypothesis?
In fact the BEST global temperature curve going back to 1800:
http://www.bbc.co.uk/news/science-environment-15373071
reveals some interesting oscillations. There is an appearence of ocsillations of increasing wavelength, suggesting something like an interferance of phase shift effect, that would merit further research.
While the claim that temperature has increased since 1950 may be accurate, that should be no surprise – if it is the case. But, so what? Recall that the period from about 1940 to 1975 was cooling. (Even Obama’s science adviser, (a warmist) should have to concur on that cooling period because, in the 70s Holdren was busily searching for ways to warm the planet to forestall the oncoming ice age.)
Understand that it is alleged that due to increased green house gases in the atmosphere, heat is trapped that cannot escape from earth. So if an increase in green house gases is to blame for the warming, it should be minimum temperatures (that occur during the night) that must show the increase (of modern warming). In that case, the observed trend should be that minimum temperatures should be rising faster than maxima and mean temperatures. That is what would prove a causal link.
What I have discovered so far from my (silly?) carefully chosen sample of 15 weather stations is that the overall increase of maxima, means and minima was 0.036, 0.012 and 0.004 degrees C respectively per annum over the past 35 years. So the ratio is 9:3:1. Assuming that my sample is representative of all those stations listed, I have to conclude that it was the maximum temps (that occur during the day) that pushed up the average temps. and the minima. So either the sun shone more brightly or there were less clouds. Or, even, perhaps the air just simply became cleaner (less dust? Are there records on that?).
I also noted that the warming on the NH is totally different to that of the SH. There is virtually no warming in the SH as seen by the means and minima whereas in the NH, the ratio of the increase in maxima, means and minima is about 1:1:1, amazingly.
Again, if it were an increase in CO2 or GHG’s that is doing the warming, you would expect to see the exactly the same results for NH and SH because these gases should be distributed evenly in the whole of the NH and SH hemisphere. So, even here, we again must conclude that it never was the increase in CO2 that is doing it. The only logical explanation I can think of is the difference in the rate by which the earth is greening. In South America we still had massive de-forestation over this period whereas Australia and Southern Africa have large deserts. Obviously, the NH has most of the landmass and here everyone seems to be planting trees and gardens. A recent investigation by the Helsinki university found that 45 countries were more green then previously out of a sample of 70.
Paradoxically, the increase in greenery is partly due to human intervention, partly due to more heat coming available (increase in maxima!) and partly due to the extra CO2 that we put in the air which appears to be acting as a fertilizer/ accelerator for growth.
For my data, see:
http://www.letterdash.com/HenryP/henrys-pool-table-on-global-warming
(make a copy for yourself of the tables)
Now, if we could have the 3 plots Maxima, Means and Minima for the BEST figures? That would help.
I have been revising the maps on my site by increasing the resolution to 3 mile square grids, the increased detail shown compared to the 30 mile square gridded maps now on site are able to resolve the natural and man made sheltered areas that are responsible for the UHI effects.
from the sample maps shown at the link below there is an area greater than 10-15 miles in diameter out from the population centers that clearly show the UHI effects, so the 7 mile limit used by BEST IMO is lame at best.
Excerpt;It becomes easy to see not all of the warmer and cooler spots are due to cities alone, most are due to sheltering from weather due to surface textures that were preexisting before human occupation. People tended to settle in sheltered areas along water ways, so the natural heat islands have over the years, been human enhanced by urban growth. A fact of life not mentioned in the research literature?
Valleys in slow wind flows patterns can be over 10 degrees warmer than on windy days, like in the Dakotas in this screen shot, finished maps will be masked to block the random noise out side of the borders. Click to expand view and zoom in for more detail.
http://research.aerology.com/project-progress/map-detail/
DirkH says:
October 22, 2011 at 8:34 am
You can find examples that illustrate the necessity of smoothing and the liabilities of smoothing. Muller’s response to Keenan is essentially correct: it is necessary to use smoothing and judgment (i.e. explicit tests of various kinds of known problems) together.
phlogiston says:
October 22, 2011 at 6:37 am
I am with Richard Muller on the question of smoothing.
No you aren’t. You are with yourself on the question of smoothing, as you do not understand
“Keenen’s assertion that smoothing is incompatible with subsequent modeling seems a bit exaggerated.”
Keenan did not make that assertion.
“There have been several shrill proclamations here at WUWT that any kind of smoothing renders time series data completely unanalysable and devoid of any predictive value.”
Not on this thread. In fact, at the time you posted that accusation, there weren’t any.
“This must be nonsense. Richard Muller has provided a valuable service to WUWT by explaining why statistically smoothing does not empty data of any meaning.”
Richard Muller has given no such explanation of why his use of smoothing does not invalidate the statistics he computes.
“I work in micro-tomography where reconstructed images are sometimes noisy and require singificant smoothing to allow real structure to emerge from noise. It would make no sense to assert that such images were devoid of real information following smoothing.”
How very nice, and completely irrelevant to this thread. Keenan made no assertion that smoothing renders any data ‘devoid of real information’. Nor did he say anything about microtomographic images. What he did say, which you dont comprehend, is that smoothing time series data should not be performed before statistical analysis of those data, specifically the computation of correlation statistics.
“This spurious assertion that smoothing invalidates timeseries data …”
Does not appear anywhere but in your ignorant interpretation of what Keenan said. Keenan said that smoothing data invalidates certain statistics and staistical inferences calculated from those data. Muller did nothing but wave his hands and make appeals to authority – always fallacious but egregiously so when the authority is dead and cannot have given explicit support to Muller’s claims.
JJ
Well I’m no statistician and no doubt exaggerated my case about smoothing, so your rebuttal is fair up to a point. However Keenan’s views on smoothing of time series are not the only views in the field – in his reply Muller cites the influential work of John Tukey in justifying pre-smoothening (“whitening”) in certain circumstances.
I was not referring to comments on this post but other threads in the last year or so, where interesting empirical correlations between astrophysical parameters and climate, or correlations involving multidecadal oceanic oscillations, have been dismissed perhaps too hastily on the grounds of over-demanding statistical technicalities, considering all the unknowns in the system.
phlogiston says:
October 22, 2011 at 6:37 am
I work in micro-tomography where reconstructed images are sometimes noisy and require singificant smoothing to allow real structure to emerge from noise. It would make no sense to assert that such images were devoid of real information following smoothing.
Yes, but if you are talking about imaging such as in CAT scans and MRI’s, we already know what the basic structures should look like, so removing the presumed noise according to this well guided smoothing makes sense. On the other hand, it is exactly this question which is ‘begged’ – if not also routinely contradicted – in ipcc-style Climate Science smoothing and which apparently still plagues it in Muller’s work.
For those non-statisticians like myself, I would use the following example of how smoothing too early can give you the wrong answer (or a less correct answer).
If you teach basic maths (like I do from time to time), when you teach ’rounding’, you have to emphasize that you do not ’round’ intermediate answers, otherwise you get an incorrect final answer.
Time and again I have to supply examples where an ’rounded’ (or ‘smoothed’) intermediate result, when further operated on by a mathematical operator will give you an incorrect final result.
One can imagine that when you ‘smooth’ thousands of numbers in a variety of ways, errors cumulate.
I can see why stats naive people like me would smooth early, you smooth some data, you look at it, you like/dislike the ‘look’ of the results, then you carry on with your next operation on this set of smoothed data.
You really need a hard nosed stats expert to crunch your figures, operating without fear or favour if you are trying to do world class research. I guess that’s not climate science though.
Ps I am not sure why climate researchers, when using programs like MatLab etc do not include a couple of extra lines to their scripts to produce the output graphs using a selection of smoothing techniques i.e. AR(1) AR(4) etc at no cost to themselves.
I don’t know if you are old enough, to have experienced the world of “slide rule” mathematics and engineering (before digital calculators/computers). All calculations were rounded and approximated then. It required strict attention to significant numbers criteria and rounding rules. The world class research seemed to progress nicely, despite this.
Having said this, I do understand your point, and agree. GK
phlogiston says:
October 23, 2011 at 1:50 am
JJ
Well I’m no statistician …
And yet you felt OK making strong pronouncements about those who are, on a topic of which you clearly do not have even a basic understanding. And having been called on that, you persist.
and no doubt exaggerated my case about smoothing,
You did not exaggerate your case, you fabricated your case. You attributed to Keenan things he did not say, and then attacked those strawmen with your gross misunderstanding of the issue.
However Keenan’s views on smoothing of time series are not the only views in the field – in his reply Muller cites the influential work of John Tukey in justifying pre-smoothening (“whitening”) in certain circumstances.
And you understand none of it. All you personally understand is that someone thinks statistical analysis of smoothed data is OK, and someone else thinks it is not. You have no legitimate basis on which to make any decision whatsoever about who is correct. Yet you have formed an opinion, and picked a side. You have extolled the virtues of its proponent, fabricated disparagement against its opponent, and declared his position ‘nonsense’.
Muller even has you inventing accolades for the objects of his fallacious appeals to authority. He didn’t say that Tukey’s work on smoothing was “influential” – why did you? For all you know, Tukey’s work on smoothing is widely regarded as an egregious error by the statistical community, and is only given any credence whatsoever by one of his his former students. But you refer to that work as “influential” and tell that tale as if it were a fact that you learned and have reason to believe rather than a figment of your own imagination.
You are exactly the person that the BEST propaganda is aimed at, and you have swallowed the hook.
I was not referring to comments on this post but other threads in the last year or so, where interesting empirical correlations between astrophysical parameters and climate, or correlations involving multidecadal oceanic oscillations, have been dismissed perhaps too hastily on the grounds of over-demanding statistical technicalities, considering all the unknowns in the system.
“Perhaps too hastily” … LOL. Perhaps not hastily enough. You have absolutely no legitimate means of deciding between the two, and yet you have made such a decision and are acting as advocate for that position. “Over demanding statistical technicalities” … you have no basis for declaring anything such. What is your criterion for deciding which statistical methods are “absolutely fundamental assumptions which must be met for valid results” and which are “over demanding technicalities”? Yet you refer to “spurious assertions obstructing discussions” and proclaim that an objection that you do not comprehend should no longer be made.
Ask yourself why you do such things.
Yes, the globe has fortunately been warming since it started to recover from the “little Ice age,” an inhospitable period when the Thames was frozen over and (the then unbrainwashed) citizens of London were ice-skating across that famous river. There are plenty of honest historical records over many millennia that indicate Earth’s climate, due to purely natural causes, has endured a number of huge temperature variation cycles that were far greater than those now underway. But the recent cycle actually may be reversing due to changes in the Sun’s heliomagnetic field, so soon Earth may need all the global warming it can get from increasing CO2’s trivial contribution.
The current AGW panic is being created with completely unproven theories backed up with fudged computer programs designed to frighten scientifically uneducated dupes This hoax was created to provide enormous windfall profits to “experts” who continue to propagate it. But not to worry — London’s bitters-pint pub will eventually wake up to the scam that is costing him plenty in higher fuel bills and unnecessary extra costs when driving to work.
Tamino responds to Doug Keenan…
http://tamino.wordpress.com/2011/10/23/fake-skeptic-criticism-of-decadal-variations-in-the-global-atmospheric-land-temperatures/
Mr Briggs has written further on the BEST “results” and Mr Keenan’s comments on them:
http://wmbriggs.com/blog/?p=4530
FD
Phil says: @ October 21, 2011 at 2:59 pm
“On page 5 the authors state:
‘We consider two sets of stations, a complete set and a set restricted to sites that are far from urban regions.’ (emphasis added)
…….The authors have used terms that may be misleading, as “far” and “very-rural” are not terms that I would submit most people would use to describe sites that are as close as 7 miles to built-up areas. I would submit that most people would probably use the term “suburban” to describe sites that close to built-up areas…..
It is well known that suburban areas have higher growth rates than urban areas. Urban areas are simply already developed. Accordingly, one would expect the higher growth areas in the suburbs to have higher warming trends due to UHI than urban areas, since the urban areas are already hot and are probably not getting much hotter due to UHI. In short, BEST may have simply compared suburban UHI trends to urban UHI trends and found that suburban UHI trends were a little higher overall. No surprise there.
Unfortunately, BEST did not disclose the data used for the UHI study……
Anthropological effects are not just limited to buildings and pavement. Irrigation, clearing of land and changes in vegetation can also have important effects on temperature trends and are most prevalent in suburban areas. Further evaluation will have to wait until BEST releases their data and methods, but it appears that this UHI study may have some serious flaws.
____________________________________________________
I agree there may be some very serious flaws. For example I am in a “Very Rural” area, 12 miles from the closest city. Our road and the surrounding roads have gone from gravel to black top in the last decade and several farms have been bought up and turned into housing developments shopping malls and office parks. WORSE the closest “weather station” is at a little rural airport (built 15 years ago) that has recently been turned into “The Raleigh Executive Jetport” see aerial photo at : http://www.airnav.com/airport/ktta
How many of these “rural stations” are actually sitting at or have been moved to airports as happened to the weather station near my home?
Without the actual data we have no way of knowing.
I agree with others the BEST study is nothing more than a finely crafted propaganda piece designed to help push the Carbon Trading Scheme. It is no coincidence that the “Economist” (Think financiers) was where the story was “Leaked” and that Anthony was carefully muzzled by the promise of confidentiality before hand.
In Anthony’s own words:
“….The case twists around an emerging multibillion-dollar market trading carbon-credits under the Kyoto Protocol, which contains mechanisms for outsourcing environmental protection to developing nations.
The company involved, New Forests Company, grows forests in African countries with the purpose of selling credits from the carbon-dioxide its trees soak up to polluters abroad. Its investors include the World Bank, through its private investment arm, and the Hongkong and Shanghai Banking Corporation, HSBC…..” http://wattsupwiththat.com/2011/09/25/they-had-to-burn-the-village-to-save-it-from-global-warming/#comment-751952
Berkeley of course is home to Berkeley Model United Nations: http://www.bmun.net/ A training ground for future “Globalists”
Septic Matthew says:
October 21, 2011 at 3:48 pm
This problem seems to invalidate much of the statistical analysis in your paper.
To me, Muller adequately addressed that problem…
By admitting that the BEST results are roughly right, he implicitly affirms that they are better than what went before, and that the averaging did not increase any roughness unacceptably, if at all.
Citing Briggs is an “appeal to authority” when Muller already had the advice of a greater authority.
_____________________________________
Everyone is missing the point. Muller is providing a great example of “How to lie with statistics”
The REAL LIE is the starting point of the data set and fitting a LINE to a SINE CURVE!
Here is a graph with two whole cycles of that sine curve that prove the whole study is nothing but a well timed propaganda tool to move the wealth in YOUR pocket to the pockets of the financiers reading The Economist. I am sure they are laughing at us as they pulled this fast one and get ready to setup the next big economic bubble, carbon trading.
AMO & HCUT (USA temps): http://img689.imageshack.us/img689/9449/hacrut3detrendedandthea.png
(Chart from Bill Illis http://wattsupwiththat.com/2011/10/21/sceptical-berkeley-scientists-say-human-component-of-global-warming-may-be-somewhat-overstated/#comment-774453 )
Legatus says:
October 21, 2011 at 6:35 pm
Well, I suspected I smelled a false flag operation (see my previous post), then I come here and discover Lucy Skywalkers post, and now see that there is proof. Lets look at some:
GreenGov™ is a service offered by Muller & Associates
Hmm, GreewnGov, sure sounds like he already believes in “green”, not usually associated with ‘skeptics”. And he combines “Green” and “Gov”, sure sounds like pro AGW to me.
We know that in order to be effective, solutions must be sustainable…..
___________________________________________
Legatus you missed one.
A KEY WORD is sustainable that is the code word for UN Agenda 21 which is tied to CAGW/Climategate e-mails:
From my old notes:
Climategate e-mail on Global Governance & Sustainable Development (B1)
Here is more on the (B1) scenario IPCC Emissions Scenarios
Here is who wrote scenario (B1) Ged Davis is (Shell Oil executive with IPCC connection)
(I ROTFLMAO every time some one accuses skeptics of having “Oil connections)
Sustainable Development => Agenda 21
* Sustainable Development ====================
B1] 13th session of the United Nations Commission on Sustainable Development
(Source: Earth Negotiations Bulletin, Vol. 5 No. 218, 11 Apr 05)
The thirteenth session of the United Nations Commission on Sustainable Development (CSD-13) takes place from 11-22 April 2005, at UN headquarters in New York. CSD-13 is the second session to be held since the new multi-year programme of work was adopted at CSD-11 in 2003. The new work programme restructured CSD’s work on the basis of two-year “Implementation Cycles.” Each Implementation Cycle is comprised of a Review Year and a Policy Year, and focuses on a thematic cluster of issues. Building on the outcomes of CSD-12 (which was the Review Year of the first cycle), CSD-13 will focus on policies and options to expedite implementation of commitments in the areas of water, sanitation and human settlements, as contained in Agenda 21, the Programme for the Further Implementation of Agenda 21, the Johannesburg Plan of Implementation and the Millennium Declaration. Various cross-cutting issues will also be addressed. “
UN Division for Sustainable Development – full text of Agenda 21
UN REFORM – Restructuring for Global Governance
Our Global Neighborhood – Report of the Commission on Global Governance: a summary analysis
a lot of research and links about Agenda 21 in the USA
When talking of CAGW you can not forget the goal of the United Nations and that goal is GLOBAL GOVERNANCE! Nor can you forget that Berkeley is front and center in the drive toward global governance.
BEST is a team from the University of California and UC is neck deep in promoting “Globalization”
There is : UC Berkeley Model United Nations: http://www.bmun.net/
and http://www.tc.umn.edu/~unsa/integrationofdevelopingcountriesintotheworldeconomy.pdf
In looking into another matter I found these people at the University of California, San Diego.
Jill Richardson who is “UC San Diego” Sustainability Coordinator and is working on the practical aspects of UN Agenda 21 as far as I can tell.
Raymond Clemencon another faculty member, was one of the negotiators on the Rio Declaration and Agenda 21
http://irps.ucsd.edu/faculty/faculty-directory/raymond-clemencon.htm
Interesting that, based on BEST’s figures and confidence intervals, HadCRU’s figures are too high for portions of the mid and late 19th century, and too low from about 1971 onwards.
Another example of how failing to take account of UHI gives an inflated trend?
>>
Steve Richards says:
October 23, 2011 at 7:40 am
If you teach basic maths (like I do from time to time), when you teach ’rounding’, you have to emphasize that you do not ’round’ intermediate answers, otherwise you get an incorrect final answer.
<<
This goes against standard scientific/engineering calculation policy. When you do calculations, you have to remove the extraneous digits. Otherwise you’re including bogus precision into your calculations. If you multiply a number with n significant digits by one with m significant digits, then you get a number with n+m significant digits (in general). If n < m, then your final answer should be a number with n significant digits. To carry all n+m significant digits is false precision. For example, if we perform the following product:
(9.9 ± 0.5)*(9.999 ± 0.005) = 98.9901 ± 4.999745 . . . .
The correct answer is 99 ± 5, not 98.9901 ± 4.999745.
(The formula for computing error for a product is A*B*sqrt((EA/A)² + (EB/B)²), where A and B are the numbers to be multiplied, and EA and EB are their respective errors.)
>>
G. Karst says:
October 23, 2011 at 9:47 am
I don’t know if you are old enough, to have experienced the world of “slide rule” mathematics and engineering (before digital calculators/computers). All calculations were rounded and approximated then. It required strict attention to significant numbers criteria and rounding rules. The world class research seemed to progress nicely, despite this.
<<
A slide rule was limited to 3 significant digits. You might squeak out four digits near the lower end of the scale and only two at the upper end, but 3 significant digits was the basic limit of a standard length slide rule.
Rounding to the proper number of digits is always a requirement. A slide rule enforced an automatic rounding to 3 significant digits.
Jim
Jim Masterson:
If you are old enough for slide rule protocol, then you like me, have been “rounding” your age downward, for many years. If I extrapolate far enough, I may re-attain my youth. If I could only program my shaving mirror to reflect reality better. GK
>>
G. Karst says:
October 26, 2011 at 12:14 pm
If you are old enough for slide rule protocol, then you like me, have been “rounding” your age downward, for many years.
<<
Actually, I haven’t been rounding my age down. I’ve been using two significant figures for my age and still have a while to go before using three significant figures.
Jim