By William M. Briggs, professional statistician
“J’accuse! A statistician may prove anything with his nefarious methods. He may even say a negative number is positive! You cannot trust anything he says.”
Sigh. Unfortunately, this oft-hurled charge is all too true. I and my fellow statisticians must bear its sad burden, knowing it is caused by our more zealous brethren (and sisthren). But, you know, it really isn’t their fault, for they are victims of loving not wisely but too well their own creations.
First, a fact. It is true that, based on the observed satellite data, average global temperatures since about 1998 have not continued the rough year-by-year increase that had been noticed in the decade or so before that date. The temperatures since about 1998 have increased in some years, but more often have they decreased. For example, last year was cooler than the year before last. These statements, barring unknown errors in the measurement of that data, are taken as true by everybody, even statisticians.
Th AP gave this data—concealing its source—to “several independent statisticians” who said they “found no true temperature declines over time” (link)
How can this be? Why would a statistician say that the observed cooling is not “scientifically legitimate”; and why would another state that noticing the cooling “is a case of ‘people coming at the data with preconceived notions’”?
Are these statisticians, since they are concluding the opposite of what has been observed, insane? This is impossible: statisticians are highly lucid individuals, its male members exceedingly handsome and charming. Perhaps they are rabid environmentalists who care nothing for truth? No, because none of them knew the source of the data they were analyzing. What can account for this preposterous situation!
Love. The keen pleasures of their own handiwork. That is, the adoration of lovingly crafted models.
Let me teach you to be a classical statistician. Go to your favorite climate site and download a time series picture of the satellite-derived temperature (so that we have no complications from mixing of different data sources); any will do. Here’s one from our pal Anthony Watts.
Now fetch a ruler—a straight edge—preferably one with which you have an emotional attachment. Perhaps the one your daughter used in kindergarten. The only proviso is that you must love the ruler.
Place the ruler on the temperature plot and orient it along the data so that it most pleases your eye. Grab a pencil and draw a line along its edge. Then, if you can, erase all the original temperature points so that all you are left with is the line you drew.
If a reporter calls and asks if the temperature was warmer or colder last year, do not use the original data, which of course you cannot since you erased it, but use instead your line. According to that very objective line the temperature has obviously increased. Insist on the scientificity of that line—say that according to its sophisticated inner-methodology, the pronouncement must be that the temperature has gone up! Even though, in fact, it has gone down.
Don’t laugh yet, dear ones. That analogy is too close to the truth. The only twist is that statisticians don’t use a ruler to draw their lines—some use a hockey stick. Just kidding! (Now you can laugh.) Instead, they use the mathematical equivalent of rulers and other flexible lines.
Your ruler is a model Statisticians are taught—their entire training stresses—that data isn’t data until it is modeled. Those temperatures don’t attain significance until a model can be laid over the top of them. Further, it is our credo to, in the end, ignore the data and talk solely of the model and its properties. We love models!
All this would be OK, except for one fact that is always forgotten. For any set of data, there are always an infinite number of possible models. Which is the correct one? Which indeed!
Many of these models will say the temperature has gone down, just as others will say that it has gone up. The AP statisticians used models most familiar to them; like “moving averages of about 10 years” (moving average is the most used method of replacing actual data with a model in time series); or “trend” models, which are distinct cousins to rulers.
Since we are free to choose from an infinite bag, all of our models are suspect and should not be trusted until they have proven their worth by skillfully predicting data that has not yet been seen. None of the models in the AP study have done so. Even stronger, since they said temperatures were higher when they were in fact lower, they must predict higher temperatures in the coming years, a forecast which few are making.
We are too comfortable with this old way of doing things. We really can prove anything we want with careful choice of models.
Lies, damn lies and statistics.
Applause.
About 40+ years ago, a book came out “How to Lie with Statistics”. It’s a great book, and you can still get it from Amazon.
Has anyone seen the data provided by the AP?
‘He is the very model of a modern climate statistician.’
Most excellent. In other words,
“The devil can cite scripture for his purpose.”
“Th AP gave this data—concealing its source…”
Seth Borenstein told me he got the data that he gave to the statisticians from Dr. Christy. Wouldn’t Dr. Christy be able to tell us exactly what it was?
Nicely done.
So true.
Figgers lie and liars figger.
The moving average of x-years is a nice tool, telling us about what has happened, but not about what will happen. To understand that, you need to understand all the various factors that impact the climate and how they interact and affect each other. Lots a luck in that department! We’ve still a ways to go before we can get a handle on that messy mass of factors. Looking again at the graph of temperature data for a number of years, the moving average becomes useless toward the end of the data series; it is still a work in progress.
Again, “Standard Deviation”? Is it legitimate to call a least squares linear fit
valid, if the SD is high enough? Can we only compare GROUPS of data (say,
1950-1970, 1970-1990, 1990 to 2010. Take the SD of each data set, and
THEN see if there is a STATISTICALLY SIGNIFICANT DIFFERENCE?
Where the H-E-double toothpicks is “basic statistical practice” in this?
Too bad “statisticians” are not licensed like Medical Doctors. We could then
remove some licenses now!
Thank you for giving such a pleasant intro into statistics.
Have to explain my wife now the loud laughing here.
Moderator: Note I became too zealous and added my own carraige returns instead of allowing automatic work (in previous post). Sorry.
I wanna see those graphs and linear trends in.
http://www.woodfortrees.org/plot/gistemp/from:1998/plot/gistemp/from:2002/trend/plot/gistemp/from:1998/to:2000/trend/plot/gistemp/from:2008/trend/plot/gistemp/from:2000/to:2002/trend/plot/gistemp/from:2007/to:2008/trend
“How to Lie with Statistics” by Darrell Huff is a classic, wonderfully illustrated by Irving Geis. I didn’t know it was still available.
However, the tricks it teaches have nothing to do with the current issue replacing data with models.
I think this short article on models says it all.
Here’s Monty Python’s take on a particular model…
How true. “fitting” a curve to a dataset, however, IS the #1 objective of any data modeler. Even a randomly-generated dataset may have “fittable” (is that a word?) curves along most of its plot. (19th century mathematicians even imagined curves that had a different function at every point….very complex system indeed.)
Anna is always harping on this: Why do we humans insist on linear regression? The answer might be “cuz one of the basic tenets of science is that it be simple.” The problem with that is (and this goes back to the article here), when handed a random set of data (might even look more like a cloud than any line or curve), a simple straight-line fit is possible in ANY DIRECTION.
OK, so maybe we need polynomials (curves)…not straight lines. But how many degrees? Again, any degree will do, as long as it fits your pre-conceived notion. The more degrees, the more ups-n-downs. This is basic calculus. What if there are some points of discontinuity (Ha! missing data!)? What if the FUNCTION ITSELF is actually changing at every point?
I don’t think those possibilities are even discussed by today’s modelers.
Any one of us could play with data-fit using polynomials in MS Excel. Make a chart, and then choose “Add Trendline.” You’ll see all kinds of simple methodologies that assume continuous function data. This is as far as modern modeling goes, I swear! And your “guess” is as good as the Statistician’s, IMO.
As a professional in process risk analysis, the old saying goes: If you torture data it will tell you anything you want to know! Works every time.
Is temperature rising? Or cooling?
I cannot believe the amount of time wasted on this entirely subjective question. The answer is “it depends” It fully depends on what two dates are used to measure the time period over which the trend is to be determined and the length of time measured. Are we looking at a 5 year timescale? 10? 50? 100? 800? 12,000? 400,000? 100,000,000?
For ANY graph of global temperature one can pick numerous start points and end points that will show either warming or cooling. Even during the last decade I can show rapid warming, OR rapid cooling, depending on the start and end dates.
Picking a start date during 1998 and an end during 2008, one can show a strong cooling trend, however using the date range from 2001 – 2007 would show a significant warming trend.
The one thing that cannot be found is any time where climate was in stasis. The default nature of climate is one of change. And that has always been the case.
What’s the best kind of ruler to use when you want to draw a roller coaster?
AP?
“It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.” Sherlock Holmes.
In the case at hand it seems that the AP statisticians were able to twist the facts to suit their theories even though they had the data. All of which evokes another Doyle quote “His ignorance was as remarkable as his knowledge.”
Thanks to Dr. Briggs for putting it in perspective, much appreciated.
I have a ruler I love that I inherited from my Grandmother. It was made by Nabob Coffee in 1912 (or there abouts).
Good analysis. I imagin a 10 year moving average would indeed hide the last 7 years cooling fairly well
The global temperature was not increasing in the satellite era prior to the 1997/98 El Nino. And it has not been increasing since that El Nino. The El Nino resulted in a net change of about 0.3 degrees.
Larger image here:Global Temp Since 79
A four-year warming / cooling cycle of about +/- 0.3 degrees can be seen in the data over the last 30 years.
Thanks for an enjoyable read William, and good to see the truth coming from the horses mouth… 🙂
However, it’s worse than that. Temperature is the outcome of a set of interlinked dynamic chaotic processes, where trends mean nothing. Global average temperature is an even more ridiculous and meaningless measure, as only the total energy balance sheet of Earth’s climate system that has any meaning. Unfortunately we have no method of accurately measuring that yet, nor will we be able to do it for the foreseeable future.
Currently all predictive climate science is a sham.
Come on. You guys are reading too much into this. We all know that 69.43% of all statistics are made up on the spot. *;)
Psychologists did a study of statisticians and found that their attitude, on average, was mean.
OT
For those following the Science Museums support Copenhagen pole, the results have changed drastically yet again.
* 764 counted in so far
* 5223 counted out so far
Curiouser and curiouser!!!
Poll is open until December – link below if you want to vote:-
http://www.sciencemuseum.org.uk/proveit.aspx
Outstanding !!!!!
Good stuff Briggs. This exercise by the AP appears to be just another version of the bogus Rahmstorf exercise of a few years ago (“it’s even worse than we thought!”). The real fun is to contemplate what the AP is thinking by going to the statisticians in the first place. Apparently, regular people are not capable of looking at two numbers and discerning which is the higher number. Only with a doctorate in statistics and a resume replete with publications is it possible to compare numbers.
In kindergarten, children are asked to identify which number is larger. Most handle the task with ease. Apparently, none of those who managed the task are employed by the AP.
Hmm..don’t get nervous now. Cooling might not be so impressive, there is a big sunspot, arctic temperature refuse to dip, ice extent threatens to lag behind.
Things don’t go the climate sceptics way in the short perspective.
It can’t do every moment.
Monty Python jokes and even raljant and witty articles like the above seems to me a sign of desperation.
There is no need yet. Just wait and see and save the jokes till AGW is indisputably discredited.
As I have said before, borenstein is the worst. It’s not what he writes, but what he leaves out. In essense, he only writes half the story. Of course, he knows what he is doing, thus the reason for him being the worst newspaper writer out there today.
The AP article makes a rather stunning error of terminology in it’s headline. “Statisticians Reject Global Cooling”. This implies that the data was given to them, not without any information, as Borenstein claims but with the idea given to them that they should test it for negative trends as a null hypothesis. Worse than that, it would be impossible, if they were doing things right, for them to have “rejected” cooling in the last twelve years-that’s what RSS UAH and Hadley show!-this is a terminological inexactitude-the statisticians actually failed to reject warming. Why didn’t anyone think to test against the IPCC projections instead? .2 degrees per decade is probably easy to reject after a dozen years with none.
I wonder what these statisticians would have done if for each global annual average they were also given a calculated estimate of varience or if they were given global monthly averages with varience? Also, since they assumed the deviation from the streight line based on ten points was random error ( which it is not) what do the confidence limits on a true line look like and with what confidence can they declare either warming or cooling?
When I was in college, I had a stats class and we were taught how to make the stats say what we want them to say. Here is an example of what I mean.
The population of City A grew to 200% of the original. The population of City B grew to 105%. Without any more knowledge, which do you think gained the most people? However, what if the number of people in City A was 10 and grew to 20 but the number of people in City B was 100,000 and grew to 105,000.
Lawyers use this kind of stuff to manipulate juries. You change the scope or you use selective data to get the desired impression. For instance, in the above example, I can chose varying years to compare the two cities and only choose the ones that give the impression that I desire. It is very easy to make stats mislead, even when it is completely true. You better believe there are statisticians out there who are paid to make the numbers mislead.
Very good. What are the good Statistician’s views on Rahmstorf’s triangular filtering and other “innovative” methods used to “prove” the full horrors of AGW?
An enjoyable read.
Sean (09:27:25) : “What’s the best kind of ruler to use when you want to draw a roller coaster?”
A UC Twit of the Century nutter. Long live the Queen!
Matt:
My guess is that it is Borenstein that really needs to read this clearly stated explanation. But perhaps he already knew exactly what the statisticians would have to say given his poorly articulated charge.
I do think it would be helpful to clarify that most good looking statisticians would be loathe to undertake such a task without knowing what the data represented at since the model chosen should have some relationship to the physical, social or behavioral processes represented by the data.
Just a reminder to you folks. The data given to the researchers was with the satellite data removed.
However, data from the National Oceanic and Atmospheric Administration and NASA show 2005 has topped 1998. Published peer-reviewed scientific research generally cites temperatures measured by ground sensors, which are from NOAA, NASA and the British, more than the satellite data.
So they doctored the data especially for the article to get the result they wanted before Copenhagen. This is the data used.
ftp://ftp.ncdc.noaa.gov/pub/data/anomalies/monthly.land_ocean.90S.90N.df_1901-2000mean.dat
Obviously this:-
http://wattsupwiththat.files.wordpress.com/2009/10/junkyard_mmts_org.jpg
Is superior to this:-
http://wattsupwiththat.files.wordpress.com/2008/03/noaa-n_satellite.jpg
There’s quite a few models that I would like to draw…
All cousins are distinct, but some are distant?
Gary (08:47:51) :
“Seth Borenstein told me he got the data that he gave to the statisticians from Dr. Christy. Wouldn’t Dr. Christy be able to tell us exactly what it was?”
Seth Borenstein used NOAA and NASA data for his conclusion. He didn’t use all data sets from around the world. He did mention one set, but didn’t use that set, that shows cooling in the earth, to make the conclusion of his article.
The NASA set, i.e., GISTemp, is the data that James Hansen distributes. NOAA data is government data. There are problems with both entities.
“What’s the best kind of ruler to use when you want to draw a roller coaster?”
A mobius ruler
OT: For the sake of argument, the latest “Prove It” numbers from the British Science Museum are 767 in and 5239 out; More than 87% out.
I think that I can safely say than they won’t post another online poll.
Temperatures rose in the 20th Century.
Stamp prices rose in the 20th Century.
Stamps caused temperatures to rise.
FedEx and UPS are doing fine. The problem comes from the Post Office.
ralphzillo (10:13:12) :
I’ve dated a few. They’re not like people think. 🙁
But ya, great to look at.
Great article.
Sorry for the OT – what’s happening back at the science museum “prove it” survey? It’s now (5:35 PM GMT) reading 771 in, 5249 out?
Cheers
Mark
Here’s a question that will immediately reveal my complete lack of knowledge of anything in climate science. Why do we use the temperature anomaly? Why not the man global temp? Based on the above graph, if I understand what is being graphed, it is indeed continuing to warm, but at a much slower rate than in the past. The 0.1 degree anomaly means that we are still higher than the average over the past x years, right? Or am I missing something?
Of course, if we just looked at temperature it would be easier. That’s one of my complaints about this whole field of climate science. We build models based on data that is not direct experimental data, but either derived from models of proxies or based on models of actual data with n layers of indirection before we get to direct measurement.
Yanno, a lot of grad school guys try to model stock prices in similar, simplistic ways. They quickly learn better, if they want to stop losing money for themselves or their clients.
Interestingly, a lot of aggressive speculators are also fascinated by weather and prediction. I don’t think that this is coincidental. The climate prediction and “modeling” and the stock market prediction and “modeling” games are very similar.
The big difference is that there’s very real and immediate pecuniary reward and punishment in the stock market prediction game. In the climate game? Well, it seems to me that there’s little relationship between accuracy and pecuniary reward, unless it is an inverse one.
From my position, if global temperatures were a market that I would trade on price (temperature data) alone, I’d view it as having broken up the uptrend unambiguously by 2001. I’d be looking to short rallies since then, but I’d not be making big, long term bets on the down side just yet. I’d view temperature as rangebound but vulnerable to further declines.
I think that most of my profitable peers would be looking at this data the same way (if temperature were price).
The average price for a gallon of gas $2.99.34689734221. Where can I get gas at this price?
A) The price of gas has gone down over time.
B) The price of gas has gone up over time.
C) Both.
D) Neither.
E) All of the above.
True or False : I can use data to show you gas prices are always going down.
You didn’t know there’d be a pop quiz, did you!
” Rob Vermeulen (08:03:16) :
the problem is that it is well known in dynamical theory that negative feedbacks, when coupled to positive feedbacks, can lead to instabilities as well.”
Point to the significant positive feedback. The one that is certainly present is quite small, pretty monotonic, and well bounded. The AGW models aren’t based on chaos theory instabilities. They’re based on the non-existence of any significant cooling systems – which would cause fatal warming from the addition of any net increase in warming.
The shortform simplification of AGW is:
1) An increase in atmospheric carbon dioxide causes a quite small direct net retention in heat. This warming – in itself – isn’t directly relevant. (We can calculate the direct greenhouse effects directly, and even an overwhelming amount of carbon dioxide isn’t exciting directly. Even amazing levels of carbon dioxide don’t cause us to “Go Venus.” (Or we would have before the oceans even cooled off enough to liquify.) But…
2) The fact that the atmosphere is slightly warmer means there is more water vapor due to equilibrium with the oceans.
3) More water vapor means more clouds.
4) Clouds act as a strong radiation blanket causing more warming.
5) Return to #2 – with amplification. Strong amplification.
Looking at the well understood #1, there is a hefty chunk of unexplained warming remaining. There’s only one other spot to assign the apparent warming to in this model. So when you -fit- this model, you calculate a strong positive feedback from cloud cover.
What Lindzen is saying is: When one actually puts satellites up and observe step four directly, as opposed to projecting via models – one finds that step four should read “4) Clouds insulate the ground from the incoming radiation substantially better than they retain the existing heat. Thus they cause a net cooling.”
More CO2 -> More heat -> More Clouds -> More cooling isn’t a runaway feedback loop. Well, unless the first step is wrong. But we have the physical chemistry for that step pretty well nailed down. The entire reason we were looking for a positive feedback loop in the first place was that the quite solid science of the “More CO2 -> More Heat” steps was dramatically failing to answer why, precisely, we were getting more heating than that.
This fundamental parameter “How much warming does an increase in cloud cover cause?” is one of the inputs in the circulation models. Extensive computer modeling occurs here – with the basic assumption that clouds are a net warming effect.
What a clever, witty, piece. Why do we elect politicians (and even some scientists ( not all ) who have not learned to think in such obvious ways of logic, such as this excellent article suggest we do. Sadly. plain common sense seems to be the main victim every time a politician gets elected, and claims to be a success.
Bernie (10:12:47) :
But perhaps he already knew exactly what the statisticians would have to say given his poorly articulated charge.
That and,
perhaps he thought saying “the National Oceanic and Atmospheric Administration and NASA” would be enough to settle it and keep people from looking at the other data sets. Because after all “the National Oceanic and Atmospheric Administration and NASA”, well, how could they be wrong. 😉
Good one, thanks Mr Briggs.
However, so what? Temperatures go up and down, we live in a fluid dynamic environment. Does it say anything about the difference between mans influence and that of nature? Not that I can see.
The AGW disseminators say we control our weather much like the Victorians thought they could conquer it. It seems to me both were/are wrong. Heck, most people I talk to do not understand where the carbon comes from in organic life!
“First, a fact. It is true that, based on the observed satellite data, average global temperatures since about 1998 have not continued the rough year-by-year increase that had been noticed in the decade or so before that date.”
When you say a “decade or so”, you actually mean 2 years don’t you?
http://data.giss.nasa.gov/gistemp/graphs/Fig.A2.lrg.gif
Do you think a statistical significance test yields a statistically significant cooling temperature trend between 1998 and present. It certainly doesn’t and the entire premise of this article is flawed.
The human body has it’s own statistical memory. It knows, for example, if the weather has been getting warmer or colder year-on-year. The warmist statisticians and marketeers try to overwhelm this human memory, but they succeed only in alienating as the memory of what is happening grows stronger.
Hysteria goes so far and no more, as does emotional drive.
Ask a football coach how long and far that type of game plan will last.
Well the good Dr Statistician puts it about in perspective. Part of the fault must lie with the mathematics (of statistics, that is).
I can create a set of data (numbers) which have the property that not one of those numbers bears any relationship to anything real; I simply make them up in my head, and write them down on the list as they come to mind; in no particular order.
Now I can present my data set to a statistician; well why not Dr Briggs’ set of statisticians; and I can ask them; “what do you make of this data ?”
Each of them can now apply the mathematics of statistics to my set of data; and produce means and medians and standard deviations; and any or all of the trappings of statistical mathematics; and their results are as valid as if I took a sequential set of readings of the official NIST Atomic clock time Standard, and gave them those numbers instead.
And therein lies the rub. The mathematical processes may be faithfully carried out; but give no assurance that the result actually means anything. Which means I can do the same analyses on the numbers in a telephone book, and extract the same amount of nonsense.
So Dr Briggs has an adorable straight line ruler. It might be quite useless, since no such thing as a straight line exists anywhere in the universe; that is simply a figment of our imagination, as is all of mathematics.
So what about the polynomial fit to the data; how many real physical processes actually follow a polynomial theoretical model. Well there are some that come close to doing that. I know that the mechanical resonant frequencies of some cuts of quartz crystal resonators, change with temperature in polynomial fashion such as parabolic; but then only over certain temperature ranges.
One is much more likely to encounter an exponential function in nature than a polynomial one. Radioactive decay for example, would give you a headache trying to fit a polynomial expression to measured data; exponential decay is much more common.
Statistics is supposed to give us “better” answers for experiments that are repeated many times; and the more the merrier; if Statisticians are as emotional as Dr Briggs suggests.
I still love the example of the first SSS Draft Lottery where calendar birth dates for conscription were selected by lottery. Overzealous statisticians immediately declared the result of that lottery to be not random, and therefore unfair; because more earlier calendar dates were selected , than later ones. I maintain that the draft lottery could have come up; Jan1, Jan2, Jan3…..Dec 29, Dec 30, Dec 31 ; a result which is no more unlikely than the result they actually got, which was one out of 366! (factorial) possible results.
But I submit that Dr Briggs concerns are somewhat irrelevent; because the real problem is in the data sampling methodology.
The theory of Sampled Data Systems, and the reconstruction of continuous functions from sampled data, is very well understood, and well developed. Our entire modern data communication, and telephone communication; as well as all our digital recording of music or images; even movies, is all dependent on sampled data theory.
And the most fundamental theorem of Sampled Data Theory, is the Nyquist Sampling Theorem; which establishes the requirement for adequacy of sample spacing, in order for the sampled function to be properly reconstructed. The theory goes on to explain the aliassing noise, errors that arise, and the creation of false in band error signals that corrupt the reconstruction irretrievably. No filtering process can completely remove aliassing noise errors, without also removing valid signal information which also corrupts the reconstruction.
So all the statistical manipulation is irrelevent Dr Briggs, because the data itself is pure garbage; just like the random stream of consciousness data set I wrote down as it occurred to me.
Global temperature data is a continuous function of two variables; time and space, and the Nyquist theorem requires that to correctly map that function in a retrievable fashion so the data can be processed to yield a global long term mean temperature; requires that the timing and spatial separation of sampling locations conforms to the Nyquist Criterion, that requires at least one sample taken for each half cycle of the highest frequency present in the presumably band limited function. Failure to do that by a factor of two or more means that aliassed noise will occur at the zero frequency which is the average that is being sought.
The min/max twice daily temperature sampling method in wide use for land based stations, already fails that test, since the diurnal temperature cycle is NOT a pure sinusoidal waveform; nor is it even time symmetric on a 24 hour basis, and that means even the daily time average is unrecoverable for any single site. The spatial sampling fails the Nyquist test by orders of magnitude, so that is simply a joke; to suggest that GISStemp or HADcrut is related with any accuracy to the mean global temperature of the earth.
The satellite measurments might be doing better on that score; but even that system has its problems.
So take heart Dr Briggs, you statisticians are not the only varmints in the pantry; the data gatherers are guilty of egregious errors.
And the computer modellers don’t seem to understand the garbage in; garbage out relationship.
George
Mark Young (10:38:30) :
The climate prediction and “modeling” and the stock market prediction and “modeling” games are very similar.
Making it look like a stock is going up and temperatures are going up can be the same :
http://www.thedailyshow.com/watch/thu-march-12-2009/jim-cramer-extended-interview-pt–2
The video will continue to a part 3 at the end of part 2
There’s no doubt that the satellites give a cooling trend by linear regression e.g. from 1998 and many other moments including 2001 – beginning of the century. Wolfram Alpha is enough to check it.
It can also be demonstrated that one gets warming with different choices, especially of the initial moment.
More importantly, most of these trends up to 15-year-long intervals are not statistically significant.
At any rate, it’s bizarre to fight against a “global cooling” straw man. As Roy Spencer correctly said, wasn’t this discussion about global warming as opposed to cooling? The absence of statistically significant cooling doesn’t imply the presence of warming.
Dr. Briggs is always a fun read, and always an educational read as well. His nmberwatch site is in itself a great education as to the wiles of those who play around with numbers.
What these warm-mongers are doing is exactly what my professors in college when I majored in physics lo those many years ago warned me specifically to not do.
Oh well, those whiz kids made all of the financial computer models which extrapolated past data bet our bankrolls on their future extrapolations. Disaster has been the result.
Well, according to my “model,” the AP will be going out of business shorty.
Maybe the answer to what has been going on is a lot simpler. Maybe GIStemp has been dropping cold stations and adding/keeping warm ones. Then gridcells with no stations are interpolated/filled in from other (warmer?) stations. All but one station north of 65 degrees in Canada have apparently disappeared. Only coastal stations remain in California (the inland and mountain ones are apparently gone). Similar issues apparently show up in Australia and Brazil (stations further away from the Equator have been dropped). The stuff is still preliminary but very interesting. See http://chiefio.wordpress.com/.
Mark Fawcett (10:35:57) :
Great article.
Sorry for the OT – what’s happening back at the science museum “prove it” survey? It’s now (5:35 PM GMT) reading 771 in, 5249 out?
Cheers
Mark
Maybe they are using only votes that have comments?
anyway now it is
# 773 counted in so far
# 5279 counted out
Personally I love bicubic splines, you can prove anything with those.
Jeff in Ctown (Canada) (09:35:52) :
“Good analysis. I imagin a 10 year moving average would indeed hide the last 7 years cooling fairly well.”
That is because a 10 year moving average has a 5 year group delay. You would not even begin to see the change until then, and it would be smoothed out.
Alan S. Blue (10:44:33) :
” Rob Vermeulen (08:03:16) :
the problem is that it is well known in dynamical theory that negative feedbacks, when coupled to positive feedbacks, can lead to instabilities as well.”
I don’t know from which board this came. I guess Alan had multiple windows open. But, this is certainly not well known, or at least, the problem needs to be defined more carefully. Assuming a series plant, the basic rule is that the gain of the negative feedback must be greater than the positive feedback gain to stabilize the mode. I.e., the bandwidth of the negative feedback must be greater than the frequency of instability. In this case, increasing negative feedback always improves stability. On the other hand, the bandwidth of the negative feedback must be less than the frequency of non-minimum phase zero dynamics, or it will induce instability.
This is something which bothers me about the entire analysis enterprise to date. I have seen lots of assertions of positive feedbacks which are, in fact, zero feedbacks (pure integrators). I have never seen any discussion of the zero dynamics. In fact, I am doubtful that the climate modelers are that advanced in their understanding of feedback theory. But, I have a hunch the existence of NMP zero dynamics within the bandwidth of known negative feedbacks could invalidate some of the models, since it would have led to instability in the past.
Don’t take this as an assertion that there are such deficiencies. I don’t have the time to expend any of my own effort in this direction and have not done so. But, maybe, someone else will have an inspiration to try it.
Alan Cheetham (09:39:10) :
The global temperature was not increasing in the satellite era prior to the 1997/98 El Nino. And it has not been increasing since that El Nino. The El Nino resulted in a net change of about 0.3 degrees.
http://www.appinsys.com/GlobalWarming/GW_Summary_files/image004.jpg
An interesting step change caused by an El Nino?
Total OT, belongs to an older topic:
From Greenie-Watch: http://antigreen.blogspot.com/
The museum’s Prove It! website, which is designed to influence politicians at the Copenhagen climate summit in December, allows members of the public to pledge their support, or lack of it, to the environmentalist cause. But so far those backing the campaign are out-numbered nearly six-to-one by opponents.
By Saturday, 2,385 people who took the poll said “count me out” compared to just 415 who said “count me in”, after being asked whether they agreed with the statement: “I’ve seen the evidence. And I want the government to prove they’re serious about climate change by negotiating a strong, effective, fair deal at Copenhagen.”
LarryOldtimer (11:28:54) said:
Unfortunately, Oldtimer you have a case of mistaken identity:
Number Watch John Brignell
http://www.numberwatch.co.uk/number%20watch.htm
William M. Briggs
http://wmbriggs.com/blog/
BOTH are excellent reads. Dr Brignell has given us the list of all the things caused by AGW. Dr Briggs has given us several chapters of a useful introductory text on statistics.
best wishes
anna v:
I voted, prompted by the mention of the survey here. The numbers are astounding. Given the obvious bias of this Prove It posting, namely pro-CAGW, the results will be embarrassing in the extreme. I somehow doubt that we will hear much about these results come December.
You do have to identify yourself with a valid email address for them to count your vote.
Politicians use statistics like a drunkard uses a lamp-post – not for illumination but for support…!
“If your experiment needs statistics, you ought to have done a better experiment.”
Ernest Rutherford
Hey, today is Climatefoolsday:
http://climatefoolsday.com/
Here is the confernce program:
http://climaterealists.com/attachments/database/Climate%20Fools%20Day%20List%20of%20Speakers.pdf
Ever notice when a reports come out that shows earth is cooling ,which it is just geting started,or when the report that showed where the global temperature had dropped by 1 degree that the global warming crowd tries to find ways to discredit the report.Now i’m wondering if where they show a record high temp or back in 2007 when the ice was suppositly at lowest level ever in the Artic do they try to find out if it is true or,do they just accept it as FACT????? My guess is they will never try to dispute warmer temps or less ice and just accept that as gospel.
“”” aylamp (12:53:08) :
“If your experiment needs statistics, you ought to have done a better experiment.”
Ernest Rutherford “”
Now there’s another smart Kiwi for you.
George E. Smith (11:12:38) :
The issue of using min/max to characterize the “average” temperature of the day is one that has ben addressed in studies before. However, if you like you can do the study yourself using CRN stations yourself which measure the temperature every few minutes. You can find this data online. Simply, use that continous stream of data and integrate it over time to find an average. Then compute max+min/2. What is important is not that max=min/2 = the integrated value. what is important ( since max+min/2 is used as an estimate of the integrated value) is that max+min/2 not be a biased estimator or rather that its bias should not change over time. This is important because what we are interested is the TREND in the data not the absolute temperature itself. Does max+min /2 = the integrated temperature?
No. it does not. It is not intended to. It is intended to provide an unbiased
( over time) estimate of the integrated value such that trend estimates are not biased. There are plenty of ASOS sites that also provide continuous data so you can calculate this quite easily.
1. Pick some sites ( either CRN or ASOS or both) say 30.
2. Download the sampled stream for a few years ( in some cases 5 minute data in other cases less frequent)
3. Create integrated average temps per day.
4. Create a monthly average.
5. from #2 select a daily max and daily min.
6. compute (max+min)/2 for each day
7. Create a monthly average.
8. Compute a trend line for #4 from start month to end month
9. Compute a trend line for #7 from start month to end month.
10. Subtract the slope values for 8 from the slop value for 9.
11. Report the results for all 30 stations.
Do that and you have something that goes beyond merely stating txxhat
(max+min)/2 is not the same as the integrated temperature.
With respect to nyquist and temperature the issue is the frequency of the signal and the correlation of the spatial field. The high correlation of the spatial field will allow for a characterization of the trend ( or estimate of the trend) with fewer stations than a simple calculation of the nyquist requirement would indicate. You can see this by taking a full dataset and then decimating it. Actually George it would be kinda cool to do a simulated experiment of this where you just used artificial weather over a spatial grid.
Habibullo Abdussamatov makes a distinct forecast. He risks falsification. That’s scientific. The late Karl R. Popper would have liked that.
The article right or wrong is clear and comprehensible too.
If he is wrong it will be revealed in the next few years, won’t it?
I’m dreaming of a White Thanksgiving,
With all the AGW stuffed turkeys,
Where the pumpkins glisten,
And the children finally listen,
To someone who doesn’t have rocks in their head.
George E. Smith (13:16:02)
His dad was fae Perth in Scotland.
Wha’s like us?
Let’s not overlook the possibility that the computers used to derive these ‘models’ were powered by the ‘Infinite Improbablility Drive’.
Maybe we should run this whole thing through the Great Computer on Magrathia. Why, I’d trust analysis from Slartibartfast more than from AP. Heck, I trust Zaphod Beeblebrox more than AP now I think about it.
Chuckle…
Alec Kitson
“First, a fact. It is true that, based on the observed satellite data, average global temperatures since about 1998 have not continued the rough year-by-year increase that had been noticed in the decade or so before that date. The temperatures since about 1998 have increased in some years, but more often have they decreased.”
From 1988-1998, there were four years in which the temperature was lower than the previous, and six where it was higher, according to GISS. That is not really a “rough year-by-year increase”.
From 1999-2008, there were five years in which the temperature was lower than the previous, and five where it was higher. The pattern looks rather identical to that seen in the previous ten years – no sign of a “rough year-by-year increase”, just stochastic variations. Ten years is, of course, too short a time span to see the trend about which these stochastic variations are taking place.
Why start your piece with a “fact” which is trivially shown to be false?
But the American Statistical Association say case closed?:
http://www.aaas.org/news/releases/2009/media/1021climate_letter.pdf
I once heard the saying:
In life, as in art, as in science, one should never fall in love with one’s models.
As a professional ecological statistician, I’m almost embarrassed by some of these ‘crucial’ analyses that are floating around in cyberspace; they appear so rudimentary! Any model that is fit to data should have some basis in accepted theories and/or scientific (as opposed to statistical) hypotheses about how a system might work (or at least a reasonable approximation to these). Where there is debate or uncertainty about the underlying theories/hypotheses, this can be represented as different models that can all be fit to the same data. There is then a myriad of different ways of formally comparing the different models to see which are better or more believable, which in turn indicates the level of support (or lack thereof) from the data for the different scientific hypotheses. However, as noted in William’s post, the true test of any model(s) is prediction of future outcomes.
I’d also suggest (like a previous poster), a real statistician shouldn’t only be concerned about fitting models to data, but where the data come from in the first place and how representative it is of the system of interest (and the more I read, the more I cringe). It’s an inescapable fact, that how much faith one can have in one’s conclusions from a model based on data, depends upon the quality of the data in the first place. It’s GIGO once again. Now if you have a model that isn’t based on data, ….
alec kitson (13:50:03)
Didn’t they say in 42 years the Arctic will be ice-free? {chuckle}
Honestly I guess the Models are based on Bistromatics …
Why don’t you skip the AP and go straight to the original research papers that were published earlier this year?
“”” steven mosher (13:22:17) :
George E. Smith (11:12:38) :
The issue of using min/max to characterize the “average” temperature of the day is one that has ben addressed in studies before. However, if you like you can do the study yourself using CRN stations yourself which measure the temperature every few minutes. You can find this data online. Simply, use that continous stream of data and integrate it over time to find an average. Then compute max+min/2. What is important is not that max=min/2 = the integrated value. what is important ( since max+min/2 is used as an estimate of the integrated value) is that max+min/2 not be a biased estimator or rather that its bias should not change over time. This is important because what we are interested is the TREND in the data not the absolute temperature itself. Does max+min /2 = the integrated temperature? “””
Well Steven, maybe what you are interested in is the “trend” but Gaia could care less about the trend. She responds not to trends; nor to daily averages; but to instantaneous (temporally and spatially) values.
And when it comes to the EFFECT of temperature, in that it establishes the rate and spectrum of the LWIR cooling of the planet, that is compensating for the solar TSI, then MN is more interested in the 4th power of the temperature; and when it comes to the capture of LWIR photons by say CO2 molecules at some wavelength other than at the peak of the LWIR emission, then MN pays more attention to the fifth power of the temperature because of the Wien displacment.
And if you integrate either the fourth or the fifth power of the temperature over that 24 hour diurnal cycle or more importantly over the greater amplitude annual cycle, then you ALWAYS get a positive offset compared to what the average would suggest.
There is also that slight matter of the passage of clouds across the landscape, which has a nasty habit of raisng the bandwidth of the temperature signal.
The whole idea of recording “anomalies” rather than the temperature itself, is tantamount to a differentiation process; which is a well known method of raising the noise level of any signal.
I only have to watch the bay area six PM news weather report to see that the spatial frequencies of temperature data, are much higher than is represented in any global sampling methodology.
No I can’t tell you what the magnitude of the errors are, that result from inadequate sampling; nobody can because the real global temperature function has never been properly measured; well nobody besides Gaia has measured it; and she always gets the right answer.
In any case it is a futile exercise anyway; because the average temperature of the planet tells you nothing at all about the heat flow processes or fluxes.
The weather is a consequence of the differential conditions that set up winds and currents and convections and all the other processes that go into messing with our daily lives.
Climatologists define climate as the long term average of weather; there’s that “trend” rearing its ugly head. I prefer to think that climate is more properly the integral of weather, and not the average of weather.
But then what the hell do I know; I’m just a Physicist, and Mathematician.
Years ago I learned that a statistician can put your head in an oven and your feet in a bucket of water and tell you that you are comfortable.
That is not to say that statistics are not important. We live in a statistical universe, but we must know and UNDERSTAND what we are doing with them.
“ice water” – damn!
By the way, let’s not lose sight of the fact that the ‘statisticians’ didn’t apparently find signs of warming!
‘Our’ side of the issue didn’t run around screaming that the earth is cooling, the earth is cooling, just mentioned that it’s possible to infer a trace cooling in recent years, somewhere between a plateau (I’ll settle for that) and a modicum of cooling. No hysteria there, nothing to defend and ‘we’re’ not asking for trillions and world government! No, the claims were of inexorable warming, that’s what the alarmists must defend.
If it can’t be found in the data the claim is, how to put this … er, wrong.
But I’ll check with the mice.
And thanks for all the fish.
Alec Kitson
George E. Smith (11:12:38) : “The mathematical processes may be faithfully carried out; but give no assurance that the result actually means anything.”
Two excellent posts in this thread, well worth remembering in any discussion of catastrophic AGW.
For a more whimsical take on the problem, I like one of Isaac Asimov’s short stories:
http://en.wikipedia.org/wiki/The_Machine_that_Won_the_War
Well, I’ve read about halfway through the comments. Seems to me that most commenters miss the whole point of this blog.
HOW DO YOU DO GOOD STATISTICS WITH BAD DATA????
?????????????
I was once told that all people who drank beer in 1896 are now dead thus confirming that beer kills people. I was also told that there was a direct correlation between increasing sales of coca cola in the ’50’s and increasing population thus confirming that coke causes pregnancy.
Now I discover that co2 is to blame for everything.
Interesting that both beer and coke contain co2. Hmmmm
Suppose I have a tea kettle warming up for five minutes, after which I turn off the burner for one minute. I have taken water temps every few seconds.
If I do a regression on the water temperatures over time, I will find an increasing trend, provided I use the entire data set. I will miss the slow cooling after the fifth minute.
If of course I only regress the last minute, then I am “cherry picking.”
Statisticians should be required to practice for some time as quality engineers in manufacturing plants. They would learn the one primary difference between academic statistics and statistics in the world. And that truth is this:
Do not assume that something is constant.
The primary interest of the quality engineer is whether a production process is constant (that is, due to random, common causes), and the usual tests (such as Shewhart charts) are designed to detect the presence of assignable, special causes.
In this case, the assumption is that the slope of the regression line is constant. But the slope of the line may be as variable as any other variable. The key question they ought to have asked is whether the trend line really does reflect “a constant system of common causes” or whether the parameters themselves are subject to change.
However, academic statisticians are often accustomed to sampling colored beads from an urn; and — as A. C. Rosander once said, “the world does not consist of a fixed number of balls in an urn.”
I should add:
An Exponentially Weighted Moving Average chart or a Cumulative Sum chart are especially sensitive to small shifts in the process mean and would be nice tools for detecting a change in direction in a graph.
@george Smith
The extremal average (M+m)/2 may not be an unbiased estimator if the distribution of values within the day is not symmetric. Even if it is, and even if it is approximately Gaussian*, it is still an inefficient estimator. If there are outliers in the daily data, due to measurement system error or actual +/- spikes, those outliers will be one or the other or both of the extreme values, and so will affect the extremal average, making the estimator subject to considerable variation.
If it is impractical to “integrate” over the entire series of daily temperatures — perhaps temperatures are not taken continuously — it would make more sense to take samples at random times of the day and compute a sample average and sample standard deviation. (Of course, there is the problem of serial correlation…)
The temperature trend lines are flat.
You can believe your own eyes or if you are an pro-AGW’er, you can gain some comfort with an analysis that says “the temperature trend is still up even though it looks flat. This is the settled science. The deniers are so far out there trying to claim the flat line is flat.”
To be clear though, we really need to take into account the impact that El Ninos and La Ninas and the changing AMO had on the trend lines. If you pull these impacts out, there is still an increasing trend. It is much lower over the last 10 years than it was before and it is much, much lower than predicted by the theory and the climate models, but it is still an increase.
And the pro-AGW set would rather believe an abstract or an statistical analysis that contradicts the actual data available – why? because it provides comfort that their position is still viable. It might go on like this for decades no matter how flat the line is.
Jeff in Ctown (Canada) (09:35:52) :
I have a ruler I love that I inherited from my Grandmother. It was made by Nabob Coffee in 1912 (or there abouts).
———————-
Interesting ruler. I wonder if there is a picture of it on the internet.
———————-
Good analysis. I imagin a 10 year moving average would indeed hide the last 7 years cooling fairly well
———————–
Wrong averaging has covered over other things too :
However, Nir Shaviv explained that it should be expected that such a signal is not seen in the averaged monthly data they had used.
http://wattsupwiththat.com/2009/08/04/a-link-between-the-sun-cosmic-rays-aerosols-and-liquid-water-clouds-appears-to-exist-on-a-global-scale/
I have sat through WAY too many lectures that started with:
“Assume X1, X2, X3, … ~ … i.i.d. …” (*yawn*)
…Wo – back up there prof – you just started and you already:
a) turned your back on reality.
b) lost my trust.
Step 1: Sell a lie.
Success during step 1 underpins all later “reasoning” (quotation-marks mandatory).
784 counted in so far 5394 counted out so far
PWNED
As an engineer (originally) by trade it was always quite simple:
1) 2 data points = straight line
2) 3 data points = a curve
Then I went and did a bunch of statistics (and econometrics) in a Masters and discovered my creative talents. Now I can prove anything you want on demand.
I miss the good ole days 😉
It’s refreshing to see a statistician being honest about his field of expertise.
Now if only the statisticians who work for the TV ratings outfit would be so honest. There’s no way their relatively tiny sample size can be scaled up to accurately calculate how many viewers each TV show had out of the 300+ million people in the most diverse country on Earth.
Well said William.
Hope you are well.
Best regards, Allan
You’ve convinced me to dig out my old Master’s coursework on long-term forecasting. I think I mentioned before in another blog here that I used about 50 or so years of US aluminium (two i’s darn it!) consumption data and applied all kinds of lovely forecast modeling techniques to it.
Things to note:
1) The data was known with a high degree of accuracy (far better than thermometer readings I bet);
2) All models fit past data extremely well (R^2 in the region of 95-98%);
3) Some of the models were purely time trend in nature, others were econometric models (intensity of use models and the like).
The forecast range was phenomenol. Everything from a doubling in US aluminium consumption over the forecast period to dropping down to almost zero.
Modelling is fun!
From the instant post:
“Love. The keen pleasures of their own handiwork. That is, the adoration of lovingly crafted models.”
So true.
And which one of these model lovers is going to come out and say their model is wrong?
What is also important to remember is that mathematical equations are in effect “models”.
String theory is a mathematical model…of what?
The answer is nothing that has been observed & measured.
So, what happens when mathematical models are applied to properties and relationships of physical elements and energy in our environment?
Unless the mathematical models are rigorously defined by actual observation & measurement and the observation & measurement is rigorously quantified and the mathematical models (also called equations) are rigorously and consistently related to the quantified observations & measurements, it is likely, no, it is more than likely, it is assured the mathematical models will be wrong.
Meaningful mathematical models can only be made after observation & measurement have been accomplished.
The mathematical model is analogous to the forward pass in football: Three results can happen; an incomplete pass, an interception, or a completed pass.
So, three things can happen and two of them are bad, but when the third thing happens it is very advantagous:
When mathematical models are correct (based on repeated testing, i.e., further observation & measurement) representations of properties and relationships of physical elements and energy, they help us predict future behavior and allow engineering to manipulate (or harness) these properties and relationtionships of physical elements and energy to create technology for the betterment and enlightenment of Man.
But remember, just like the forward pass in football, two out of three results are bad.
And what’s worse, sometimes an interception (the worst result) can erroneously be called a completion.
Mathematics is a very powerful tool, but like all powerful tools, if misused either intentionally or by mistake, it can lead to very misleading or dangerous outcomes.
When statisticians apply mathematical equations to non-linear, unstable, and complex physical relationships and processes.
Beware of the pick-six going the other way for a touchdown.
Since we are free to choose from an infinite bag, all of our models are suspect and should not be trusted until they have proven their worth by skillfully predicting data that has not yet been seen.
That will be difficult for a model of a random or chaotic system.
James F. Evans (22:57:31) “Beware of the pick-six going the other way for a touchdown.”
Indeed, it is the linear-correlation-destroying phase-reversals that are most interesting, reminding us that “completions” do not have the same definition in all games.
Useful analogy – thank you.
This article basicly supposes data collection is a waste of time. No conclusions can be pulled from them anyway.
RR Kampen (04:18:18) :
“This article basicly supposes data collection is a waste of time. No conclusions can be pulled from them anyway.”
No – I think it is saying that any desired conclusion can be pulled from data.
Re : Jimmy Haigh (05:24:40) :
No – I think it is saying that any desired conclusion can be pulled from data.
That is why data collection is supposed to be a waste of time, of course.
Your remark is fortunately not true as it stands. It is only true that any desired conclusion can be pulled from data given abuse of statistics and statistical methods, and crooked or forged interpretation of these.
In fact this type of remark (or the infamous ‘there are lies, damned lies, statistics’) is characteristically made by people who simply do not understand statistics.
Speaking of models vs. data – has anyone noticed that the October update (of September hemispheric and global temperatures) over at CRU is, as has often been the case of late, very late? The general impression I’ve been getting is that the later the update is reported the more “exotic” we can expect the number to be (by which I mean, the earth will be running a particularly high fever). Also, has anyone else noticed that no matter how late the update is posted (possibly into November this month) that the “Update” is reported as sometime around mid-month?
Just curious…
George E. Smith – the only difference between and integral and an average is a multiplier – but what do I know, I’m only a chemist and a physicist.
It’s a dumb question, obviously, but can somebody tell me what ‘the AP’ is?
Funny how a group of uninvolved statisticians all arrived at the same conclusions, yet are vilified here. I can’t for the life of me understand how people can’t look at the data objectively, and continue to entrench themselves further with confirmation bias (in this case, ignoring substantive findings that conflict with their belief system).
Amazing what the human psyche will go through to maintain it’s version of reality…
Author: Jimmy Haigh
Comment:
RR Kampen (04:18:18) :
“This article basicly supposes data collection is a waste of time. No
conclusions can be pulled from them anyway.”
No – I think it is saying that any desired conclusion can be pulled from data.
That’s not true; but conclusions can often be stated floating free of the assumptions. The idea that the increase in temperature has ceased since 2000 cannot be falsified by performing a linear regression on all the data since 1987. Basically, you would be assuming that there has been no change in the slope; i.e., you are assuming that the Null Hypothesis is true and are simply trying to estimate what that slope is.
Hidden behind this assumption is the assumption that the slope is necessarily linear and constant.
This is not a property of statistics, but of the carelessness or tendentiousness of those using it.
Keep in mind, too, that statistics cannot prove anything; it can only disprove. You can show that the facts are incompatible with the hypothesis, and thus the hypothesis in not true (“true to the facts”); but you can never prove that the hypothesis is true.
We have here a failure to distinguish between interpolation and extrapolation. See Mark Twain, Life on the Mississippi, for details. Twain proves conclusively, by using the measured shortening of his river over the last 200 years, that at one time the Mississippi stretched out into the Gulf of Mexico like a fishing pole, while in a few thousand years New Orleans will be a suburb of Chicago. Just a simple linear mapping!
Only when the underlying mechanism is fully understood (which means full knowledge of the limitations of the model) can one meaningfully extrapolate.
Alas for statisticians. Statistics does not deal in certainties. Statistics only deals with the assignment of likelihood (for example, a 100 year flood plain). This does not mean that if one has gone 99 years without a flood, there will be a flood in year 100.
And significance tests only allow us to accept or reject a null hypothesis. We can assign a likelihood of being wrong on our decision. And that is all.
*****
We could accurately forecast the weather. The expense would be monumental. Simply consider the surface area of Earth and the number of data collection points needed for one per square kilometer. And one would need data collection for at least 10 altitude points above each point on the surface. Talk about petabytes! Talk about gigadollars! Never happen.
After all, we are talking about a thin film here.
Here’s what I said about this AP (Associated Press) story in the prior thread on the topic:
*******
Let’s parse that AP article:
“The statisticians, reviewing two sets of temperature data, found no trend of falling temperatures over time.“
Strawman. 2009 is warmer than 1979 and 1880. But the period between those two start points is not what skeptics have in mind by “over time.” They are referring to the most recent trend.
“And U.S. government figures show that the decade that ends in December will be the warmest in 130 years of record-keeping.”
Another technically correct pseudo-refutation. Since the first half of that period preceded heavy man-made CO2, and therefore warmed from another cause, it indicates there’s a non-anthropogenic component to the long-term warming trend—a component that could still be active. (I.e., the rebound from the LIA.)
“Global warming skeptics are basing their claims on an unusually hot year in 1998.”
Another strawman. Most skeptics (here on WUWT, anyway) don’t choose 1998 as their starting point. Instead, they claim it’s been cooling during the present century, or since 2002, or 2004.
“They say that since then, temperatures have fallen — thus, a cooling trend. But it’s not that simple.”
A red herring (diversion). It IS that simple, because a short-term flattening and cooling trend falsifies the IPCC’s prediction for this decade, casting doubt on its models’ reliability; because it casts doubt on the implacability (and the urgency of the threat) of CO2’s alleged “forcing”; and because the PDO has flattened and turned negative at about the same time, which suggests that the PDO is the climate “forcer,” not CO2.
**********
“when I asked him [Borenstein] why he felt it necessary to *make* news, and then report it, he answered that he was simply fact-checking against recent “internet memes””
The primary “meme” here on WUWT and CA has been that the globe has been cooling slightly for the past five years or so. Borenstein merely knocked down a strawman (a caricatured version of an opponent’s argument) by pointing out that the globe has not been cooling since 1880 and 1979. The fact that this obvious dissembling hasn’t been caught demonstrates the CAWGers lack of critical thought.
PS: Here’s the link to the Amazon thread on How to Lie with Statistics.
http://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/0393310728/ref=sr_1_1?ie=UTF8&s=books&qid=1256832335&sr=1-1
“”” Merrick (06:23:29) :
George E. Smith – the only difference between and integral and an average is a multiplier – but what do I know, I’m only a chemist and a physicist. “””
Well Merrick, let’s test your thesis with the often cited absurdity. Shall we place your legs in Dry ice at -80 deg C, and simultaneously put your head in superheated steam at 120 deg C. On average you are at a very comfortanble +20 deg C; but if we integrate (add up) the effects of the “weather events”, the results are quite uncomfortable; fatally so. So what multiplier would you suggest to equate the two situations.
Climate is the sum total of ALL of the weather events that have ever occurred.
Under the average conditions; there would be no weather at all.
“”” Mike the QE (16:37:03) :
@george Smith
The extremal average (M+m)/2 may not be an unbiased estimator if the distribution of values within the day is not symmetric. Even if it is, and even if it is approximately Gaussian*, it is still an inefficient estimator. If there are outliers in the daily data, due to measurement system error or actual +/- spikes, those outliers will be one or the other or both of the extreme values, and so will affect the extremal average, making the estimator subject to considerable variation.
If it is impractical to “integrate” over the entire series of daily temperatures — perhaps temperatures are not taken continuously — it would make more sense to take samples at random times of the day and compute a sample average and sample standard deviation. (Of course, there is the problem of serial correlation…) “””
Well Mike, the simplest situation would be if the daily temperature cycle at a single loaction were a pure sine wave. (M+m)/2 ; to use your terminology, would in fact be the correct average, and moreover would meet the Nyquist criterion for two samples per cycle of the highest signal frquency (in this case only one frequency). But in general exactly two samples per cycle (not M and m) is not sufficient to reconstruct the signal; it’s a degenerate case. For a non sinusoidal but periodic signal, there must be at least a second harmonic component or higher frequency present, so at least four samples per day would be required to at least get the average correct, although once again not to reconstruct the signal. Bear in mind that at exactly two samples per cycle, the samples would be phase locked to the waveform, and would never reveal the waveform; but a slightly higher sample frequency, would slue the sampling, and eventually capture the complete waveform.
But the real daily cycle is not sinusoidal so two samples per day even M and m cannot give a correct average. More importantly the radiated thermal emission, that is a consequence of the temperature, varies more as the 4th power of the temperature, so the daily integrated radiation is always greater than what is calculated from the average temperature; even if you do have the correct average temperature. It’s a small but not zero effect for the daily cycles; but is quite significant for the annual cycle.
No you don’t have to take the temperature every second or minute; but more often that twice daily M and m is worse than crude; and as I have said in the past; that completely ignores the effect of random cloud changes. Gaia does NOT ignore the effect of cloud changes on even the average temperature; let alone on the weather and climate.
sustainableloudoun (08:15:42) :
“Funny how a group of uninvolved statisticians all arrived at the same conclusions, yet are vilified here. I can’t for the life of me understand how people can’t look at the data objectively, and continue to entrench themselves further with confirmation bias (in this case, ignoring substantive findings that conflict with their belief system).
Amazing what the human psyche will go through to maintain it’s version of reality…”
We have seen the actual data, do we need unnamed “uninvolved statisticians” to tell us what we see? We see temperatures not following the predictions of the vaunted climate models. Skeptics favour the null hypothesis in the face of what could charitably be called insufficient evidence for AGW. One only has to look at the statistical atrocities committed by “confirmation bias” prone scientists their pathetic attempts to get rid of the medieval warming period. Appeals to authority get no traction here, skeptics have seem too many authorities disgrace themselves over the so called “settled science”.
@george
That’s pretty much what I said: the extremal average is unbiased under strict conditions not likely met by the distribution of temperatures during the day, and even if it were, it is extremely sensitive to outliers in the data. A proper daily average needs to take samples at multiple times during the day.
From the lack of response to my question, I judge that no-one here is much interested that the original post begins with “facts” that are trivially shown to be false. Do you not wonder what would motivate someone to mislead you so?
Briggs is spot on about statisticians’ love affair with their models. However I wish that he had gone into more detail. A few commentators on this thread have already mentioned the null hypothesis. In my opinion, that’s the elephant in the room. How so?
When we translate from Statisticalese into plain Engish, there’s not always a one-to-one correspondence. Take the word “no”. When a statistician says “no recent cooling trend”, what does he mean? Here’s my educated guess about the meaning of the word “no” in this context:
Null hypothesis: Global temperatures in recent years have not changed appreciably. Given the paucity of global temperature data over the last 11 years, and given the high noise level within that data set, we cannot reject the null hypothesis at the 95% confidence level.
If the statisticians in the AP story had been able to reject the null hypothesis with even 90% confidence, that would still not pass muster; 95% is the magic number. In other words, “no” does not always mean no.
Surely no capable statistician would agree to this analysis before knowing if the figures were truly independent of each other, or if a dependency existed between them. Different statistical tests are appropriate in each case.
Well, George E. Smith – let’s test your “hypothesis” – A head in the oven and feet in ice water. This defines an integral or an average how? If you’re a mathematician, as you claim, why don’t you use descriptions which actually apply?
Now, by definition, an integral is a sum. An average is a sum multiplied by the inverse of the number of elements in the sum. So the difference between a sum and an average is a multiplier. We can’t get into calculus if you want, but no matter how you slice it my statement is exactly and unquestionably true.
Your image is colorful, but it’s fairly meaningless.
Your complaint is that the (min+max)/2 definition of average temperature is at best inadequate and at worst ludicrous. I coudn’t agree more. Your argument was still poorly made, your counter to criticism was equally so. You immediately assumed I disagree with you, and I don’t. What I was criticizing was what I saw and still see as poor arguments.
An integral is a sum. An average is a weighted sum. Talking about heads in ovens and feet in ice water doesn’t change that.
“”” Merrick (16:14:54) :
Well, George E. Smith – let’s test your “hypothesis” – A head in the oven and feet in ice water. This defines an integral or an average how? If you’re a mathematician, as you claim, why don’t you use descriptions which actually apply?
Now, by definition, an integral is a sum. An average is a sum multiplied by the inverse of the number of elements in the sum. So the difference between a sum and an average is a multiplier. We can’t get into calculus if you want, but no matter how you slice it my statement is exactly and unquestionably true.
Your image is colorful, but it’s fairly meaningless. “”
Well Merrick, I give up; you clearly didn’t read what I said; and since you didn’t read that and understand it, then you didn’t get the point of my “frairly meaningless” example.
To reiterate; I said that climatologists (in their writings) define climate as the long term average of weather. Note the two terms climate and weather. Nowhere is the word “temperature” mentioned. That is their definition not mine.
And I asserted that I believe that climate is the integral of weather. Once again no mention of the word temperature.
Weather and climate encompass a whole lot more variables than simply temperature. And the planet as a whole reacts to the continous sum of THE EFFECTS of all those variables that change during the course of what we call weather.
I’m sure there are some aspects of both weather and climate that might react to the average of temperature; but what about the many effects of both climate and weather that are a direct result in DIFFERENCES of temperature from place to place and from time to time.
It is the integral of THE EFFECTS of weather that makes its mark on the planet.
So if you believe that the integral of weather (not temperature) is just a factor times the average of weather (not temperature); what value do you presently have for that constant (or varying) factor that relates the integral to the average.
And to revisit my fairly meaningless example; I quite deliberately chose the extremes that I used so that it was quite clear that the EFFECTS of some of the “weather” items; such as the superheated steam, and the dry ice were of such a nature that they would never be cancelled out by some other extreme weather event; to yield a benign average result.
If you want to believe that we can get the climate by simply multiplying the weather (averaged in some way of course) by a factor; well you are welcome to that point of view.
I wonder what the average of a Hurricane and a Tornado is, in climate science terms of course.
And as to my “credentials” which you see fit to impugn there is more than one bio to be found out there in webland. I didn’t write any of them; so they are probably about like what typical Journalists might write from information they get from various sources. I’m sure if you contacted my alma mater, you could easily get what are the public records; they aren’t secret.
“”” On average you are at a very comfortanble +20 deg C; but if we integrate (add up) the effects of the “weather events”, the results are quite uncomfortable; “””
There you are Merrick; a direct quotation from my post ; exactly what it was I said; that evidently you completely misunderstood. I was extremely careful in the choice of words that I used; I usually am.
It is not my fault if people substitute their own words for mine.
In “other words” lies “other meaning”. So I choose to not use “other words”.
Funny when Mann was attacked because his statistics wasn’t strong enough, this crowd crooned over statisticians. Now this crowd is vilifying them. Can’t have your cake and eat it too, especially when the decadal and interdecadal trends are still pointing up;
http://www.woodfortrees.org/plot/gistemp/mean:10/plot/gistemp/from:1940/trend/plot/gistemp/from:1999/trend/plot/gistemp/from:1994/trend/plot/gistemp/from:1989/trend
sustainableloudoun (18:57:17) wrote:
“this crowd crooned over statisticians. Now this crowd is vilifying them. Can’t have your cake and eat it too,”
That’s an inappropriate analogy. We skeptics are consistently applauding and deriding, respectively, the proper and improper use of statistics. (And we aren’t focusing primarily on the statisticians, but on the way their methodology can be used inappropriately.)
“the decadal and interdecadal trends are still pointing up;”
Sure, because the PDO has a 30-year cycle. Now it’s turning. Hit page-up 12 times and see my post that parses and critiques the statistics and argumentation used in the AP article. In particular, here’s its key point/counterpoint:
“They say that since then, temperatures have fallen — thus, a cooling trend. But it’s not that simple.”
That’s a red herring (diversion). It IS that simple, because a short-term flattening and cooling trend falsifies the IPCC’s prediction for this decade, casting doubt on its models’ reliability; because it casts doubt on the implacability (and the urgency of the threat) of CO2’s alleged “forcing”; and because the PDO has flattened and turned negative at about the same time, which suggests that the PDO is the climate “forcer,” not CO2.
And here’s what I posted as a follow-up to it on another thread on this topic:
“If a patient has a fever and the fever “breaks,” that breakage can’t be waved aside with the diversionary argument that the temperature decline hasn’t lasted long enough to be a long-term trend. No one is claiming it is a long-term trend–-just that the fever (most likely PDO-driven) has broken.”
Very good article by Dr. Briggs. As for you, sustainableloudoun, keep in mind that the criticisms of Mann have more to do with selectively chosen data and improper use of recognized statistical methods in order to draw his conclusions. The sum of all the criticisms in the posts and the observations of Dr. Briggs are not inconsistent with this analysis. Borenstein only gave these stat guys one part of the data, and probably knew these guys well enough to know what kinds of bees were in their bonnet. His article proves nothing other than what we already know – you can cherry pick anything you want to prove what you want to prove.
My basic argument has been, and remains, what is really happening as opposed to what the IPCC and its cronies have predicted would happen based upon their so-called “models?” The answer is that they haven’t even been close. That is the reason for all the “cooling” talk. As George E. Smith pointed out, what is worse is that the surface temperature data that the alarmist scientists use to try to bolster their arguments is highly questionable if not unreliable.
Further, let’s face facts: all climate predictions are shams. Talk of “trends” are useless, and it is time guys like you recognize this. These guys couldn’t predict tomorrow’s weather much less predict long-term climate “trends.”
sustainableloudon
Funny when Mann was attacked because his statistics wasn’t strong enough, this
crowd crooned over statisticians. Now this crowd is vilifying them. Can’t have
your cake and eat it too, especially when the decadal and interdecadal trends
are still pointing up
There is nothing in “being” a statistician that bestows wisdom. It lies in whether the statistical analysis is done properly. If the AP story correctly presents what the “four statisticians” said, there is a great deal missing from the account.
For example, was the only statistical analysis made a simple linear regression on the entire data set? Such an analysis implicitly assumes that the slope (if any) is constant. No one denies that the trend has been generally upward since the ending of the Little Ice Age. But the hypothesis to be tested is that the slope has changed in the past ten years or so. For that, a different sort of test is required.
However, the AP is remarkably silent on what was actually done. Moreover, if the statisticians were not told what the data was, how could one of them then make a statement about temperature trends? Unless they were told afterward. Left unstated is whether they found an upward trend in the past ten years of data and, based on past data, whether such a flat 10-year period is likely.