Guest essay by Douglas J. Keenan
Temperatures on Earth’s surface—i.e. where people live—are widely believed to provide evidence for global warming. Demonstrating that those temperatures actually provide evidence, though, requires doing statistical analysis.
All such statistical analyses of the temperatures that have been done so far are fatally flawed. Astoundingly, those flaws are effectively acknowledged in the IPCC Fifth Assessment Report (AR5).
The flaws imply that there is no demonstrated observational evidence that global temperatures have significantly increased (i.e. increased more than would be expected from natural climatic variation alone).
Despite this, one of the main conclusions of AR5 is that global temperatures have increased very significantly. That conclusion is based on analysis that AR5 itself acknowledges is fatally flawed. The correct conclusion is that there is no demonstrated observational evidence for global warming.
Full critique at http://www.informath.org/AR5stat.pdf
===========================================================
He closes with these quotes:
The reason for so much bad science is not that talent is rare, not at all; what is rare is character. People are not honest, they don’t admit their ignorance, and that is why they write such nonsense. — Sigmund Freud
Half the harm that is done in this world is due to people who want to feel important. They don’t mean to do harm — but the harm does not interest them. Or they do not see it, or they justify it because they are absorbed in the endless struggle to think well of themselves. — T.S. Eliot
You should, in science, believe logic and arguments,carefully drawn, and not authorities.—Richard Feynman
“The flaws imply that there is no demonstrated observational evidence that global temperatures have significantly increased.”
And Santer’s deadline for human attribution of the insignificant increases is knocking at the door.
I love the quotes.
They go a long ways towards explaining the “sheeple” behaviour we see in so many individuals, especially surrounding this topic.
Especially applicable is the end of the quote that says “…they justify it because they are absorbed in the endless struggle to think well of themselves.”
Monte Carlo methods use pseudo-random number generators to simulate random processes. Both Monte Carlo and random number generators have been around for many decades.
The Mersenne Twister is a particularly good random number generator which I have used myself, but I fail to see how it could have caused a revolution in Monte Carlo methods. Perfectly adequate random number generators were around long before the Mersenne Twister arrived.
“The correct conclusion is that there is no demonstrated observational evidence for global warming.”
Yes douglas there was no little ice age and its not warmer today.
In fact if you use Keenan’s approach you can show that you have no evidence for natural variability of any kind much less warming since 1750.
Douglas J. Keenan:
Thankyou for providing your superb summary and I congratulate you on its presentation in plain English.
I commend everyone to read it especially its Section 10 and I copy your link to here so as to assist people going to it
http://www.informath.org/AR5stat.pdf
Richard
Steve Mosher I did not know that the Little Ice Age was between the 4th assessment and fifth assessment report I thought it was much earlier.
” Steven Mosher says:
October 30, 2013 at 8:02 am
Yes douglas there was no little ice age and its not warmer today.”
Thus proving that there is no “Anthropogenic Global Warming”, it is completely [natural], which is of course what Doug meant, as you well know.
You are a very devious person.
Steven Mosher says:
October 30, 2013 at 8:02 am
I guess you ignored this from above.
“The flaws imply that there is no demonstrated observational evidence that global temperatures have significantly increased (i.e. increased more than would be expected from natural climatic variation alone).”
Thanks, Douglas.
This is a very good post and your “Statistical Analyses of Surface Temperatures in the IPCC Fifth Assessment Report” deserves to be read.
With some of the other models that were nearly as likely as the chosen model, the increase in temperatures is not significant. To summarize—some likely model
DRAFT ⁞ Page 17
shows the increase as significant and other likely models show the increase as significant. Hence, we cannot determine whether the increase is significant. The main conclusion of Breusch & Vahid, however—based on their choice of model—is that the increase is significant. The main conclusion is thus actually baseless.
I think one of the significants in the above paragraph should be ‘not significant’
Stephen Mosher: Thus amplifying a point that is commonly unobserved. We are in an interglacial period in which TEMPERATURES RISE UNTIL THEY DON’T. Then we have another ice age.
Much and appreciative thanks to WUWT for posting this.
@rabbit
My critique gives a link for that, which you could click on for more details. Briefly, the linear congruential generators that you, presumably, were using sometimes led to inaccurate results. The BlumBlumShub generator was available, but so slow that hardly anyone used it.
@Ceri Phipps
Kind thanks—fixed.
I have also been questioned privately about something, and I thought it was worth leaving the reply here.
The question is this: if almost all statistical analyses of climatic data are wrong, does that affect the work done by McIntyre & McKitrick on the hockey stick? The answer is “No”. The hockey stick was a temperature reconstruction spanning the past several centuries; it was derived via statistical analysis by Mann-Bradley-Hughes (MBH). In other words, MBH drew a statistical inference. MM showed that there was an error in the analysis that underpinned the drawing of the inference. MM did not have their own temperature reconstruction; i.e. they did not draw a statistical inference themselves. For that reason, the issues discussed in my critique do not apply to MM.
Since publishing on the error by MBH, McIntyre & McKitrick have often done similar work, i.e. showing that the inferences drawn by other researchers are based on flawed analyses. Occasionally, however, they have claimed to draw statistical inferences themselves. Here is an example.
Ross McKitrick, Stephen McIntyre, Chad Herman (2010), “Panel and multivariate methods for tests of trend equivalence in climate data series”, Atmospheric Science Letters, 11: 270–277. doi:10.1002/asl.290
Preprint: http://www.rossmckitrick.com/uploads/4/8/0/8/4808045/mmh_asl2010.pdf
The paper is based on assumptions that are wholly unjustified by the authors, and might well be wrong. Hence the claims of statistical inference in the paper are invalid. Note that understanding the invalidity does not require specialist training in statistics; the invalidity is more basic than that.
This is another example of the problems discussed in my critique. An underlying issue here, with McIntyre, McKitrick, and almost all climate scientists, is that authors appear to not understand what the question is. The question is this: what statistical model should be chosen? That is the question that must be answered, before statistical inferences can be drawn. This is basic.
Mosher: “and its not warmer today”
All of the 128 yellow lines are Septembers warmer than September 2013. Some all the way back to the 1660s.
http://sunshinehours.files.wordpress.com/2013/10/hadcet-since-1659-monthly-mean-sep.png
http://sunshinehours.wordpress.com/2013/10/03/hadcet-september-2013-only-128-years-were-warmer-2/
That’s rich: quoting Sigmund Freud about nonsense.
Superbly written overview of the statistical assumptions and problems in IPCC AR5. Required reading.
We don’t know what the null distribution (the distribution absent a CO2 effect) for climate variability is. This has been written repeatedly.
One way to guess at a null distribution would be to take the most recent ensemble of GCM outputs, compute their mean, then compute the difference between each realization and the ensemble mean. That variation has been shown to be huge. With that estimate, which credits the models more than they are really worth, the natural variation is so large that the CO2 “signal” over the last decades is indistinguishable from 0.
What is the correct (or best) null hypothesis for a particular problem? That question can not be answered by statistical analysis. Taking a particular “effect size” to be 0 is merely a frequent convenience, no more.
Notice, however, that if you take the GCM model mean as the null hypothesis, the data since the ensembles were run strongly reject that hypothesis: the model mean is not an accurate representation of the process generating the data. There is a question, as rgb at duke has written, whether the ensemble of model trajectories can be reasonably treated as a sample from any population. So a hypothesis that the data come from a process of which that mean is a reasonable representative may not be testable.
In regard to Vaughan Pratt’s model I wrote that if you know the correct CO2 signal then you can estimate the natural variability; and if you know the correct distribution of the natural variation (aka “noise”) then you can estimate the CO2 signal. On present data, there is no good reason to conclude that either of those is “known”.
Matthew R Marler says:
October 30, 2013 at 9:53 am
“One way to guess at a null distribution would be to take the most recent ensemble of GCM outputs, compute their mean, then compute the difference between each realization and the ensemble mean. That variation has been shown to be huge. With that estimate, which credits the models more than they are really worth, the natural variation is so large that the CO2 “signal” over the last decades is indistinguishable from 0.”
********
Very interesting comment. A rough estimate from the various climate model predictions between 1980 and 2025 gives a range of +/-0.75C over a 45 year period. That is comparable to the warming between 1950 and 2000 that has been IPCC-certified to be mostly due to anthro CO2 emissions.
Which of course means that news of IPCC’s dead-certainty has been greatly exaggerated.
Keenan just disappeared the LIA.. pretty funny.
Steven Mosher says:
October 30, 2013 at 8:02 am
“The correct conclusion is that there is no demonstrated observational evidence for global warming.”
Yes douglas there was no little ice age and its not warmer today.
In fact if you use Keenan’s approach you can show that you have no evidence for natural variability of any kind much less warming since 1750.
==================
Well, since the LIA ( NASA defines the term as a cold period between AD 1550 and AD 1850) is not in the range of the period examined by the IPCC (1880 – 2012) or the author, I don’t see how pointing out its existence offers anything useful about the paper.
I think that was the point of the paper. That there is no valid statistical model used so far (especially the linear models used predominantly) that allows for a valid conclusion that there is demonstrable observational evidence to be drawn about ANY trend in global surface temperatures up, down, or none. That is NOT saying that there isn’t one, nor that temperatures have not increased.
You need to show your work.
Typo in Excursus 3–“be” should (I think) be “by”:
Steven Mosher says:
October 30, 2013 at 8:02 am
The author deals only with HadCRU figures for 1880-2012, not with the earlier 19th & 18th centuries (let alone 17th & 16th), so says nothing about the LIA, for most of which time good instrumental temperature data are lacking. Proxies however show that the LIA cold phase was global. I’ve just been studying centennial-scale glacial advance & retreat on the scenic Villarrica volcano, Chile, for instance.
Dear Mr. Mosher – re: 10:41am — (eye roll)
It was in the FIRST instance, perhaps, a careless mistake made in your eagerness to refute Mr. Keenan… but after several other commenters clearly showed you your error, one can only conclude that you:
1. really do not understand; or
2. are impaired intellectually (perhaps, by pride) — no one unimpaired would intentionally make oneself look idiotic.
In case you would like to learn (even more, to prevent your succeeding in leading others astray), here is what Keenan said with my emphasis to point you to an accurate reading of his language:
I hope that, one day, Mr. Mosher, you will be free of the darkness that clouds your heart and/or mind and which so obviously impedes your thinking abilities. Instead of persisting in trying to think well of yourself, you would be wiser (and happier, no doubt) if you would strive to have those who are
honest and logical think well of you.
Hopefully,
Janice
Any data from desert regions that show night time temperatures are increasing?
Janice Moore says:
October 30, 2013 at 11:24 am
>>>>>>
You have just (partially) redeemed yourself….
I see two major issues with climate analysis that are basic to all further considerations.
1) Temperature data is the result of chaotic processes and can not be accurately predicted. This results in the prohibition of parametric analysis. Assuming observations drawn from today’s “population of temperatures” will be drawn from the same population tomorrow are false. Temperature has guaranteed black swan events – essentially 1% chance for a new record each year for any 100 year period. There are no many sided dice that encompass all possible temperatures.
2) ALL of the currently available data stinks ! Any rational analysis of current data sets will conclude that the errors are larger than any “average” trend. ALL current analysis have laughable estimates of ALL error types. There are no random samples. There are generally no replicates or an inadequate number of replicates. Data discontinuities are glossed over. Bullsh*t adjustments are de rigeur.
SUGGESTIONS (from a non-scientist)
Dear Dr. Keenan,
Given that the above excellent analysis was largely written to educate non-scientists, I suggest:
1. Page 9 – re: “… the 90%-confidence interval for the global temperature in that year: [−0.10 °C, −0.36 °C]… ” — a brief explanation of anomalies would be helpful (otherwise, the figures are undecipherable and meaningless to a non-scientist). Chapter 2.1 of Bob Tisdale’s new book Climate Models Fail is one source to link for an explanation of anomalies.
2. To prevent misunderstanding, I suggest reconciling the following two clauses:
“The flaws imply that there is no demonstrated observational evidence that global temperatures have significantly increased (i.e. increased more than would be expected from natural climatic variation alone).” (Executive Summary, p. 1)
With:
“Simply put, no one has yet presented valid statistical analysis of any observational data to show global warming is real. Moreover, that applies to any warming—whether attributable to humans or to nature.” (Sec. 7, p. 11)
3. “Having the confidence interval so far away from including 0… .” (p. 11) — Need to re-word to prevent non-scientist confusion.
4.
“… SPM ignores what is said in Chapter 10. It does that even though responsibility for the statistical analysis lies with Chapter 10.”Suggest: “… SPM ignores Chapter 10 even though per Chapter 2 the only valid statistical analysis is done in Chapter 10.”
5. p. 12 — correct for capitals in title: “paper is entitled “Statistical Models and the Global Temperature Record”.
6. minor vocabulary clarification: “tabled” – Does that mean to pose a question? In the U.S., it would mean to set aside and, possibly forever, ignore it. (Can’t Lord Donohue simply issue a subpoena and force the witness to appear and testify? — never mind my queries here if this paper is completely aimed at a U.K. audience — not important, in any event, to the fine discussion you present in your paper.)
7. While your bending over backward (on p. 16) to portray the climastrologists as innocently mistaken does credit to your good nature, it is so implausible (not a justified assumption, ahem!) that I would suggest you delete it. It is obvious to a layperson that these Ph.D.’s know good and well that what they are doing is wrong. You need not say so explicitly, but to toss a high-necked, full-length, flannel nightgown on a prostitute in an attempt to portray her as naive is a little ridiculous and, thus, detracts a little bit from an otherwise well-written expose.
8. p. 17 — capitals in title: ““Global Temperature Trends”
9. Fascinating reporting of the religious-like attachment of those who believe in Neo-Darwinism to a mistaken radiocarbon dating regime. You were treading on hallowed ground, there!
EXCELLENT REPORT.
Thank you for sharing it with us for free.
Your grateful student (and humble proof reader),
Janice
Note: I apologize if I have duplicated suggestions above — I did not take the time to review the comments written while I was writing this.
Hi, Geran. Re: my redeeming myself — huh?
Keenan is entirely correct in pointing out that the critical question of assumed time-series model strongly prejudices all statistical conclusions about “trend” drawn from short measurement records Those assumptions are typically made ex cathedra–on both sides of the AGW debate. Based upon model-free spectral analyses of actual meaurements as well as proxy data, I have long argued that linear-trend plus AR(1) models are simply inappropriate. What is fitted to the data by linear regression or by nonlinear methods cannot be taken as a SECULAR trend, persisting far beyond the time-frame of the records.
I must quibble, however, with the claim that Koutsoyiannis, who resorts to Hurst-Kolmogoroff persistence models married to the unproven notion of self-similarity at all time scales, advances our insight very far. The power density spectra almost invariably show the presence of strong oscillations over a very wide range of frequencies that are incompatible with simplistic H-K “red noise” spectra. Sadly, virtually none of the statisticians whose voices are repeatedly heard in “climate science” are truly adept at geophysical time-series analysis and modeling.
Mosher
Instead of carping from the sidelines, perhaps you can answer Doug’s central point, and tell us what evidence you have that any warming recorded is “statistically significant”.
We know you are much cleverer than the rest of us, so that should not be terribly difficult.
Janice Moore says:
October 30, 2013 at 2:20 pm
Hi, Geran. Re: my redeeming myself — huh?
>>>>>>
You’re on the “come back trail” from calling me a screaming monkey….
Paul Homewood says:
October 30, 2013 at 2:30 pm
Mosher
Instead of carping from the sidelines, perhaps you can answer Doug’s central point, and tell us what evidence you have that any warming recorded is “statistically significant”.
We know you are much cleverer than the rest of us, so that should not be terribly difficult.
>>>>>>>>>>
Paul, Mosh will not be returning any calls tonight. He has no evidence, only his personal wishes. He wishes so much that mankind was ruining the planet. He is angry because CO2 has gone up but we are not yet “boiled”. To him, his computer models cannot be wrong….
My dear Geran,
I believe I said that you were acting “like” a screaming chimpanzee.
#(:))
Did I just make it three feet farther along the “come back trail?”
I hope so.
Your truth in science ally,
Janice
Janice Moore says:
October 30, 2013 at 3:20 pm
….
Did I just make it three feet farther along the “come back trail?”
>>>>>>>
We’re talking centimeters here, but stay with the program. Rehab will set you free….
As an example of the long-term natural variability manifested by paleo data, see the power density of GISP2 del18O-isotope data at:
http://i1188.photobucket.com/albums/z410/skygram/graph1.jpg
P.S. Some day, i’ll have time to organize my photobucket account.
– – – – – – –
Douglas J. Keenan,
Thank you for making it clear; inappropriate statistical models were chosen in AR5 and the authors of AR5 admit it.
The time is fast approaching for a formal independent statistical expert panel to do a comprehensive audit in full public view. An audit with full access to all internal docs, communications and archives of their non-open forums. N’est ce pas?
John
I do wonder about IPCC’s claimed accuracy of global average temperatures, especially as listed by CRU to 3 decimal places. They rely on Jones et al and the √n rule. The surface stations project shows that at least 6% of these measurements have errors greater than 5 degrees C. Now this isn’t much better than putting a finger in the air. If we could get all 2,400,000,000 internet users to put their fingers in the air to estimate their local temperature, we should be able to calculate an average global temperature and get an error in the average of 0.001 degrees C. I’m not a statistician – what’s wrong here ?
Dear Geran,
I did not realize how deeply I had offended you by my exaggerated description of your comments on that thread. I have learned a lesson (heh, well, I hope I have) about hyperbolic and carelessly (and poorly) executed repartee. I am sorry for the hurt I have caused you. I was wrong.
Please, forgive me.
Your wiser, humbler (we can only hope), friend,
Janice
Janice, no harm done here.
“Words” don’t hurt me. I am not harmed, and you owned up to your loss of control. So, all is level. We skeptics fight amongst ourselves, but it only makes us stronger to fight the REAL enemies.
Now go get the Warmists, and take no prisoners! (They FEAR the TRUTH.)
Thanks, Geran.
Douglas, you have already been labelled as a “denier”, therefore your entire analysis is invalid; at least to the warmists and the great unwashed masses. Too bad too, because everybody needs to understand this issue.
Is anyone able to explain why Mosh isn’t happy that we’re not boiling ourselves to death?
I garnered from Met Office’s replies to Parliament that they chose the AR(1) model because everyone else was using it. In other words, they had no theoretical basis to their choice. Maybe I shouldn’t be, but I am somewhat surprised that the IPCC is clinging to first order autoregression for determining significance, especially after Met’s admitting it was unsuitable. Don’t they not learn from those behind the woodshed experiences?
Ugh… I meant “Don’t they learn from those behind the woodshed experiences?”
Steven Mosher: “The correct conclusion is that there is no demonstrated observational evidence for global warming.”
Yes, that was a mistake. Touche’
His other punchline is correct: The flaws imply that there is no demonstrated observational evidence that global temperatures have significantly increased (i.e. increased more than would be expected from natural climatic variation alone).
@ Janice Moore, October 30 at 2:15 pm
Thank you much and very kindly. About your first three points….
1. Good point—will amend.
2. The Executive Summary is intended to be a true executive summary; in particular, it is not an abstract. I am unskilled at writing something like this; so the Summary will be written jointly with someone else (Lord Donoughue). For now, I decided to include a rough version. That was perhaps a mistake, though, because you, and others, have found valid problems with it.
3. Thanks—will do.
@ 1sky1, October 30 at 2:26 pm
About the work of Koutsoyiannis, I agree that the model that he has found does not fit the data well; indeed, I had a brief conversation with Koutsoyiannis about that last year. The critique, though, does state that he has not found a viable model.
What makes the work of Koutsoyiannis especially important is that it provides the only example of attempting to use physics to constrain the set of candidate statistical models. Using physics to constrain the candidate models is an obvious strategy, but implementing that strategy is difficult. Virtually all other researchers who attempt to statistically model climatic time series constrain the candidate models by assuming that the models are analytically simple, e.g. by assuming linearity. That is wrong, but it allows those researchers to claim “results”. Koutsoyiannis has taken the correct, but much more difficult, path.
Dear Mr. Keenan,
You are so very welcome. And, thanks, for letting me know that I could be of some little help. Re: Koutsoyiannis, his work IS a good example for the point you were making; I guess you just need to make the purpose of citing K. a bit more clear for readers like 1sky. Of course, you can’t write a paper carefully enough for those who read it in haste. Just have to ignore the criticisms of such readers.
Best wishes for success with this, your latest endeavor on behalf of truth.
Yours,
Janice
Doug, what you write for me, both in your essay and in comments, is extremely flattering. Thank you very much.
Also, thank you Janice for your kind comment.
1sky1, I appreciate your criticism. Indeed, self-similarity at all time scales (using just one parameter) may be too simplifying. However, I find it extremely useful to derive simple and parsimonious models by using the simplest possible constraints. Then by adding more constraints, one may approximate reality better. Of course, there is much more to do than what is reported in a single paper to which Doug mostly refers. We are continuing our studies in this direction (in very tough conditions though—note, our university has been closed for the last couple of months due to substantial cuttings and firing of half of the personnel, and its future is uncertain). Anyhow, you may find additional stuff and replies to many comments in a recent discussion in http://www.climatedialogue.org/long-term-persistence-and-trend-significance/
Dear Professor Koutsoyannis,
I’m so sorry to hear that you are working under such tough conditions. A world-class scientist like you should have the finest that academia has to offer. You come from a land of heroes, who gave to the world great science, art, and literature…….. who fought for democracy………. who ran 14 kilometers in bare feet and cried,
“We have won!”
Just a little encouragement from an American admirer. An American man singing in a rather ordinary way, but with that optimism which has been and always will be the gift of America to the world. You — can — do — it!
Truth will win.
Take care,
Janice
Dear Janice, it’s a great feeling to hear from so far away that “You’ll never walk alone”; thank you indeed for this friendly encouragement.
Demetris
I just completed an analysis of a representative sample of the ~5,500 surface air temperature records used in the construction of the HadCRUT4 global time series, which is why I’m late in commenting.
I reviewed a total of 638 individual records with at least 50 years of data located between the high Arctic and the South Pole and found that 531 of them showed an overall warming trend. Coin-toss statistics show that the chances of getting 531 heads in 638 tosses are on the order of a trillion to one against, so obviously the coin is very heavily weighted.
Does this result confirm that the warming is statistically valid? I’ve been trying to think of a reason why it doesn’t but so far have had no success. Maybe someone else can come up with one.
D.J. Keenan:
Dr.K’s work may be the only example that you know from reading “climate science” literature that appeals to a physical principle (max. entropy) in choice of statistical model, but that is certainly not the only such case in geophysics. Entirely different models are used as a matter of course in wave problems than, say, in those of turbulent diffusion. The crucial difference is that each of those models has been verified and refined by extensive field experiments. By contrast, the H-K model, which has found some verification in hydrology, is patently inadequate for understanding the long-term behavior of terrestrial temperatures.
It should be apparent from the spectral graph I posted that quasi-millenial and trans-centennial oscillations dominate the GISP2 record, with multidecadal oscillations accounting for only ~1/3 of the total variance. The former thus introduce what are perceived as “trends” in records that are only ~100yrs long. No such oscillations can be accounted for by any H-K model, which produces the “red noise” model spectrum shown in the graph. Enough said.
Dr. K:
The discussion you reference is quite revealing, in that you express belief that there is no intrinsic difference between mathematics and physics. Having made the (often painful) transition from the former field to the latter many decades ago, I assure you that the difference is categorical. If I had time, I could supply quotations from great physicists to that effect. For reasons sketched above, the H-K formalism, which may be useful in physical processes governed entirely by first-order DEs, will provide misleading results when second-order, oscillation producing processes are at work. Although the drivers for these oscillations are not clear, their presence in empirical records is unmistakable. This is especially true if one stays away from manufactured or corrupted indices that de-emphasize such oscillations at the expense of LTP-increasing “trends.”
Dear 1sky1,
I can tell from reading and comparing both the discussion thread linked by Dr. Koutsoyiannis and your comment at 5:27pm today that either you misread what Koutsoyiannis wrote in the linked thread, or you are deliberately mischaracterizing what he said there. I cite as evidence for your error another commenter’s (Bart Verheggen, who was arguing vigorously with Koutsoyiannis in that thread) response to the math-physics dichotomy issue Koutsoyiannis discussed in that thread:
B. Verheggen.
Before I can take seriously anything you write from now on (and I WANT to — you are a bright, articulate, obviously well-educated person), 1sky1, your credibility will need some rehabilitation. I would LOVE to hear from you that you simply misread what Koutsoyiannis wrote.
With high hopes that all is truly well,
Janice
Dear Demetris,
You are so very welcome. And, thank you for the privilege of using your first name. Hang in there.
I hope you spend a LOT more time with your grandchildren than with people on threads like the one you linked above and this one. Colleagues are great, but family is what makes life worth living.
God bless you,
Janice
1sky1,
What you can assure me is about your own feelings, painful or not, about mathematics and physics. These do not necessarily represent an objective truth.
Mathematics has helped to try to be accurate in what I am saying. In fact you misquote me as I did not say “there is no intrinsic difference between mathematics and physics”. Rather, I said “I think the language of physics is (or at least includes) mathematics”. Since you are invoking quotations, here is one relevant: “[The] great book which ever lies before our eyes — I mean the universe — […] is written in the mathematical language (Galileo).” By the way, when you mention first-order DEs and second-order, oscillation producing processes, aren’t these DEs mathematical objects?
For combining oscillating behaviours with HK see “Climatic variability over time scales spanning nine orders of magnitude: Connecting Milankovitch cycles with Hurst–Kolmogorov dynamics”, http://itia.ntua.gr/ 1297/
Otherwise, I am afraid I have been off topic and distracted the discussion, so I stop here. So, feel free to misquote me again—I don’t think I’ll reply 🙂
Corrected link: http://itia.ntua.gr/1297/
Janice, thanks for your support. Also, thanks for your suggestion–indeed sometimes I forget the importance of family and focus on minor things,
Demetris
Roger Andrews: “Does this result confirm that the warming is statistically valid?”
in the absence of human interference is given by 
, where 
is a zero-mean sequence of independent identically distributed random values of a certain (e.g., unity-standard-deviation Gaussian) distribution and 
and 
are functions you specify. If you had done that and then asked whether the chances of seeing a trend as high as you saw exceeded one in twenty, I would have understood the question. But I don’t understand “significantly valid.”
What do you mean by “statistically valid”? (And I could just as well have asked Mr. Keenan–or any of the other adepts who’ve made similar pronouncements over the years–what he meant by “significant.” Being a statistics naif, I have heretofore kept silent in these discussions, but my frustration at what seems a total lack of clarity has finally reached the breaking point.)
Now, you might instead have postulated that temperature values are generated by a random process such that the (temperature) output
Thank you for the thread and your paper regarding AR5 Mr Keenan. It must be extremely frustrating to have to work so hard just to get people to see things correctly when you have demonstrated the facts. I certainly hope you and Lord Donoughue succeed in getting Dr. Slingo and others to respond to your questions because if you do it will remove one of the piles the AGW edifice has been constructed on i.e. the climate has and still continues to warm in an unnatural way.
@Joe Born . . my understanding of significance in statistics is that the null hypothesis has been breached. The null hypothesis being a trend line, coefficient, that is at or very near zero.
Keitho:
At November 1, 2013 at 10:38 am you say
Oh dear, sorry, but no.
I will explain the matter again.
The Null Hypothesis says it must be assumed a system has not experienced a change unless there is evidence of a change.
The Null Hypothesis is a fundamental scientific principle and forms the basis of all scientific understanding, investigation and interpretation. Indeed, it is the basic principle of experimental procedure where an input to a system is altered to discern a change: if the system is not observed to respond to the alteration then it has to be assumed the system did not respond to the alteration.
In the case of climate science there is a hypothesis that increased greenhouse gases (GHGs, notably CO2) in the air will increase global temperature. There are good reasons to suppose this hypothesis may be true, but the Null Hypothesis says it must be assumed the GHG changes have no effect unless and until increased GHGs are observed to increase global temperature. That is what the scientific method decrees. It does not matter how certain some people may be that the hypothesis is right because observation of reality (i.e. empiricism) trumps all opinions.
Please note that the Null Hypothesis is a hypothesis which exists to be refuted by empirical observation. It is a rejection of the scientific method to assert that one can “choose” any subjective Null Hypothesis one likes. There is only one Null Hypothesis: i.e. it has to be assumed a system has not changed unless it is observed that the system has changed.
However, deciding a method which would discern a change may require a detailed statistical specification.
In the case of global climate no unprecedented climate behaviours are observed so the Null Hypothesis decrees that the climate system has not changed.
Importantly, an effect may be real but not overcome the Null Hypothesis because it is too trivial for the effect to be observable. Human activities have some effect on global temperature for several reasons. An example of an anthropogenic effect on global temperature is the urban heat island (UHI). Cities are warmer than the land around them, so cities cause some warming. But the temperature rise from cities is too small to be detected when averaged over the entire surface of the planet, although this global warming from cities can be estimated by measuring the warming of all cities and their areas.
Clearly, the Null Hypothesis decrees that UHI is not affecting global temperature although there are good reasons to think UHI has some effect. Similarly, it is very probable that AGW from GHG emissions are too trivial to have observable effects.
The feedbacks in the climate system are negative and, therefore, any effect of increased CO2 will be probably too small to discern because natural climate variability is much, much larger. This concurs with the empirically determined values of low climate sensitivity.
Empirical – n.b. not model-derived – determinations indicate climate sensitivity is less than 1.0°C for a doubling of atmospheric CO2 equivalent. This is indicated by the studies of
Idso from surface measurements
http://www.warwickhughes.com/papers/Idso_CR_1998.pdf
and Lindzen & Choi from ERBE satellite data
http://www.drroyspencer.com/Lindzen-and-Choi-GRL-2009.pdf
and Gregory from balloon radiosonde data
http://www.friendsofscience.org/assets/documents/OLR&NGF_June2011.pdf
Indeed, because climate sensitivity is less than 1.0°C for a doubling of CO2 equivalent, it is physically impossible for the man-made global warming to be large enough to be detected (just as the global warming from UHI is too small to be detected). If something exists but is too small to be detected then it only has an abstract existence; it does not have a discernible existence that has effects (observation of the effects would be its detection).
To date there are no discernible effects of AGW. Hence, the Null Hypothesis decrees that AGW does not affect global climate to a discernible degree. That is the ONLY scientific conclusion possible at present.
Richard
Thank you for that full and detailed explanation Richard, I am much obliged.
I was simply referencing the meaning in a purely statistical sense.
Joe Born:
Okay, let me rephrase the statement in a form that a “statistics naif” – which you clearly aren’t – can understand.
“Does this result confirm that the warming is REAL?”
As I said, I’m having difficulty in coming up with a “no it doesn’t” answer.
Thanks to Keitho and richardscourtney for the attempts to respond to my inquiry. Particularly since such inquiries from us plodders are usually ignored, I’m reluctant to exhibit such ill grace as to admit explicitly that those attempts left me none the wiser. Still . . .
It seems to me as a matter of logic that unless the data’s probability given the null hypothesis is zero the data do not disprove that hypothesis. In the unlikely event that my logic is sound, “significance” would at best be a measure of how close to zero the data place that probability. So I’m having trouble warming to Keitho’s proposition that significance means that the null hypothesis has been breached–at least if “breached” means disproved.
In any event, I suppose my real difficulty in most such discussions is getting a solid purchase on what the “significance” user considers the null hypothesis actually to be. I am thankful in this regard for the benefit of richardscourtney’s thoughts–and not eager to question them since on so many matters of definition he seems more certain than I am on any. Indeed, his identification of the null hypothesis here as that man has not affected global average temperature does indeed seem tacitly to be implied by many such users.
But if there is indeed some probability threshold that they use to decide whether their data are significant, I have been unable to infer their mathematical basis for calculating the probabilities against which the threshold is compared and what connection that basis has to the hypothesis that man has caused no warming.
I suppose that this is what Mr. Keenan is driving at, and I join others in commending him for his efforts. Unfortunately, though, this returns me to Roger Andrews’ question that gave rise to my previous comment. Mr. Andrews noted his difficulty in avoiding the conclusion that global average temperature has exhibited a positive trend over some (unless I missed it, unspecified) time interval. Modulo the irregularities in the temperature records’ maintenance, I share that difficulty, But I suspect that Mr. Andrews’ question arose from comments such as this from Mr. Keenan: “Simply put, no one has yet presented valid statistical analysis of any observational data to show global warming is real.”
Now, perhaps Mr. Keenan did not mean that literally. Indeed, Ms. Moore and Mr. Marler seem to imply so, and there is some basis for that conclusion, including the excerpts in Ms. Moore’s second suggestion and Mr. Keenan’s response to that suggestion.
But Mr. Keenan said it more than once, and he has no Jay Carney-type exegete to place a more-congenial gloss on his writings. So I remain unsure of just what he means.
Joe Born:
Both Keitho and I tried to explain the null hypothesis to you but – at November 1, 2013 at 12:57 pm – you say you remain “none the wiser”. This leaves me at a loss because I do not know how to state the Null Hypothesis more clearly than
The Null Hypothesis says it must be assumed a system has not experienced a change unless there is evidence of a change.
If you can provide evidence of AGW having made any change to the behaviour of the climate system then I assure you that you will obtain at least one Nobel Prize. This is because 3 decades of research conducted worldwide at a cost of more than $5 billion per year has failed to find any such evidence. Ben Santer claimed to have found such evidence but that was soon shown to be false (he deliberately cherry picked a part of a data series).
You quote Douglas J. Keenan as saying, “Simply put, no one has yet presented valid statistical analysis of any observational data to show global warming is real”. In his above essay he actually says
He is simply correct when he says
“The correct conclusion is that there is no demonstrated observational evidence for global warming.”
because
the IPCC itself says the statistical analyses it uses are fatally flawed.
The clearest demonstration of that Douglas Keenan is right is that the IPCC says it is 95% certain the major cause of the recent warming is anthropogenic. If the IPCC had evidence – statistical or otherwise – for AGW then they would not say they are 95% certain of it.
People say gravity exists.
People say DNA contains genetic information.
People say an action has an equal and opposite reaction.
The knowledge of those things is provided by observational evidence, so nobody says they are X% certain of those things: people say them.
Simply, there is no empirical evidence that AGW exists; none, zilch, nada.
There may be effects of AGW but – if so – they are so trivial that they cannot be distinguished from previously experienced natural climate variability. So, as I explained to you
To date there are no discernible effects of AGW. Hence, the Null Hypothesis decrees that AGW does not affect global climate to a discernible degree. That is the ONLY scientific conclusion possible at present.
Richard
Roger Andrews:
re your post at November 1, 2013 at 11:11 am.
Nobody disputes that there has been warming from the Little Ice AGE (LIA). The warming from the LIA started centuries before there could have been any significant AGW. At issue is whether AGW has contributed to that warming, and there is no evidence of any kind that AGW has contributed to it.
Warming is not evidence of the cause of warming.
Richard
Richard:
You would be perfectly correct if Keenan had confined himself to AGW, but he doesn’t. He states that we have no proof that there’s been ANY warming. The relevant quote reads “Simply put, no one has yet presented valid statistical analysis of any observational data to show global warming is real. Moreover, that applies to any warming—whether attributable to humans or to nature”. He more or less confirms this by adding the Figure 5 time series plots, which he claims show that all of the observed warming could be simply a random effect.
Well, it can’t. To obtain representative results Keenan should have combined thousands of random runs, one for each of the individual surface station records that contribute to the global time series. Had he done this the chances that any one of his “models” would have replicated the observed warming would have been at least a trillion to one against according to my results.
Roger Andrews:
I take your point in your post at November 1, 2013 at 3:29 pm, and I thank you for your correction of my misunderstanding.
However, your correction of me raises two issues.
Firstly, as I now understand that you and I agree, there has been warming over e.g. the twentieth century but we make no comment on the cause(s) of it. However, the evidence for that is various (e.g. your analysis of individual station records, predominantly retreat of glaciers, etc.).
Secondly, that does not alter the validity of Keenan’s point about the time series analyses of global temperature not providing valid statistically significant indication of that warming. Indeed, it raises the importance of his point.
Again, thankyou for correcting my misunderstanding.
Richard
RIchard: No problems with your first issue. I agree there has indeed been warming over the 20th century and as far as this discussion is concerned we needn’t specify the cause of it.
But I’m a little confused on your second issue, where you seem to imply that while there has indeed been warming it isn’t statistically significant. Is this in fact what you are saying, or have I misunderstood you?
Roger Andrews:
I apologise for my lack of clarity. It was not intentional: I always try to be clear often to the point of bluntness. But at November 1, 2013 at 4:26 pm you say to me
No, actually I was expanding on a point which you made in the post I was answering.
You wrote
Perhaps, but Keenan’s paper is not discussing that. His paper discusses the compilations of global temperature time series provided by i.e. HadCRU, GISS, etc. and concludes that it is not possible to validly determine warming by statistical analyses of those compilations.
As I understand your comment which I here quote, you agree with him.
You are saying that “individual surface station records” (n.b. NOT the compilations of global temperature time series provided by i.e. HadCRU, GISS, etc.) should be used as the basis of an analysis to determine warming. So, I wrote my paragraph which you query; i.e.
As I see it, your argument “raises the importance of his point” for two reasons; i.e.
(a) as you seem to be saying, the time series analyses of GLOBAL temperature are not the appropriate data for determining a statistically valid significant indication of global warming,
and
(b) the the time series analyses of GLOBAL temperature have little – if any – practical value when they cannot provide a statistically valid significant indication of the global warming which you and I agree has happened.
I hope my meaning is now clear. And I assure that my lack of clarity was not intentional.
Richard
Dr. K:
Pray tell, where have I ever directly quoted you? In the context of my
ongoing critique about your lack of adequate empirical verification for “HK
dynamics,” I simply abstracted your ultra-academic view that “there is no
dichotomy between physics and statistics” and that “statistics is physics”
[stated on May 1 at 8:07pm in your linked discussion]. Nor is my
parenthetical remark about the “often painful” transition from mathematics
to physics an expression of personal “emotion,” as you would have it.
It merely ecapsulates Einstein’s humble recognition that a single
experiment can prove an entire theory wrong. Feynman put this issue much
more pungently: “Physics is to mathematics, as sex is to masturbation.”
The H-K formalism is plainly a kinematic, rather than dynamic, methodology.
It expresses no universal law. Its applicabilty to any physical problem
needs to be verified empirically on a case-by-case basis, instead of being
assumed as a given. Your novel “climacogram” effectively wrings most
information of possible physical interest out of a motley lot of proxy
time-series in reducing everything to a single variance metric. And then
the results are plotted on a log-scale, which always visually diminishes any
discrepancies. The results may impress novices, but it scarcely constitutes
geophysical verification.
That you may not respond further here is not unexpected. Only crickets were
heard when, a few years ago at Climate Audit, I numerically posted the acf
of the aggregate average of a geographically representative set of USA
stations, which contradicted all H-K expectations of functional form.
richardscourtney
Thank you for your response.. My reading of it is that you have conflated global warming with anthropogenic global warming–and that is my problem with Mr. Keenan’s paper. But your answer to Mr. Andrews suggests that you may have recognized that error. Although I was and still am thankful for Mr. Keenan’s efforts, which have clarified some things for me, his paper does not make it clear that he is distinguishing the two concepts.
As to your definition of what a null hypothesis, I had not intended to ask for elucidation. Truly, I appreciate your setting it in bold face so that we less gifted observers would be more likely to grasp it. However, I had already known what you thought the definition was; I am just unable to bring myself to accept it. I have identified no reason in your remarks not to consider a null hypothesis simply to be any assumption upon which one could base a calculation of observed data’s probability.
Again, though, thank you for the attention.
Dear Janice Moore:
I regret your impressions. But there are nuances in a highly technical discussion that are often missed by those unfamiliar with the science. Please read my response to Dr. K above. Let’s all enjoy the weekend.
Joe Born:
In your post at November 1, 2013 at 5:06 pm you say to me
I find that strange because in my “remarks” at November 1, 2013 at 10:54 am I wrote
In other words, the reason one cannot “consider a null hypothesis simply to be any assumption upon which one could base a calculation of observed data’s probability” is because that would be a rejection of the scientific method.
Richard
@ 1sky1 (re: your 5:10 sneer) — With dismay, I read your response to Dr. Koutsoyiannis (at 5:03pm this evening); sadly, you only confirmed my bad impression of your character for veracity. And your attempt to prevaricate by quibbling over whether you directly quoted Koutsoyiannis or not is disgusting. Since I won’t know whether what you say is accurate or not, I will be ignoring anything you have to say from now on — what would be the point? Why should ANY of us here now take you seriously?
Indeed, “there are nuances…,” your mischaracterization of what Koutsoyiannis said, however, was blatantly obvious.
No WONDER you don’t reveal your real name, here.
Please know, 1sky1, that you are not disgusting; your behavior is. You are better than that.
You are loved. And you are worth redeeming.
I, even now, have hope for you.
Even if you snarl at me in response to this,
there is always — hope.
Janice
Mr. Keenan, read that pdf start to finish and learned a bit about assumptions that I never realized, that assumption are so tightly coupled to statistical analysis and MODELS!. Nice work.
Thank you Doug Keenan for a clear, well written article.
I was especially interested in your paper on calibrated and uncalibrated C14dates. It would be interesting to know the error ranges between the two,if, as you say, they increase in time, 40-50 thousands years back starts to look very wobbly..
I can understand your criticism of C14dating methods being unpopular, but, good on you for persistence.
It seems tree rings do have their uses !
richardscourtney:
‘s are independent values identically distributed randomly with a zero-mean distribution of known variance, the 
‘s are known coefficients, and the process is subject to boundary conditions in accordance with which the average trend over a large number of runs tends to zero.
Having in other contexts been in your current shoes, I sympathize with your frustration at being unable to bring us unwashed to enlightenment. Still, I am unable to give you the satisfaction that appearing edified would have afforded, and I doubt that any further effort on your part will result in any greater return; I fear your explanation’s level of abstraction exceeds that with which I can cope consistently.
Just to let you know that I did not dismiss your input lightly, though, I’ll describe the hurdle I am unable to surmount. Let’s say that Mr. Keenan had assumed for the sake of argument that the annual-global-average temperature series resulted from, say, a process such as the following:
where the
This (arguably arbitrary) assumption is what this untutored observer would consider a null hypothesis; from such an assumption, I am told, statisticians can compute the probability that an $n$-length series generated by such a process will, despite the process’s average-trend value of zero, exhibit a trend at least as high as that of some $n$-length data sequence of interest, such as the global-average temperature record. Although I can’t profess to understand how all that is done, I can accept that it probably can be, and I can (with reservations) accept that statisticians are wont in those circumstances to pronounce a trend significant if the thereby-computed probability is less than 0.05. Moreover, I would be open to an argument based on a higher probability that drawing inferences from the observed trend is unjustified.
But for me it’s a bridge too far to accept the proposition that repeatable probability calculations can be based reliably on so impressionistic a formulation as that “a system has not changed unless it is observed that the system has changed.” A result, no doubt, of my inability to think as abstractly as you, but, well, there it is.
Nonetheless, I thank you again for your input.
Joe Born:
I am replying to your post addressed to me at November 2, 2013 at 6:58 am.
You rightly observe that I adhere to the principles established at the Enlightenment. I don’t care if you are “unwashed” or not, but you correctly observe that I suffer “frustration” when confronted with people who reject enlightenment thought. And that brings me to the problem your post provides to me.
My problem is that I adhere to the scientific method and, therefore, I lack your ability to invent a new scientific method as and when it suites me. Indeed, I flatly refute any such invention unless it is fully justified and certainly when – as in your case – it is simply asserted for convenience.
The scientific method defines the Null Hypothesis. As I said in my post above at November 1, 2013 at 10:54 am
Enlightenment thought as expressed by the scientific method has provided great benefits, and I am not willing to claim I can supplant it with anything better. Also, I reject your unsubstantiated assertions that you can improve on it.
In the specific case of Keenan’s statistical analysis of GASTA time series, the statistical Null would be ‘no change in temperature anomally’. Either a statistical model of the data can determine a change or it cannot. And if it can determine such a change then that determination is only valid to the degree of its validly calculated statistical significance.
Importantly, if the analysis does not disconfirm that statistical null then that disconfirmation does not mean GASTA has not changed: it only means the available statistical methods to assess the evaluated time series cannot discern a change. Hence, the null hypothesis has not been overcome by the statistical analysis.
As I also said in my explanation at November 1, 2013 at 10:54 am
In the case of a change to GASTA there may be “empirical observation” which shows a change whether or not statistical analyses of the GASTA time series can determine it. And, as I agreed with Roger Andrews in my clarification to him at November 1, 2013 at 4:53 pm, I think there is such “empirical observation”.
None of this is – as you claim – a “level of abstraction”. It is all completely practical.
And that is why I am unwilling to consider your request that “Let’s say that Mr. Keenan had assumed for the sake of argument that the annual-global-average temperature series resulted from, say, a process such as the following: etc.”: that assumption is a “level of abstraction” and I am dealing with the practical issues which are expressed by the Null Hypothesis.
Which, of course, does not deny my true statement that “deciding a method which would discern a change may require a detailed statistical specification”. Any such specification is open to debate.
In conclusion, I thank you for this conversation.
Richard
Janice Moore says:
October 30, 2013 at 11:24 am
Dear Mr. Mosher – re: 10:41am — (eye roll)
It was in the FIRST instance, perhaps, a careless mistake made in your eagerness to refute Mr. Keenan… but after several other commenters clearly showed you your error, one can only conclude that you:
1. really do not understand; or
2. are impaired intellectually (perhaps, by pride) — no one unimpaired would intentionally make oneself look idiotic.
In case you would like to learn (even more, to prevent your succeeding in leading others astray), here is what Keenan said with my emphasis to point you to an accurate reading of his language:
… no demonstrated observational evidence that global temperatures have significantly increased (i.e. increased more than would be expected from natural climatic variation alone).
I hope that, one day, Mr. Mosher, you will be free of the darkness that clouds your heart and/or mind and which so obviously impedes your thinking abilities. Instead of persisting in trying to think well of yourself, you would be wiser (and happier, no doubt) if you would strive to have those who are
honest and logical think well of you.
Hopefully,
Janice
++++++++++++
Well (and thoughtfully) written. Mosher’s fly by shootings are merely a source of tiresome distraction to otherwise interesting reading here at WUWT. Kudos to you for not being as lazy as I often get. I find him tiring when I’m trying to get to the meat of the matter here.
Mario
Janice Moore:
The nuance that continues to elude you is the dichotomy between physics as a
mental construct and physics as the concrete manifestions of nature. The
instrumentality of mathematics in human comprehension is indisputable. But
nature operates majestically without any recourse to it. Only in a poetic
sense does nature speak to us through that language. First-rate physicists
have long recognized that academic theory is not enough. If reliable
empirical data contradict the theory, it is wrong.
Dr. K’s didactic list of dichotomies offers precious little such
recognition. And it virtually ignores the crucial issue of climate data
reliability. In fact, based upon misguided notions prevalent in “climate
science,” it draws the uncircumspect conclusion that there is no real
dichotomy between signal and noise.
What you fail to realize is that, throughout this thread, I was directing
attention of scientific readers not to Dr. K’s words as such, but to the
evidentiary requirements for sound science. Nowhere have I knowingly
misrepresented his views. Your contrary verbal impression is, perhaps,
understandable, but the “evidence” you present is illogical. All it shows
is Bart Verhaggens concurrence on the instrumentality of mathematics.
Your irrational attack on my personal integrity launched on such flimsy
grounds is reprehensible. You owe me, and other targets of your
self-indulgent ad hominems, an apology.
P.S. Personal identity may be paramount on Facebook, but should not be a
factor in any serious scientific discussion. FYI, it was Gravitar that
stuck my e-mail account with the odd WUWT log-in monicker. Internet
anonimity is not shameful hiding.
Joe Born:
In signal analysis there is a well-developed concept of detection threshold. But this requires some knowlege of the signal spectrum and employs advanced analysis techniques unfamiliair to most classical statisticians. The upshot is that the current climate state is far from reaching any such threshold that would indicate the presence of an AGW signal.
Richard Telford’s blog has written about this paper. See his comments below. He says the IPCC’s statistical model is “reasonable for the question being asked.” I wonder if Keenan has read the post or has anything to say about it?