Ross McKittrick writes via email:
A UK-based math buff and former investment analyst named Douglas Keenan has posted an intriguing comment on the internet. He takes the view that global temperature series are dominated by randomness and contain no trend, and that existing analyses supposedly showing a significant trend are wrong. He states:
There have been many claims of observational evidence for global-warming alarmism. I have argued that all such claims rely on invalid statistical analyses. Some people, though, have asserted that the analyses are valid. Those people assert, in particular, that they can determine, via statistical analysis, whether global temperatures are increasing more that would be reasonably expected by random natural variation. Those people do not present any counter to my argument, but they make their assertions anyway.
In response to that, I am sponsoring a contest: the prize is $100 000. In essence, the prize will be awarded to anyone who can demonstrate, via statistical analysis, that the increase in global temperatures is probably not due to random natural variation.
He would like such people to substantiate their claim to be able to identify trends. To this end he has posted a file of 1000 time series, some with trends and some without. And…
A prize of $100 000 (one hundred thousand U.S. dollars) will be awarded to the first person, or group of people, who correctly identifies at least 900 series: i.e. which series were generated by a trendless process and which were generated by a trending process.
You have until 30 November 2016 or until someone wins the contest. Each entry costs $10; this is being done to inhibit non-serious entries.
Good luck!
Details here: http://www.informath.org/Contest1000.htm
When Doug Keenan talks of “random natural variation,” he seems to have a very restrictive AR(1) “random walk” in mind. Such Markov processes have power densities that decline monotonically with frequency . In fact, natural variation of temperature manifests much more structured spectral characteristics, with pronounced peaks and valleys. These can produce far more systematic rises and falls over limited time-spans, which are mistaken as “linear trends” by signal analysis amateurs. The trick to Keenan’s challenge is the very limited time-span of his simulations of random walks coupled with the introduction of non-linear artificial trends. His money thus seems to be safe.
Your first sentence is not correct. With most of the rest, though, I agree.
Doug:
It’s the random walks illustrated in your figure that prompted my speculation. Random natural variation is far better illustrated by a Gaussian random phase process, which does not produce the appearance of secular divergence from a common starting point, as in AR(1) diffusion.
Are they mistaken as short-lived linear trends, or are they actually short term linear trends? You appear to be claiming that there is some process which leads to short term linear trends but then chastising “amateurs” for finding them and saying: “Look, there is a short term linear trend here!”.
A linear trend can always be fit to ANY time series, including arbitrary stretches of totally trendless random processes of wide variety. This computational exercise reveals virtually nothing, however, about the inherent structural characteristics of the process. It’s the lack of recognition of these analytic truths that betrays amateurs, who fall prey to the presumption that the obtained “trend” is an inherent property, instead of a mere phenomenological feature, of the process. Linear regression simply is not an incisive time-series analysis tool.
1sky1: “Linear regression simply is not an incisive time-series analysis tool.”
Aside from the fact that linear regression produces a straight line – a phenomenon almost totally unknown in nature – from what are generally clearly cyclic processes.
In climate “science”, a suitable portion of the cyclic function is carefully cherry-picked, the resulting linear regression line is then extrapolated to Armageddon, and we all know what happens next.
catweazle666:
Indeed, as an analytic example, sin(x) is well approximated, for small x, simply by x. The R^2 of the fitted linear “trend” then can be made arbitrarily close to unity with a short-enough records.
To Janice Moore and John Whitman
I think Keenan would not reveal if he could solve the puzzle, and i tend to believe it is not possible, but maybe. I am not going to try, my skills are not up to that, but the quest is intriguing.
Is it a time series or not, and what is the difference? I don’t see any difference, any random series of numbers could be a time series or several or just random numbers whith some filtering/processing.
Electrical “white” noise is a good analogy (for me at least). If you sample it faster than the bandwith, each sample depends a little on the earlier samples and a little on “real” noise, but noise it is.
Noise is what you can not foresee, wether it is lack of knowledge or understanding.
The temperature of the real world is not a single lowpass filter on white noise, but the combination of a lot of filters on a lot of drivers with their own filters with vastly varying timescales, and then comes the interdepencies. Climate in a nutshell.
In digital communication you define the signal to noise as the energy in each bit relative to kT. The bit could last for micoseconds or years, but the relation holds. It is all about energy.
I know of some tests for chaos, and different temperature series have been found to excibit chaos at any timescale up to the lenght of the series.
I really like the idea to give Mann a free trial with his amazing software. 🙂
The prize goes to:
The Annals of Applied Statistics
Ann. Appl. Stat.
Volume 8, Number 3 (2014), 1372-1394.
Change points and temporal dependence in reconstructions of annual temperature: Did Europe experience a Little Ice Age?
Morgan Kelly and Cormac Ó Gráda
We analyze the timing and extent of Northern European temperature falls during the Little Ice Age, using standard temperature reconstructions. However, we can find little evidence of temporal dependence or structural breaks in European weather before the twentieth century. Instead, European weather between the fifteenth and nineteenth centuries resembles uncorrelated draws from a distribution with a constant mean (although there are occasional decades of markedly lower summer temperature) and variance, with the same behavior holding more tentatively back to the twelfth century. Our results suggest that observed conditions during the Little Ice Age in Northern Europe are consistent with random climate variability. The existing consensus about apparent cold conditions may stem in part from a Slutsky effect, where smoothing data gives the spurious appearance of irregular oscillations when the underlying time series is white noise.
You have to read the whole article to find out that they identify “structural breaks” in the late 19th and early 20th centuries, a different year for each temperature series. If the “pre-break” temperature series adequately satisfy “random variation” then the “trends” since the breaks are “statistically significant” in each temperature series. On this analysis, the “Little Ice Age” label applies equally well throughout the interval between the Medieval Warm Period and the 20th century warm period.
You can put this into a hierarchical framework in which the “breaks” appear to have happened “randomly”, in which case the authors would not win the prize.
AAAND the statistical methods for finding structural breaks are flawed in the presence of non-white noise. Too many false positives. Whooops. Something I’ve pointed out to some of the ground-station analysis folks. Haven’t heard a response yet.
Citation: http://journals.ametsoc.org/doi/pdf/10.1175/JCLI4291.1
Didn’t have a chance to see if they cited this paper or not
Peter
Peter Sable: AAAND the statistical methods for finding structural breaks are flawed in the presence of non-white noise.
Thank you for the link to Lund et al.
ABSTRACT
Undocumented changepoints (inhomogeneities) are ubiquitous features of climatic time series. Level
shifts in time series caused by changepoints confound many inference problems and are very important data features. Tests for undocumented changepoints from models that have independent and identically distributed errors are by now well understood. However, most climate series exhibit serial autocorrelation.
Monthly, daily, or hourly series may also have periodic mean structures. This article develops a test for
undocumented changepoints for periodic and autocorrelated time series. Classical changepoint tests based
on sums of squared errors are modified to take into account series autocorrelations and periodicities. The
methods are applied in the analyses of two climate series.
Like Lund et al, Kelly and O’Grada used autocorrelated noise as one of their models.
Kelly and O’Grada did not cite Lund et al, and the reference lists of the two papers have surprisingly little overlap (surprising to me).
All statistical methods are flawed. If Doug Keenan responds to every submission with the comment, like yours, “that method is flawed”, then he’ll never pay the reward. There needs to be some guidance about what noise/natural variation assumptions are considered “good enough”. My later post asserts that if the noise/natural variation model incorporates a period of about 950 years, then no statistical method is going to reject the natural variation hypothesis.
Another note about Kelly and O’Grada, is that their “headline message” is that a proper change-point analysis reveals no change-points supporting the idea of any particular short (a few decades) “Little Ice Age”. “Only” one change point is supported by their analysis, namely at about 1880.
the prize will be awarded to anyone who can demonstrate, via statistical analysis, that the increase in global temperatures is probably not due to random natural variation.
The result will depend heavily on whether the “random natural variation” is assumed to have a spectral density with power at a period of 950years. If that is specified, then the recent warming has almost for sure resulted from “random natural variation”.
Doug:
Are you going to tell us what percentage of the data sets had trends added to them?
If not, knowing when we’re done is harder. Just want to know if you are going to tell us that or not.
I’ve positively identified 21 data sets have added trends in a random sample of 500 of the data sets. Yeah, I know, slow start. Had a stupid bug in my code.
Peter
Why would somebody who has the ability to detect these trends require to know how many trends he can see in the data? If he can see the trends so clearly as the alarmists claim they can, then they will instantly know how many of the series have trends!
Ha, just trying to save ten bucks.
Seriously though if you are doing differential analysis it does help to know the frequency distribution of the underlying data. Cryptanalysis uses this all the time.
Doug hasn’t responded, so I assume the answer is “sorry you figure this out”. I would like Doug to mention in public that any response concerning the data he gives via email will be publicized so that all may benefit..
Peter
Peter Sable: Seriously though if you are doing differential analysis it does help to know the frequency distribution of the underlying data.
In this context, “help” is a humorous understatement. If the data series spans a time less than one of the periods in the spectral density of the noise then you are just plain out of luck if you do not know the underlying frequency distribution. Even having data covering multiple periods may be insufficient, as shown by Rice and Rosenblatt in a paper in Biometrics about 1988.
If we give $10 to every African and ask them to write 1000 selections of TRUE and FALSE each… then we are bound to prove that these trends can be detected.
If a trend is so difficult to detect then by definition it is too weak to worry about. That is what I cannot get my head around! Why people care so much about this virtually undetectable thing!
I am really surprised that no-one (unless I missed it) has mentioned the 300 year old Central Limit Theorem in this comment stream. An annual global temperature is not a measured temperature but an average of over a million daily min/maxs x 365 days x (say) 2000 stations. There are of course local correlations over short time and space in that population but enough effectively independent ones that the Central Limit Theorem will operate. That is, when annual differences are looked at as a population then the result will tend towards a Gaussian distribution, independent of the underlying distributions of the raw data population. (Reminder: summing or averaging is an information destroying process!).That is why every plot ever seen of global temperature index data is jagged with the annual differences being extremely difficult to distinguish from a Gaussian distribution.
That of course does not preclude there being, additionally, a trend component which is a common component of all million+ raw measurements which is what AGW theory demands. But that is Doug’s challenge; can you repeatedly tease out the natural (Gaussian) from the trend?
Actually a statistician with time on their hands could probably work out, by looking at raw data variance at shorter time and space frames, what the effective number of independent measurements is in that million plus raw data population and from that compute the expected variance of the year-to-year differences and from that…. the probability of detecting a trend.
Maybe Doug has already done it that’s why he is confident to put $100k on the line!!
The way this competition is done you can not rule out selection bias of the seeds used to create the data. By which I mean he could have generated 10,000s of datasets and then picked the sets that randomly had long term trends. This would make a statistical analysis exercise like this impossible.
I am not saying that’s what’s been done but there is nothing here to safeguard against it.
I way to possible protect against this it for the entries to be code that can analysis the 1000 trends and identify which contain a trend and which do not.
After the competition closes the code to generate the series is verified and a new set of 1000 series is generated from randomly generated seeds. With the trend randomly added to some of them. Then the winner is the code that can correctly identify at least 90% of the series in the newly generated data. This way you can rule out any selection bias.
The way this competition is done u can rule out selection bias of the seeds used to create the data. By which I mean he could have generated 10,000s of datasets and then picked the sets that randomly had long term trends. This would make a statistical analysis exercise like this impossible.
I am not saying that’s what’s been done but there is nothing here to safeguard against this.
I way to possible protect against this it for the entries to be code that can analysis the 1000 trends and identify which contain a trend and which do not.
The code to generate the series is verified and a new set of 1000 series is generated from randomly generated seeds. With the trend randomly added to some of them. Then the winner is the code that can correctly identify at least 90% of the series in the newly generated data. This way you can rule out any selection bias.
I don’t see the point of this contest.
Suppose you have 4 dices, two fair, one loaded to have a “positive” bias and one a “negative” bias.
Now suppose you have record for 1000 series of these dices, without the information of which dice produced it. This contest is basically about trying to guess that.
Can it be done ? Obviously it depends on how much unfair dices are loaded .
I guess the climate load is not enough to distinguish dices at the 900/1000 level.
However, our climate problem is quite different. It is : “given the unique serie i have, which is trendy, what is the most reasonable to believe between
H0) this trend is a produce of randomness and has no reason to keep going on [climate dice is fair]
H1) this trend has some causes that didn’t disappear so it will keep going on [climate dice is loaded]
For sure we cannot rule out H0 at 95 or even 90 % level. However H1 remains more probable (at may be 51 Vs 49).
This of course do not provide any hint at the causes of this slow probable trend. Since this trend began several centuries ago, GHG are most probably out of cause …
All of the models assume that the data are accurate. A major problem is that of data coverage – it has changed over time. We would normally expect recorded temperatures to creep upwards over time due to the location in urban areas – there are much more buildings concrete, parking lots, airports, roads than there were 135 years ago. I any case the original question is not how temperatures increase with time but rather how temperatures increase with the addition of CO2 to the atmosphere by burning fossil fuels. We are using the wrong independent variable.
I think this is the proper question, though we should also include all of the other possible sources of warming, even if we don’t think they exist.
But, my premiss is that the main effect is a loss of night time cooling, not warming. I have been calculating the difference between today’s day time warming, and tonight’s night time cooling for all stations that measure at least 360 days a year(at 360 almost all stations are 365/366). And then averaging each of these stations, and there’s no sign of a loss of cooling. In fact since 1940 there’s a slight cooling, if you include measurement uncertainty the average of all years is 0,0F +/- 0.1F
https://micro6500blog.wordpress.com/2015/11/18/evidence-against-warming-from-carbon-dioxide/
I’m not sure the problem to be solved is clearly stated. Looking at the first few I find two interesting series:
1) There is a very pronounced upward linear trend for the first half and then no trend for the second half. (Like the UAH satellite data). How should one reply to this.
2) There is no trend for the first half, then a jump and then no trend for the second half. This will baffle a least squares fit – or a ranked sign test. (This type of behavior used to be very common in interest rates data. The series would jump every time the FED changed the rate).
Doug is making a very clear point here. I don’t thing that you can run these series through a “trend detector” and just spit out the results. You will probably need to check each series individually.
His critique on statistical methods used in climate modeling is scathing.
Odd that anybody thinks this ‘challenge’ has anything to do with the study of nature. Nature might be operating by a quasi-random process, but is always limited by the laws of conservation of energy, and so forth. The ‘challenge’ has no such constraint. 1000 time series selected by unstated methods, from an unstated statistical generating method (or methods, there being an infinity thereof; one could easily produce every series from a different generator), with unstated types of functions added to an unstated fraction of those series … that’s not even a remotely interesting question in mathematics. Much less science.
As I observed elsewhere, not a bad method to collect the entry fees and buy yourself a present.
Digressing:
Many comments have mentioned Nyquist frequency, almost all of them incorrectly. The Nyquist frequency is the most rapid cycle that can be detected without aliasing, and its period is 2*dt. For annual data, 2 years is the shortest period. A comment mentioned something ‘about length/5’. This isn’t Nyquist frequency/period. Rather, it’s a fair rule of thumb as to the longest period one can make reasonable statements about in a spectral analysis. For 135 years, that’s a cycle of 37 years. Hence talk of 30 year cycles solely from the data, is perhaps doable, though shaky. 50-60 year cycles are too long.
oops, 135/5 = 27 years. The rest stands.
I’m the one talking about this, let me defend my comments.
Robert, I believe Nyquist is symmetrical. A window of data such as the 135 year temperature record is missing both low and high frequencies – you can’t resolve anything useful above or below a set of 2 frequencies.
Everyone knows that the high end cutoff of resolulution period is 2/sample rate. the literature on this is overwhelming, a simple google search gives bazillions of results, and everyone is subjected to marketing literature on their audio equipment.
However, there has to be a low frequency resolution cutoff as well. What is it? Google is a complete fail on this. In fact low frequency signal processing isn’t really well studied AFAICT. Look at how bad Mann screwed it up…
I haven’t created a formal mathematical proof, but I’ll try English.. With that entire window of 135 years, we effectively have one sample for low frequencies. By Nyquist we need two samples. Ergo, we can only resolve 67.5 years. I’ll also add that the math involved is pretty symmetrical, which also justifies my claims (short of a formal proof).
The 5x sampling “rule of thumb” applies at both the low end and the high end. On the low end, it’s due to being unable to resolve multiple overlapping signals of slightly difference frequency and phase. I arrived at the number via Monte Carlo simulation and thinking about the problem a lot.
At the high end, due to sampling error and filtering issues, you actually need better than Nyquist. For example most digital oscilloscopes start at 4x oversampling, the good ones 8x. Your CD player scrapes by with 2.2x, but when I still had good (young) ears I could hear the distortion in the cymbals due to the high slew rate in combination with high frequencies. That’s why pro audio is 96Khz to 192kHz (5x-10x).
I’d really love if someone could find some formal work on low frequency resolution of a limited sample length. It’s just not well studied by signal processing type folks. At least, I can’t find a good reference. Statisticians, AFAICT, are completely ignorant of this entire issue…
Finally, as to your comment on 30 year cycles. There are not (major, known) 30 year natural cycles, there are 30 year HALF cycles (as defined by signal processing folks), the actual cycles are on the order of 60-75 years. People get confused and think going from High to Low is a cycle. It is not. You have to go back to High to complete the cycle… I note Wikipedia is complete confused on this matter…. but good look editing anything climate related there… don’t trust anything you read that doesn’t have a graph where you can visually verify what a cycle is.
Peter
I completely forgot to add another justification that also shows the symmetry in English instead of math.
Nyquist was originally studying to see how many symbols could be transmitted per sampling period. Which is a different use of the term Nyquist than we use today. It’s called the Nyquist Interval:
from: https://en.wikipedia.org/wiki/Nyquist_rate#Nyquist_rate_relative_to_signaling
We have a cycle rate of B = 1/135 cycles per year (the length of the sample window). We wish to resolve a symbol (the underlying signal). The symbol rate we we can resolve is 1/(2*1/135) = 67.5 years.
Of course I could have inverted the logic and be off by 4x. Then explain why you can’t see it on an FFT… Sometimes the numerical simulations are useful, for example in proving you didn’t screw up by a factor of 4. Unlike climate simulations, that try resolve 10x finer grain details…
Also note the comment about “peak interference”. I’m pretty sure that’s sampling noise or other confounding signals…
Peter
The Cartesian mathematician will argue that everything is deterministic, and that any collection of N data points can be exactly described using an orthonormal model that spans the data domain and includes N contributing factors; the theoretical extension of the adage ‘it takes two points to determine a line.’
Statistics is that branch of mathematics that provides tools for handling problems where the number of data points in a situation far exceeds the number of known factors.
Statistics has absolutely no connection with causality, however. That issue is handled in the selection of the model, and is best facilitated by an understanding of the physics involved.
Given a set of data points (values of a dependent variable paired with values of an independent variable), I could fit the set exactly with a Taylor series, a Fourier series, or any number of orthogonal polynomials. The choice I make will be influenced by what I understand the data to represent.
If I expected a combination of periodic and secular (non-periodic) influences, I might even try a full Fourier decomposition and make a fit to the transformed data to separate the periodic and non-periodic functions.
The determination of whether or not there is a truly random influence would involve fitting a subset of N-1 data points with the physically most appropriate model, and then checking whether that has any effectiveness in predicting the value of the missing data point. Comparing the predicted and observed values of each point for all N data points and then examining this new set for an r^2 correlation will tell me whether there not there is randomness present.
Deterministic does not mean predictable.
This was the key insight of Ed Lorebz in his foundational 1961 paper “Deterministic Nonperiodic Flow”.
I wonder if Doug is using Lorenz’ DNF61 code. That would be very cool.
It should first be noted that in the IPCC Third Assessment Report – Chapter 14: Advancing Our Understanding, the following statement appears in sub-section 14.2.2.2:
“In sum, a strategy must recognise what is possible. In climate research and modelling, we should recognise that we are dealing with a coupled non-linear chaotic system, and therefore that the long-term prediction of future climate states is not possible. The most we can expect to achieve is the prediction of the probability distribution of the system’s future possible states by the generation of ensembles of model solutions. This reduces climate change to the discernment of significant differences in the statistics of such ensembles. The generation of such model ensembles will require the dedication of greatly increased computer resources and the application of new methods of model diagnosis. Addressing adequately the statistical nature of climate is computationally intensive, but such statistical information is essential.”
Now go win the $100,000!
There has been an update to the Contest. Briefly, the 1000 series have been regenerated. The reason is discussed at
http://www.informath.org/Contest1000.htm#Notes
Everyone who plans to enter the Contest should ensure that they have the regenerated series.
Thanks, Doug. However, I have a problem with your claim:
However, rather than requiring that people can demonstrate it via “statistical analysis”, you have a problem which requires that we get an “A” (90% correct). I know of no statistical analysis which requires this kind of certainty. Here’s what I mean.
Suppose for the sake of argument that half of your data has an added trend. If I could identify say 60% of the trended ones correctly, that would have a p-value of 2.7E-10. This is far more than is needed to “demonstrate, by statistical analysis”, that we can tell the difference between trended and untrended datasets.
Of course, your test is even harder because you haven’t said how many are trended and how many are not. As a result, you are demanding much, much, much, much, much more from your challenge than is required to distinguish a series with a trend from one without a trend.
As a result, I fear that as it stands your challenge is useless for its stated purpose. It has nothing to do with whether or not we can distinguish trended from untrended data via “statistical analysis” because the threshold you’ve set is far above what is needed by statistical analysis.
However, none of this means that I’ve given up on your puzzle. I estimate I can get about 80% accuracy at present, more for subsets. And I just came up with a new angle of attack, haven’t tried it yet. I suspect that this problem is solvable, I just have to get more clever.
My best to you, and thanks for the challenge,
w
This competition is fraudulent. What Keenan says on his web page in the 3rd paragraph is “I am sponsoring a contest: the prize is $100 000. In essence, the prize will be awared to anyone who can demonstrate, via statistical analysis, that the increase in global temperatures is probably not due to random natural variation. ”
In fact this can be demonstrated quite simply by considering the decadal variation in the temperature anomaly. Every single decade since 1950 has been warmer than the previous decade. The probability of getting such a sequence by chance is less than 2%. However, Keenan invites us to solve a mathematical problem without providing any evidence whatsoever that it has any bearing on global climate. In fact, if you dissect his fallacious logic, it goes like this:
1. Assume that climate is random
2. Devise an insoluble problem involving random numbers and trended vs trendless process
3. conclude that since the problem is unsolvable, assumption (1) must be true
TomP says:
Every single decade since 1950 has been warmer than the previous decade.
That is neither unprecedented, nor unusual. The same thing has happened repeatedly prior to the rise in human emissions:
http://jonova.s3.amazonaws.com/graphs/hadley/Hadley-global-temps-1850-2010-web.jpg
Once again: THERE IS NOTHING UNUSUAL OR UNPRECEDENTED HAPPENING.
Thus, the “dangerous man-made global warming” scare is debunked.
It’s not even particularly true.
The daily rate of change since 1950 is slightly negative and with measurement uncertainty the temp is 0.0F + / – 0.1F
https://micro6500blog.wordpress.com/2015/11/18/evidence-against-warming-from-carbon-dioxide/
I think you and he are saying slightly different things:
You are saying that periods of aggregate multi-decadal temperature rise have been seen prior to assumed AGW “start time” (1950) of the same order of magnitude as the rises since 1950. Some have been as long as 50 years. You are right and I have never seen an effective rebuttal. However within a substantial multi-decadal rise there will usually be one or more decades (if you choose the decade) where it didn’t rise, interrupting the contiguous sequence on which he bases the claim of very low probability
He is saying that, choosing decades starting 1950, there have been 5 sequential “up” decades. If the system was largely random with roughly 50% chance of up or down and no autocorrelation that is indeed a 2% chance event. He is right. However, the year on year variance is quite large in relation to the year on year trend change so I suspect I could start the decades at slightly different years and easily produce a several possible down decades which would interrupt the 5 UPs and become, say a mixture of 4 UPs 1 Down in some sequence which is a much more common occurrence. However I would then have to try every year as a start year and see how many showed that pattern to calculate the equivalent to the 2% chance for each. And then weight them by the (equal) probability of starting the decade in that year to calculate overall the probability of the observed result, independent of the start year. It would be higher than 2% but I don’t know how high. It is Sunday afternoon after all.
Jonathan Paget
I told already that this competition is pointless. It doesn’t make it fraudulent and it doesn’t aim at proving that climate is random. This competition aims at proving that we just cannot be reasonably sure ( at 90 % ) that the climate dice is biaised toward warming, since we are not able to distinguish between loaded dice.
Actually, remember that a canonical definition of randomness is “Meeting of two independent causal series”, and indeed there are two (and even more: I leave it to you to name some of them that are known) independent causal series in climate . So, no discussion, there is randomness in climate, period. No assumption here, simple fact.
“Every single decade since 1950 has been warmer than the previous decade. The probability of getting such a sequence by chance is less than 2%. ”
NO, In long enough a random process the probability of getting such a sequence by chance is 1 (100 %). It had necessarily to happen some days. And temperature series are indeed a long enough for this to apply.
So “less than 2%” refers to the probability of you being alive in an era when you can say that ? But we KNOW for sure that you live in our present era, as opposed to many others era when people couldn’t have say that . The probability is 1, again, not “less than 2%”.
From the GISS global anomaly, take the average of each decade centered on whole decades,
meaning 1885-1894, 1895-1904 and so on with the last decade being 2005-2014.
When you do this you get:
1890, -0.273
1900, -0.224
1910, -0.353
1920, -0.259
1930, -0.185
1940, 0.031
1950, -0.041
1960, -0.028
1970, -0.018
1980, 0.147
1990, 0.298
2000, 0.510
2010, 0.652
As you can see, every decade since 1950 is warmer than the previous one. In fact in the entire series there are only two decades that buck the rule, 1910 and 1950.
Jonathan Paget speculates without any actual evidence that you might get quite different results by shifting the central year. In fact the only effect of doing so is that for a few choices, 1950 is then warmer than 1960. The sequence from 1960 onwards remains the same, even at the cost of omitting some data from the past decade. If anyone is interested I can paste the code. (python).
paqyfelyc objects that if you try long enough, tossing an unbiased coin will eventually turn up the same sequence. However, that is not the problem at hand. We are not given an infinite number of tries. If you have 13 coin throws and asked to conclude, based on these coin throws, whether the coin is biased or not, what is your conclusion as a statistician?
And this is exactly what you would expect based on the knowledge that global average temperatures have been rising ever since their low point in 1650. The biggest question is: Will today’s Modern Warm Period peak in 2000-2015 be the maximum between 1650 and 2550’s Modern Ice Age?
Or will today’s Modern Warming Period max out at the next 66 year short cycle peak in 2070, or 2140?
When do we begin going back down to the next ice age?
Hello TomP
Thank you for the courteous response. I was reacting to what I initially saw was the faux precision of the 2% figure in the light of some ambiguity around start dates, decadal starts, etc etc and that really quick “non rigorous” probability analyses sometimes have stings in tails.
However,I’ve learnt pragmatically that truth is almost never further illuminated by calculating probabilities beyond one decimal point and often not even TO the first decimal place!! So I readily concede that the notion that GISS Global decadal surface anomaly change from 1950 to 2014 is generated from a purely random (Gaussian) “walk” of decadal steps would be a VERY LOW probability hypothesis and any rational analyst would look for a different starting point.
There clearly is an upward trend in decadal changes that has a persistence in time beyond that which pure chance can sensibly account for.
So, while I have your attention, can I ask your critique of the following elementary logic, which is in principle a simpler approach to the same question?
The standard deviation of the annual anomaly changes 1950 to 2014 is 0.14 degrees. The time period n is 64 years (convenient for mental arithmetic!). A Gaussian random walk with 0 mean and n steps will produce a distribution of end points with 0 mean and standard deviation = root(n) x 0.14 = 8 x 0.14 = 1.12 degrees.
The actual increase in anomaly in the 64 years was 0.89 degrees. What is the probability that the end point meets or exceeds that actual result?
Brief pause whilst consult old fashioned Normal Distribution tables
Answer….. 24%
Presumably the answer rests in the choice of time periods varying between annual (me) and decadal (you). But isn’t my annual approach equally valid and how would we rationalise the difference? By the way I would characterise 24% as LOW not VERY LOW!
regards
Jonathan
good one,
former investment analyst named Douglas Keenan!
Regards – Hans
talking about sports – not that aqquainted with baseball –
but FIFA, formula 1, skiing …
each season brings new regulations to hold suspence and weld public to the view:
really ‘self regulating’ systems.
Regards – Hans
leaves the question what’s FIFA on this blue environmantled system –
lives persistance vs. Entropy,
real diesels stronghold vs. EPAs imagined *pony farms, …
Hans
*worldwide ressorts