Guest Post by Basil Copeland
Figure 1
Each month, readers here at Watt’s Up With That, over at lucia’s The Blackboard, and elsewhere, anxiously await the latest global temperature estimates, as if just one more month of data will determine one way or the other the eternal destiny of AGW (the theory of “anthropogenic global warming”). For last month, July, the satellite estimates released by UAH and RSS were up sharply, with the UAH estimate up from essentially zero in June, to +0.41°C in July, while the RSS estimate was up from +0.081°C to +0.392°C. Does this sharp rise presage the resumption of global warming, after nearly a decade of relative cooling? Or is it just another in a series of meandering moves reminiscent of what statisticians know as a “random walk?”
I have not researched the literature exhaustively, but the possibility that global temperature follows a random walk was suggested at least as early as 1991 by A.H. Gordon in an article in The Journal of Climate entitled “Global Warming as a Manifestation of a Random Walk.” In 1995 Gordon’s work was extended by Olavi Kӓrner in a note in the Journal of Climate entitled “Global Temperature Deviations as a Random Walk.” Statistician William Briggs has written about climate behaving like a random walk on his blog.
Now even I will confess that the notion that global temperature, as a manifestation of climate processes, might be essentially random is difficult to accept. But I am coming around to that view, based on what I will present here, that monthly global temperature variations do, indeed, behave somewhat like a random walk. The qualifier is important, as I hope to show.
So, what is a “random walk” and why do some think that global temperature behaves, even if only somewhat, like a random walk? And what does it matter, anyhow?
While there are certainly more elegant definitions, a random walk in a time series posits that the direction of change at any point in time is essentially determined by a coin toss, i.e. by chance. As applied to global temperature, that is the same as saying that in any given month, it is just as likely to go up as it is to go down, and vice versa. Were global temperature a true random walk, there would be no underlying trend to the data, and any claimed evidence of a trend would be spurious. One of the best known “features” of a random walk is that in a time series it appears to “trend” up or down over extended periods of time, despite the underlying randomness of the direction of change at each point in time.
So why might we think global temperature follows a random walk? One reason is suggested by a close look at Figure 1. Figure 1 is the familiar HadCRUT3 time series of monthly global temperature anomalies since 1850, with a simple linear trend line fit through the data. When we look close, we see long periods, or “runs,” in which the data are above or below the trend line. If the data were truly generated by a linear process with random variations about the trend, we’d expect to see the deviations scattered approximately randomly above and below the trend line. We see nothing of the kind, suggesting that whatever is happening isn’t likely the result of a linear process.
On the other hand, when we perform what is a very simple transformation in time series analysis to the HadCRUT3 data, we get the result pictured in Figure 2.
Figure 2
A common transformation in time series to investigate the possibility of a random walk is to “difference” the data. Here, because we are using monthly data, a particularly useful type of differencing is seasonal differencing, i.e., comparing one month’s observation to the observation from 12 months preceding. Since 12 months have intervened in computing this difference, it is equivalent to an annual rate of change, or a one month “spot” estimate of the annual “trend” in the undifferenced, or original, series. When we look at Figure 2, it has the characteristic appearance of a random walk.
But we can do more than just look at the series. We can put a number to it: the Hurst exponent. Here’s a very understandable presentation of the Hurst exponent:
“The values of the Hurst Exponent range between 0 and 1.
-
A Hurst Exponent value H close to 0.5 indicates a random walk (a Brownian time series). In a random walk there is no correlation between any element and a future element and there is a 50% probability that future return values will go either up or down. Series of this type are hard to predict.
-
A Hurst Exponent value H between 0 and 0.5 exists for time series with “anti-persistent behaviour”. This means that an increase will tend to be followed by a decrease (or a decrease will be followed by an increase). This behaviour is sometimes called “mean reversion” which means future values will have a tendency to return to a longer term mean value. The strength of this mean reversion increases as H approaches 0.
- A Hurst Exponent value H between 0.5 and 1 indicates “persistent behavior”, that is the time series is trending. If there is an increase from time step [t-1] to [t] there will probably be an increase from [t] to [t+1]. The same is true of decreases, where a decrease will tend to follow a decrease. The larger the H value is, the stronger the trend. Series of this type are easier to predict than series falling in the other two categories.”
So what is the Hurst exponent for the series depicted in Figure 2? It is 0.475, which is very near the value of 0.5 which indicates a pure random walk. And when we exclude the data before 1880, which may be suspect because of a dearth of surface locations in computing the HadCRUT3 series, the Hurst exponent is 0.493, even closer to 0.5. So by all appearances, the global temperature series has the mark of a random walk. But appearances can be deceiving. In Figure 3 I fit a Hodrick-Prescott smooth to the data:
Figure 3
In the upper pane, the undulating blue line depicts the smoothed value derived using Hodrick-Prescott smoothing (lambda is 129,000, for the curious). In the lower panel are detrended seasonal differences, i.e., what is left after removing the smoothed series. Conceptually, the smoothed series can be taken to represent the “true” underlying “trend” in the time series, while the remainder in the bottom pane represents random variations about the trend. In other words, at times, the annual rate of change in temperature is consistently (or persistently, as we shall see) rising, while at other times it is consistently falling. That is, there are trends in the trend, or cycles, if you will. And while it is not obvious, because of the scaling involved, these are essentially the same cycles that Anthony and I have attributed to a lunisolar influence on global temperature trends. That should not be so surprising. In our paper, we smoothed the data first with Hodrick-Prescott smoothing, and then differenced it. Here we’re differencing it first, to show the random walk nature of the series, and then smoothing the differences. But either approach reveals the same pattern of cycles in global temperature trends over time.
Looking more closely at the smoothed series, and the random component (labeled “Cyclical component” in Figure 3), we have an interesting result when we compute the Hurst exponents for the two series. The Hurst exponent for the smoothed (blue line) series is 0.835, while the Hurst exponent for the detrended random component (bottom pane) is 0.383. The first is in the range associated with “persistent” behavior, while the second is in the range associated with “anti-persistent” behavior. Let’s discuss the latter first.
Anti-persistence is evidence of mean reversion or what is also sometimes called “regression toward the mean.” Simply put, when temperatures spike in one direction, there is a strong probability that they will subsequently revert back toward a mean value. Ignoring all other factors, this property would suggest that the dramatic rise in the temperature anomaly for July should lead to subsequent declines back toward some underlying mean or stable value. I think this is probably more what Gordon or Kӓrner had in mind for the physical processes at work when they proposed treating global temperatures as a random walk. I.e.,shocks to the underlying central tendency of the climate system from processes such as volcanism, ENSO events, and similar climate variations create deviations from the central tendency which are followed by reversions back toward the mean or central tendency. Carvalho et. al (2007), using rather complicated procedures, recently laid claim to having first identified the existence of anti-persistence in global temperatures. We’ve identified it here in a much simpler, and more straightforward, fashion. (I’m not trying to take away from the usefulness or significance of their work. Their procedures demonstrate the spatial-temporal nature of anti-persistence in global temperatures, especially on decadal time periods. I think WUWT readers would find their Figure 10 especially interesting, for while they do not use the term, it demonstrates “the great climate shift” of 1976 rather dramatically.)
With respect to the smoothed series, the Hurst exponent of 0.835 indicates persistent behavior, i.e. if the series is trending upward, it will have a tendency continue trending upwards, and vice versa. But that is to be expected from the cyclical undulations we observe in the smoothed series. As to the possible physical processes involved in generating these cycles, after Anthony and I posted our paper, comments by Leif Svalgaard prompted me to perform a “superposed epoch analysis” (also known as a “Chree analysis”) on these cycles:
While Leif contends that the analysis should be performed on the raw data, in this case I would beg to differ. As shown in Figure 3, the raw data is dominated by the essentially random character of the monthly changes, completely obscuring the underlying cycles in the data that emerge when we filter out (detrend) the raw data. Arguably, what we have in the blue line in Figure 3 is a “signal” that has been extracted from the “noise” depicted in the bottom pane. Now as such, the “signal” may mean something, or it may not. That is where Figure 4 comes in to play. The peaks in the cycles depicted by the blue line in Figure 3 show a strong correspondence to maximums in the lunar nodal cycle (the “luni” part of our suggestion of a “lunisolar” influence on global temperature trends). They also show a strong correspondence in solar maxima associated with odd numbered solar cycles, especially beginning with solar cycle 17. Are these correspondences mere coincidence? Anthony and I think not. While each may play an independent role in modulating global temperatures, since the 1920’s the solar and lunar influences appear to have been roughly in phase to strongly influence temperature trends on a bidecadal time frame. In other words, Figure 4 may be revealing the physical processes at work in explaining the persistence revealed by the Hurst exponent for the blue line in Figure 3.
Taken together, the two Hurst exponents – one for the true “signal” in the series, and the other for the “noise” in the series – essentially offset each other, leaving us with a Hurst exponent for the unsmoothed, raw, seasonal difference of ~0.5, i.e., essentially a random walk. And so on a monthly basis, the global temperature anomalies we await anxiously are essentially unpredictable. However, if the cycles in the smoothed series can be plausibly related to physical processes, as Anthony and I believe, that gives us a clue as the “general direction” of the monthly anomalies over time.
In our paper together, Anthony and I presented the following projection using a sinusoidal model based on the same cycles shown in the blue smooth in Figure 3:
Figure 5
The light purple line in Figure 5 is, essentially, a continuation, or projection, of the blue smooth in Figure 3. From this, we derived a projection for the HadCRUT3 anomaly (light blue in Figure 5) which has it essentially meandering between 0.3 and 0.5 for the foreseeable future (here, roughly, the next two decades).
But the monthly values will vary substantially around this basically flat trend, with individual monthly values saying little, if anything, about the long term direction of global temperature. In that sense, global temperature will be very much like a random walk.
We all know that the strong La Nina of a while back has been fading away and we are now on the verge of weak El Nino conditions.
The observed movement of the air circulation systems slightly more towards the poles (but with strong resistance from certain cold air masses) and the increased rate of energy transfer from ocean to air can be expected to result in exactly that which we now observe.
The sole question is whether any of it is caused by human influence. I see no evidence of that unless one can demonstrate that the changed rate of energy release from the oceans is affected by human activity.
We can only affect the air. The air cannot affect the oceans significantly.
Humanity is acquitted.
Not being any kind of statistician, nor any kind of climatologist, but my reaction is to be sceptical of this theory.
A random walk has the feature that future course changes are independent of its history. Like a roulette table.
Physical systems just don’t work that way. They have a state, and they respond to physical laws. The current state of any system that includes any kind of feedback (+ve or -ve) will affect the future course of the system. Lorentz attractors and butterfly wings taught me that at high school.
Craig Allen (22:23:39) :
Hang on minute! I regularly read here the assertion that due to negative feedbacks in the climate system, increasing greenhouse gasses could not possibly have a significant affect on global temperature. Now you’re asserting that it is a random walk. Seems to me that it’s your sequence of climate theories that most resembles a random walk.
Do you mean to say the AGW thermogeddonists didn’t test various null hypotheses before they led us down the road of dismantling modern civilisation?
Caramba!
Basil Copeland
could you please add a constant 0.008°C/year to the seasonally differentiated temperature data and report the Hurst exponent of that new series? I’d be interested to see the changes in the Hurst exponent induced by doubling the warming during the 20th century, i.e. whether your analysis is sensitive to this order of magnitude of linear trend at all.
Another question, why do you differentiate instead of simply detrending the data with a linear fit or other suitably smoothed version of the data? Seasonal differencing seriously changes the spectrum of the noise, adding multiple dips in the FFT spectrum. Furthermore, why use seasonal differences at all instead of simple monthly differences, after all we are dealing with temperature anomalies that have the annual cycle removed already.
1. The Hurst exponent is used to analyse the behaviour of self-similar or scaling systems, NOT random walks. The two are quite different and unrelated. A random walk is a storage measure and should be differenced prior to Hurst-type analysis. As noted by Leif, evidence suggests global temperature does not follow the characteristics of a random walk (although I note in your presentation you do acknowledge this)
2. The Hurst exponent is a fixed value applied to a series, not a variable parameterised across scale. If the Hurst exponent did vary with scale, it would suggest non-Hurst like behaviour of the system.
3. The Hurst exponent is VERY difficult to estimate. Furthermore, because it assesses the behaviour of a time series across scales, it is quite wrong to smooth a series and then estimate the Hurst coefficient. The estimator will not work correctly in these conditions.
4. The Hurst exponent for global mean temperature has been estimated to be in the region 0.9 to 0.95, and this number is consistent across all temperature series (HadCRU, Satellite) as well as historical proxies (e.g. Greenland and Antarctic ice core measures of temperature).
For more info, please see some of the following presentations by Demetris Koutsoyiannis. For an estimate of H over different scales, see pages 28-32 of the following presentation:
From climate certainties to climate stochastics (Opening Lecture)
And for a nice general overview on the Hurst phenomenon and climate:
The Hurst phenomenon and climate
Each month, readers here at Watt’s Up With That, over at lucia’s The Blackboard, and elsewhere, anxiously await the latest global temperature estimates, as if just one more month of data will determine one way or the other the eternal destiny of AGW (the theory of “anthropogenic global warming”).
No – not quite. Looking at the monthly figures can give an indication whether what we think might be happening in the oceans and atmosphere is actually happening. For example – do the satellite readings ‘lag’ the surface readings in response to ENSO fluctuations.
Also the GISS July anomaly of +0.6 implies that the surface is (or was), relatively speaking, cooler than the lower troposphere – yet GISS are continually being accused of ‘fraud’.
I’m confused. A few weeks ago, you were telling us that temperatures in the USA were below average:
http://wattsupwiththat.com/2009/08/10/noaa-july-temperature-below-average-for-the-u-s/
and then you told me of these record cold temperatures for July:
http://wattsupwiththat.com/2009/08/02/roundup-of-some-interesting-july-weather-records/
And at the end of June, you told me the the June global temperature anomaly was important:
http://wattsupwiththat.com/2009/07/03/uah-global-temperature-anomaly-for-june-09-zero/
“Given the U.S. Senate is about to vote upon the most complex and costly plan to regulate greenhouse gases, while the EPA suppresses earlier versions of the chart shown below from a senior analyst, this should give some pause to those who are rational thinkers.”
and now, all of a sudden, the argument has changed to “But the monthly values will vary substantially around this basically flat trend, with individual monthly values saying little, if anything, about the long term direction of global temperature.”
So please, please explain to the rational thinkers on why the data in June matters but the data in July does not, and why the articles about record cool temperatures in certain cities in the USA in July was important but the global average temperature for July is not?
Please, please, explain that.
Sorry, on re-reading my previous post (01:46:17) I noticed my point (1) from my previous post is not well written, and does not read as I intended it. What I was attempting (poorly) to say, is that I recognise you acknowledge the relationship between the random walk and the storage element, but what you are missing is that the behaviour of the raw data is not consistent with that of a random walk, which is the point Leif makes.
That said, a conventional Hurst analysis of the raw global temperature data IS interesting, and I would strongly recommend people read the presentations that I linked above.
Doesn’t this discussion lead back to the understanding that our climate/weather is a chaotic & non-linear system. The idea that predictability is possible rather than just pattern recognition does suggest that people are still struggling with the concept of chaos.
Even if we could control every variable which affects the Earth’s climate/weather system, variation would still occur as a chaotic, non-linear system can shift without external changes to forcings.
That global temperature is a random walk is no surprise.
Relax Dr Jose Sanchez, it’s what we call ‘discussion’. Someone presents an idea and we talk about its merits or otherwise. I know that may be a difficult concept, especially if you are a regular at Real Climate, but that’s how we do things here.
Dr Jose Sanchez (00:50:04) : At the end of the month of June, this website and heaps of right-wing journalists like (such as Andrew Bolt and Mudoch) were claiming that the 30 years of warming had been erased and now the world is cooling. But as soon as the data makes a sudden change, the story changes to “random walk”. Honestly, the only random walk is by those who are changing their theories on a monthly basis on why man is not responsible for climate change! All credibility is thrown out the window!
Here, maybe this will help you:
http://chiefio.wordpress.com/2009/08/13/gistemp-quartiles-of-age-bolus-of-heat/
At the bottom I added a bonus section. When you look at 150 years of the best 10% of the thermometers in the GHCN record (27% of the data) you get no warming.
Not in summer. Not in winter. Not over time. Nada. Nothing. Zip.
Temperatures are stable to within a few 1/10C over that 150 years.
All your “30 years of warming” are concentrated in a few short lived thermometers, flitting into existence, then evaporating.
The temperature rise only happens in winter, not in summer.
The temperature rise only happens in a subset of the the stations.
The temperature rise does not happen in the stable stations.
The spatial domain is not uniformly heated.
The time domain is not uniformly heated.
The heating has a strong SEASONAL pattern.
All of these things say it can not be CO2.
Further: the pattern of thermometer variation strongly implies that all of the “global warming” is simply an artifact of changing what thermometers are read and in the record at any given time. A semi-random event.
I can think of little that would confirm the thesis of this posting more than the observation that on any given day we take a random sample of the air temperatures, but that the central tendency is very stable since (as the top 10% of stations show) the mean just has not changed significantly in 150 years.
Or to put it more bluntly, the climate is astoundingly stable, but the weather tomorrow is nearly random. Central tendency is solid, any given day and place, not so much.
Dermot O’Logical (01:35:19) : A random walk has the feature that future course changes are independent of its history. Like a roulette table.
Physical systems just don’t work that way. They have a state, and they respond to physical laws.
You may well be right on the physical level, but this article talks about the reported temperature. We have a strongly non-Nyquist compliant set of thermometers in the spacial and in the time domain. At any given time you may have dramatically more, or less, of them under a hot or cold mass of air on any one day.
That daily “average temperature” can vary dramatically just from how many thermometers are on the cold vs warm side of the Canada Express…
But over a long period of time, the average tends to a constant.
Thus we have the random daily component, but the strong central tendency in the longer term.
It’s not about the physical system, it’s all about a very poorly distributed set of thermometers, not read frequently enough.
Dr Jose Sanchez (02:13:35) :
“I’m confused… Please, please, explain that. ”
Dr Sanchez, the articles you gave links for are all from different sources, and this latest article is a guest post by Basil Copeland. It might surprise you that different people actually have different opinions. WUWT publishes articles and information from many sources. You do realise that not every single article is penned by Mr. Watts don’t you?
Presumably you write to New Scientist or Scientific American on a regular basis asking them to “Please, please, explain that.” whenever they publish a new article with a different viewpoint?
TonyB (01:22:51) :
This is a very interesting and thought provoking article and Basil deserves our thanks for putting it together. So is global temperature rising sharply due to mans activities? It depends greatly on start point and methodolgy. The following comes from a longer article I wrote recently;
Do we accept that Hadleys start point of 1850 (the LIA) in which 20 poorly distributed stations worldwide is meaningful (or as meaningful as a ‘global’ temperature gets.)
Or do we take Hansens 1880 figures based on slightly better station distribution (but still very poor) that relies on 1200km gridded cells?
====
Er, uhm, no. We DON’T accept Hansen’s tortured and corrupted data as indication of anything but a lesson on “how to manipulate data and hide a methodology of corruption”.
A few points: 1850 did NOT mark the lowest point of the LIA, temperatures have been rising naturally and with no assistance from man since about the 1750- 1780 point, with another dip in 1816 from volcanic activity. Prior to that, they had been declining (again, with no assistance from man!) from the much earlier Medieval Warm Period. Temperatures have been rising sinicie the LIA, oscillating with a 70 year period about a sinusoid baseline that id itself now rising.
Hansen’s methods of smearing data, of recalculating and erasing old data, and of multiply re-calculating and re-averaging vaguely sourced and well-hidden data are NOT clear, nor are they explained, nor are they repeatable. They are NOT based on logic nor on valid and validated computerprogramming techniques, nor have they EVER been audited or passed through ANY review.
Further ..”Link 6 the link below is Hansen’s original 1987 paper which is still much used by the climate industry as proof of temperature change since 1880.”
Hansen’s computer-gaming scenarios (his “climate models”) do NOT provide
ANY proof of anything except “garbage in = garbage out” – though I do agree with your term using “the climate industry” – today’s 60 billion dollars spent on climat epolitics – and the 1.6 trillion demanded by that industry, are far, far larger than all but a few industries worldwide. Hansen’s models approximate output values based on assumed inputs, assumed constants, and assumed changes. NONE of which assumptions can be “proved” correct now, in the model, or in the future.
SOME of his assumptions can be ASSUMED correct under some circumstances as coarse approximations of ideal simplifications of a few carefully selected physical processes. Most important, NONE of Hansen’s predictions have proven true in even the short 10-20 real-world cases, and he can offer NO reason why his predictions are such failures – but he demands we extrapolate using HIS approximations not in 5-20 years, but over 200, 400, and 800 years.
A very reliable rule, when reading scientific papers, is that if the title is a question, the answer is no. When reading ~snip~ science blogs, that rule holds even more strongly.
Dr Sanchez
Dr of pyscology ? peut etre. Surely not physics!! or economics
So monthly temperature data is reproducible by taking a normal distribution round 0 sigma .5 say and picking a random figure from it and calling it a monthly anomaly.
Do this for a big enough dataset and stretches would compare (within alarmist error bars) favourably with the existing records?
This article and Tonyb’s response are very helpful to me as I was pursuing similar lines — but felt someone must have already done it. Now, I can head in a more productive direction based on this information.
Tonyb, at this moment I tend to lean in the direction you indicate:
“…deriving anomalies from fairly meaningless information will come up with a fairly meaningless and suspect ‘global’ temperature which in itself is a strange concept.”
Nonetheless, I would like to prove (or disprove) this to myself in a defensible way. I am in the “baby step” mode right now. Again, thanks to the authors for saving me some time.
I’ve always had a problem with the anomaly approach to global temperature since it is a second order differencing scheme that introduces noise due to measurement error and lack of resolution. What if you applied this analysis to a true global average temperature, i.e., for each day add up all the temperatures and simply divide by the number of stations?
I am not sure that I follow your argument.
In deciding if the temperature record represents a random walk would it not be better to give the Hurst Exponent of the raw data to begin with. I expect it will show long term persistence. Can you give us this value?
Your seasonal differencing must act as a high pass filter removing the long term trend and its associated persistence. I can not see any need for this step, the HadCRUT3 data has had its seasonal variations removed when the climatology was subtracted from the actual temperatures. Given that there is no need to remove monthly details, choosing to make the subtraction X(i)-X(i-12) seems arbitary and just defines the pass band of the filter.
I think that one of us must have got something wrong.
Alexander
I remember the January 2009 temp anomalies showing a rather large positive spike. Everyone then spoke about the return of AGW.
No doubt there is a major random component and each month’s data is rather irrelevant – however much partisan observers emphasise deviations which favour their point of view and ignore others! But it passes the time while the serious trends build up 🙂
However Basil’s posting seems to be shading dangerously close to trying to suggest that there is no underlying linear trend. Maybe that wasn’t the intention but some people seem to be taking it that way. The choice of the differential series to analyse certainly masks the underlying trend, since the eye (and possibly the random walk analysis?) can’t easily see whether the sum of all positives is greater than the sum of all negatives. I don’t know the details but I suspect the Hurst analysis only looks at short-term sample-to-sample differences rather than long term integrations.
To me the most likely analysis of temperature data is the sum of:
a) A more-or-less linear trend (C02, LIA rebound?)
b) Several long-term cycles (PDO, solar)
c) A whole pile of random noise
For me, the only real question is whether the “more-or-less” is indeed more than linear (positive feedback) or less than linear (negative feedback). I think we’re just going to have to wait to find out…
I have been interested in the fact that whatever happens to the worlds ‘temperatures’ there seems to be almost no good correlation with any theory…after the theory is gone over with a fine tooth comb, anyway.
Despite vociferous arguements on either side strongly trying to prove that their sides evidence is overwhelming, the resulting cause and effect of such influences is very weak regarding the actual climate response. Solar cycles seem to have a weak influence, as does CO2. All related ocean oscillations seem to have modest, but temporary effects both cooling and warming. Particulate pollution seems to both warm and cool the atmosphere depending on the type and action. There are conflicting actions of cloud type, cloud cover and relative humidity. Large dust storms and volcanic eruptions are ‘random’ and apply temporary forcings. Ice cover/extent is variable and apparently unpredictable. Cosmic rays have weak and variable influences, Etc.
In the end, from my ‘stand back from the trees and look at the forest’ point of view, it seems as though the strong arguments are actually weak and the strong forcing theories are actually weak as well. It seems as though the random walk theory is about as good as any existing theory at this point in time.
Looking at past temperature cycles (as good as any proxy can be anyway) long term and short term temperature fluctuations are as normal as can be and that todays temperature cycles can easily be fit into ancient cycles absolutes and periods. It makes little sense to me that anyone can say with a straight face that they understand climate/climate change or that they can predict changes or modify the climate natural progress.
There probably will be some climate model that eventually makes some reasonably accurate predictions, but that day is obviously a long ways off. I was attending many climate modelling presentations back in the mid-70’s where attempts were being made to predict future weather/climate. Clearly, after 30+ years the difficulty in accurate modelling is as onerous as ever despite the incredible advances made to date.
This is one field where natural effects are slow and minimal, man’s effects are slow and minimal and predictions are consequently heavily inaccurate over such small time frames.
Patience here is an obvious virtue, here.
Jim, too.
On reading a bit more about Hurst exponents (but still not really understanding them!) it seems they can detect long-term drift. But like “bluegrue” above I’m worried that the differentiating step turns the linear trend into a fixed offset which the Hurst process will probably then ignore (what’s the betting the first thing it does is subtract out the series mean?).
Lots of interesting comments to respond to.
First, a thanks to
Stef (03:13:17) :
for pointing out that one cannot casually respond “you said this here, and you said this there” to posts up here on WUWT. Not everything posted here comes from Anthony directly. And he frequently puts up posts from people with differing perspectives.
That out of the way.
Craig Allen (22:23:39) :
Hang on minute! I regularly read here the assertion that due to negative feedbacks in the climate system, increasing greenhouse gasses could not possibly have a significant affect on global temperature. Now you’re asserting that it is a random walk. Seems to me that it’s your sequence of climate theories that most resembles a random walk.
Did you read it all? I suggested that while the aggregate, or raw, data, looks like a random walk, when we decompose it into a “signal” and residual “noise,” the “signal” exhibits persistence, and the “noise” exhibits anti-persistence. If you want to find your GHG effect, look for it in the “signal” (not that it is necessarily there, just that that is where you’d find it in the framework of this analysis). And what Stef wrote applies too.
Leif Svalgaard (22:27:11) :
It works the other way around: the find the cycles, you detrend the raw data and do the Chree analysis on the residuals [i.e. raw – trend].
Except that we do not need Chree analysis to identify the cycles. That’s already been done by differencing and smoothing the differences. Why should it matter how we’ve identified the cycles (and just to be clear, we — Anthony and I in our paper together — demonstrated that the same cycles can be derived using wavelet transforms, and MTM spectrum analysis)? Having identified some cycles, Figure 4 is what it is. If you want to chalk the apparent correspondence of the peaks in the cycles to solar cycle and lunar nodal cycle peaks as just fortuitous circumstance, what more can I say?
Arden Halliday (23:12:30) :
Hey, thanks for your work!
I’m trying to explain this to my son – can you help me? I’m wondering why you use the transformed HadCRUT3 data – why don’t you just apply the Hurst exponent to the normal HadCRUT3 time series? Does that make a random walk too?
A Hurst exponent applied to the untransformed data will undoubtedly be high. But that does not rule out a random walk. The high Hurst exponent in the untransformed series is an artifact of serial correlation. The conventional approach to determining whether or not a random walk exists is to difference the data, as we’ve done here.
John Tofflemire (00:18:00) :
Basil,
I don’t understand the reason for this analysis. One can easily have a random walk component with a drift component that would be non-stationary in the untransformed data and stationary in the differenced data. Whether or not the global temperature anomaly time series has a random walk component is not particularly interesting.
What is interesting is whether the underlying stochastic (random) process that produces a given time series (such as the global temperature anomaly) is invariant with respect to time. To understand whether this is the case, one differences the time series, that is, one subtracts the value in a given period from that of a chosen earlier period. If the resulting transformed time series is white noise (that is, random) then we may say that the original time series is stationary.
Well, in the end, I did what you are saying, but didn’t come up with the result you might have been looking for.
Yes, one can have a random walk with drift. But that (as normally done) implies an underlying linear process. I do not think that applies here. We do not have (linear) drift, we have a stochastic process varying around a series of oscillations (cycles) in the underlying system. It is of interest whether or not those oscillations have a physical basis or explanation. We (Anthony and I) do.
As to the character of the stochastic process, a Hurst exponent of 0.383 suggests pink noise, if not white.
You also wrote:
That being the case, you did not explain well why differencing a given month’s anomaly with a 12-month moving average of the anomaly makes sense for the global temperature anomaly.
I didn’t difference a 12 month moving average. No moving average was applied before differencing.
TonyB (01:22:51) :
This is a very interesting and thought provoking article and Basil deserves our thanks for putting it together. So is global temperature rising sharply due to mans activities? It depends greatly on start point and methodolgy. The following comes from a longer article I wrote recently;
Do we accept that Hadleys start point of 1850 (the LIA) in which 20 poorly distributed stations worldwide is meaningful (or as meaningful as a ‘global’ temperature gets.)
I, too, question inclusion of data prior to 1880. You can see some obvious difference in Figure 3. That said, it would not change anything in this analysis to have restricted the data to 1880 and on (except to give some who are looking for anything to criticize something to harp about. Note, that doesn’t impugn all critics. Some critics, perhaps most here, if not all, are honorable in their intentions).
Spence_UK (01:46:17) :
4. The Hurst exponent for global mean temperature has been estimated to be in the region 0.9 to 0.95, and this number is consistent across all temperature series (HadCRU, Satellite) as well as historical proxies (e.g. Greenland and Antarctic ice core measures of temperature).
But that number isn’t all that meaningful because of the autoregressive nature of the time series. Yes, there is persistence on that order of magnitude in the raw data, but as a correlation against time, some of it is spurious. By differencing the data, we control for serial correlation. Now we still see persistence, but we also see now where it comes from — the oscillating blue “trend” line in Figure 3. Note, besides the persistence that comes from the cycles, there may be a small underlying trend. I’ll deal with that in a follow up response specifically on that.
Dermot O’Logical (01:35:19) :
Not being any kind of statistician, nor any kind of climatologist, but my reaction is to be sceptical of this theory.
A random walk has the feature that future course changes are independent of its history. Like a roulette table.
Physical systems just don’t work that way. They have a state, and they respond to physical laws.
Please read the whole thing, or if you have, read more carefully. Your “state” is in the bottom pane of Figure 3. It is not random. It is mean reverting.
Thanks to all who are commenting, whether you agree or not.