Guest essay by Sam Outcalt
Introduction: The object of this document is to present a brief and concise introduction to Hurst ReScaling. More detail is presented in a paper by Outcalt et.al. (1997), which is posted on the WUWT
Website ( http://wattsupwiththat.files.wordpress.com/2012/07/sio_hurstrescale-1.pdf ). A extensive reference list in this paper can be consulted so references are omitted here.
Background: During a study of the hydrology of the Nile a British Engineer discovered that the annual runoff appeared to have a memory. The Hurst Exponent (H) named in his honor is calculated using Equation 1, in which R, S and n are the rescaled range, standard deviation and observation number.
Equation 1. H = Log[R(n)/S(n)] / Log (n)
The rescaled range is the amplitude of the integral trace of deviations from the mean of a serial data vector. Hurst had anticipated an exponent near 0.5, which is termed Brown Noise and can be simulated from a series of random numbers. Values above 0.5 indicate increasing auto-correlation in the data with 1.0 termed Black Noise indicating extreme correlation with the past or strong “memory”. In rather obscure paper (Outcalt et.al.(1997)) discovered that the extremes and inflections of the integral trace used to determine the value of the rescaled range flagged regime changes in the data. These regimes were discovered to pass tests for a statistically normal distribution at significance levels where the bulk data failed. Linear trend lines fit to the “regimes” also displayed significant slope differences at the transitions.
Example: A useful and instructive example calculation can be carried out on the NASA GISS Data used to document “Global Warming” or “Climate Change” or the pressing need for new Carbon Taxes.
Before presenting the example let’s outline the steps in the calculation.
1. A calculation of the Hurst Exponent can be made to estimate the level of “memory” in the data.
2. The mean is subtracted if the data is not presented as deviations from the record mean.
3. The integral trace is calculated as the accumulated deviations from the record mean.
4. Any slight trend is removed from the integral trace.
An upward linear trend will produce a parabolic trace of negative values below the zero level as the early deviation sum is downward and the later values upward terminating near zero. A downward linear trend will produce a positive parabolic trace. More complex functions with upward and downward trending sectors will display an integral trace with sectors both above and below the zero level.
The NASA GISS data is displayed in Figure 1. It should be mentioned that this data vector is already in the deviation from record mean format.
Figure 1. The data show a “strong memory” with a Hurst Exponent of 0.787.
There are two major inflections on the integral trace in 1936 and 1976. The latter inflection is the base of the “hockey stick” warming regime. A 5 year moving average trace indicates that the end of the warming trend may have occurred early in the 21st Century. Before moving to a consideration of the early 21st century it is necessary to mention that there is an alternate method for estimating the Hurst exponent. The Hurst Exponent can also be calculated from a log-log linear fit to the data FFT as the X,Y axes of an FFT have the following for listed in Equation 2.
Equation 2. Y = a + Exp [H X] or Ln Y = Ln a + H Ln X or Log10 Y = Log10 a + H Log10 X
Figure 2. The slope of the Log-Log transform of the FFT estimates H at 0.774 compared to 0.787 in Figure 1.
To explore the transition into the early years of the 21st Century the period from 1980-2010 was lifted from Figure 1 and the mean subtracted and the analysis carried out on the data. The results are displayed as Figure 3.
Figure 3. The data shows a strong “memory” similar to the 1880-2010 data set and a single inflection at the integral trace minimum value in 1997.
In Figure 3 the integral inflection in 1997 and leveling of the “warming trend” in 2004 hint that the warming regime which began in 1976 may have ended with the turn of the century.
The reader is encouraged to run thru the calculations using a spreadsheet program or other resources. I use Dplot software because it has automated functions to create integral traces and FFT’s. I strongly believe that it is impossible to understand analytical procedures without actually carrying out the calculations or better still writing the source code.
Data from Fountain Hills, AZ:
There is some controversy about the requirement for a strong memory to display regime transitions. To test this hypothesis I have analyzed a data set created from readings collected from a WiFi temperature probe. The probe was located on a north facing wall of my winter house in Fountain Hills, AZ. The data were initially collected a 30 minute interval and decimated to yield only noon readings of air temperature. The decimated data is displayed as Figure 4.
Figure 4. Data collected with a Lascar WiFi Probe from 19 March thru 16 April 2013. This data set was decimated to preserve only noon readings.
Even with a Hurst Exponent just above the threshold of Brown Noise and well below the value of the NASA GISS data the integral trace inflections flag air mass transitions in the Phoenix Region. Although there are only 29 observations in the set the integral information content remains in tact
indicating air mass transitions. A data set from 3 probes at radically different sites at my winter house produce some interesting conclusions. This data is displayed as Figure 5.
Figure 5. Data from 3 USB Probe sites at the Winter House in Fountain Hills, AZ.
The traces in Figure 5 indicate that the integrals flagged major air mass transitions at 3 probe sites. The strong diurnal signal is still present in the integral but the regime changes are still evident. This indicates that probe location is not critical for airmass transition analysis. The spike in the Wall Plot is due to the beam radiation striking the probe but has little impact on the integral.
Conclusion: Hurst Rescaling provides a unique method for detecting regime changes in weather and climate data. The detection of strong airmass control of the integral in Brown Noise indicated that strong regimes may be detected even in data with almost no auto correlation. Probe location appears to have little effect on integral transitions. The major consideration in probe location appears to be placing the probes at sites where there in no unnatural time dependent influence. A probe placed in the exhaust area of a clothes dryer running on a irregular schedule is one thing but the diurnal passage of shadows due to buildings may have a minimal impact on the integral trace.
The reader is encouraged to collect data using WiFi or USB Temperature Probes. The Lascar USB-Lite (under 50$US) will record data at 30 minuted intervals for a month before a battery replacement. I waterproof these tiny probes with a section of old 23C road bike inner tube trimmed to the probe length and mount the probes inside a small styrofoam coffee cup attached to a length of stiff wire. The wire is then attached to a tree limb.