Extreme Times

I think this goes to what Lindzen says – one would expect our times to be warmest in a warming climate.

So…any idea why this happens? It seems counter intuitive to say the least

This seems to be an example of Benford’s distribution, or Benford’s Law as it is sometime called. If you take, say Bill Clinton’s tax forms, or any of hundreds such data, the number 1 will occur most frequently as the first number in the the data set and 9 will be the least frequent. It is why, in the old days, that the first several pages of a book of log tables get worn out and dog-eared.
http://en.wikipedia.org/wiki/Benford%27s_law
weird stuff

That was very well explained Willis. Thanks.

When I grow up, I wanna be a statistician. Then I won’t have to tell my Mom I’m a piano player in a whorehouse (kudos to HSTruman).
You, Willis, are a man among men!

I suspect that the explanation might be as simple as follows: a) in a dataset such as you describe, it is generally true that there will be will be long-term variations with period longer than the time period of the dataset. That is, a Fourier analysis of the “full” data series (i.e. the data before a chunk was cut out) would not be band-limited to the period of the sample. b) When you cut a chunk from a long-time-period Fourier component, there is a good chance that you will cut a chunk that is either increasing or decreasing throughout the chunk. When that happens, the end-points of the chunk will be extrema relative to all other points in the chunk.
Sorry – not as simple to explain as I had hoped. A drawing would be easier.

Gary Bucher’s reference is exactly on-point. Thanks.
Willis: this is another very relevant and surprising observation from your fertile mind. I enjoy your work very much.

Benford’s law may be just the tool to reveal fiddled data.

You don’t even need to do a Monte Carlo experiment to see why this is the case. Draw a parabola. Now pick a random interval on the x-axis. No matter what interval you pick, at least one endpoint of that interval will be an extreme (if the vertex is not in your interval, then both endpoints will be extremes).
Realize any functional relationship that goes up, down, or both, will have subsets of that relationship that are somewhat parabolic in shape.
So, yeah, the endpoints tend to be extremes.

just providing another laugh:
24 April: Bloomberg: Julie Bykowicz: Steyer Nets $10,050 for $100 Million Climate Super-PAC
Billionaire Tom Steyer is trying to enlist other wealthy donors in a $100 million climate-themed political organization, pledging at least half from himself.
So far, he’s landed one $10,000 check.
Mitchell Berger, a Fort Lauderdale, Florida, lawyer and top Democratic fundraiser, was the lone named donor to NextGen Climate Action Committee in the first three months of the year, a U.S. Federal Election Commission filing shows…
The report notes another $50 in contributions so small that they didn’t need to be itemized.
“Well, if I’m the only donor, I guess it won’t be the last time I’m a donor,” said Berger, chuckling, in a telephone interview. “Although I certainly hope that I’m joined by others at some point.” …
***Berger has spent much of his adult life raising political money and has worked for decades with former Vice President Al Gore, another advocate for addressing climate change. His assessment of Steyer’s goal of securing $50 million from others: “It’s not going to be easy.” …
The donor compares the climate issue to the Catholic Church’s condemnation of Galileo in the early 1600s after the astronomer disputed its pronouncement that the Sun orbits the Earth.
“Things that will appear to be obvious to us in 100 years are not as obvious now,” Berger said. He said he admires Steyer’s goal “to create an undercurrent on climate where it’s possible for politicians to say the Earth travels around the Sun without being excommunicated.”…
Steyer, a retired investor who lives in California, didn’t solicit the donation, Berger said. Rather, Berger volunteered the $10,000 while Steyer was visiting in Florida. Steyer and Berger’s wife, Sharon Kegerreis Berger, are high school and college classmates…
http://www.bloomberg.com/news/2014-04-24/steyer-nets-10-050-for-100-million-climate-super-pac.html

Wonderful explanation of a wonderful insight, Willis. Just what we expect from you.

For example, currently we are at a low extreme in hurricanes
For additional things that are extreme now, see:
http://www.theweathernetwork.com/news/articles/extreme-weather-events-not-making-headlines/25948/
Extreme weather events NOT making headlines
Dr. Doug Gillham
Meteorologist, PhD
1. GREAT LAKES ICE COVERAGE
“Current ice coverage is over double the previous record for the date (April 23) which was set in 1996.”
2. SLOW START TO TORNADO SEASON
3. LACK OF VIOLENT TORNADOES
4. QUIET 2013 TROPICAL SEASON

“Usual Request. If you disagree with what someone says, please have the courtesy to quote the exact words you disagree with. This avoids misunderstandings.”
=================
Please define a “misunderstanding” 🙂

Is the effect stronger for shorter series? Eg what about a 160 point long series (to reflect the hottest year on record claims), or 16 point long series (to reflect hottest decade)

Robert of Ottawa wrote; (CAPS added by myself)
“Any mathematical issue that depends upon an integral from minus to plus infinity (correlation, Fourier transform, etc.) is NOT ACCURATE WITH A FINITE SERIES. Hence the great interest in Window Functions:”
Exactly correct, this is one of the limits taught early in signal processing. Most signal processing (especially digital, versus analog computing) is an APPROXIMATION to a closed form equation (you know the ones with the integral sign). That is why there a dozens of windowing functions. These artifacts can easily be mistaken for real information, but they very rarely are.
As an interesting historical aside; the old Soviet Union was far behind the “West” in terms of digital computing power. But they had many quite good mathematicians. They solved many integrals with closed form equations (i.e. to get the accurate answer for the integral of function “abc”, plug the limits and the values into this closed form equation). The “West” just hammered it with digital signal processing. I have an old Russian reference text (translated version) from the 1970’s (long out of print) that has closed form equations for hundreds of integrals, 20 per page, 700 or so pages. And the closed form solutions are exact (up to the number of decimal places you use, of course). Finding the closed form solution for an integral is like a puzzle, there is no exact algorithm to follow, you just try hunches, I wonder if the derivative of function “qwy” is the answer?
The whole mathematical basis behind stealth radar technology applied to warplanes was done in closed form equations by the Russians and published in open math journals. The US defense industry found it and used it to create the F-117 and the B-2. The first time they took a scale mock up of the F-117 out to a radar test range they (US scientists) thought their instrumentation was broken; “how come we cannot see that metal object over there with our radar?”. The secret was the shape of the plane, then they applied radar absorbing coating and the plane virtually “disappeared” from the radar screen.
Another example of these artifacts is the use of digital random number generators (like the rand()) function in Excel ™. It does not produce a true random number sequence, it is good enough for most work, but you can see frequency components in the data that are artifacts from the random number generator algorithm. At one time there was a company that had a electronic circuit which digitized the noise across a resistor (designed to maximize the noise) and sold it as a “true random number generator. The digital versions have become better with time (more bits to work with) so I think that device is no longer on the market.
Cheers, Kevin.

When you study ‘highly correlated data or red noise’, I’m am fairly certain you will find that it exhibits all the characteristics of ‘highly correlated data or red noise’.

“”””””……tchannon says:
April 24, 2014 at 6:12 pm
I suggest Willis you use pink noise, not red noise…….””””””
Well 1/f noise is pretty common in analog MOS or CMOS circuits; and it is inherent. PMOS transistors, tend to have lower 1/f noise than NMOS, so analog designers (good ones), tend to use PMOS in input stages, even though NMOS gives higher gm values for a given gate area.
It is common to use very large area PMOS devices, in analog CMOS to reduce the 1/f noise corner frequency.
I designed (and built) an extremely low noise, and low 1/f corner frequency CMOS IC, using truly enormous PMOS transistors. It was a very high current gain feedback amplifier for an extremely sensitive high speed photo-detector.
1/f noise seems to defy logic, since it tends to infinity as f tends to zero. Actually it is not a catastrophe, since you can prove that the noise power is constant in any frequency octave (or decade; so the noise power doesn’t go to infinity, since the lower the frequency, the less often it happens.
I have often claimed, that the “big bang” was nothing more than the bottom end of the 1/f noise spectrum. Get ready for the next one.
As to Willis’ new conundrum; is not a truncated data sequence akin to the transient portion of the startup of an inherently narrow band signal.
An ideal sine wave signal, only has zero bandwidth, if you disallow for turning it on, or switching it off. When you do either of those things, you get a transient, that dies out with time leaving the steady state signal.
So if your signal begins with an off to on step function, which it does in Willis’ chopped chunks, you are going to get the overshoot, of a brick wall filter response.
Is that not what is going on here ??

davidmhoffer says:
April 24, 2014 at 5:56 pm
In other words, assuming the data is an undulating wave, it doesn’t much matter how you cut it up into smaller segments, you’re pretty much guaranteed to have extrema at both ends of the segment.
Thanks, David … I think that was very well worded and well explained.

If you’re looking at any given time window on an autocorrelated time series, the extreme values are more likely to be at the beginning and the end of the time window.
=====================================
??
It appears to me that the “beginning” and “end” need to be defined. The last 5 years of the T record are less extreme then the preceding five years and they are not random. If the end and beginning are defined as the first third and the last third, then you are covering 2/3rds of the series, and so more likely to have extremes within those segments.
(Likely I do not follow this at all.)

What if you start and end the undulation on the mean?

David A;
What if you start and end the undulation on the mean?
>>>>>>>>>>>>>>>>.
Hmmmm. Well in my thinking out loud thoughts above, I presumed that each segment was smaller than the over all cycle. So if I understand your question, you’d be using a segment size that equals the entire cycle rather than a segment size that is only part of the cycle. But you could only do that if the over all trend across an entire cycle was zero, and you’d be able to manipulate what extrema showed up at what end simply by choosing where the start point was (ie peak to peak or valley to valley) Or, assuming that there is an underlying trend that is positive (and in Hadcrut there is a positive trend) by choosing a segment that starts and ends on the mean would be tough to do. You’d essentially have to choose an artificial start and end point over a part of the data that where those end points are at the mean value, which would be less than a complete cycle. Could you find such a segment? Probably. But it would be vary rare in comparison to other segments of the same length with random start and end points.

Here is my logical explanation…
An auto correlated time series is similar to a continous function in mathematics, since neighbouring points are more likely to be near each other.
For a continuous function, all global maxima occur either at local maxima or at the endpoints.
All local maxima occur at critical points (places where a function is either non-differentiable or the derivative is zero).
If you consider the space of all continuous functions, all points in the domain are equally likely to be critical points..
So that means that endpoints are more likely to be global maxima. They are equally likely as all other points to be critical points, and in addition, there are classes of continous functions where they are the maxima even when they are not critical points.

Walpurgis says: “Interesting. Also, how likely would it be that you would want to go into a career in climatology if you believed the climate isn’t changing much and won’t until a few thousand years after you retire. What would you write your thesis on? What would you do every day?”
Well you could do paleoclimatology. That’s a lot of fun. Or you could do prehistorical-historical climatology (Discontinuity in Greek civilization Paperback, Rhys Carpenter; also R.A. Bryson). Interesting summary here: http://www.varchive.org/dag/gapp.htm
Or you could do cosmoclimatology. Hendrik Svensmark, Nir Shaviv, Jan Veizer, Eugene Parker and Richard Turco manage to keep busy. http://www.thecloudmystery.com/The_Cloud_Mystery/The_Science.html
Or you could do anthropological climatology. Elizabeth Vbra found that interesting enough to edit Paleoclimate and Evolution, with Emphasis on Human Origins. http://yalepress.yale.edu/yupbooks/book.asp?isbn=9780300063486
During the last 20 years climatologists, geophysicists and other scientists have revealed a few pages of the book of Earth’s climate system.
Still, our ignorance is greater than our knowledge and will continue to languish until scientists free themselves from the view that the science is settled..

I think I may have found a mathematical explanation for this.
For a Wiener process (a random walk comprising infinitesimally small random steps), the “Arcsine laws” apply: http://en.wikipedia.org/wiki/Arcsine_laws_(Wiener_process)
Per that page, the arcsine law says that the distribution function of the maximum on an interval, say [0,1], is 2 / pi * arcsin(sqrt(x)).
Differentiating that expression yields the probability density 1/(pi*sqrt(x)*sqrt(1-x))
This yields a plot that looks quite like your histograms!
https://www.wolframalpha.com/input/?i=plot+1%2F%28pi*sqrt%28x%29*sqrt%281-x%29%29

Sorry, from 1850 to 1890.

I do not follow the logic in a true random series. When throwing a fair die, each of the six values 1 to 6 has the probability 1/6. Assume that each throw generates a different number for six throws. Is the one and the 6 any more likely to be the first or last throw?

In 1st year physics we learned about the “drunken walk” and soon understood that after n random steps on a one dimensional line, one ended up a distance proportional to sqrt(n) from the starting point. Intuitively most people guess you would have traveled a distance of zero (on average), which is wrong. Is this not the same as saying the extremes are more than likely at the beginning and the end of the series.

Geoff Sherrington says:
April 24, 2014 at 10:57 pm
While the general pattern derived from the statistics of red noise shows more extremes in the end bins, this is a generalisation. Can I surmise that the actual case, rather than a general or synthesised case, should be adopted for making statements about recent climate extremes?
====
The point is that in making statements about recent changes being “weird” , “unprecedented” or unusual, we should be making probability assessments against Seattle’s graph , not the layman’s incorrect assumption of a flat probability.
The point is to compare the actual case to general synthetic case.

So whenever I look for power spectral density graphs of temperature from different sources, I see similarities to red or pink noise, but basically never blue noise. So I think it’s quite plausible that the effect pointed out in this article is applicable to temperature time series.
I’m not sure about other kinds of time series.

Seattle: “But which kind of power spectrum does the climate have?”
This assumption that red noise (which is the integral of white noise) is the base-line reference against which all climate phenomena should be measured is erroneous. It is often used to suggest that long period period peaks in spectra are not “statistically significant”.
Since many climate phenomena are constrained by negative feedbacks (like the Plank feedback for temperature) the red noise assumption is not valid for long term deviations which tend to be less than what would be expected under a red noise model. It may work quite well for periods up to a year or two.
http://climategrog.wordpress.com/?attachment_id=897
The end effect will still be there but may be less pronounced than the simplistic red noise model.

Draw a sine curve with long period. Give it a completely random phase. Now that’s a beautiful stationary and highly autocorrelated time series for you.
Observe it through a short independently chosen window. What do you see? A line going up, or a line going down.
If the peaks mean a global baking Saharan desert and the valleys are massive ice-ages, then seeing the series go up and up and up could be cause for concern if you don’t like the heat.

https://www.ipcc.ch/ipccreports/tar/wg1/446.htm
Bearing in mind this is log-log it should be perfectly straight for red noise.
It looks very straight from 2 to 10 years. Then breaks to a very different slope for 10-100 and is basically flat beyond 100y.
Bear in mind that much of this is quite simply artificially injected ‘red’ or other noise that it pumped into the models to make the output look a bit more ‘climate-like’. It is a total artifice, not a result of the physical model itself.
This is done to disguise the fact the models are doing little more than adding a few wiggles to the exaggerated CO2 warming they are programmed to produce.
They then do hundreds of runs of dozens of models , take the average (which removes most of the injected noise by averaging it out ) and say LOOK, WE TOLD YOU SO! Our super computer models that cost billions to produce and a generation of dedicated scientist working round the clock PROVE to that within 95% confidence it’s all caused by CO2.
We must act NOW before it’s too late ( when everyone realises it’s a scam).

Willis, excellent thinking and very clear explanation. However, I am curious. Does this phenomenon hold true regardless of the size of the second sample? You chose 2000. What if you chose 1000? 1717? 3000? Does it hold true if the second datasets aren’t sequential, but rather overlap? or are chosen randomly? This is going to bug me now until someone comes up with the proof, and explains it to me.

“I used two levels of AR and MA coefficients (lag-1 and lag-2). ”
Just out of interest , could you post details of the actual model that you used?

And how much I learn from Anthony and Willis!

“… in fact twice as likely, to be at the start and the end of your window rather than anywhere in the middle.”
____________________________
I have to disagree with this statement. I did a simple graphical analysis of your graph:
http://i.imgur.com/X0ht7Ch.png
and my conclusion is that chance that extremes will be within 15% of the length of the interval from either edge (covering 30% of the interval length; 6 out of 20 columns) is approximately 44%. It is definitely more than 30% which would be the case if the distribution was flat. But it is about half more probable over uniform distribution, definitely not twice.

If you’re looking at any given time window on an autocorrelated time series, the extreme values are more likely to be at the beginning and the end of the time window.

That depends on the width of the window. Wiki gives the following definition for autocorrelation:

… It is a mathematical tool for finding repeating patterns, such as the presence of a periodic signal obscured by noise, …
wiki

If the window is narrow compared to the period of the underlying signal then the window will usually not contain the signal’s peak values. In other words, the underlying signal will either have a positive or negative slope for the whole window.
On the other hand, if the window is wide enough we can easily find examples where the extreme values do not come at the beginning and end of the window.
Example – The window contains exactly one cycle of the underlying periodic signal. In that case the signal’s waveform at each end of the window will have the same value and, depending on the phase, the extreme values will be somewhere within the window.

Economist Eugen Slutsky showed that random processes can result in cyclic process, such that Fourier analysis would find the cycle, but it is just random noise. The article’s title is The Summation of Random Causes as the Source of Cyclic Processes, translated to english around 1936. Here is a brief article discussing the impact on economics: http://www.minneapolisfed.org/publications_papers/pub_display.cfm?id=4348&
Perhaps this has something to say about all the oscillations connected to weather and all the speculation about future cycles in the weather.

charles nelson says:
April 24, 2014 at 10:34 pm
“…..Has anyone subjected Warmist Climate Data to the Benford Law test?…..”
That thought has occurred to me.
However, I doubt if any climate scientists simply sat down and invented the data. It’s not necessary. They can use cherry-picking and ignore any inconvenient data, or they can ‘adjust’ the data. I imagine that, while Benford can detect purely made-up numbers, it may not detect data that has been systematically adjusted (by systematic, I mean the same adjustment was applied to all the data).
I’ve thought about it, and I think that Benford probably doesn’y apply to Willis’ fascinating findings.
One (rather boring) explanation of Willis’s findings did occur to me, other posters may have arrived at a similar explanation:
If you cut out part of a long data series, the selected section will almost certainly have an overall positive or negative trend, even if it’s random (e.g. the drunkard’s walk). So, if the trend is positive, the early numbers will tend to be lower and the later numbers will tend to be higher, and vice versa.
I’m not sure if this real effect is needed to explain some of the recent claims about ‘records’. Although there has been no global warming in this century we’re still very near the top. Therefore it’s very easy for short-term temperature excursions (which are often large) to set new records. Of course, it’s a complete scam: they know that most people, when they hear about new records being set, will assume the climate is still warming, when of course it isn’t.
Records should have no place in science. The only thing that matters is the trend.
By the way, for anyone not familiar with Benford’s Law, I suggest you Google it – now.
Chris

@Willis – You really didn’t need to ask. Of course you have full permission, and thanks for the flowers!

Wow. Just when I think my troglodytic brain is beginning to get a handle on things, Willis comes along and says the inside of a table tennis ball is exciting. And then he explains it, and he’s right. Reading this blog continually humbles me, usually when I’m preening over what I had previously supposed was a clever thought of my own.
Once again Willis, thank you for opening up a new vista.

To me this makes perfect sense considering the definition of autocorrelated data.
Consider: an event, n, where the measure T=f(n) is autocorrelated. By the definition we know that ΔT for f(n – (n+1)) is small therefore the probability of T0 = T1 is high. The same probability holds for each delta step, n+1 – n+2; n+2 – n+3; etc. However the probabilities drop with each successive event from the original event, n, such that the probability of T0=T10 is much lower or ΔT for f(n – (n+10)) is much greater. The function is symmetric about n so that the probability for T1 = T-1, T2 = T-2, etc.
So what we end up with an inverse probability distribution centered about event n (the middle of the graph). All that is required is that the data be autocorrelated.

http://bishophill.squarespace.com/blog/2013/5/27/met-office-admits-claims-of-significant-temperature-rise-unt.html

Willis Eschenbach says:
April 24, 2014 at 8:42 pm
(checked out pink noise)
Thanks for looking, not that then.
Result you have is nonsensical, why so being the question. I think several comments give clues and is about the validity of the test. I agree with those who point out the known “non-stationary” will produce a convolution kind of result aka fourier transform hence the large items both ends.
Adding noise to something leaves something plus noise.
To me this suggests the result is dominated by the large slow excursions.
Recently I had something similar giving a strange statistics answer. Eventually I figured what, removed it from the “signal” and that left normal stats stuff. The point perhaps is non-stationary is more literal than it might seem.
It might be interesting to band-split the hadcrut at say 4 years and then do the same analysis on each portion. (low pass will do and subtract to produce the complement)
I recall another ploy involving chopping into sections and reordering.

The effect may be due to the generated time series being red (as has been suggested in several of the comments). Have you tried it with a white time series to see if the result holds for that?

It’s been 34 years since I was involved in statistics, so I will avoid commenting on most of this discussion. I just want to note, that people only tend to look at things when they are at an extreme. If there is an even distribution of cancer in a certain area, people don’t question unless it is clustered, i.e. extreme. That perception of extreme is based first on their personal mental database. (i.e. wow, I don’t remember having that much snow before.) so the other extreme is likely to pretty distant from the extreme that caught your attention. The fact that most things run in waves or cycles, means that most observations in the cycle are going to be close to the norm and the extremes are going to be distant from each other. Why this works on random numbers I don’t know, but why it works in real life I get.

Willis, I think you made a small error on your simple example:
” Even this very simplified example follows the math. From his math we’d expect 3.2 out of eight at each end (40%), and 1.6 out of eight in the middle (20%). Instead of 3.2 and 1.6, the results are 3 and 2, which are the closest whole numbers for the actual situation.”
Unless I am missing something – the 0’s of the middle two possibilities each land at BOTH ends meaning that we have a total of 10 extremes (instead of 8) since the 0’s which are extremes are repeated. 4 extremes at each end (40%) and 2 extremes in the middle (20%). So we don’t even need to round to get exactly the same answer as the formula.
I really have appreciated both the mathematical and common sense expalantions that have made this seem much more intuitive after all.

WayneM asked at April 25, 2014 at 2:54 pm
“The effect may be due to the generated time series being red (as has been suggested in several of the comments). Have you tried it with a white time series to see if the result holds for that?”
Good suggestion. If Willis is right (bite my tongue!!!) than the histogram for White should be flat. It is:
http://electronotes.netfirms.com/ac.jpg
The figure shows red, white, and white with a (pinkish but not strictly pink) feedback of 0.8 (feedback for red is a=1.0, for white a=0). Also on the figure is the Matlab code that produced the figures, and with other options, just for documentation and to show how simple this is.

Bernie Hutchins
A random walk? What I know of random walks is that they can have a drift (as in multiple Brownian motion simulations reveal). In this sense they are not necessarily stationary. This is not normally true of the algorithm I presented. But then I’m no expert on random walk algorithms/methods?
Also I can’t see how multiple random walk would give you typically more extreme values at the start and end. Nor can I see how with this would be the case using the simple algorithm I presented above – unless all my assumptions are correct. In short, there is a bias in the algorithm – why?

Willis spoke precisely of (in his opening paraphrase) “an autocorrelated time series”. I think we do not need the “auto” part of that – it is just correlated (a property – as opposed to uncorrelated). Red noise (brown, integrated white, random walk – all the same) is the first such example that comes to mind. Keep in mind that what Willis plots is NOT an antocorrelation, but a histogram of occurrences of the maximum of many sequences with a PRE-EXISTING correlated property. With a red noise sequence of any length, there is always a frequency of period longer than the chosen length that is not only present but stronger than any “wiggles” we suppose we are seeing in the particular window (instance). This, in many cases, trends the segment to tip up or down. Hence the extremes at the ends. Lovely! Obvious – when we have someone like Willis to point at it! Here was my repeat of the experiment as I posted above.
http://electronotes.netfirms.com/ac.jpg

cd –
You need to run some code of your own. It’s simple and you will know exactly what was done because you yourself did it.
You said as well: “Autocorrelation suggests that the degree of correlation between two sets (both derived from the same series) is a function of the lag (the distance between the sampled pairs used to compute the correlation/covariance).”
I think you have described a cross-correlation since you derive both as sub-sequences from the same (presumably much longer) time sequence. If I have a sequence of length 1000 and I correlate samples 200-299 with samples 550 to 649, this is a cross-correlation. The process does not “know” whether it is being correlated with a later part of it own self, or perhaps sunspot numbers. If you insist on autocorrelation, you must do the whole sequence.
And, I emphasize, correlation (or not) is a pre-existing property of the series we are looking at (like red, white, pink) and we are examining it in the time domain – just inspecting the samples. No one is computing and sort of correlation anyway.
This is much simpler than you are making it – I think.

Bernie Hutchins says:
April 26, 2014 at 2:47 pm
You need to run some code of your own. It’s simple and you will know exactly what was done because you yourself did it.
Maybe. But I have very little time and was hoping for something more akin to a technical link (maths), as being able to reproduce the same kind of results does not explain why.
I think you have described a cross-correlation since you derive both as sub-sequences from the same (presumably much longer) time sequence.
No I’ve described an autocorrelation. You’re bivariate statistic comes from the same series. Cross-correlation samples two different series.
And, I emphasize, correlation (or not) is a pre-existing property of the series we are looking at (like red, white, pink) and we are examining it in the time domain – just inspecting the samples. No one is computing and sort of correlation anyway.
I never said they did. Autocorrelation is a product of any Markov Process such as a random walk as each new value is conditioned on a previous result (did you not mention random walk?).
This is much simpler than you are making it – I think.
It may be, and if Willis is right which I have no reason to doubt, no one has explained why yet. But thanks for your efforts.

Nick Stokes
But some will occur at weekends, and will provide maxima for two weeks. Sundays aren’t warmer per se, but will show up more often in the statistics.
Sorry, if the week runs from Monday to Sunday, why would a warmest day on Sunday count as two and if it occurred in Wednesday it would count as one. If your suggesting the heat carries over (for an autocorrelated series this is a fair assumption) but then the warmest day would be at the start not the end of the weak. So unless your warmest days are more likely on a weekend or end/start of window then your explanation doesn’t work.
In short, I don’t think that works at all. For example, your argument would mean for the same year, (and exact same record) if we decided to start our dissects on a Wendesday that result would change.
Personally, I think the result depends on the window size. If your first order statistic (the mean) is only stationary for a given sub-sampled window length (i.e. the sample mean is invariant under translation for windows above a certain size), then windows less than this size will likely be in part of the series with a local trend. This drift will run across the window so that high and lower values at either side of the window. Therefore, this result only holds for certain window sizes.

Nick

I happened to have on hand Melbourne daily max from may 1855 to Nov 2013. I counted the days on which the weekly max occurred (omitting 17 weeks with missing readings). The results were:
Sunday 1657
Monday 1185
Tuesday 896
Wednesday 814
Thursday 917
Friday 1224
Saturday 1581

That is a very interesting meteorological result, but it doesn’t prove your point, quite the opposite. If I were to dissect my time series from mid-week to mid-week then the result would look like this:
Wednesday 814
Thursday 917
Friday 1224
Saturday 1581
Sunday 1657
Monday 1185
Tuesday 896
The part of the window with the weekly max are in the middle not the ends.

cd –
After I said that the issue of cross- vs auto- is a matter of terminology, you provide a link to definitions! Not only do I know the definitions, I know what they MEAN.
Please apply your definitions to the following two sequences:
w1 = -0.3708 0.8942 0.0703 0.4039 0.7501
w2 = 0.6957 0.0537 0.6148 -0.2130 0.9235
These may, or may not, be extracted from a longer sequence:
w3 = -0.3254 -0.3708 0.8942 0.0703 0.4039 0.7501 0.9725 0.7706 -0.1903 -0.2291 0.6957 0.0537 0.6148 -0.2130 0.9235 -0.9398 0.9075
Is the correlation between w1 and w2 auto- or cross-? Would the results be different, or tell you anything?
As for a computer package, you hardly need anything like that. Have you looked at my Matlab code? Not the details, you don’t even have to know Matlab to see that it is simple, short, and that just about anyone could read it (like BASIC). No fancy functions, only a dozen lines, since most of it is commented out or for display.
Sorry if I am sounding sanctimonious to you. Apologies if I have crossed the line between persistently trying to be helpful (a habit as an educator) and showing impatience with a lack of progress.

Willis
The question is not why, as the effect is assuredly real.
As I said I’m quite happy to take your word for it.
Note that the step size in my loop (window = 1:1000) is one. This means the window steps just the way you specified, “continuously through the series” one step at a time.
Thanks for doing that. But there was no need as I took all your points. What strikes me though is the exact symmetry in your histogram. This is almost starting to look as if, in a round about way your actually producing the series’ non-centred spectrogram (for the windows temporal resolution). I know there are many methods of producing certain types of power spectra using moving windows and since the series is red noise then more signal (i.e. exponential decay with wavenumber for w = 1 to N/2) is contained in the end members of the of the non-centred power spectra. I hasten to add that I only know of them, but have never used them and don’t know exactly what they do so I could be talking out of my hat.
Because statistically, the moving window is no different than randomly selected windows.
I understand all that as stated, I don’t refute anything you say. And I’m not suggesting that it isn’t real. But I want to know is why – mathematically, not just that it is.