The easiest way to determine the lag is to drive the filter with an impulse and observe the delay (in time steps) before the output maximum appears. This may be a fractional time step in which case, interpolation is required to properly do the time alignment.

If you decide to learn wavelets, I recommend this site as an efficient starting-point:
http://www.ecs.syr.edu/faculty/lewalle/tutor/tutor.html

-
Dear, Dr. Sanchez,

Is that what you really think?

WUWT is a place where people come to discuss the complexities of natural climate variation.

Unlike other climate forums, participants here are not barred simply for being non-alarmist. If journalists are looking here it is probably in-part because the site welcomes a variety of comments about natural climate variation. On other climate sites you can’t reliably get honest, factual comments past moderation; if & when that changes, the journalists you are so worried about might have more potential sources of information.

If some of the comments here have been offensive, that is regrettable. Anthony & the moderators are being extraordinarily tolerant of nut-job alarmists posing as nut-job “deniers” to deliberately make WUWT look bad.

Do you have any interest in discussing natural climate variation? If so, please join us frequently. Feel welcome to ignore the partisan comments and focus on the complexity of nature. I think you will find there is lots of work to do – and that it is very interesting. Nature is far more beautiful & complex than any anthropogenic computer fantasy.

Regards,
Paul Vaughan
Ecologist, Parks & Wilderness Advocate

“… journalists around the world are using this website as a source to spread misinformation about the theory of global warming.

It is not misinformation; CO2=AGW is misinformation. No one has falsified the theory of natural climate variability. The CO2=AGW hypothesis was an attempt to show that the natural warming of the planet is caused by increased carbon dioxide. That hypothesis has taken a couple more torpedoes with E.M. Smith’s demolishing of any unusual warming, and CO2 as a cause of anything but increased agricultural yields.

It’s best to be careful with words: CO2=AGW is not a theory. It is a hypothesis [actually more of a conjecture], and it is unable to predict the climate. The long accepted mainstream theory, which makes reliable predictions, is the theory of natural climate variability. Here and here are two good references.

I do like a bit of number-crunching, but I can’t help thinking that you are not using the best technique for determining whether or not global temperature follow a random walk. As I understand it, the Hurst exponent is most useful for detecting long-term predictability in a time series. While that means that there are implications for what you would calculate as the Hurst exponent for series that follows a random walk, it does not follow that it is a particularly powerful (in the statistical sense) for detecting random walks. Why use binoculars to look at bacteria when you have a microscope in the lab? To me, an obvious example of the microscope we have to hand is the Phillip-Perron (PP) unit root test. Since a random walk has a unit test, if this test allows you to reject the hypothesis of a unit root, you can dismiss the hypothesis of a random walk. According to my calculations, the seasonally differenced series has a PP test p-value of 0.01 (the p-value for the original series is also 0.01), so I would suggest that we can reject the random walk hypothesis with 99% confidence.

Sorry to have seemed to ignore you but my last posting was started an age back.

I am no expert on either Hurst or wavelets but both are of interest to me. I do what I can, and it is always good to learn new things even if it does take me a lot longer than it once did.

Thanks for thanks, I am sure we all like to hear that it is not all a waste of time.

To Basil,

As you no longer wish to discuss it and have not given a justification for the particular choice of filter you used to get a H value close to 0.5 I will presume you do not have one.

And you are correct I do tend to stop at the point where an argument seems to be weak if things that follow reference it and do not seem to strengthen it.

I have moved on to HP filters which as I said I was not familiar with. Why should I get overly concerned with a section that I was not equiped to reason about.

I have posted a breif overview of how I see the performance of HP filters above which may or may not be correct, either way I expect, or at least hope, that it will interest others, maybe even your.

Hopefully that post and my long post on various filters will be leason to the circumspect.

Once I have done some more background, I will see if I can implement HP filtering and see If I get the same results.

Now you may or may not be interested in the nuts and bolts of what filtering does to signals; I certainly am.

Anyway I look forward to figure 3, hopefully I am saving the best to last.

Alexander Harvey

Directly from the definition it appears to me to be a low pass filter with the folowing gain.

G(w) = 1/(1+L*(w/A)^4)

where L is the lambda value and “A” is the number of elements per unit time.

Hence its slope is to 1/f^4, with a cut-off frequency given by:

fc =A/(2*pi*L^(.25))

In this case a=12 (months in a year), and L=12900 giving fc=0.1 cycles/yr.

It does appear to have one very useful property in the the phase lag is zero.

It appears to be have the Buterwortth property of maximal flatness but differs in that G(w) is of the form 1/(1+f(w)) as opposed to the 1/Sqrt(1+f(w)) for the Butterworth filters I know. Also the zero phase lag is not typical.

At first sight it does not seem to be a real time realisable filter in that the minimisation process that describes the HP filter is over the entire series and would require foresight to impliment.

I also suspect that it generalises into a range of filters in a fashion similar to the Butterworth family.

That said it basically is a moderately steep low phase filter. In this case it attenuates signals with a periods less than 10 years.

By the time it gets to down to 2 years its gain is down to ~1/600. I expect that this may be why it does not care if it is coupled to a 12 monthly differenced filter or a 1 monthly differenced filter. Either combination will have a pass band somewhere around 10 years and similar performance both above and below that period.

Alexander Harvey

Alex,

I would like to discuss this with you, but Hurst coefficients are new to me and my plate is already full for the foreseeable weeks. I’ve downloaded the papers others have cited above. I thank you & others for sharing your enthusiasm for the methods under discussion – I hope to look into this in the future.

Also: I hope we (WUWT participants) might start having some in-depth discussions about wavelet methods moving forward. It’s tricky when not everyone in the audience knows a method. For example, I suspect many readers might better-understand the info in my wavelet plots (upthread) if I simply present the following plain time-plots instead:

http://www.sfu.ca/~plv/(J,N)o2&Pr.png
http://www.sfu.ca/~plv/PhaseConcordancePxySI.png
http://www.sfu.ca/~plv/(J,N),r..png
http://www.sfu.ca/~plv/Pr,JN4,r..,m4..png

Perhaps these will lead readers unfamiliar with wavelet methods to a better appreciation for the post-1940 stability of the Chandler Wobble period.

We are certainly well-into an era when specialized statistical-computing is leading to communication break-downs.

Regards,
Paul.

could you please reply to my questions, too? You seem to have missed them. I’d just like to see, whether your analysis is sensitive to long-term trends.

Regarding Jose’s silly challenge, let me put that out of its misery. He linked to a list comprised of less than one-tenth the number of signers of the OISM Petition, which is limited to those with degrees in the hard sciences.

AAAS has about 125,000 members, 84% of which is 105,000, if you really want to play the numbers game.

But Jose’s list? His list includes poseurs with degrees in things like Community Ecology, Advertising, Prediction [heh], Politics, Downscaling, Market
Research, Economics, etc.

Community Ecology has nothing to do with your local town hall, but is relevant for impact assessment; do you have anything specific to say against GC Hurtt? Or FW Zwiers or C Rosenzweig? You mocked them for being listed under “prediction”. And why would you, Smokey, of all people, mock JS Armstrong, the only one listing advertising? He signed the 2009 newspaper ad by the Cato Institute. His most cited paper on climate is published in E&E and his paper on Polar bear population is published in Interfaces, a curious choice, given that the journal’s scope is “Learn how to overcome the difficulties and issues encountered in applying operations research and management science to real-life situations.”. Too bad that most of his citations are not for his work on climate, otherwise you would have had a winner. I have not checked further.

Given how shallow your reading seems to be, I’m stepping out of this discussion.

Thankyou for your respectful responses.

And thanks to you, too. I sometimes get impatient, as in my previous reply to Alex, but I do appreciate vigorous discussion and challenge, so long as I think something constructive can come of it. So again, thanks.

I do not think that you have given us a H value for this filter of the HadCRUT3. It you have I apologise but could you give it again.

Yes I did post this above, but in a reply to PaulM, who wrote:

3. If I do a month-to-month difference, as some people have suggested, instead of the seasonal difference, I get a much smaller number, around 0.1 – I don’t understand why this is!

To which I responded:

“I get 0.24. What software are you using? (I’m using gretl.) As for the number being noticeably smaller, I’m not sure that should be so surprising. That is simply saying that the degree of anti-persistence is greater in monthly fluctuations than it is in annual fluctuations. That makes sense to me, in that monthly fluctuations will tend to revert to the mean more quickly than shocks measured on an annual basis. Actually, given my numbers — 0.24 for monthly, and 0.38 for seasonal — the difference is about what I would expect.”

I think the explanation holds up as well for any point you are trying to make of this.

At this stage I am concentrating on figure 2 not figure 3. I have not commented on the H values for figure 3, so need to restate them.

No, I do not need to restate them. You need to do me the courtesy of reading the whole post, and get off of your fixation over Figure 2. I’m not discussing this any more with you, if after all this time you haven’t even bothered to read the entire original post (or read it so cursorily that you do not recall that what you now ask me to restate in in the post, as was a major focus of the post). Frankly, you have just vindicated my previous post (in reply to Stevo). You are wasting my time. I’ve got more to do on this, so I shall consider our discussion at an end, and move on to other matters.

With all the false-alarmist huffing and puffing, they still can’t come up with more than a small fraction of 31,000 scientists who say:

There is no convincing scientific evidence that human release of carbon dioxide, methane, or other greenhouse gases is causing or will, in the foreseeable future, cause catastrophic heating of the Earth’s atmosphere and disruption of the Earth’s climate. Moreover, there is substantial scientific evidence that increases in atmospheric carbon dioxide produce many beneficial effects upon the natural plant and animal environments of the Earth.

What part of ‘no convincing scientific evidence’ do you people fail to understand?

Regarding Jose’s silly challenge, let me put that out of its misery. He linked to a list comprised of less than one-tenth the number of signers of the OISM Petition, which is limited to those with degrees in the hard sciences.

But Jose’s list? His list includes poseurs with degrees in things like Community Ecology, Advertising, Prediction [heh], Politics, Downscaling, Market Research, Economics, etc.

Furthermore, about 40% of Jose’s list is comprised of UN/IPCC political appointees, who have their marching orders, no matter what they privately think. Those IPCC individuals have traded their credibility for job security.

And Pew? Give me a break. They were easily the most error prone, inaccurate polling in the last election; Pew polls are used to skew results. In other words, for propaganda. How else can we take an all-or-nothing question like “Warming is due to human activities”? Does that mean 100% of warming? Hang your hat on that push poll if you want, but it is not credible when it frames questions like that.

I won’t embarrass the inept Jose Sanchez for his foolish challenge: “I challenge you to find one scientists in the top 500 most published global warming researchers that claims we are now in a global cooling trend.”

Jose’s own list includes internationally esteemed names like Ross McKitrick, Roy Spencer, Lubos Motl, Fred Seitz, William Gray, Freeman Dyson, Nils-Axel Morner, Pielke, Sr. & son, Edward Wegman, Roger Revelle, Chris Landsea, Syun-Ichi Akasofu, Nir Shaviv, Bob Carter, Craig Idso, Tim Ball, Willie Soon, Henrik Svensmark, Piers Corbyn, Benny Peiser, Hans Erren, Joe D’Aleo, Vincent Gray, and plenty of other AGW skeptics. The list also has the names of disreputable individuals like Caspar Amman, Rajendra Pachauri, William Connolley and Michael Mann, who wouldn’t recognize integrity if it bit ‘em on the ankle.

Jose was simply winging it with his fingers crossed, hoping that no one would check the list he posted. I’d be willing to wager $10,000 that I can find someone on his list who thinks we’re in a cooling trend, if you or Jose are game. Otherwise, why bother? We all know the answer to Jose’s foolish challenge.

So you and Jose lose both the “consensus” claim and the challenge, hands down. You couldn’t even come up with one-tenth the number of scientists that the skeptic side has. Which is the way it should be: skepticism is an absolute requirement of the scientific method — a requirement that the false-alarmist side has given up and surrendered to greed. Now, they’re just grant hogs with both front feet in the public trough.

Perhaps we have a terminology problem in case so, I will clarify.

I am referring to 12 month differencing X(13)-X(1) etc., of the HadCRUT3 time series for which I get H=0.415 (You gave us H=0.475)

and to 1 month differencing X(2) – X(1) for which I get H=0.228.

I do not think that you have given us a H value for this filter of the HadCRUT3. It you have I apologise but could you give it again.

So for me one filter gives a series with H close to 0.5 and the other definitely does not.

I felt that you were drawing some significance to its closeness to zero and I am saying that it could be seen as happenstance as H values differs if you use different filters.

I have tried examples of other filters quite arbitrarily just because I or others have your choice might be seen as arbitrary and suggested such filters as alternatives.

For X(25)-X(1) I get H=0.533.

This filter has 11 stop bands (inside the pass band) with amplification in its pass bands.

Also for simple high pass filter (time constant = 1/(2pi)yrs) I get H=0.238.

This filter definitely does not have any stop bands (inside the pass band), only a minima at f=0, also and at no frequency does it amplify the variance.

Another method of minimising the seasonal component that, for me, leaves the H value largely unchanged and close to 1.

Here I am referring to subtracting a climatology based on the means (over the whole interval) for each month,

I get H=0.980 after removing a climatology. This is the same value (3dp) I get for the unfiltered series.

This type of filtering has the same stop bands as 12 month differencing but they are much finer. It does remove the seasonal cycle without doing a lot else. In this case it removed about 27% of total variance whereas 12 month differencing removed > 60%. It is also not a high pass filter. It is flat across the spectrum except for narrow stop bands at 12,6,4, … month periods. This is almost certainly why it leaves the H value almost unchanged.

As I mentioned above 12 month differencing also amplifies the variance across the majority of the width of the pass bands so the 60% does not give such a good guide to how much more targeted subtracting a climatology is. After adjusting for this amplification by dividing the series by 2 to reduce the maximum gain in the pass band to unity, >90% of the variance has been removed by 12 month differencing).

So the filtering effect of the 12 month interval does produce stop bands at 1,2,3,4,5 Cycles/Yr whereas I find a 1 month interval does not.

If you are saying that one month differencing produces stop bands can you tell me where they are?

For clarification I have not been included the behaviour for f->0 as this minimum is common to high pass filters I am referring to stop bands inside the pass band.

For reference: The number of stop bands (inside the pass band) is given by N/2 – 1 (N=interval in months) for even months, and (N-1)/2 for odd months. I doubt that I am wrong here. If you wish to count minima at the top of the pass band (6 cycles/yr in this case) you add one for even months.

So I definitely get different Hurst coefficients depending on my choice of filter.

Now I make no bones about what I do and do not know. I do know the results I get and if I discover I am wrong I will take it on the chin.

You say that you can get there without seasonal differencing yet you chose to do it, and I thought that you were drawing some significance from a H value ~0.5. If you no longer think that was a point worth making just snip it out.

I would do so because I think it is not sound. I find it to be highly dependent on your choice of a filter that for me seems to serve no purpose other than to provide this value. I am not sure why you added this stage which you say you do not need, and except for saying that people commonly do this I am not sure you have done much more to justify doing it.

Now I think you were asking for some specifics so I have provided some, hopefully this will clarify the discrepancy between our points of view on the choice of filters.

BTW at no point have I referred to your processing beyond this point, as I have no idea what Hodrick-Prescott smoothing would do to Hurst coefficients, so I could not comment. But I shall look to it.

At this stage I am concentrating on figure 2 not figure 3. I have not commented on the H values for figure 3, so need to restate them.

Alexander

Thankyou for your respectful responses.

Part of the reason I don’t criticise your main point is that I don’t see any reason to disagree with it. That’s partly because I’m already of the view that a lot of the short term ‘trends’ are spurious randomness anyway, the result of stochastic processes with memory, but partly because the issues with methodology obscure precisely what you’re doing and what you mean by it for me.

But the main reason I raise such objections is in the hopes of improving the arguments. When I start reading through an argument and suddenly run into something I know is wrong, it jerks me up short. I find it distracting and off-putting. Even when it is irrelevant to the rest of the argument, it is still an error. And if the rest of the argument continues to refer to and rely upon it, it makes things very difficult to follow. And worse, even irrelevant points get picked up and remembered as background knowledge by other readers, and repeated elsewhere.

It’s one reason why I keep ranting about it every time I see someone repeat the line that greenhouse gases “trap” radiation, even if it’s just by way of a scene-setting intro and not relevant to the main argument, because other people then go on to waste huge amounts of effort to debating and debunking a mechanism that isn’t even the ‘official’ understanding of the greenhouse effect anyway. Letting the small errors pass perpetuates the myths.

Criticism isn’t always opposition. I do sometimes find that when I criticise a sceptic’s argument, people assume I’m therefore an AGW-believer. I had hoped that citing Climate Audit might allay that, but it doesn’t always work. Many of the arguments in this area, on both sides, are in some small detail wrong. That includes many of my own. Nobody likes that; but it’s unfortunately inevitable. On the whole, I’ve generally found sceptics to be more open-minded with regard to improving them.

You have paid more attention than many would, for which I thank you. I apologise for the misunderstanding. Your conviction that there is something worthwhile in what you are doing is not something I have any wish to argue against. My aim is to help, because I think the details do matter.