Crowdsourcing A Full Kernel Cascaded Triple Running Mean Low Pass Filter, No Seriously…

Why bother with HadCrut and GISS? They are worthless. Use UAH and RSS, but get rid of that annual version and use monthly versions. And draw the trend with a semi-transparent magic marker. See Figure 15 in my book “What Warming?”

Arno 100% correct GISS, HADCRUT are trash data I don’t understand why any posting using that data can be taken seriously *”adjustments”, “UHI ect…”. This is just feeding the warmist trolls.

Very nice elucidation of the 60 year cycle in the temperature data, Now do the same for the past 2000 years using a suitable filter. A review of candidate proxy data reconstructions and the historical record of climate during the last 2000 years suggests that at this time the most useful reconstruction for identifying temperature trends in the latest important millennial cycle is that of Christiansen and Ljungqvist 2012 (Fig 5)
http://www.clim-past.net/8/765/2012/cp-8-765-2012.pdf
For a forecast of the coming cooling based on the 60 and 1000 year quasi periodicities in the temperatures and the neutron count as a proxy for solar “activity ”
see http://climatesense-norpag.blogspot.com
this also has the Christiansen plot see Fig3. and the 1000 year cycle from ice cores Fig4
The biggest uncertainty in these forecasts is the uncertainty in the timing of the 1000 year cycle peak.
In the figure in Richards post it looks like it is about 2009. From the SST data it looks like about 2003. See Fig 7 in the link and NOAA data at
ftp://ftp.ncdc.noaa.gov/pub/data/anomalies/annual.ocean.90S.90N.df_1901-2000mean.dat
It is time to abandon forecasting from models and for discussion and forecasting purposes use the pattern recognition method seen at the link
http://climatesense-norpag.blogspot.com .

Lance Wallace says on March 16, 2014 at 1:42 pm:
“The CO2 curve (seasonally detrended monthly) can be fit remarkably closely by a quadratic or exponential curve, each with <1% error for 650 or so consecutive months…"
So what. CO2 is not the cause of any global warming and is not worth that expenditure of useless arithmetic. The only thing important about it is that it is completely smooth (except for its seasonal wiggle) during the last two centuries. It follows from this that it is physically impossible to start any greenhouse warming during these two centuries. We already know that there has been no warming for the last 17 years despite the fact that there is more carbon dioxide in the air now than ever before. That makes the twenty-first century greenhouse free. Since this constant addition of CO2 is not causing any warming it follows that the theory of enhanced greenhouse warming is defective. It does not work and should be discarded. The only theory that correctly describes this behavior is the Miskolczi theory that ignorant global warming activists hate. But the twentieth century did have warming. It came in two spurts. The first one started in 1910, raised global temperature by half a degree Celsius, and stopped in 1940. The second one started in 1999, raised global temperature by a third of a degree in only three years, and then stopped. Here is where the smoothness of the CO2 curve comes in. Radiation laws of physics require that in order to start an enhanced greenhouse warming you must simultaneously add carbon dioxide to the atmosphere. That is because the absorbency of CO2 for infrared radiation is a fixed property of its molecules that cannot be changed. To get more warming, get more molecules. This did not happen in 1910 or in 1999 as shown by the Keeling curve and its extension. Hence, all warming within the twentieth century was natural warming, not enhanced greenhouse warming. Cobsequently we now have the twentieth and the twenty-first centuries both entirely greenhouse free. Hence that anthropogenic global warming that is the life blood of IPCC simply does not exist. To put it in other words: AGW has proven to be nothing more than a pseudo-scientific fantasy, result of a false belief that Hansen discovered greenhouse warming in 1988.

Is it possible to humor some of us old-timer digital filter designers by specifically saying what the (Finite) Impulse Response of the CTRM is; in basic terms such as what simpler elements are being cascaded (convolved for the IR response, multiplied for the frequency response, etc.)? This is the key to understanding filtering – old or new. Thanks.

I have a couple of questions regarding the initial filter.
1. A 365 day filter has an ongoing problem due to leap-years. There are, roughly, 25 leap days in a century, which pushes a 365 day filter almost a month out of alignment. How is this catered for? It’s not a problem with a 12 month filter, but then you hit the problem that each month is not equal in length.
2. You’re doing filtering on anomalies to remove the seasonal component. Aren’t the anomalies supposed to do that themselves, since they’re anomalies from the average for each part of the season. That is, the anomalies themselves are trying to remove the seasonal component, and the filter is also trying to remove the seasonal component. How do ensure that any result we get isn’t an artifact of these two processes interacting?
Please forgive me if these questions show my ignorance, but they’ve been a concern of mine for awhile.

I am not too fond of the triple running mean filter, mainly because it is not easy to implement generically in Excel. By that I mean if I wanted to change the number of points in one stage, it is a fair amount of work. Also there are three smoothing parameters to tinker with rather than a single number of regression points, and I have not seen any good theory on how to pick the three ranges to average over.
I prefer either a LOESS or Gaussian filter for this kind of data. Both can be implemented easily in Excel with macros. Both have the advantage of being able to handle points near either end of the data set. LOESS also works when the points are not equally spaced. By selecting the proper number of regression points, either one will be able to produce results virtually indistinguishable from the triple running mean filter. Also with LOESS it is possible to estimate the derivative at any point.
I have not experimented much with the Savitzky-Golay filter yet because it requires recalculating the coefficients with each change in the order or number of regression points. Can be done of course, just makes the macros more complicated. And I am not sure it is worth the effort. One of these days I’ll probably get around to it.
Any smoothing like this is mainly just a data visualization tool. How it looks is largely subjective. Rarely do I see a smoothed signal being used in later analysis, such as correlating it with something; and when I do it raises a red flag.

RLH The Christiansen data I linked to are I think archived. Read the paper and see why I think they do provide a temperature record which is good enough to be useful for analysis. It should replace the Mann hockey stick in the public perception of the NH climate over the last 1000 years.
Again check Figs 3 and 4 at http://climatesense-norpag.blogspot.com.
Forecasting is fairly straightforward if you know where you are on the 1000 year quasi-periodicity.
If you don’t take it into consideration you are lost in a maze of short term noise.

It’s hard to tell what this is from the description in words. Can you show a block diagram or provide a spreadsheet? Also just showing the time response to a step input, or the frequency response would be helpful.
It sounds like it’s an FIR filter with no feedback of the output. If so, you should be able to multiply out the nested means and the coefficient and get 12 coefficients, one for each tap of an FIR.
One quibble: 75 years is period, not a frequency. It would be more correct to say “…frequencies with periods greater than 75 years…”

RichardLH says: March 16, 2014 at 3:30 pm
“Bernie Hutchins says:
March 16, 2014 at 3:14 pm
“Is it possible to humor some of us old-timer digital filter designers by specifically saying what the (Finite) Impulse Response of the CTRM is”
Basically the triple cascade allows for a near to Gaussian response curve. This can be seen in greater detail on Greg’s thread where the various curves can be compared. CTRM is just slightly better than Gaussian in that it completely removes the corner frequency whereas a true Gaussian does leave a small portion still in the output.”
Fine – thanks. But we need a “formula” or design procedure that leads to h(n) = …. so that we can calculate the frequency response in a known way (sum of frequency domain cosines in this case). It seems to me that SG with its optimal flatness around DC couldn’t be beaten. Because the data is noisy, spurious resonances (if any) matter. Hence the advantage of SG.

Assuming monthly data points: 1, 2, etc. Is this how it works?
M1 = mean(1,2,3,4)
M2 = mean(5,6,7,8)
M3 = mean(9,10,11,12)
M4 = 1.2067 x mean(M!,M2)
M5 = 1.2067 x mean(M2,M3)
OUTPUT = 1.2067 x mean(M4,M5)
Shift data one place and continue…

RichardLH says:
March 16, 2014 at 4:09 pm
Thanks Richard – that’s what I thought.
But when you cascade Moving Averages (running means, rectangles, “boxcars”) you get all the nulls of the original rectangles in the cascade. When you cascade Gaussians, you get essentially the smallest bandwidth of the set (longest average). With SG, you get an attractive wide low-pass bandwidth (optimally flat) at the expense of less high-pass rejection (never a free lunch).
SG is easy to do. By coincidence I posted an application note on it just a month ago. Note my Fig. 6b for example compared to the two frequency responses you posted for me.
http://electronotes.netfirms.com/AN404.pdf
Without due regard for the frequency response, you COULD BE seeing “colored noise” due to a spurious resonance. You are certainly NOT seeing something totally spurious, because for one thing, we can “see” the 60 year periodicity already in the original data. Less noise – but no new information?

RLH
Here’s the link . You will have to read the paper carefully to see which data was used in the paper and how Check on Christiansen
at
ftp://ftp.ncdc.noaa.gov/pub/data/paleo/contributions_by_author/
To look at the longer wavelengths you really need FFT and wavelet analysis of the Holocene data See Fig 4 at http://climatesense-norpag.blogspot.com
1000 year periodicity looks good at 10,000, 9000,8000,7000- then resonance fades out comes back in at 2000,1000 0.

I think your blog is one of the top conservatarian blogs out there and I put you in my links. Keep up the good work. http://the-paste.blogspot.com/ , http://thedailysmug.blogspot.com/

Off the subject:
Most of the time, just read here go elsewhere on the net and seek out those who need to join in and link back here. The first word of this post “Crowdsourceing” struck me.
What we are missing is a “Crowdsourceing” person from the warming side to focus more voters to this site and others like this site and or to pull people out of their daily lives and into the struggle.
Back in say 2006 to 2008 old Al Gore was all about and never ever shut his yap.
Seems we need to find a way to get someone as polarizing Al Gore together with others like him to once more and get them all back into the fray and go all bombastic and over the top with wild claims. Think up a way to gig his or some other ones of their over sized egos and get them in the press and then use that to pull in more people of reason to the battle ground.
Not that these types will ever go away but we need to get this lie based fraud redistribution of wealth pulled way down to earth.

RichardLH says:
March 16, 2014 at 1:58 pm
“It is useful to compare how the CO2 curve matches to the residual after you have removed the ~60 ‘cycle’ and there it does match quite well but with one big problem, you need to find something else before 1850 to make it all work out right.”
There’s a bigger problem. It doesn’t really match well at all. The match is between the integral of temperature and CO2.
And, that’s GISS. The match with the satellite record is even better.
CO2 is not driving temperatures to any level of significance at all. It is, instead, itself accumulating due to a natural process which is modulated by temperatures. Once you remove the trend and the ~60 year periodicity, which have been around since at least 1880, well before CO2 is believed to have increased significantly, there is very little left of temperature to be driven.

RichardLH said in part March 16, 2014 at 6:38 pm
“Too many animations of images in the work I have done elsewhere I suppose, it just looks like a caterpillar to me 🙂 Sorry.
http://en.wikipedia.org/wiki/File:Lissage_sg3_anim.gif”
Richard – Thanks yet again. That helps. Glad we are LTI.
But you are completely wrong (misled by that animation I think) about whipping. None of those yellow polynomials in the animation you posted whipping about are ever even computed. Instead the SG fit is a FIXED FIR filter. Now, to be honest, the first time I learned this, the instructor’s words were exactly: “Astoundingly, this is an FIR filter.” I WAS astounded – still am. No whipping.
[ Aside: In general, polynomials as models of signals are going to be unsuited. Signals are mainly horizontal, while polynomials are vertically in a hurry to get to + or – infinity. Your whipping. ]
All you need do is populate a non-square matrix with integers (like -5 to +5 for length 11), raise these integers to successive powers for the rows below, take the least-squares (pseudo) inverse for the non-square matrix, and use the bottom row as the impulse response. (It is basically “sinc like”.) That’s the nuts and bolts – no polynomial fitting, and always the same answer for a given length and order. See my previous app note link. Yea – magic!
So it is no more complicated (once rather easily computed) than Gaussian. And it’s a much better filter.

Matthew R Marler said in part: March 16, 2014 at 7:23 pm
“Does anybody know what the result will look like when the “correct” smoothing method is used?”
No. Nobody knows! Unless you have a physical argument of other evidence, curve fitting is near useless. And the physics – let’s call it “incomplete”. Again, data fit to a curve is not THE data. Curve fitting destroys information. You can always get the proposed curve (again), but you can’t go back.
Many years ago I did a physics lab experiment and plotted data on every possible type of graph paper I could find in the campus store. My professor (Herbert Mahr) was kind enough to compliment my artistic efforts, but then assured me that none of it necessarily MEANT ANYTHING. Indeed.

Sorry guys but this is absolutely worthless. There’s no data-set worth an iota pre-satellite era and post-satellite era lacks enough data to be useful. Even if there were a data-set that was accurate back to circa 1850 that’s still an incredibly short snippet of time wrt Milankovitch Cycle lengths and even the supposed Bond Cycle lengths.

jai mitchell says:
March 16, 2014 at 2:39 pm
. . .hilarious. . .
—————————————————-
You quote skepticalscience, and you laugh at other people?
Now that is hilarious.

jai mitchell says:
March 16, 2014 at 2:39 pm
“Since this constant addition of CO2 is not causing any warming it follows that the theory of enhanced greenhouse warming is defective. It does not work and should be discarded.”
. . .hilarious. . .
——————————————————————————————————–
Even more hilarious is if you do the calcs for the volume of ocean down to the depth quoted then the graph you quote represents only a few hundredths of 1 degree Centigrade rise in temperature. i.e. 2/5ths of bugger-all. And there is no evidence that the CO2 bogey is responsible.

wbrozek says in part March 16, 2014 at 8:33 pm
“In that case, how would you prove that the warming that is occurring now is not catastrophic?”
You ask a good question and since no one else is apparently still up, I will give it a shot.
In physics, particularly with regard to chaotic non-linear systems (like the climate), but still constrained, there is really no such thing as “proof”. There is instead evidence. The various so-call “global temperature” series, although surely imperfect, are evidence (not proof) that the catastrophic “hockey stick” warming is likely wrong.
The closest thing to establishing the truth is almost certainly the 2nd Law of Thermodynamics. Essentially heat moves from where there is more to where there is less. If the path of flow is not obviously in place, Nature is OBLIGED to provide it. In consequence, certain negative feedback thermostatting mechanisms are mandatory. No engineer or HVAC technician required. The great physicist (is there any other kind of physicist) Eddington famously and amusingly said, often quoted here I believe:
“If someone points out to you that your pet theory of the universe is in disagreement with Maxwell’s equations—then so much the worse for Maxwell’s equations. If it is found to be contradicted by observation—well these experimentalists do bungle things sometimes. But if your theory is found to be against the second law of thermodynamics I can give you no hope; there is nothing for it but to collapse in deepest humiliation.”
Any folks we know!

Bart : http://s1136.photobucket.com/user/Bartemis/media/CO2GISS.jpg.html?sort=3&o=0
That is very interesting.The short term (decadal) correlation of d/dt(CO2) to SST has been discussed quite at bit and is irrefutable. The ice core records seem to show direct CO2 vs temp correlation. So the question is where in between does it change? There will be a sliding mix as the response changes from one extreme to the other.
If earlier data diverges it may be the first indication of the centennial relationship or equally likely an indication of spurious data adjustments. That would be worth investigating.
Do you have a similar graph that goes back to say 1850?

Here is an example of the lanczos filter used on daily satellite TLS data, to remove annual signal.
http://climategrog.wordpress.com/?attachment_id=750

Willis: “As tempting as it may be to read a “cycle” into it, there is no “~ 60 year cycle”. It’s just a wiggle. ”
There is a long term change in there too Willis. Cooling at end of 19th c , warming since beginning of 20th. When you add a slope to a pure cosine you will find that it shifts the peaks and troughs, when you have differing slopes behind it they will get moved back and forth. That is the cause of the irregularity of the intervals you have noted.
Once again, you display your lack of understanding and over-confidence in your own abilities, then start telling others the way it is. You, like our host, seem to have adopted “the science is settled” attitude to cycles. Just remember that science is never settled and keep an open mind.
It is true that there is no guarantee that this will continue but so for it seems to fit quite well to the “pause” now becoming a down turn since 2005.

two much = too much 😉

Filters throw away information.
The raw data, is the most information you can ever have.
Filters simply delude one into believing that something else is happening; other that what the measured sampled data values tell.

Bernie Hutchins. “… the instructor’s words were exactly: “Astoundingly, this is an FIR filter.” I WAS astounded ”
So am I.
I’ve always been dubious of LOESS filters because they have a frequency response that changes with the data content. I’ll have to have a closer look at SG.
It does have a rather lumpy stop band though. I tend to prefer Lanczos for a filter with a flat pass band ( the major defect of gaussian and the triple RM types ).

Richard or others
Nice concise post. I have three questions – sorry.
1) Are the cascade filters based on recursive operations using some basis functions? If yes, then is this not akin to curve fitting? I’ve probably misunderstood but I think you’ve taken issue with this before.
2) In the field I work in we tend to use the Butterworth filter extensively; but in light of the recent post on “signal stationarity” in WUWT, does such an approach seem inappropriate if the composed data series is the sum of many non-stationary signals. I suspect that there will be major issues with significant localised phase shifts (something I understand to be a problem with the Butterworth filter even with “perfect” data sets).
3) Finally, for the purposes of quant operations, is the filtered data any use beyond signal analysis? For example, even if one uses the best filtering method, does the resulting processed series not take with it significant bias: when one tries to statistically quantity something like correlation between two filtered data sets. This seems a little open ended I know, but you see this type of analysis all the time.

Richard, can you design a filter to remove comments from the people who don’t believe the data in the first place?
Why they bother commenting is a mystery to me ;(

What happens when you apply the same filters to the sunspot data over the last 150 or so years?

I have used Fourier analysis successfully to detect periodic and pseudo-periodic effects on data. When viewed in the frequency space, the peaks in the Fourier Transform identify such effects, while shallows, regions in the transform space where peaks are notably absent, identify ‘sweet spots’ where the cutoffs for filters can be tuned.
When applied to sunspot numbers. for example, there is a group of peaks corresponding to 11-13 year periods and a secondary group at about a 100 year period, but relatively little amplitude in between.
That is about the limit of applicability of the Fourier transform, however. The periodicity inherent in the mathematics makes it useless for extrapolations or for dealing with secular trends.This will allow optimization of tuned filters, however.

Richard,
John Linsley Hood was one of the greatest analogue electronic design engineers the UK has ever produced. I wish I’d known him. I spent a lot of my time at university back in the 80’s building his designs when I should have been using my project budget on, well, my project.
I am sad he’s gone, but I don’t think he’ll be forgotten.

RichardLH
Thanks for your reply.
With regard to your answer to my question 2. I appreciate that the Butterworth filter (frequency domain) can be seen as an approximation of a Gaussian convolution given complementary parameters. And I’m not talking about the relative merits of either. I guess what I’m really asking is: Is there a problem in designing a filter based on spectral profile (“hard” approach ), as opposed to eye-balling the parameter in the host domain (“soft” approach), if the composed signal (the raw data) is the sum of many non-stationary signals?
In my field of study, the Butterworth is effectively a passband filter where the higher frequencies are typically removed, according to the filter parameters which are determine from the series’ spectral profile. But the recent discussion, with respect to signal stationarity, would deem such an approach foolhardy rather than proper, and yet it’s used extensively.
I was interested in hearing your views on this if you have time.
I hope it doesn’t seem off topic.

“Now the question is how I can improve it. Do you see any flaws in the methodology or tool I’ve developed?”
Because HADCRUT4 data by CRU is greatly incorrect (such as within parts of the 1930-1980 period), as is that from Hansen’s GISS, unfortunately any analysis based on HADCRUT4 is also greatly incorrect. While such has the global cooling scare of the 1960s-1970s occur without substantial cooling beforehand in the global or Northern Hemisphere average, as if it just happened with little basis, I would challenge you or anyone (if inclined to defend it, though you might not be) to find any publication made prior to the CAGW movement of the 1980s onward which shows the 1960s-1970s without showing far more cooling relative to the warmer period of the 1930s-1950s.
For instance, a 1976 National Geographic’s publication of the temperature record of scientists of the time, in the following link, is one I have literally seen in paper form in a library, unlike those repeatedly rewritten-later electronic versions which by strange coincidence (not!) happen to be respectively by a group infamous in Climategroup and by a department which has been under the direction of someone so much an activist as to have been arrested repeatedly in protests:
See that and other illustrations of original temperature data around 40% of the way down in http://tinyurl.com/nbnh7hq
“Are there any particular combinations of data sets that you would like to see?”
While it would be a significant amount of work, digging up original non-rewritten temperature data for history up through the 1970s (prior to the political era, back when there wasn’t motivation to fudge it), digitalizing it, and then carefully joining it onto later temperature data (from more questionable sources but when no alternative) would be a better starting point. Joining together two datasets like that isn’t in theory ideal but the best option in practice; several years of overlap could be used to help check the method. The Northern Hemisphere average, the Southern Hemisphere average, and the global average could best be each generated, as there are reasons, for instance, Antarctic temperatures follow different patterns than the arctic (as the prior link implies).
Of course, like me, you might not have time to do so in the near future even if you desired. But that is something needed yet utterly lacking, as browsing enough skeptic websites indirectly illustrates. The ideal for convenience of other analysis would be both producing plots and producing a list of the values by year (e.g. a data spreadsheet).

To add, regarding this:
I would challenge you or anyone (if inclined to defend it, though you might not be) to find any publication made prior to the CAGW movement of the 1980s onward which shows the 1960s-1970s without showing far more cooling relative to the warmer period of the 1930s-1950s.
The Berkeley BEST set made in the modern day by someone who pretended to be a skeptic without pro-CAGW-movement bias but was an environmentalist (as found out in looking at some of his writing beforehand IIRC) does not even remotely meet that challenge. Only original paper publications (or a clear scan not looking rewritten in any way) made prior to the existence of the global warming movement would count.

MR Marler. “Filtering” is nothing more than than fitting data by a method that uses a set of basis functions, and then separating the results into two components (as said by Greg Goodman),
I said nothing of the sort. Don’t use my name to back up your ignorant claims.

RichardLH says:
March 17, 2014 at 6:55 am
“Well they all draw from the same set of physical thermometers.”
So did the data published prior to the 1980s, but that can be seen to be drastically different. When depicting average changes of a tiny fraction of 1% in absolute temperature, of tenths of degree, they depend utterly on the interpolation between widely spaced apart stations, the choice of specific stations used, and, when applicable, hidden adjustments. The practical, convenient way to be certain of no bias in favor of the CAGW movement, without spending thousands of hours personally examining everything, is just to use data published prior to its existence.
In addition to the examples in my prior link, there are others, such as the Northern Hemisphere average history of the National Academy of Sciences, illustrated at http://stevengoddard.files.wordpress.com/2014/03/screenhunter_637-mar-15-11-33.gif in http://stevengoddard.wordpress.com/2014/03/15/yet-another-smoking-gun-of-data-fraud-at-nasa/ , which is utterly different from the CRU/BEST rewritten versions of NH average as well as global average history.
“The head figure is one where I have aligned the 4 major series over the 1979 onwards era and shown how they agree and disagree.”
Obviously, and similar has been seen before, like the undercover CAGW movement supporters on Wikipedia publish a similar plot. However, assuming that RSS or UAH are relatively accurate for the sake of argument, correspondence with them 1979-onwards has absolutely jack to do with disproving rewriting of the pre-1979 section easier to get away with.
If you wish to argue this, then try to meet my challenge:
Find (and link or upload) any publication made prior to the CAGW movement of the 1980s onward which shows the 1960s-1970s without showing far more cooling relative to the warmer period of the 1930s-1950s.
Since you’re arguing this is merely a matter of them all using the same thermometers, that should be easy rather than impossible. Again, what is shown must have been published back then, not a BEST publication of decades later for instance.

Greg Goodman says:
March 17, 2014 at 12:34 am
“Do you have a similar graph that goes back to say 1850?”
I only really trust the CO2 data to 1958. Proxies are… proxies, i.e., not direct measurements. There are no means truly to verify them.
But, this was a hangup with Ferdinand Englebeen. He was keen to point out that, if you accept the ice core measurements as accurate, the relationship should predict too low a CO2 level before 1958, as here.
Besides the fact that I do not trust the ice core data to deliver a faithful reproduction of CO2 levels, I pointed out it was moot anyway, because knowing what happened since 1958 is enough to discount the influence of human inputs during the era of most significant modern rise.
But, if one really must insist on matching a dodgy set of data farther back, there is no reason that the relationship
dCO2/dt = k*(T – To)
must be steady. The fact that it has been since 1958 is actually quite remarkable. But, there could easily have been a regime change in, say, about 1945 which altered the parameters k and To. I showed the effect of such a regime change to To could make the records consistent here. What could that signify? Possibly an alteration in the CO2 content of upwelling waters at about that time. Maybe Godzilla emerging from hibernation stirred up a cache of latent CO2 at the bottom of the ocean. Who knows?
But, in any case, it is moot. Knowing what happened since 1958 is enough to discount the influence of human inputs during the era of most significant modern rise.

Thanks Bart, That was not a challenge to what you said, you just mentioned something about it varying earlier and I presumed you had a graph that went further back that may be been interesting. I agree pre-1958 is a different story in CO2 records.
Those who wish to infer something from ice-cores showing stable millennial scale relationships and modern climate are comparing apples to oranges. The magnitude of the long term , in-phase relationship does not tell us anything about the magnitude of the short term orthogonal relationship.
Ferdi did suggest some ice core data for circa 1500 with 50 year resolution, that may be relevant but I won’t digress too far into that discussion here.
Your graph was very interesting. I may have another look that, Thanks.

Does throw up another question though. Why is it that GISS and HadCRUT are so far apart in the middle? They are close together at the start and the finish, why so different in the middle? I am not sure GISS (or HadCRUT) will thank me that much.
GISS has a (broken, often backwards) UHI correction. HADCRUT does not. Probably other reasons as well. The UHI correction used by GISS maximally affects the ends.
You cannot really say that they are close at the start and finish and different in the middle, because they are anomalies with separately computed absolute bases. That is, there is no guarantee that the mean global temperature computed from the corrected GISS data will correspond to that computed from the uncorrected HADCRUT4 data. Consequently, you can shift the curves up or down by as much as 1C. So perhaps they should be adjusted to be close in the middle, and maximally different at the start and finish.
There are multiple data adjustments in the two series, and while there is substantial overlap in data sources, the data sources are not identical. And then, as Steve M. notes, there are other long running series with their OWN adjustments and (still overlapping) data sources. It is amusing to see how different they are when plotted on the same axes via e.g. W4T, and then to imagine how their similarities and differences would change if one moved them up or down as anomalies around their distinctly computed global average temperatures on an absolute scale. It is even more amusing to consider how they would change if one actually used e.g. personal weather station data that is now readily available on a fairly impressive if somewhat random grid at least in the United States to compute an actual topographical UHI correction with something vaguely approximating a valid statistical/numerical basis for all of the series. It is more amusing still to consider what the series would look like with error bars, but that, at least, we will never see because if it were ever honestly plotted, the game would be over.
This is a partial entre towards not exactly an explicit correction to your approach, but rather to a suggestion for future consideration.
If one does look at your figure 2 above (HADCRUT4 vs smoothed), you will note that the removed noise scales from the right (most recent) to the left (oldest). This is, of course, a reflection of the increasing underlying uncertainty in the data as one goes into the more remote past. In 1850 we knew diddly about the temperature of the world’s oceans (which cover 70% of the “surface” in the global average surface temperature) and there were whole continental-sized tracts of the planet that were virtually unexplored, let alone reliably sampled for their temperature.
This leaves us with several chicken-and-egg problems that your filter may not be able to compensate/correct for. The most important one is systematic bias like the aforementioned UHI, or systematic bias because places like the Arctic and Antarctic and central Africa and central Australia and Siberia and most of the Americas or the world’s oceans were pitifully poorly represented in the data, and some of those surface areas are make dominant contributions to the perceived “anomaly” today (as maps that plot relative anomaly clearly show). Your smoothed curve may smooth the noisy data to reveal the underlying “simple” structure more clearly, but it obviously cannot fix systematic biases, only statistically neutral noise. I know you make no such claim, but it is important to maintain this disclaimer as the differences between different global average temperature anomalies are in part direct measures of these biases.
The second is that your smoothed curve still comes with an implicit error. Some fraction of the removed/filtered noise isn’t just statistical noise, it is actual measurement error, method error, statistical error that may or may not be zero sum. It might be worthwhile to do some sort of secondary computation on the removed noise — perhaps the simplest one, create a smoothed mean-square of the deviation of the data from the smoothed curve — and use it to add some sort of quasi-gaussian error bar around the smoothed curve. Indeed, plotting the absolute and signed difference between the smoothed curve and the data would itself be rather revealing, I think.
rgb

A better result can be obtained using an Ormsby zero phase filter:
http://www.xsgeo.com/course/filt.htm
Industrial scale treatment of seismic data uses Ormsby filters, applied in the Fourier domain, almost exclusively to frequency limit the data.
The frequency content and filter slopes can be established in the design of the filter operator, and the operator is always run from end to end of the data without truncation of the output.

RichardLH says:
”This does allow quite a detailed look at the temperature series that are available. It allows for meaningful comparisons between those series.”
A detailed look at manipulated junk is still manipulated junk (IMHO). I’ll concede it does allow for comparing one manipulated junk data-set to another manipulated junk data-set and thereby revealing they’re different manipulated junk datasets. (JMHO)
Just look at your figure 4 between 1945 and 1970. Does that look like something that would cause an ice age scare?
Look, I really do appreciate your effort but I’m afraid those that have betrayed us and objectivity have ruined any attempt at data analysis for many more years decades to come.

Smoothing is generally the destruction of at least some of the data. If you argue that you still have the original data stashed away – then true enough (obviously) you haven’t lost anything completely. If you argue that you can recover the original from the smoothed alone, in the general case, you are wrong. Inversion is often at least ill-conditioned and usually impossible due to singularities. Most useful filters have what are intended to be stopbands. As such, they generally have nulls (unit-circle zeros). The inverse filter has unit-circle poles and is useless.
A moving average has nulls. SG has nulls. Continuous infinite-duration Gaussian, I believe, has no nulls, but I’m not sure about a truncated Gaussian which is convolved with a sinc? Successive passes (cascading) through moving-average rectangles have nulls – they accumulate.
Smoothing may be curve fitting (such as intentional interpolation), but it need not be. But if you claim insight from a smoothed curve, all you can really claim is that the data provides some evidence for what you feel is a proper model of the data, and the rest you assume is noise, worthy of being discarded. This is certainly circular reasoning at least in part. Such a claim requires considerable caution and full analysis and disclosure.

Richard –
Please read what I wrote, and then think. You said:
“Not true in the case of a digital low pass/high pass band pass splitter filter such as a CTRM. You can do a 1-x by subtracting one band from the input source and get the other.”
I said:
“Smoothing is generally the destruction of at least some of the data. If you argue that you still have the original data stashed away – then true enough (obviously) you haven’t lost anything completely.”
You say you haven’t thrown data away BECAUSE you kept a copy of the input. Didn’t I say that?

RichardLH says:
March 16, 2014 at 1:58 pm
Lance Wallace says:“Can the Triple whatsis somehow compare the two curves and derive either a lag time or an estimate of how much of the CO2 emissions makes it into the observed atmospheric concentrations?”
RichardLH :Not really. Low pass filters are only going to show periodicity in the data and the CO2 figure is a continuously(?) rising curve.
A LPF by definition, passes frequencies below it’s cut-off frequency. Since D.C. (i.e. frequency = 0) is always below the cut-off, the above statement by RLH is incorrect. A ramp-like input into a LPF will cause a lagged ramp-like output, with the lag determined by the filters group delay characteristic.
Note that it is incorrect I think to speak of an LTI filter (such as the one described here) as causing “distortion”, which as a term of signal processing art refers exclusively to artifacts caused by system non-linearity. If you can sum the output of your filter with the residual (i.e. the rejected signal) and reconstruct the input, no distortion has occurred.
Another way of saying this is that an LTI filter can not create energy at a given frequency that is not present in the input signal, even if the filter response peaks badly at that frequency. Such peaking/sidelobe/pb-ripple may extenuate frequencies you wish to extinguish (an so in a sense “distort” the variance present at that frequency relative to some other frequency band of interest), but this is not distortion as the term is normally used.

Richard –
Except as you can construct a proper inverse filter, smoothing (filtering, reduction of bandwidth) will remove information, and here you can’t construct this inverse. You say that you have not lost information because you kept the input separately. That doesn’t count!
Nobody is talking about numerical issues of the processing. Measurement issues (systematic) are a problem as is often discussed on this blog. It is very difficult to see how smoothing helps these.
And we all already agree we can see the apparent 60 year periodicity. What does your method offer that is better, let alone new?
If you are instead suggesting that you are not discarding any important information, then make that case, both in showing that you have chosen the correct parameters for a correct model, and that you understand the errors involved.