UPDATE: See the note at the end of the post.
Date: June 10, 2015
Subject: Karl et al. (2015) Possible artifacts of data biases in the recent global surface warming hiatus and the Sea Surface Data that Support It
From: Bob Tisdale – Independent Researcher
To: Tom Karl – Director NOAA/NCEI
Dear Tom: I’m writing to you with respect to the recent paper, of which you were lead author. The paper is, of course, Karl et al. (2015) Possible artifacts of data biases in the recent global surface warming hiatus (paywalled.) It presented the impacts on the recent slowdown in surface temperature warming of the not-yet-implemented changes to the NOAA/NCEI global land+ocean surface temperature dataset. The changes to the ocean surface portion (the NOAA ERSST.v4 sea surface temperature reconstruction), not the land surface portion, played the larger role in your findings. The changes to that sea surface temperature data are supported by the papers:
- Huang et al. (2015) Extended Reconstructed Sea Surface Temperature version 4 (ERSST.v4), Part I. Upgrades and Intercomparisons, and
- Liu et al. (2015) Extended Reconstructed Sea Surface Temperature version 4 (ERSST.v4): Part II. Parametric and Structural Uncertainty Estimations.
The intent of this letter to present when and how the new NOAA sea surface temperature data differ during the hiatus from the night marine air temperature data, upon which it is based, which are used for bias adjustments over the term of the data.
WHEN AND HOW THE NEW NOAA SEA SURFACE TEMPERATURE DATA DIFFERS FROM THE NIGHT MARINE AIR TEMPERATURE DATA, UPON WHICH IT IS BASED, DURING THE GLOBAL WARMING SLOWDOWN
Figure 1 is Figure 3 from this weekend’s post More Curiosities about NOAA’s New “Pause Busting” Sea Surface Temperature Dataset. The new NOAA ERSST.v4 sea surface temperature data for the latitudes of 60S-60N have a noticeably higher warming rate during the hiatus than the UKMO HadNMAT2 data, which served as the reference for bias adjustments in your new ERSST.v4 data. 
Figure 1
From January 1998 to December 2010, the last month of the HadNMAT2 data, the new NOAA ERSST.v4 data basically doubled the warming rate of the reference HadNMAT2 data. In other words, the HadNMAT2 data do not support your claims of no slowdown in global warming.
Someone might want to try to claim that the higher warming rate of the NOAA ERSST.v4 data is caused by the growing number of buoy-based versus ship-based observations. That logic of course is flawed (1) because the HadNMAT2 data are not impacted by the buoy-ship bias, which is why NOAA used the HadNMAT2 data as a reference in the first place, and (2) because the two datasets have exactly the same warming rate for much of the period shown in Figure 1. That is, the trends of the two datasets are the same from July 1998 to December 2007, a period when buoys were being deployed and becoming the dominant in situ source of sea surface temperature data. See Figure 2. 
Figure 2
Note that magnitude of the trends during that time period. Back in 2008, few people were discussing a slowdown in global warming.
In reality, the differences in the trends shown in Figure 1 are based on the responses to ENSO events. Notice in Figure 1 how the night marine air temperature (HadNMAT2) data have a greater response to the 1997/98 El Niño and as a result they drop more during the transition to the 1998-01 La Niña. We might expect that response from the HADNMAT2 data because they are not infilled, while the greater spatial coverage of the ERSST.v4 data would tend to suppress the data volatility in response to ENSO. We can see the additional volatility of the HadNMAT2 data throughout Figure 2. At the other end of the graph in Figure 1, note how the new NOAA ERSST.v4 sea surface temperature data have the greater response to the 2009/10 El Niño…or, even more likely, they have been adjusted upward unnecessarily. The addition additional response of the sea surface temperature data to the 2009/10 El Niño is odd, to say the least.
What caused that unusual behavior, Tom? Could it possibly be the Lowess filtering you’ve elected to use for the bias adjustments, instead of the linear smoothing you used in the previous ERSST.v3b data? The support paper for the new NOAA sea surface temperature data, Huang et al (2015), states with respect to filtering as shown in its Figure 5 (my boldface):
The monthly fitting coefficients (gray lines) are shown in Fig. 5, which overall fits the fifth assumption that the biases vary slowly with time. To filter out potentially spurious high-frequency noise in the fitting coefficients, a linearly fitted coefficient was used in ERSST.v3b (Smith and Reynolds 2002). Subsequent to ERSST.v3b several analyses have highlighted the likely presence of substantive multidecadal bias variability throughout the record (e.g., Kennedy et al. 2011) rather than simply around the transition from mainly buckets to mainly ERI measures around the early 1940s. In ERSST.v4, a Lowess filter (Cleveland 1981) has been applied on Ay (Fig. 5) and allowed to vary the bias adjustments throughout the record. A filter coefficient of 0.1 is applied to the Lowess, which is equivalent to a low-pass filter of 16 years and represents the low-frequency nature of the required bias adjustment. The reason to apply a filter is to make the bias adjustment smoother so that it may be more consistent with the assumption of applying a climatological SST2NMAT pattern of Am,y. However, we stress that higher-frequency changes in SST biases are virtually certain to exist as indicated in Thompson et al. (2008), Kennedy et al. (2011), and Hirahara et al. (2014). Shorter windows or use of annually averaged data would be noisier by construction because the estimate at any given point would be based upon a smaller sample and it is not clear at what point there becomes a risk of fitting to random sampling noise rather than systematic bias signal. The preference is for robust estimation of the multidecadal component of the bias adjustments using a coefficient of 0.1 but may come at a cost of accurately portraying biases at times of rapid transition (e.g., the WWII era).
And as illustrated and discussed above, could the filtering you’ve elected to use also “come at the cost of accurately portraying biases at times of rapid transition” in response to strong annual perturbations from El Niño events during the hiatus? Something appears it may have definitely added unnecessarily to the buoy-ship bias adjustments of your new sea surface temperature dataset after 2007.
CLOSING
UPDATE: See the note at the end of the post.
Judith Curry commented in a recent post here that that the findings of your recent Karl et al. (2015) paper were based on cherry-picked methods:
This new paper is especially interesting in context of the Karl et al paper, that ‘disappears’ the hiatus. I suspect that the main take home message for the public (those paying attention, anyways) is that the data is really really uncertain and there is plenty of opportunity for scientists to ‘cherry pick’ methods to get desired results.
I would tend to agree. The results of the statistical methods used on the earlier version of the NOAA sea surface temperature data (ERSST.v3b) did not provide the results NOAA was looking for now, so NOAA/NCEI, under your direction, mixed and matched methods until they found the results you wanted (ERSST.v4).
I look forward to your reply.
Sincerely,
Bob Tisdale
# # #
UPDATE (June 16, 2015):
There continue to be comments about my introduction to a quote by Judith Curry in the closing of the post. See the comments starting here on the cross thread at WattsUpWithThat (WUWT) and the comments starting here on the thread of Tim Ball’s recent post at WUWT. I apologize for my poor choice of the word “commented” in the lead-in to the quote. Because Judith Curry didn’t specifically state that Karl et al cherry picked methods, I should have used the word “implied” or “suggested” in place of “commented”. It should have read Judith Curry commented implied (or suggested) in a recent post here that that the findings of your recent Karl et al. (2015) paper were based on cherry-picked methods:
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
“….so NOAA/NCEI, under your direction, mixed and matched methods until they found the results you wanted..”
Ouch…Brother Bob!! Having a comprehensive understanding of the motives of someone you don’t know – is that a gift you were born with, or can anyone learn it??
You certainly know how to tweek a presentation so as to encourage a gainful reply.
Under your standard we would have no need for courts because we could never ever be allowed to determine the mind of the defendant.
Bob did give Karl et al. the benefit of the doubt: Given that the authors must be aware of the implications of the fiddling, and in full realization that these implications should be discovered within hours of publication, they can only have come to the desired (and falsified) result through the use of incompatible/unwarranted/misleading/… methods. Perhaps you believe there is another way this dubious paper could have been produced honestly using real data and statistical best practices?
Yes lets just throw “motive” out of the whole legal system, since it must be completely impossible to determine motive. Those darn prosecutors and their tricks.
So if I started assigning motives to the bloggers here – Brother Bob, Sir Christopher, even the Master – do you reckon that would get past the Mods on guard duty around the Village??
People were talking about a slowdown in global warming in 2008. There was another hide the decline e-mail by Mick Kelly where he discussed how he was leaving the last few points off his presentations to keep the public from seeing a decline in temperatures.
Have they not covered all sins, simply by inserting the word “possible” into title of the paper?
If in ten years time the results of this paper have been discarded, then it, and the road to Paris will be forgotten. If anyone were to even bother to ask, then the fact that the title explained that the paper concerned only “possible artifacts of data biases” would make
complaint seem irrelevant.
The real deception here was the widely published presentation that a single paper suggesting “possible artifacts” has wiped out the need for an explanation of the global warming slowdown during a period of high GHC emissions.
They’ve covered themselves.
I recommend using the word “possible” more often in life.
I’ve already spent most of this morning giving my wife a range of “possible” reasons why I was late home from work last night.
The lipstick on my shirt and the smell of perfume can thus be explained away, by reference to the new “possible” explanations.
She’s not buying any of them, but then she isn’t a gullible fool with a fixation upon maintaining her condition of motivated self-delusion.
No doubt Karl et al will be seen as a loyal contribution to Obama’s Legacy and rewarded accordingly.
Ask your friends how come we have never heard “The Pause” in the news when all this time we have been hearing about record temperatures over the past decade?
If global warming is really such a problem, how come the scientists didn’t tell us about this 15 year “Pause or Hiatus”?
Doesn’t the “Pause or Hiatus” make it look like information has been withheld; that scientists have not been telling us the whole story? Can you trust someone that only tells you 1/2 the story?
Actually the announcement that 2014 was the warmist ever included a statement that the pause could continue another 5-10 years Per G S.
Somewhat on topic. This morning I posted the May 2015 sea surface temperature anomaly update:
https://bobtisdale.wordpress.com/2015/06/11/may-2015-sea-surface-temperature-sst-anomaly-update/
Cheers
The Blob off the west coast of North America has brought fantastic summer weather to the Pacific North West.
For those that don’t live here, we normally get about 180 days a year of measurable precipitation. In other words, it rains 1 out of 2 days of the year. June is often damp and cold. So if this is global warming, bring it on.
Most of us would welcome San Diego weather year round.
The salmon and farmers may not agree with you.
ferdberple —
This has been a terrible hay fever year. I want damp and cold back. In fact I want snow. Or the entire Pacific Northwest concreted over.
Eugene WR Gallun
I moved to Salem, Oregon, in late 2004, from San Diego, at age 45, and couldn’t disagree more. If you didn’t grow up in San Diego, you might think the climate ideal, but not only is near-daily sunshine as boring as near-daily overcast, but what natural precipitation there is leads to only a trickle of water to fill reservoirs. I don’t see how the west-of-Cascades PNW could thrive with a high desert climate.
Ferd, depends on where you are exactly. On Whidbey Island it’s been a typical june, mostly sunny most of the time. We got a little wam a few days ago, but right now it’s right about average.
“Possible artifacts of data biases in the recent global surface warming hiatus”
===============
Interesting title. does this mean that the paper is itself an artifact of data biases? a Freudian slip?
As a theoretical linguist, I find this title interesting as well. The semantic structure of the title makes “the recent global surface warming hiatus” old information, and by definition an assumed fact. How can there logically exist bias in a fact?
perhaps the authors were given their marching orders to produce a specific result, but as scientists they still had some tiny scrap of integrity they wishes to preserve. thus the title”
“Possible artifacts of data biases” – suggest there may be bias in the(ir) result
“recent global surface warming hiatus” – acknowledges that the hiatus actually happened.
Thanks, Bob, for your clear words. I wish us all the best outcome in this controversy and agree that it looks like the choices made during the design of ERSST.v4 predetermine the results.
When does the climate change paper from Lois Lerner come out?
Bob,
As far as I know, the normal procedure for anyone who has (what they consider to be) valid criticisms of a paper can submit these as a ‘comment’ to the publishing journal, in this case Science. Alternatively they can submit their own paper for peer review as a ‘rebuttal’.
Do you intend to do either of these things?
David R, see my comment at M Courtney June 11, 2015 at 3:15 am
Do you doubt that?
M Courtney
Yes, I do doubt it. It would be highly unethical for a reputable publisher such as Science to allow the authors of a paper that was the subject of criticism to review the critical paper or comment.
If the alleged errors are so glaring, as has been alleged by several published authors here, including at least one climate scientist, then people may be at a loss to understand why a more formal rebuttal isn’t published via the normal channels.
It’s not unusual at all to let the authors challenge the challenges to their paper.
They know the paper best, after all. Who else should the Journal go to?
And I didn’t say the challenges wouldn’t get published. Only that they wouldn’t get published until after Paris.
Comments in science journals in general are not a favored form of communication on published work by both scientists and the journals. Which puts science about 20 years behind the times in terms of communication.
There are a number of reasons for this, a large one being journals aggressively discourage comments using petulance and random bias to determine which comments are allowed. The link below by Prof. Rick Trebino of the Georgia Institute of Technology(which ranks number 4 in the country for Best Engineering Programs or in other words applied physics) described in detail the level of nonsense which can attend making a comment. It is an easy read and worth reading.
http://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comment-in-123-Easy-Steps.pdf
Alx
We don’t yet know whether anyone has attempted to submit a comment to Science re this paper. In any case, there are other methods of publishing criticisms of papers, such as a formal rebuttal, which doesn’t have to be submitted to the journal that published the offending article.
I just wanted to know whether Bob, or indeed Judith Curry, intended to do this, given the apparent confidence they have in strength of their criticisms of Karl et al. If not, then why not? That seems like a reasonable question.
Perhaps they are doing both.
Waiting for a rebuttal to be published could take months. Meanwhile the “inaccuracies” of this paper get more firmly entrenched.
If he asks you to tea
As he did with Antnee,
Pack a long spoon,
Push politely the ruin.
===============
Read Judith again.
She said opportunity.
K15 said “Possible”. JC has, perhaps, connected the dots correctly? Why the weasel word(s) if the finding is robust?
It’s a magical elevator. And there’s two versions of this.
In the Wikipedia version, the reference source produces artifact A; which is the tuple of the claim Q and provenance P. Some media organization picks this up and reiterates claim Q, but with provenance R — from Wikipedia. This then is the artifact A’. A later edit to Wikipedia converts from provenance P to provenance S — from the Media Organization. This then is the artifact A”. Wikipedia has rather infamously gotten itself afoul of this for following this chain of events when the original provenance P was null or not provided. But even if P was initially correct and well validated, the later edits are still erroneous in nature.
In the Science version, there are two initial artifacts; A and B, both directly instrumental in nature. At some point a team comes along and notes that A and B are in disagreement and creates the new artifact B’ on the basis of A. This creates a new claim as well as modifying the provenance. Which now moves from the instruments I to the instruments I1 as well as A and the adjustment process P.
At a later date a team, perhaps even the same one, notes a discrepancy between A and B’. And decide to rectify the discrepancy by adjusting A. This creates the new artifact A’ with a new claim, and a provenance based on I2, B’ and the adjustment process. By obviousness, A’ is now constructed by I1, I2, A, and both adjustment processes.
At a later data still, another team notices that there is a discrepancy between A’ and B’. And decide to rectify the discrepancy by adjusting B’. And so on.
This is a general problem when we don’t scribble out and keep immutable our provenance for any notion. And/or don’t pay attention to the provenance of what we rely on. And it hardly requires Bad Faith from any party as, in fact, this happens most easily when Good Faith is presumed of the other parties and that the previous work is ‘correct’ without regard to its provenance.
What’s amusing to me about the Karl paper, is that it seems to have condensed this chain of events into the work produced in a single paper by a single set of authors.
This is a general problem when we don’t scribble out and keep immutable our provenance for any notion.
============
we see this in data cleansing all the time. the original “dirty” data isn’t preserved. rather it is over-written by new “clean” data. then this new “clean” data becomes the source for the next data cleansing operation.
since you no longer have the original “dirty” data to compare, the repeated cleansing becomes like the party game of “pass the message”. each time there is a data cleansing operation, some corruption is inevitable. carried on long enough without reference to the original source, the more likely you are to end up with a message that is completely corrupt.
When I worked in research a correction was made with a single line out so the original data never disappeared, The same in surveying.
0.089 and 0.143 C per decade. Anybody who represents they can actually measure these anomalies is totally full of it. Nothing but statistical hallucinations.
The view on my screen is regularly blowing up. I can’t easily read the text. (Screen in 21″ wide.)
If the Journal “Science” is committed to good science journalism, why would they not agree to include / publish a good rebuttal with good data to support it? It would seem that by doing so they could only increase their good reputation. Why not submit Bobs , or any other good rebuttal of the recent “Karl and company” paper to “Science”?
Because it would puncture the myth regarding the correctness and necessity of Peer Review. There are two ways to be consistently correct. The first is to never make a mistake. The second is to never acknowledge a mistake was made.
Comments like this and others seem to point out that filters are a way of getting to a desired result. To sharpen an image some photographers use unsharp-mask filter others use Photo Shops smart sharpen and still others use third party tools and this is just a small few of the filters that can be applied that all use complex math to get the “best” photo. The filters selected in all cases are the ones that gets the photographer the result most appealing to them.
Anyways various statistical filtering is a science but the results from using this science is an art. In this case the art of coming up with an expedient political position.
But filtering wasn’t enough so Carl had to also change the weighting factors until the desired results were achieved. Bravo Carl, for further debasing the already sullied field of climate.
I wonder when somebody is going to stand up and give the speech from the movie Network, “”I’m as mad as hell and I’m not going to take it anymore!””.
” To sharpen an image some photographers use unsharp-mask filter”
Unsharp doesn’t sharpen, it’s an illusion. It degrades the image but exaggerated contrast at boundaries.
BTW it’s Karl not Carl . If you want to slate someone best to get the name right.
.
Mike says:
June 11, 2015 at 4:54 am
BOB:
No she didn’t. Re-read the quotation you provided. She said there was plenty of scope for cherry-picking, she did NOT accuse Karl et al having done so. If you wish to say that, make your case, but do not misrepresent what Judith Curry or others have said to suit your own arguments.
That is quite improper and should be amended.
Bob, if you want this to be taken seriously, you really should present your findings with error margins and preferably statistical significance levels. You might very well be fitting noise, which would make your results meaningless.
IF you get a reply, please post it here so Dr. Karl can crucify himself with indescribable glory and public shame.
Bob, do I understand correctly from the second part of this letter that you are suggesting that a low-pass filter caused a short-term artifact? I would be really interested to see how such a thing might be possible.
I discussed that higher up
http://wattsupwiththat.com/2015/06/11/open-letter-to-tom-karl-of-noaancei-regarding-hiatus-busting-data/#comment-1960691
Your question was how this was possible and is answered in my comment.
Look at the beginning of thier figure 5 from Huang. The 32 and 16y filters are suspiciously flat obviously wrong. Also 32y at end.
I’m not convinced that the lowess(0.1) has distorted the end by a lot but it does not look well centred on the data.
But all the world’s scientific organizations agree. How far can they push the envelope with this nonsense? Are there any limits? I’m afraid this is a fundamentalist religion for the believers.
Bob Tisdale – “Independent Researcher”. Did you give yourself this title?
I wait for citations to your studies in the peer-reviewed scientific literature then.
harrytwinotter
Why?
(Other than your insulting but failed attempt to insult Bob Tisdale, that is.)
You are defining yourself by your words, your attitude towards “science”.
RACookPE1979.
If you mean by my “attitude” you mean preferring peer-reviewed scientific literature, then yep guilty as charged.
I love the ways those who pretend to love science go out of their way to pervert it.
Peer review means nothing more than a couple of your friends reviewed it and fixed the mis-spellings.
MarkW
To be honest, “Peer review can mean nothing more than a couple of your friends reviewed it and fixed the mis-spellings.”
Or it can be a futile waste of time for all concerned because no equations, raw data, nor methods were checked.
Or it can mean a simple way to open a cesspool of near-endless deliberate delays and arguments about trivia as one editor/reviewer/judge/jury/advocate/priest fights tooth and nail and pen to prevent any such idea from any publication at all.
But in today’s world, it does NOT mean “a valid filter to obvious errors and poor assumptions, while second-checking all calculations, all programs, and all methodologies used.”
Bob has provided more science and honest information in this site that you would have ever done in your lifetime, judging from your puerile insulting comments here, showing nothing but a sick and devious third rate mind of a no good loser. So put your name and show your qualifications and publications before you even attempt to comment on Bob’s work.
Venter.
OK if Bob Tisdale is “honest”, then he can provide evidence for his claim that a climate scientists engaged in professional misconduct:
“… mixed and matched methods until they found the results you wanted (ERSST.v4).”
And if you try and point out where Bob Tisdale is wrong (like I have in the past), you are likely to just get insults in reply. Me I do not spend much time on fake experts, there is plenty of good-quality science from dependable researchers to review.
I’m guessing that you didn’t actually read the letter. In it he provided the evidence you are demanding.
Amazing? RACook and appropriately named “venter” both claim that a simple, impassionate question “Did you give yourself this title?” is an insult.
Venter then continues :”…. nothing but a sick and devious third rate mind of a no good loser.”
Hypocracy much?
Anybody who does research and presents facts is a researcher and Bob Tisdale has done a lot of that. If you care to read through the archives or his earlier posts also you can see evidence of that. He presents his findings and data in full and answers all questions put up about his presentation.
On the other hand, harrytwinotter and you Mike have provide nothing of the sort except throw in garbage. Both of you are of the same kind of third raters who provide nothing productive and just come to disrupt threads. I’m describing your behaviour exactly and it is a statement of fact, not an insult. You are exactly what I have stated as proven by your own posts. Go away somewhere else and stay with your ” good quality science ” harrytwiotter.”. The fact that you have been hanging in here thread after thread spewing nothing but bile shows what you are.
Anyone who does research is a researcher and since he isn’t employed by anyone to do research, he is an independent researcher.
That should be easy for a thinking person to understand. Of course when your only goal is to change the subject, then arguing over the meaning of words is a good way to do it.
Mike.
Much. 🙂
Trust me Harry, Bob’s stuff is peer reviewed. They are peering at it constantly.
=========
Look at how the various posters here have reviewed any and all articles presented.
I would say that this site provides better “peer” review than any journal, even the major ones.
Is that the best you can do? Surely not.
Kar et al is now the clear outlier of all direct and indirect global temperature estimated, but Alarmists now insist it is the “one true temperature series”
It seem “consensus” is only of importance as and when it is convenient.
We have heard about “natural ” ‘global warming . We have also heard from the alarmists about their so called “man induced global warming . We now have the latest and newest form global warming created magically by new NOAA “assumptions” . These assumptions are even more powerful than the sun as the new assumptions have doubled the rate of recent global warming which the sun has not been able to do nor have greenhouse gases been able to do this .. A new form of renewable energy ? .
This makes me think that it is appropriate to draw attention to a quote by Karl R. Popper in his book “The logic of scientific discovery”. Watch out for behavior like this:
“It is still impossible, for various reasons, that any theoretical system should ever be conclusively falsified. For it is always possible to find some way of evading falsification, for example by introducing ad hoc an auxiliary hypothesis, or by changing ad hoc a definition. It is even possible without logical inconsistency to adopt the position of simply refusing to acknowledge any falsifying experience whatsoever. Admittedly, scientists do not usually proceed in this way, but logically such procedure is possible.”
This is what Wikipedia has to say about Karl Popper: Sir Karl Raimund Popper was an Austrian-British philosopher and professor. He is generally regarded as one of the greatest philosophers of science of the 20th century. … I guess it will not take long before inductivists start rewriting it.