NCDC responds to identified issues in the USHCN

The NCDC press office sent an official response to Politifact, which is below.

The NCDC has not responded to me personally, I only got this by asking around.

I’ve provided it without comment. 

=====================================================
Are the examples in Texas and Kansas prompting a deeper look at how the algorithms change the raw data?
No – our algorithm is working as designed.  NCDC provides estimates for temperature values when:
1) data were originally missing, and
2)  when a shift (error) is detected for a period that is too short to reliably correct.  These estimates are used in applications that require a complete set of data values.
Watts wrote that NCDC and USHCN are looking into this and will issue some sort of statement. Is that accurate?
Although all estimated values are identified in the USHCN dataset, NCDC’s intent was to use a flagging system that distinguishes between the two types of estimates mentioned above. NCDC intends to fix this issue in the near future.
Did the point Heller raised, and the examples provided for Texas and Kansas, suggest that the problems are larger than government scientists expected?
No, refer to question 1.

==================================================

 

Advertisements

  Subscribe  
newest oldest most voted
Notify of

NUTS…
Back to You…
Worked once, not so sure it works out as well this time.

Latitude

“No – our algorithm is working as designed”
again, no mention of zombie stations and no mention of stations that are reporting data having their data substituted for made up data….
As long as you let them control the conversation…this is the answer they plan on giving

I didn’t realize the similarity between the NCDC and the Emperor Penguins. Namely, they both leave streaks on the ground where they have tread. And the streaks are made of the same material.

David Davidovics

I don’t expect them to admit any flaws.

“Did the point Heller raised… suggest that the problems are larger than government scientists expected?”
If it’s government work, large problems are always expected.

Would like to see the letter you sent to NCDC regarding this issue, which they failed to respond to…

Mike Fayette

So does this mean that their future data tables will distinguish between:
A: Raw Data as originally reported with no adjustments
B: Estimated Data based on surrounding stations since the data is missing
C: Adjusted Data (using a Blackbox Algorithm) because we don’t like the original data
If they do that, wouldn’t that be helpful to all?

Pamela Gray

Not even close to a satisfactory answer. Way too curt and seems filled with hope that a short stern answer will stop the inquiry.

José Tomás

“No – our algorithm is working as designed”
So, case settled.
There was some debate here about if this was a case of deliberate tampering or a bug.
One commenter said that “it was a feature until discovered, then it would become a “bug”.”
Not even that.
The deny it being a bug.
So, the other option is…

Evan Jones

No – our algorithm is working as designed.
I know.

They are sweeping it under the rug..
GUILTY
“algorithm is working as designed”
Who signed off and designed the algorithm? I smell James Hansen’s dirty work!!!
When you are Gov and things do not go as you planed you make it show as you planned. Warmer then reality

DEEBEE

So it’s a feature not a bug

Niff

They mean…they changed the design to comply with the code…..and please go away now.

Rud Istvan

The answer is in one sense honest: “Our algorithms are working as designed.”
We designed them to maintain zombie stations. We designed them to substitute estimated for actual data. We designed them to cool the past as a ‘reaction’ to UHI.
But in another sense, this is as bad or worse than IRS losing Lerner’s Emails, not following the law to recover from the backups, not reporting the fact to the National Archivist. It is another, “if you like your temperature, you can keep your temperature…”.
Politicization of rigged data. When finally called to account after the next election, they will first say we misunderstood what they meant, and then say they misspoke. And then maybe we will be able to jail a few, since the coverup is usually worse than the original crime.
What strange post modern times.

Finn

I guess this rules out incompetence.

RAH

Reads like STFU to me.

Congressional inquiries are in order.
You can ask your Congressman to inquire into this issue.
In US Government offices, everything stops when a Congressional inquiry is received.
Tell your Congressman that your communications with the temperature office were curtly rebuffed, without a satisfactory answer.
If everyone reading this talked to their Congressman’s office, at least a few would follow-up with the temperature scammers.
And you might actually get some answers.

Jesus just turned water into wine. These clowns are trying to turn BS into data. The threads are unraveling.

Paul in Sweden

“No – our algorithm is working as designed”
The obvious has been stated. Do we know when the hearings and prosecutions will begin?

climatebeagle

My usual answer to “working as designed” is to ask to see the design documents.

Quinx

Decode: We’ll only panic if it looks like temps are dropping. Meanwhile, the money keeps rolling in.

“our algorithm is working as designed.”
Exactly.

Theodore

“No – our algorithm is working as designed.”
Unfortunately not surprising. So it doesn’t matter if their data is as accurate as VA wait times, it is the answer they intended to produce.

John Greenfraud

The answers from NCDC are acceptable to Politifact? Forget that it smacks of a coordinated effort between the two and they are willing to accept this dissembling for a definitive answer.
“our algorithms are working as designed”
Most of us believe that IS the problem. Your algorithm produces garbage by infilling with spurious data.

Theodore

Quinx says:
July 1, 2014 at 5:49 pm
“Decode: We’ll only panic if it looks like temps are dropping. Meanwhile, the money keeps rolling in.”
Temps are dropping, they just don’t have to admit that as long as their AlGorethm is working as designed.

Eliza

As I mentioned before they will do NOTHING. That is why the time is past talking.They have an agenda AGW.. As an aside this is what we shouid worryibg about; Note the definite almost circular shape Antarctica is beginning to form with the extraordinary ice expansion (ABOVE ANOMALY)
http://arctic.atmos.uiuc.edu/cryosphere/NEWIMAGES/antarctic.seaice.color.000.png

Was their algorithm deliberately designed to adjust the temperatures according to the level of CO2 in the atmosphere then (this to me is the final straw that should break their credibility entirely, showing them as deliberate fraudsters)?:
“US Temperatures Have Been Falsely Adjusted According to the Level of Carbon Dioxide in the Atmosphere”

Scute

Does that mean:
“No, our algorithm is working as designed.”
or
“No, our algorithm is working as [re] designed [yesterday in a hurry]”

Paul in Sweden

Crap on a Cracker! Hot Dog venders on a street, Clowns, Magicians & Kids Entertainers are licensed, regulated, held to a higher standard and much more respected than what we are seeing bilging from Climate ‘science’ these days. It is astonishing.

As a practical matter they have no choice but to defend their process. They will surely lose their jobs if they allow a change that damages the political narrative because that data infects many of the analyses the administration is using to push their agenda.

I am altering the data. Pray I don’t alter it any further.

D.I.

“No – our algorithm is working as designed”
What The F**k? Designing Temperature? Who do they think they are, GOD?

exNOAAman

The IRS gal pleads the fifth, because her answers may incriminate her.
“Our algorithm is working as designed”
Rather incriminating.
You should’ve plead the fifth, son.

resistance

“No – our algorithm is working as designed”
Looks like an outright, on-the-record admission of fraud to me…

Lawrence Todd

NCDC National Cruddy Data Commission

Apparently they only supply missing data when required by other programs. Might I suggest modifying those other programs, rather than inventing data? You can never increase acccuracy by guessing, nor can knowledge be increased simply by multiplying your current information.

Trust but verify.
I’m done trusting.

DesertYote

“No – our algorithm is working as designed.”
I am sure it is. I just wonder what the algorithms design criteria was!

mjc

And this ship is the best ever built…it’s totally unsinkable!

Doug Badgero

“No – our algorithm is working as designed. NCDC provides estimates for temperature values when:
1) data were originally missing, and
2) when a shift (error) is detected for a period that is too short to reliably correct. These estimates are used in applications that require a complete set of data values.”
I believe the ‘and’ should be an ‘or’.

Eliza

It needs to be brought to the attention of Mainstream media. This is probably the single most importantpoint about all this. Most have not even heard of this.It is quite a story.

Gary

The Titanic worked as designed.
The Hindenburg worked as designed.
The Treaty of Versailles worked as designed.
The attack on Peal Harbor worked as designed.
Federal funding of climate research works as designed.
Peer review works as designed.
The IPCC works as designed.
Climate models work as designed.
It’s what you didn’t design that you have to watch out for.

“No – our algorithm is working as designed. NCDC provides estimates for temperature values when:
1) data were originally missing, and
2) when a shift (error) is detected for a period that is too short to reliably correct. These estimates are used in applications that require a complete set of data values.”
Ah, notice the don’t say “…in applications that require an accurate complete set of data values.”
Very tricksey, these NCDC hobbits.

Latitude

It’s just their press office…….The NCDC press office

dccowboy

I hear the sound of a broom and a lifted rug somewhere. Beyond belief, even for a Bureaucracy trying to protect itself. It sounds exactly like what I would expect the ‘higher ups’ at an Agency to respond when their ‘technical experts’ show them a MASSIVE problem that would prove embarrassing to the agency. Exactly. I suspect that the ‘experts’, in their heart of hearts, know there is something rotten in Denmark, but, they have kids to feed.
I don’t see how they can make the claim that ‘the algorithm is working as designed’ when there are admitted problems of the scale that have been shown to exist in Texas and Kansas data. Are those States accorded unique treatment in the ‘algorithm design’ such that the issues raised are unique to those two States? What kind of an ‘algorithm design’ does that?
They are hoping that we just all go away and the American public accepts the constant drum of ‘denier, denier, denier, flat earther’ by way of explanation. Lots of ‘LA, LA, LA, I CAN’T HEAR YOU!! going on. I for one am growing tired of the expectation that personal insults suffice for intellectual argument.
I think we need to stop being passive and pursue this in whatever manner we can to expose the truth. If it is that the ‘algorithm’ is working ‘as designed’, then so be it, but, somehow, I suspect that this is not the case. We’re going to have to expose it for them.

Mike Fayette says: July 1, 2014 at 5:23 pm
“So does this mean that their future data tables will distinguish between:
A: Raw Data as originally reported with no adjustments
B: Estimated Data based on surrounding stations since the data is missing
C: Adjusted Data (using a Blackbox Algorithm) because we don’t like the original data
If they do that, wouldn’t that be helpful to all?”

That’s what USHCN does now. They provide a raw data file, and an adjusted file (F52), and there they mark (with an E) estimated data. They also provide a file with TOBS adjustment only.
REPLY: it is important to note that the issue here has to do with errors in the X and E flags in reporting on data. More on that here:
ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/v2.5/readme.txt
The problem is that they have GODD DATA IN HAND in the raw data file, but instead of using it in the F52 (final) data file, they are throwing in way too many estimates. Along with “estimated” data for a bunch of closed/zombie weather stations that shouldn’t be reporting at all, and have no data in the raw data file.
Nick and others want to argue like the town crier “All is well!”, but in reality, the USHCN is not only a train wreck from a raw data file standpoint due to all the inhomgenieity, its a bigger train wreck after NCDC inserts “estimated” and “zombie” data that should not be there, except that in their world “all is well”.
I had a couple of people call me today that might very well be able to get an independent investigation done, I said let’s wait and then we’ll see how NCDC handles the rest of this. Hopefully they won’t say “All is well!”.
-Anthony

dccowboy

Eliza says:
July 1, 2014 at 6:19 pm
It needs to be brought to the attention of Mainstream media. This is probably the single most importantpoint about all this. Most have not even heard of this.It is quite a story.
======================
IF you think this, you haven’t been paying attention. The ‘mainstream’ media loves nothing better than a ‘we’re all gonna die’ story, which is what they get from the current Admin and the IPCC. That sells newspapers.

Doug Badgero says:
July 1, 2014 at 6:17 pm
“No – our algorithm is working as designed. NCDC provides estimates for temperature values when:
1) data were originally missing, and
2) when a shift (error) is detected for a period that is too short to reliably correct. These estimates are used in applications that require a complete set of data values.”
I believe the ‘and’ should be an ‘or’.

Hey, it is their algorithm, and maybe it was designed to not recognize the difference between “and” and “or”.
But then, notice the “too short to reliably correct” phrase. Since it can’t be reliably corrected, they estimate and/or randomly generate data that is probably, mostly, not reliably correct.
/grin

Rob Dawg

One unfortunate consequence of filling in missing data is that it masks outlying readings. Instead of seeing a spurious result surrounded by dissimilar datums you now have averaging creating less of a clear difference.

John M

This sounds Mannian.
The algorithm is robust to the data input.