Eric Worrall writes: It looks like NOAA have found a new way to stifle FOIA inquiries from the public. According to Steve Goddard, NOAA have just demanded a $262,000 administrative fee for zipping up a few raw data files.
Steve Goddard has published a scan of the outrageous fee demand he and fellow FOIA requestor Kent Clizbe received from NOAA administrator Maria S Williams. The letter, sent on March 17th, demands $262,000 by March 24th, or further communication – otherwise Maria says they will consider the matter closed.
https://stevengoddard.files.wordpress.com/2015/03/fee-notification-letter-2014-001602.pdf
The NOAA staff directory lists Maria Williams as the Chief of Staff Support Services Branch. https://nsd.rdc.noaa.gov/nsd/pubresult?LNAME=wil
For the full story read Steve’s post – https://stevengoddard.wordpress.com/2015/03/20/freedom-isnt-free/
As an IT expert with over 20 years of experience, my expert opinion on the claim by NOAA that it would require $262,000 to gather up a few computer files and send them to Steve is that it sounds like a complete crock. Even if some of the files are in printed form, they can just be run through a scanner – my automatic page feed scanner can process a page every few seconds, even cheap scanners can process thousands of pages per day. If the files are too big to put in an email (likely), for trivial cost NOAA could publish them on a password protected web page – it would take at most a day to set up such a web page, and add the files to it.
If NOAA’s data files really are so poorly catalogued that several man years of effort would be required to find them, this is something NOAA should be fixing on their own time. If this is the case, NOAA should not be attempting to charge FOIA petitioners outrageous fees to cover NOAA’s own incompetence.
Respect is like trust, it cannot be taken it has to be earned. NOAA is forfeiting the respect that people once held for the organization. It will not easily regain the trust and respect that it has lost.
Good grief — apparently the request was not just for “a few raw data files”. As explained in the response from NOAA: “Responding to this part of the request would require retrieving, reviewing, and packaging many tens of thousands of items in at least 29 years of communications, if they can be located.”
Do we get to see the original request letter? WTW, WUWT?
Yes, without seeing the actual request there’s no way to know if all the outrage is justified.
Sad to see how few demand a complete story before jumping to conclusions, and ironic that those who demand original data fail to show their own.
The only reason to NOT show the original request is if it justifies to some degree the snarky response.
Barry:
According to the link:
This letter is in reference to your Freedom of Information Act (FOIA) request entered into FOIA
online on September 7, 2014, for records pertaining to “Temperature Data Record Adjustments:
Rationale, Methodology, Discussions–USG employees and others for the NOAA/National
Climactic Data Center”.
There is no original “letter”, it was done online.
The request, however, is spelled out in the response.
Nutso fasst:
“Yes, without seeing the actual request there’s no way to know if all the outrage is justified.
Sad to see how few demand a complete story before jumping to conclusions, and ironic that those who demand original data fail to show their own.”
That information was on the link. If you want the “complete story” then click on the links, that’s what they’re there for. 😉
Bezotch:
I followed both links (which many commenters obviously didn’t) and did not find an original request.
Since then I read through the comments to Steven Goddard’s blog post and found a comment by Mr. Clizbe that copied part of NOAA’s email response to his request. The request is more extensive than even NOAA’s official reply indicates:
https://stevengoddard.wordpress.com/2015/03/20/freedom-isnt-free/#comment-508739
Nutso, there is no rational justification to charge that obscene amount of money for what is basic administrative work. If they haven’t been cataloguing the information in any logical, organised manner whatsoever they have no right to be a scientific organisation.
“If they can be located”? You must be kidding.
Obviously they are trying to make it hard for the public to get their hands on the fiddled and /or raw data.
Ian W: “Respect is like trust, it cannot be taken it has to be earned. NOAA is forfeiting the respect that people once held for the organization. It will not easily regain the trust and respect that it has lost.”
Yes, I can testify to that. I used to work for NOAA back in the early 1970s – and was proud of it. But now, I have lost all respect for NOAA – it’s like when Noah got all drunk and naked.
They are obviously moving to stand-over tactics and extortion to add to their crimes of Fraud.
Now, now. Guilty of hysterical nonsense and that’s about it. Of course, it is nice to see they feel forced to reply at all even if it is too witness, once again, the degree of delusion they operate under.
Not really. The FOIA request as outlined in the response to Clizbe delineates a huge domain that includes lots of material from “the beginning” of the adjustments through the present. Lots of the material described such as phone logs will be in archive boxes “somewhere” rather than on electronic media, which is where the “if they can be found” comes from. Even some of the electronic media may no longer be supported by available hardware. Scientifically, all Goddard should need are the methodological assumptions behind the adjustments and the code used to produce the adjusted data. However, NOAA science is skint on the actual methodological reasoning behind determining what adjustments are needed and why.
“Methodology” is not to be confused with methods. You use arithmetic as the preferred method to balance your checkbook. The methodological reasoning behind that decision is the argument that by adding all deposits and subtracting all debits, you’ll know how much you have in your bank account and what is outstanding. Theoretically, you should know even before and with more precision than your bank. Reality of course will be different, but it always is.
His congressman ought to inquire on his behalf as to the details. Sounds like a ruse to avoid compliance with the law.
+1….and don’t let up. This is clearly an attempt to stifle dissemination of public data. Someone should look into other FOIA requests and see if like charges were assessed. I’ve written my congressman already about NOAA rewriting historical temperature records.
willybamboo has a good answer. Pester their Congresscritter and Senators. Remember the squeaky wheel concept. Write, call, email. Get others to do the same.
It’s your best shot at getting results.
Right – I’ll get a note off to Bernie right away.
Not disagreeing, but please pick another state than the Green Blob er Mountain one.
Just sayin’
Yep. Same thing here in New Jersey. Nothing but National Socialist Democrat American Party “turtles” all the way down.
Yeah, I’m sure Patty Murray and Jim McDermott will be all over this.
It is obviously a fence built around a crime scene to hide criminals working for our government at NOAA. Congress should investitage teh NOAA and make their data completely public, give names, and expose the entire scan.
A long time ago, a Republican told someone at NASA to do his job and that person went to claim he was being silenced. George W. Bush told James Hansen to stop doing public events on taxpayer’s time and that is when Hansen claimed he was being silenced. The only ones who would investigate this malfeasance are Republicans. If they dare ask NOAA to do their job, you’d better believe they would cry suppression. (But, of course, it is perfectly okay for their side to engage in smear tactics like having a member of Congress ask about your funding.)
Carl, all this in the Obama Administration? Even a direct court order would produce similar results. And do not believe that the White House is not involved in hundreds of other similar stonewalling efforts. If it gets too hot for the White House Obama will just issue an executive order denying most almost all FOIA requests.
House Speaker Boehner and Senate Majority leader McConnell will huff and puff, call press conferences, and threaten to see that justice is done. Then, they will privately roll over on a Sunday night or on a holiday when the MSN will have an excuse to ignore it. Count the times this situation has been repeated if you like large numbers.
So much for the transparency we were promised.
Leonard Lane,
Are you saying, “Wave the White Flag”? That’s what it sounds like.
Contacting representatives is free. But there’s always the alternative of filing suit in federal court. They cannot ignore that.
Fortunately, in less than 2 years there will be a Republican in the Big Seat then perhaps progress can be made. After all PROgress is the opposite of CONgress
NObama ’16
Leonard,
I am as frustrated with the Republican Congressional leadership as the next guy, but it is necessary to remember that NOTHING can now get out of Congress without 60 votes in the Senate, and there are only 54 Republican votes.
Harry Reid has blocked everything, most recently the attempt to de-fund DHS and thus Obama’s illegal immigrant amnesty. If anything, today’s news is worse, since the new Senate minority leader will be Chuck Schumer, who is notoriously one of the most partisan guys in Washington.
It would be very different if the MSM played fair, but they sang the Democrat narrative in the recent fracas over a “clean” bill to fund DHS, and that is the narrative that ultimately prevailed.
Bryan A
PROgress is the opposite of CONgress.
Beyond funny.
Eugene WR Gallun
Someone needs to request a similar set of records under the aegis of proving CAGW from Greenpeace, some obscure UN “agency” or similar eco-loon organization whose hand-in-hand work with US Government bureaucrats is already established. Then let’s see what fee would be charged then… any bets?
Fee? They’d probably enclose a check for a cool quarter million.
I’d immediately FOIA all internal correspondence between NOAA employees relating to Steve’s FOIA. I bet that’d make for interesting reading!
Good idea.
Hummmm, except that gov hard drives are notoriously unreliable. They crash and get recycled at the most (un)opportune times. Better hire somebody to track all the unlisted email addresses, as well. Oh, there are interoffice chats that are not recorded or archived at IRS… Maybe in other agencies too?
Employees in “other agencies” have their own private servers over which they can edit and delete as they please.
Rounded up, that’s around £176,000 Sterling.
Another excuse for ‘vested’ reasons why we need to charge you a Global ‘get rich quick whilst we still have a chance’ Warming fee for scanning more than 1 piece of A4 paper.
I know that researchers doing studies that require data from satellites have to pay a fee for it (studying subsidence of a region from some activity, regional agricultural or forestry studies, etc). Exactly what was asked for? Governments over the past 20 years began charging fees for visits to national galleries, museums, etc. that used to be free.d
They requested NOAA’s published monthly temperature data over the past couple of decades; not raw data from a superconducting supercollider, just simple temperature readings. Steve believes they overwrite the data in place in order to hide their ongoing manipulations. From his article:
“The data should be kept in a simple file structure like the NOAA drought data, and it should require no more than 10 seconds for me to recover it online. Instead, they want a quarter of a million dollars.”
When you read between the lines, this is what NOAA really said:
“Why should I make the data available to you, when your aim is to try and find something wrong with it…?”
-Phil Jones email Feb. 21, 2005
There is only one reason they won’t turn it over.
No there are two reasons at least
1. Fudging the data.
2. They haven’t GOT the data; they have a mess.
The second one turned out to be the real reason Hadcrut never complied with Britain’s FOIA law when McIntyre requested their raw data.
No, the request went well beyond the temperature data. Here is an excerpt from the letter
This letter is in reference to your Freedom of Information Act (FOIA) request entered into FOIA
online on September 7, 2014, for records pertaining to “Temperature Data Record Adjustments:
Rationale, Methodology, Discussions–USG employees and others for the NOAA/National
Climactic Data Center”.
Its the temperature data (not too bad to supply), the adjustments (probably not too bad either), methodology (might be easy-ish if it was well documented) and finally the kicker “discussions”.
Its this last point that was ridiculous to ask for. As if “discussions” are easy to identify! They happen across many employees past and present, in email, paper copies and heaven only knows what. To properly get “discussions” relating to the request would indeed take an enormous amount of resources and I fully side with NOAA on this.
Steve Goddard should have been much more specific. He could have asked for a more achievable search such as emails between 1994 and present sent from the following employees…and then list a few. At least that wouldn’t be open ended.
The “Discussions” claim could be satisfied with a simple summary of the adjustment. All Godard was asking for was for reasoning behind each adjustment. If they objected to that, then a simple “that last item is unspeakably broad, please clarify what you are looking for” would have been a reasonable answer. Claiming that it would take 2.5 man-years to find everything and pay up now, is not
Ben writes “The “Discussions” claim could be satisfied with a simple summary of the adjustment.”
No it couldn’t. The request is what it is and asking for “discussions” is clearly stupid because it legitimately gives NOAA an “out” based on an unreasonable request.
Hmmm…..I don’t know what was requested but if it was data and they don’t already have it on hand, then how can they declare things like “2014 was the hottest year on record!”?
(Unless it would cost that much to de-Hansen the records.8-)
Can’t we just get the Koch brothers or big oil to write a check? Isn’t that how we pay for everything? lol
If they paid for it, you know they’d be targeted, and the data would be called “tainted”.
http://unixhelp.ed.ac.uk/CGI/man-cgi?grep
zemlik,
Some words of explanation would be appreciated.
well I think if you want to search through a load of files that contain some words grep can help to do that and indicate those files that contain the expression searched for.
http://www.rexegg.com/regex-perl-one-liners.html
meaning, as others have said, with all these super-duper computers mimicking the earth’s atmosphere surely it shouldn’t be difficult to categorize everything everybody ever has said ?
LOL and remember, less is more!
Who knew that Obama’s “most transparent” Government had a Green filter on it?
Oh it’s clear that there are lots of filters…that clarity is what is transparent.
It has taken the alarmist cult long enough to notice that sceptics continually complain about pay walled papers.
The reason? Sceptics have less than 0.1% of the funds that alarmists have.
So, let’s take it up several orders of magnitude, instead of asking $30-50 to look at some dodgy research, let’s scare the bastards off with a ridiculous financial demand.
Bottom line? Looks like some second rate bureaucrats have something to hide.
I’m all for creating a general FOIA pool which, by law, must be replenished immediately if it overruns its annual budget. All in favor, say “aye”.
I hear you, but I can think of a lot of ways such a fund could be abused – government administrators worried about their budget could get a few “friends” to make fake FOIA requests. Said administrators could then use the alleged cost of such requests to cover their own incompetence, to plug the holes in their budget.
Simply capping fees to something reasonable is probably the way to go. If agencies whose data is of interest to the public have to repeatedly deliver the same or similar data, the need to keep costs down will be a major incentive for them to sort out their internal mess.
Eric,
The potential for abuse is one thing I was getting at. I thought about capping fees as well, but then the argument would turn to reasonableness of the cap, whether the cap has actually been reached or if creative accounting was done so as to deny a particularly inconvenient request.
Though you and I are opposites in this debate, I do want the same transparency and accessibility you are asking for. More of it.
The big problem I see with publicly-funded research is that there are so many private interests at stake, and not a priori nefarious for it. The major for-profit journals and their market-sensitive paywalls are always the first thing which leap to mind. I’m less familiar with intellectual rights to data, methods, and the laws protecting academic freedom from nuisance interference, but my general sense is that there are a lot of things there which cannot be simply disentangled.
My suggestion is that FOIA filings are the least constructive way to go about this. Grijalva’s stunt was not just politically stupid, but potentially harmful to any and all science being done on the public dime. Which is hands-down idiotic, plain and simple. Name-calling and taunting in the press and on the blogs is one thing. Targeted information requests made by an elected representatives with official powers, either real or implied?
That’s crossing a line. It’s a continuation of a bad precedent which really does not need further encouragement.
Mr. Gates,
In politics, as in life, turnabout is not only fair play but pretty much guaranteed.
And I ‘spect that the political payback for all this is gonna be a sharp and enduring and eye-opening slap in the face and kick in the seat of the pants, come swearing in day, 2017.
Menicholas,
Perhaps. I personally doubt it. Recall that most US voters don’t rate climate very high on their list of concerns. Latest polls still have Hillary beating any prospective Republican nominee by 10%. But two years is a long time.
On this point though, I’m far more concerned about the long-term implications for all publicly funded science than I am the normal ebb and flow of partisan sentiment in the voting booth.
I think the current method is fine. The problem is inconsistent or prejudicial application of fees. As all of the applications are public record, it’s easy to find when they are giving preferential treatment to certain groups.
The problem lies in that there is no mandate to be fair and no legal enforcement thereof. The only recourse is to sue and say that this fee is fraudulently large.
Agreed. So long as the govt is able to keep in check the incentive to grossly inflate costs.
Never happen. Kind of my point.
Not often, but I fully support Brandon’s suggestion.
Thanks, but take care to note I’m not fully committed to it. The whole idea of FOIA is somewhat ridiculous … just publish “all the data” and be done with it is the opposite, and logical, extreme. But not practical either.
It’s funny that the estimate is 1.9 years of search and review time. That would make it done almost the exact time Obama leaves office. I guess that’s their best current estimate of how long they can stonewall.
That is not a fee. That is “buzz off” written in a rather long-winded way.
Besides contacting congressmen, spread news of this far and wide. Put postings on obscure blog sites. I think it is fair to ask, “What are they hiding?”
They are getting more and more flagrant with their arrogant impunity. It has reached a degree where you don’t have to say much; just pointing out this behavior says enough to cause Maria Williams’ relatives to blush, and for her to consider changing her name, I imagine. However I suppose it was a dirty job, and someone had to do it. I wonder who ordered it done?
Steve Goddard must be stomping on some big toes, by constantly pointing out the “adjustments” NOAA makes to temperatures.
What, exactly, has Goddard requested? Without knowing that, one can not judge this fairly. Goddard doesn’t say. But look at the NOAA letter: “letters, phone logs, memos, and other communications on this subject”, “Historical internal and external emails”, “data stewardship …”, “retrieving, reviewing, and packaging many tens of thousands of items”, “investigating availability of items and code, some from obsolescent systems, evaluating content, and screening for non-responsive, deliberative, or personally identifying information”.
A “few files” indeed. I think I trust “Steve Goddard” even less than I trust the government.
OK. So they cut back a tiny bit on “sustainable subsidies” to cover the cost. In the end the taxpayers would likely save a bundle….including the prison cost. 😎
Mike,
Does the “M” stand for “Moby”?
Pay attention Mike. They requested NOAA’s published monthly temperature data over the past couple of decades. From Steve’s article:
“The data should be kept in a simple file structure like the NOAA drought data, and it should require no more than 10 seconds for me to recover it online.”
A “few files” indeed. I think you’re either an alarmist troll or your reading comprehension skills are severely impaired.
The key was that he also requested the adjustments and “discussions”. They seem to be doing literal-genie resistance. It would of course taken an astronomical effort to comb through all the e-mails for discussions about all of the adjustments. Rather than clarifying or just giving a summary of the adjustments, they are just stating the maximum bill based off a ludicrously broad interpretation of the request.
Louis pay attention Steve claims what you quote. He however does not provide the wording of his request nor any other evidence. The letter from NOAA however makes specific statements which directly contradict what Steve says. I’m sorry but as a tax payer I’m with NOAA on this if NOAA’s statement is of what was requested was accurate.
If Steve produces a the request that ONLY asks for this OK, I’ll back him, but if it is what NOAA is stating, then shame on him for misleading everyone by trying to claim it is only a few files. Steve has made an accusation, he should either put up or shut up. If he wants to go fishing then he should use his money to do so not mine.
benofhouston, there is something called the Data Integrity Act from years ago requiring any research funded by taxpayer dollars to be archived and retrievalble (other than embargoed data awaiting publication). As Steve commented, this data should be easily obtained at little or no cost. When the past data, that has been archived, is “adjusted” or changed in some fashion, the rationale for having done that should also be easily obtained. The fact that this has become such a maze to navigate indicates something is seriously wrong. Steve should not have had to ask for the emails and discussions behind this rationale to change previously recorded as official data. Transparency should be the routine, and not a gauntlet requiring FOIAs to obtain. Something is seriously wrong and it is not conspiratorial to see that something is rotten in Denmark.
I don’t understand how data from years ago can be “adjusted.” If the data can be shown to be invalid or unreliable, then it should be discarded, not “adjusted” by another arbitrary means which then can’t be tested to see if it is truly valid or reliable. Imagine anyone doing this with financial reports of business results from previous years.
I second most of your point. In any case, I like a key sentence from the NOAA letter: “As data stewardship – including homogeneity adjustments – has been central to NCDC’s mission
for decades, determining which records are responsive to this extremely broad request will require significant resources.”
Why would anybody doubt NOAA adjustments? It must hurt deeply.
The letter complains that his request is too broad while at the same time complaining how difficult it would be to search through everything to find the specific items he’s requesting. Huh?
It’s actually rather simple. He wants ALL docs on temperature adjustments. So, even with electronic docs you do a word search, say temperature and adjustment(s) in the same document. Now how many time do you think temperature comes up as a key word in NOAA docs, I would guess easily in the millions and adjust is a common word. Now someone has to read the doc and determine if they said “the temperatures were too low so we adjusted them up” vs. “here is the alert on the extreme low temperatures, I adjusted the wording”. Now factor in office notes, meeting notes, etc. that may or may not be in electronic format, or old formats say floppy disks, etc. and yah a quarter of a mill sounds about right, once you figure in the unions cut.
So what would you suggest to narrow the search? I see you suggested nothing at all.
It’s not like a warehouse of papers and floppies were just dropped on their desk. As the letter notes, they are the gatekeepers, and temperature adjustments are one of their important tasks. How is it that all of the relevant info isn’t already categorized?
How can anyone “suggest to narrow the search” without seeing the actual request?
Sue them and let them explain how they have violated the law in not making the data digital and easily accessible.
I’m sorry but whenever I see a post by Eric Worrall I think to myself, here might come another half-baked story with no supporting facts claiming… god knows what.
I didn’t enjoy saying that. FYI.
You could have just not said it.
Sunshine Week – this week or last? – celebrating transparency in govt. AP has been waiting 9 years for something they reported submitting FOIA for.
There are costs to replying to a FOI request. And the costs depend on the information requested and the competence of the archivists.
This seems to be an admission of incompetence by NOAA.
Have they described their plan for an investigation into their inability to access data?
And how many of their QA department have been sacked?
Read the actual FOI request, not what Steve claims he requested. Then let us know what you think the cost would be. And, don’t forget NOAA gets sued if they accidentally release any personal info.
Complete nonsense. They are doing government-funded work and all of the emails are public property. Anyone working for a corporation know that any of their emails are not their own property and are discoverable at any time. If this “adjusting” had any basis in actual scientific necessity it should be readily obtainable and part of the federally-required records and standards. You are dissembling just as they are. The entire federal government seems to be completely at odds with the voluminous requirements they themselves place on everyone in the private sector. If any of these people are performing government work on personal email accounts and private servers that points to another problem that needs to be brought out into the open and investigated.
I work in out-sourced local government IT in the UK. There was recently an FOI request for an email trail regarding a particular service contract that aroused suspicion. The local authority batted the request back with a very similar response, quoting man-hours per email times number of emails times hourly rate, etc..
The snag is, the company I work for does their IT, and they never asked me or any of my colleagues for an estimate of the effort or likely cost. We even have a standard call-off mechanism for small-scale versions of this sort of thing.
Oops.
You can draw your own conclusions.
Perhaps we and our American cousins need a tweak to the FOI legislation to eliminate the “f*** off ” responses. For sure, there has to be a way to deter the frivolous rubbish FOI requests, but I don’t think that’s beyond the wit of our handsomely paid legislators.
If you can find his actual request, let us know what you think the cost should be for compliance.
Again you are dissembling. Try that excuse in a private corporation that has their records subpoenaed by the federal government and see how well your excuse of expense and difficulty in locating the records that are demanded. What a bunch of malarkey.
Anyone semi-computer literate knows how to do a search with keywords. This FOI request could be done literally in a few minutes and emailed.
Conclusions:
1. They’re lying
2. They’ve got plenty to hide
Or,
3. They don’t know what they have to hide and daren’t risk finding out.
Or,
4. The records are on tape backup off site.
Brandon Gates at 4:47…… You can’t be serious. In my company we had multiple servers running multiple operating systems backing up every office in multiple locations on arrayed hard drives. Tape was only used weekly backups and local restoration. It isn’t hard to search and restore data. When I retired, we had 10 years of searchable information on line at all times and ALL open projects were on line at all times, some going back 20+ years. We had whole office crashes that could be restored in a few hours. Now, our old 1970’s VAX tapes took a little longer to take out of the archive and mount but it was like a couple of days to pull up and search 10 years or more of data. So, even IF NOAA is so backward that they are archiving data that they update monthly on tape …. No, sorry, that is completely unbelievable. Isn’t it?????
Wayne,
I’m hampered here by two things:
1) What Goddard is requesting isn’t entirely clear to me.
2) I know absolutely nothing about NOAA’s IT infrastructure.
In a list of speculations, I appended a speculation. Reading down, it seems Goddard is asking for more than just weather data, but communications. NOAA wouldn’t simply do a simple backup from the email server and pack that up on a USB drive. They’d do a keyword search for the relevant traffic, and assuredly have a human do a manual review of the results. That’s my guess where most of the personnel time estimates come from, and hence most of the retrieval cost.
But I don’t know. I wouldn’t presume to know. And very much unlike Goddard, I wouldn’t make any strong conclusions — or even hint at them — that the estimate is politically motivated and bogus simply on the basis that I thought the bigness of the figure was ridiculously inflated.
Now what is evidently clear to me is that he doesn’t need the data. He’s already decided. His lede:
NOAA is desperate to hide their data tampering, and is taking a new tack to defeat FOIA requests.
Not a whiff of uncertainty. If he’s so confident in the evidence at hand, why does he not file a lawsuit to open a full investigation? Inhofe would gobble it up, if he’s not already working on something similar.
It’s as obvious a political stunt to me as Grijalva’s was, though not nearly as foolish.
Brandon,
You may be right, what bugs me in Goddard is exactly this kind of whistleblowing. It is like if he swallowed the whistle and could not stop breathing through it.
Hugh, thanks for that image. It made my day.
MCourtney,
I think I like your #3 better. An honest man has nothing to hide. A dishonest man thinks they know everything about him, and thus tries to hide everything.
Your assuming that whatever Goddard asked for is online. After I get to the bottom of the comments, if there’s no explanation (apologies if I missed it already) I’ll go read Goddard’s post and figure out what he’s looking for. All I know that that it’s text, otherwise people wouldn’t be talking about grep and keywords instead of Ampex videotape.
Argh – You’re…
Seriously? Are you that ignorant? Let’s see he wanted the history of all decisions and discussions on temperature adjustments from FRICKIN NOAA. What is the size of the amount of data that contains the word temperature from NOAA!!! How many of those docs contain the common word “adjustment”? How many man hrs does it take to review the docs? NOAA is by law prevented from releasing personal information. So the personal review of the person who recorded temperatures, and also adjusted the wind vane would come up in that search. As a taxpayer I have to pay the resulting lawsuit settlement because NOAA followed your half ass advice and “emailed” a few terabytes of uncensored data.
Conclusion:
Think before you type
Don’t know whom you’re responding to, but your comment seems to assume that NOAA’s difficulty stems from the fact that it hasn’t kept personal and scientific data separated. That’s just nuts. . . . . wait a sec., you’re probably right.
Sorry, I’m replying to dbstealy’s comment. In which he state it should take minutes. If they had requested just the data, maybe, but they did not, they want meeting minutes etc. where any discussion occurred of temperature adjustments. That is not scientific data and frankly scientific data and personal data are probably mixed all the time. An email example, hey Jim here’s the reasoning for the adjustment and how’s the your child’s cancer doing? Scientific data mixed w/personal, don’t release one broke the law, do release the other broke the law. So it has to be filtered by law.
Before I can agree I’ll have to read “this FOI request.” Can you point me to it?
Well ignoring the slightly faux outrage (they actually say by amending the request to identify more accurately specific material, the cost would be less, and are not demanding it, but saying thats their estimate of what it would cost, charged as a commercial request) may I suggest you crowd source it. It would only take 2000 people contributing 100 dollars each, and I certainly would contribute. Call their bluff on the matter…
None,
I’ll match your offer.
Typical public servants. You pay, we play!
The reason the search will take so long is to allow time to alter the data to fit the global warming fantasy.
Typical public servants. You pay, we play……
Tell the Senate to send someone there to copy every hard drive in their possession and we’ll look for what we want ourselves.
Problem solved.
Yep, free all the data. Be transparent….
I’m in the wrong business
hmmm this makes me wonder who at NOAA is getting the money from the “NOAA Weather” app for the iphone, Also like to know; if it was created, serviced or uses NOAA computers or time….. since there is a free and pay version to whom does the money go?? The free version has pop up “Ads” so money is generated from both versions.
Cheers!
Joe
“Indeed, experience suggests that agencies are most resistant to granting fee waivers when they suspect that the information sought may cast them in a less than flattering light or may lead to proposals to reform their practices. Yet that is precisely the type of information which the FOIA is supposed to disclose, and agencies should not be allowed to use fees as an offensive weapon against requesters seeking access to Government information….”
http://www.foiadvocates.com/fees.html
To me, it sure sounds like someone has something to hide…
1.9 years ! so are they lying or incompetent ?
Both, it appears.
If NOAA is so mismanaged and disorganized that they feel they have to charge these amounts to search for data that should be readily available, then perhaps NOAA needs to be disbanded. Its obvious from their response that further tax payer monies will only be wasted.
‘The employee’s conducting the search’s salary range from….’ Good grief, these jokers can’t even write English!
At that price they should know how to use apostrophe’s. 🙂
Maybe’s they’s hasing’s charge’s’ per each’s keystrokes’s?
For GISS personnel: Bob’s Quick guide to apostrophe use:
http://www.angryflower.com/aposter3.jpg
In re. Bob’s Quick Guide to the Apostrophe, You Idiots….
Is this the work of cartoonist Phil Foglio</a?? Certainly looks like his stuff….
Tucci,
Don’t know who did it, maybe this will tell you:
http://www.angryflower.com
[Contact info at bottom]
Thank’ee. Bob the Angry Flower is the creative work of one Stephen Notley, though he sure as hell uses stylistic tricks I’d long identified with Foglio’s cartooning.
My initial reaction was to side with NOAA. In an organization that large and diverse, a broadly worded data request could easily result in expenses that high. It is a LOT more complicated than just “doing a search”. You’ve got everything from email to realtime data collection to analytic systems and so on. Even if there was once search tool that could access all these systems (which there isn’t) you also have the problem of false positives. For example, a given keyword could well turn up correspondence regarding an HR invesitgation that they’d get sued for releasing details of.
That said, I wondered over to Goddard’s site to see what he asked for. According to his site:
Kent Clizbe and I have been working for almost a year to get them to release their published monthly temperature data over the past couple of decades, which they overwrite in place-
That changes everything, if that is all that was asked for. Over writing data in place isn’t necessarily a good IT practice, but neither is it wrong. One of the purposes of backup and recovery systems in an IT environment is to be able to reproduce the data exactly as it existed at any given point in time in the past. NOAA’s response doesn’t say they don’t have that data, only that it would be very expensive to produce, so I can only presume that such backup and recovery systems do in fact exist. Casting them as “documents” that might have to be “copied” is a red herring. They aren’t documents, they are data files as they existed at previous points in time. There are many different backup and recovery architectures out there, but the most common is “weekly full, daily incremental”. Producing a copy of the data as it existed at the end of every “weekly full” is technically trivial.
Now, that doesn’t mean NOAA’s estimate is unreasonable. First of all, not knowing anything about NOAA’s IT infrastructure, I can only surmise that they have proper backup and recovery systems. But even if they do, there is a massive difference in recovering data from say five years ago compared to twenty years ago. Computer systems have changed, operating systems have changed, file formats have changed. H*ll, one of the biggest problems with trying to recover data from a twenty year old tape cartridge may well be finding a 20 year old tape drive of the correct format that still works. Having had customers in absolute panic mode due to an unexpected requirement to do just that, I can advise that the challenge is greater than one might think, and not inexpensive.
You went with this pretty much where I did. My big question is, even if Goddard got the data gratis, how would he verify that it hasn’t been tampered with?
And if he paid the asking price for it, how would he verify that it hasn’t been tampered with? Money doesn’t buy integrity.
Tampering with data of this complexity without leaving traces of it being tampered with is a LOT harder to do than one might think. In this case, Goddard would have multiple versions of the data, and any tampering would have to be carried forward from one version to the next in a consistent fashion. Very large challenge and a LOT of ways to leave traces of what you did without meaning to, not to mention that you’d need a lot of compute power and a lot of people who know EXACTLY what they’re doing. In other words, do an amateur job of the tampering, you’ll be instantly exposed. Do a professional job of it, and a lot of people have to be involved, and one of them will eventually leak….
Yeah? Have you taken a look at what’s been passed off as Obozo’s 1980 draft registration card?
Datestamped “80” (when U.S. Postal Service datestamps always include all four digits in the year) in a manner which is obviously – even to the untrained eye – the result of taking a “2008” stamp element, slicing it in half, and inverting the “08” to look as if it reads “80.”
Incredibly “amateur job” of forgery, and yet nobody in the Democrat Party Audiovisual Club (formerly called “the mainstream media”) has yet given it any play whatsoever.
And yet the present environment is such that any changes at all, even when they’re fully documented with both the putative before and processed after data made publicly available, along with the source codes used to do the processing, are considered tampering.
Goddard “wins” and NOAA “loses” no matter what happens. Even so, I side with Goddard. He shouldn’t be charged for the cost of retrieval. I’m more than willing to pay my share of it, in taxes, from a general fund. I’d prefer FOIA not be the normal mechanism for this, or any ad hoc process, but damn, there are an awful lot of gummint data out there which simply cannot be easy to retrieve on a whim, nor put on public servers in bullk without a serious amount of expense.
There really ought to be a reasonable and happy medium here, but I know better.
So NOAA has not been doing their job?
If they cannot supply the information, how can they make any statements of trends?
Sure systems have changed, but that data is the foundation of the NOAA climate prognosis scam.
Maybe it is time to fire them all.
For how is the taxpayer served by such bureaucrats?
They can make statements about trends from the current version of the data, which would contain historical calculations. For example, suppose the current data says that the temperature at a particular location on March 23, 1975 was 56 degrees F. Well, we can draw a trend line with that as one data point. What Goddard is trying to find out is what did the data say the temperature was at that location on that date in 1980? 1985? 1990? etc
The trouble with such a surmise is that when the members of “the consensus” make reference to precisely such NOAA data (or claim in their precious “peer-reviewed” publications to having secured that data in comparison with recent measurements and projected trends), it means that the data had been accessed by those who were privileged – by way of their religious orthodoxy, perhaps? – to get that information out of “NOAA’s IT infrastructure.”
If it had been done once (for the benefit of those “Mike’s Hockey Team” members in good standing), what excuse do the NOAA bureaucrats have for claiming that it’s murderously expensive to re-do the data collection from those legacy files?
No back-ups on their e-mail servers?
Ooh, how Clintonian!
@davidmhoffer – agreed but,
I would expect an outfit such as NOAA would protect the sanctity of their data with backup to contemporary media (full time work that) and I would hope they have an IT crew with that responsibility. I am not diminishing the task, but that stuff shouldn’t be on antique tape drives.
Maybe 15 years ago I worked with NASA at Kennedy Center to devise ways for database integration. I was on a different mission, but I expected they had a larger task than I and so I went to them for help. I was amazed at the mess of ways and forms in which they had the data. They had Access and Excel stuff in Houston, parts lists on every imaginable proprietary legacy system, Banner . . . plain text files, mixed OS’s, you name it.
Anyway, that is IT-wise now a trivial task, assuming they spent the huge man hours to integrate and maintain their information, particularly if the FOI simply requires a careful query of a single table however large.
I think it may be reasonable to have a fee, even though they’ve already got our money, but this is outrageous and Brandon’s point is critical – can you believe it?
I was amazed at the mess of ways and forms in which they had the data. They had Access and Excel stuff in Houston, parts lists on every imaginable proprietary legacy system, Banner . . . plain text files, mixed OS’s, you name it.
Absolutely typical of a large research organization, public or private.
Anyway, that is IT-wise now a trivial task, assuming they spent the huge man hours to integrate and maintain their information,
I would be astonished if they had. IT staff tends to focus on the most pressing problems. Converting 20… or even 10, year old data from a legacy operating system with a legacy application with a legacy file system on a legacy media that has to be read by a legacy backup software package… well you get the idea. No urgent need for the data, tasks like that just drift to the bottom of the “to do” pile and rarely get attended to.
Smart IT guys do amazing things. As a test, we ran VAX ands DEC PDP-11 programs on a Victor 9000 in about 1982. The Victor 9000 preceded the IBM PC. VAX/PDP-11 share time was expensive even internally, so a skunk works developed that duplicated almost all of the functionality of our “mainframes” – much slower of course, but at costs that were virtually zero in comparison. We also discovered “distributed” computing and within a few years centralized computing systems disappeared, replaced with centralized storage and backup, and complete searchability. We even had some folks that had figured out how to “computerize” the old AES word processing machines. All on going files were moved to new platforms as they were implemented so data access was continuous.
Victor 9000’s, AES, PDP-11 and VAX machines are still available today and there are services that will “try” to restore 30 year old data. It is amazing what actually can be recovered; including written over information. I am no computer expert, but as a manager in an engineering company, I had several “opportunities” to recover old data for clients. Often as a “business development” exercise.
The requested information belongs to the people who paid for it: the American taxpaying public.
There would be zero impact on the original information if it was simply copied electroniocally, and handed over to those requesting it. What’s wrong with that??
Instead, they are stonewalling. All NOAA needs to do is make electronic copies of what they have. Simples, no? Send DVD’s of the info; ALL of it, and then some.
Because to an outside observer seeing their billing of hundreds of thousands of dollars on the one hand for things they can provide, and on the other hand, telling folks it’s too much trouble, the Occam’s Razor explanation seems to be that they are hiding plenty. You can bet if Obama or Harry Reid or Pelsoi asked for the info, it would be promptly provided, and at no cost.
NOAA should just make copies, dump it on Goddard and Clibze, and walk away…
But, NO-O-O-O-O-O-OOO…
Why not?
that is not what NOAA in the reply says he requested, so either NOAA is lying or Steve “forgot” to mention all the other stuff he requested.
You’re jumping to conclusions without knowing what the FOIA request was. Kent Clizbe gives some details in the comments (March 24, 2015 at 9:33 pm). The request was much broader than Goddard says in his blog post. The NOAA response is understandable, reasonable and polite. They suggest a narrower request might be possible, rather than laughing in Goddard’s general direction for his monumentally pretentious letter, which I believe is the more appropriate response. We do ourselves no favors when we do ridiculous things and expect others to simply accommodate us.
davidmhoffer, the NOAA FTP server always contains the most current version, which is computed every night. However, if you want an older version of GHCNv3 you just have to send an email. No need for a FOIA request for that.
The problem of this FOIA request is that the time is not limited and that thus also correspondence is asked from a time that everything was done on paper. That is a lot of work.
Just a month ago this blog was all up in arms because of the FOIA harassment of 7 of your friends by Democrat Raul M. Grijalva. Wouldn’t it be a good idea if all sides stop their FOIA harassment? This is not how science is done. This is targeting scientists for their politically inconvenient results.
@V. Venema,
It is a completely different situation, which you would know if you had followed it. Grijalva picked only his political enemies to harass. G&C are simply asking the custodians for scientific data. Big difference.
dbstealey,
https://stevengoddard.wordpress.com/2015/03/20/freedom-isnt-free/#comment-510634
There’s a bit of confusion about what we are asking for.
Wellllll …. that DOES tend to happen when the information isn’t in the head post.
Our goal, instead of endlessly debating the direction of temperature changes, is to get directly to the point–to report on the government, non-government, private, academic, and other individuals who colluded to begin “homogenizing” the data to support the man-caused runaway global warming scenario.
That’s why we’re asking for the communications about the changes to the data.
Let’s review your reply to Victor:
The big difference here is what you say was asked for, and what Kent Clizbe says was asked for. Which you would know had you been following this.
Gates,
What was that all about?
You need to get out more. It’s a nice day, go out and relax. You’re getting fixated on things that don’t matter.
dbstealey,
My mistake. It’s not that you haven’t been paying attention. The issue here is that you simply never learned how to read.
Irony. I’m not the one making noises about suing the gummint for (“only”) weather data available for the price of a simple email. Since you’ve flunked the literacy test, let’s see how you do on the numeracy portion. Do you think a successful lawsuit against the Feds would cost more, or less than $262,000 NOAA says they require to honor the law and fulfil Goddard and Clizbe’s FOIA request? Which would be most cost-effective from the taxpayer’s point of view?
Tell you what. Since I’m a big supporter of government and scientific transparency, and willing to put my money where my mouth is: If Anthony starts a coordinated pledge drive with Goddard and Clizbe to raise the $262,000 for the FOIA to go through, I’ll contribute $500.00 out of my own funds in support. If the drive reaches 50% of that amount, I’ll contribute another $500.00 as a further demonstration of my commitment to the effort. If, after 90 days from the opening bell of the pledge drive, the full $262,000 is not met, all collected funds are to be donated to: http://climatesciencedefensefund.org/
Please run my offer up the ladder. Thanks.
Gates, AKA: Victor Venema’s Nanny says:
Since you’ve flunked the literacy test…
Anyone who reads these posts knows that my grammar, spelling and general literacy is superior to Gates’. That’s why he just elected himself as putative judge. No one else would.
Dream on, Gates, and while you’re at it, get a life.
dbstealey,
I just made you a $1,000 bet, you’re still running your mouth. Logic dictates that you must be dictating, after all. So not only can you not read, nor write, you are exceptionally stupid when it comes to tactics. I called your bluff, you turned tail and folded in a fit of cowardly buffoonery. As usual. But my offer to Anthony still stands. Let’s see if he’s got more spine than you do.
Where is the original letter with the FOIA request? NOAA’s response sounds like much more than just a few data files were requested:
“Very few if any letters, phone logs, memos, and other communications on this subject would be available. Historical internal and external emails are archived, though they are expensive to access and analyze due to unsupported technology.”
Without the original request, it is not possible to judge NOAA’s response.
Go to the linked article on Goddard’s blog. Read comments by kentclizbe on March 24,2015 at 9:33 pm: “e-mails, letters, phone logs, memos and other communications between…” Ridiculous! You have to read it to believe it. Only the most bombastic of Congressmen could rise to that level of hubris.
It’s over at https://stevengoddard.wordpress.com/2015/03/20/freedom-isnt-free/#comment-508739
It merely asks for analyzing pretty much every piece of communication at the NCDC over at least a decade and maybe for the life of the agency.
Oh, it asks for data and software too. At least that’s feasible.
He obviously has no experience using FOIA. “from the beginning of the adjustments through today” is just sloppy.
Brought to you by the “Administtation of Transparency.”
Despicable.
“According to Steve Goddard, NOAA have just demanded a $262,000 administrative fee for zipping up a few raw data files.”
It would be helpful to know what “a few raw data files” means. Are we talking about data files for a local area or for the globe? Is it for one year or multiple years? Perhaps if Steve asked for “all” their raw data, it would save them having to search though it. They could just copy everything to a password protected area on their website and provide Steve access to it.
Goddard isn’t exactly being forthcoming about his original request, to put it politely. Go to the linked article on Goddard’s blog. Read comments by kentclizbe on March 24,2015 at 9:33 pm: “e-mails, letters, phone logs, memos and other communications between…” Yeah, a whole lot more than “a few raw data files”.
yep. That’s the comment I found interesting. The first part of the request is for all communications internal and external, phone logs, memos, etc. regarding adjustments. The 2nd part is a request for some data files. There was no time limit apparent in the request, so what the request boils down to is going back through every single scrap of paper, disks, and data storage looking for discussion a regarding adjustments.
The head post is very misleading.
One of the people I used to work with was a Contracting Officer for the VA. One day, he received a FOIA request. He called the people initiating the FOIA request, and told him he’d send them what they wanted if they dropped the FOIA request. They did, and he sent them the information. He said complying with everything required under FOIA is a royal PITA.
Tell them ‘the check is in the mail”.
I’m guessing “original data” in many federal agencies is so deeply buried under a mountain of revised revisions of that data that lots of heavy machinery would be required to dig it out. That quarter $ mil is probably just staff time required to locate the general vicinity where digging should begin. But first must come a series of staff meetings to reach consensus on a working definition for “original data”, that term being almost certainly unfamiliar to many federal employees involved in any endeavor related to “climate”. Then the real expenses begin.
As a former NOAA employee, there is just no question that NOAA is a disaster zone…filled with politically correct narcissists especially in the management and administrative ranks. I was forced to file several complaints against NOAA management over a 20 year period, some of which were successful, and all I can say is I’m so thankful I’m outta there. Several other of my former colleagues feel the same, so those in charge can’t just say it was me that was the bad guy, the cancer, etc. Many of the field people do good work and are conscientious, but the management is completely in another realm, and I think that they think they are in a higher and much-deserved elevated realm. NOAA management and administration run roughshod over well-meaning employees who are trying hard to do good jobs, but at the same time these subordinate employees are trying not only to keep their jobs, but to get promotions; they are completely at the mercy of management, and also of complicit employee unions. NWS, NOAA, and Department of Commerce management need to be turned out and completely overhauled. But, of course, that applies to most of government leadership as well. A pity.
I’ve seen this somewhere before.
Ah, yes.
http://www.nytimes.com/2014/04/28/opinion/verbatim-what-is-a-photocopier.html?_r=0
Jeff Cagle,
Good one. With no judge present, the witness can play those games.
cough Hillary Clinton’s emails cough.
1.9 years x 2080 hrs/years x $37/hr x 1.16 overhead = $169,620. $77,000 for a file clerk? No wonder they can’t figure out what the temperature is.
There’s a post on Goddard’s blog that shows the request, including:
For the NOAA/National Climactic Data Center: Please provide Internal and external e-mails, letters, phone logs, memos, and other communications, from, to, and between: government employees, external consultants, experts, advisors, or other parties regarding the rationale, methodology, and other issues concerning adjustments/homogenization or other changes to both the US and global temperature record data, from the beginning of the adjustments through today.
it is complete misrepresentation to say these are “a few computer files”.
It’s a fishing expedition. No wonder it is so labor intensive and expensive
Missed that, thanks for pointing it out.
If that’s what he asked for, it would be very hard to produce.
That’s a lot different than what he said in his blog post.
Kent Clizbe and I have been working for almost a year to get them to release their published monthly temperature data over the past couple of decades, which they overwrite in place in order to hide their ongoing manipulations.
If what he said above is what he is ACTUALLY after, then the request needs to be rewritten.
just saw this – yup that’s a different ball game, although I appreciate what they’re after . . .
David Sanger: I tried to verify your claim. I searched Goddard’s blog for FOIA, NOAA, Clizbe, and emails. I can’t seem to find the post you’re referring to.
I think you’re just talking smack. Put up or shut up. Provide a link to the post.
Not that hard to find
https://stevengoddard.wordpress.com/2015/03/20/freedom-isnt-free/#comment-508739
David Sanger,
The post immediately below that one makes the central point:
That should all be publicly available in a transparent government.
What’s the problem? If NOAA is halfway competently organized and supervised, it would surely take a low-level employee less than a day to produce everything requested to comply with the law.
You make it sound onerous, but with everything electronically recorded and backed up these days, the actual ‘work’ involved amounts to some browsing for files, and a few clicks of a mouse.
Contrast that with the multi-$millions funneled through NOAA every month. Do you think a little transparency is such a difficult thing?
NOAA is treating Clibze and Goddard exactly like they are ememies, instead of fellow taxpaying citizens. Someone is wrong here. Want to take a guess as to which one it is?
Louis couldn’t even find the source David quoted in few minutes yet dbstealy thinks NOAA should be able to comply with that lengthy request in a few minutes and email it.
This stuff just cracks me up some days.
David thanks for putting up, sure we will both get flamed for using logic and being critically minded.
For clarity ” This FOI request could be done literally in a few minutes and emailed” dbstealy March 27 3:22 PM. I am curious what the search terms dbstealy thanks would cover this and still be able to be filtered for personal info in a year no less a day.
David Sanger: I stand corrected. My apologies.
ironargonaut: you read this blog enough to know that alarmist trolls show up and make bizarre and often false claims. I asked David Sanger to put up or shut up, and to his credit, he did. I didn’t flame him, I challenged him (although I would have flamed him as a troll if he hadn’t).
Most readers of WUWT thrive on logic and critical thinking. As long as you continue to display both, you won’t get flamed here.
Try a little harder, Louis. Go to the linked article on Goddard’s blog. Read comments by kentclizbe on March 24,2015 at 9:33 pm: “e-mails, letters, phone logs, memos and other communications between…”
Louis, no offense intended towards you. Put up or shut is a perfectly acceptable request in most cases. I in fact stated Goddard should have done the same. I was merely illustrating that searching for even specific info is not always as easy as many claim. I know at my work I couldn’t search the companies entire database, I don’t have the correct permissions, I doubt any single person does. Even finding what a policy is on the company intranet is a hair pulling experience some days. David had a specific quote so asking for the link is OK and you apologized for the smack comment, which since you did try to find the quote before making the comment it was not over the top.
NOAA has finally lost all my respect. That cost is way out of line with the true cost, and is meant only to quash the FOIA. Isn’t that illegal?
Well so much for the “Free”-dom of Information Act, and transparency in government.
I became a partner in a business with paper records going back to the 1920’s, I came in in 1985, not a computer in sight.
If you wanted me to chase down data since 1920, I’d give you a number resembling $262K, and if you wouldn’t relent I would hire 10 people to work on it (at public expense).
Try to imagine what Barack Obama would do if he wanted to get the same information. And do the same thing.
Ah, but he’s the President? He didn’t learn his tactics in the Oval Office. He learned them before he got there.
Even if he only wants the raw temperature record, the cost is probably accurate. They probably just overwrote the raw record with the corrected, so finding the file version with the raw record will be very difficult.
All govt agencies are gonna be using this tactic.
Monthly temperature data???? That is all he is asking for???
No. Here is the relevent letter. You can see the request is MUCH broader than temperature data
https://stevengoddard.files.wordpress.com/2015/03/fee-notification-letter-2014-001602.pdf
OK, case closed.
Are there no whistle blowers at NOAA?
I’m confused.
1.9 yrs at $37/hr times 1.16 (16% overhead) is only $163,096
Assuming the low value of $25/hr, it is only $110,200.
And they quoted $262,000 !!!
What am I missing?
those pesky “indirect” costs for golfing vacations?
Actually had a dean once that did that . . . and got caught.
Adjustment.
Union fees, average pay for the workers doing the slow down because they aren’t paid enough at the port of Portland 80K a year. That’s a lot of money for a HS diploma and driving a forklift, I would think someone who can use a computer would rate more.
Overhead and profit.
Thanks, Eric Worrall and Steve Goddard.
This is a ridiculous and inept response from NOAA.
I agree as to the quality of the reply.
But, just out of curiosity, what was the request?
Yepp the FOI game!
Here in the UK we FoI `d The Environment Agency (think EPA)- first time around $2000 = many embarrassing disclosures
Second time around 20% of the data and a price tag of $55000 duh!
FoI needs fitting with some sharp teeth – willful misconduct in delivery should attract personal sanctions onto the perpetrator(s)
I just read the request. It’s a pretty big request.
Hi evan,
We pay pretty big taxes. On top of that, what do you think of the charge of $262,000?
Maybe that’s chump change for the NOAA. But it would buy a lot of employee time — and with about $260,000 left over, IMHO.
Tell the truth, now: do you really think that’s an honest charge for what was requested?
Do you even know what was requested dbstealey?
In most cases I’m a big fan of full and complete transparency but this time the request was stupid. As if any organisation can effectively identify all the documentation relating to “discussions” about a broad and long term subject like temperature adjustment.
YES
timthetoolman,
Apparently they can provide the info. They gave their cost, didn’t they?
This is just the government’s version of stonewalling. Either there is transparency, and the government must follow its own laws, or the whole thing is a sham.
Which?
Too many computer savvy people here have said they can provide the requested data with no problem. Who are you gonna believe? Them, or the bureaucrats?
Frankly I think the request could never be properly satisfied as it stands. There will always be something that will be missed when the request is so broad. What is “discussion” on temperature adjustment and what isn’t? Where do I look to find it all?
You added… “Too many computer savvy people here have said they can provide the requested data with no problem.”
Irrelevant. What about memos? What about written documents? What about archived records? On old media that cant easily be read now? Too many computer savvy people here aren’t old enough to remember life before computers…
Under normal circumstances I agree, this is an unreasonable request. The request was answered with a matching unreasonable cost, although the reasonable cost would surely still be cost prohibitive.
Having said that, the request is (unfortunately) born from what many feel is data tampering. It would be reasonable to request the original raw temperature values, but many of us feel that we can’t trust what is provided due to pure manipulation of numbers to fit a desired outcome. Therefore, there is a need to verify and validate. This is a government agency which operates via tax payer money, and there should be a purity in terms of operational protocol.
I doubt it, but I don’t know.
The raw and and adjusted NOAA USHCN data, which is all I need, I got for free. I can figure out the “tampering” from that. And I have — it’s real, but not intentional; it’s a mistake I could easily have made. But it is a mistake.
Do you speak COBOL! Safe to say, no, and no NOAA manager can speak nor has knowledge of COBOL.
And then there are the “hard copy” from the late-40s through late-90s (USA Archives).
And that is just the tip of the iceberg problems in the ‘Federal.’
“Hoy Hoy! Berg Off Port And Closing.”
Helmsman [comparing Wishy to Vodka]: “Did E say Ort ? Ain’t know Ort! Bloody Liverpoolie!”
Shipmate sweating buckets [the only one looking out the window]: “Just spins der dial and pray to God.”
Ha ha.
NOAA could ask the US Department of the Treasury to print a 100 billion dollar bill. This bill would be wrapped around and glued on a USB flash drive. This flash drive would contain the password for an encrypted file in which all the data of NOAA and other organizations who want to stonewall FOIA requests would be stored. The password could only be obtained by unpacking the flash drive and destroying the bill. Therefore any request would have a price tag of 100 billion dollars.
FOIA was set up to have pretend that there is some sort of transparency. To be serious it would require continuous disclosure- no silly request process required- they put the all information straight out with some other body doing the vetting of what should be openly available.
I guess the science has settled to the point that its now buried.
Is NOAA configuring their data servers to block archive.org?
For shame, for shame.
A good discussion from various perspectives that gives food for thought.
A broad request from a known, possibly hostile source, received an indeterminate answer. Perhaps a more simple request would have opened a door. I wonder why NOAA didn’t say we can provide “X” for “$Y” but everything requested is difficult/impossible. They didn’t say that. So the question arises: “Do they actually have all the information requested?” They might, but probably in a very mixed format. Phone logs for example, may well be in handwritten diaries or computer notes depending on the individual or the group procedures. It might be that the request was guaranteed to fail with its broad scope. Undoubtedly NOAA knows who Goddard is, so they would be cautions in any case.
Hey – they could have just destroyed the hard drives, like the EPA administrator did.
Or, better yet, don’t keep any ‘official’ information on the NOAA servers. Just keep the ‘official’ data and communications on a private server….. and wipe it clean, if anyone tries to subpoena the information, the same way ex-Secretary of State Hillary Clinton just did!
Open corruption and lawlessness, by the ‘most transparent administration’ evah….. yeah.
I can’t dispute the likelihood that at least part of NOAA’s motive in stonewalling Goddard’s request is to hide the process of data manipulation from professional scrutiny, but that may not be the most accurate reason such data was first made inaccessible to ordinary citizens.
Weather and climate are personal hobbies and for fun Some years ago, I used to scan the NCDC web site for historical( and recent) rainfall and temperature data to detect trends, make comparisons, etc. or for no particular reason at all :). Then, all of the sudden, that information was largely inaccessible, except, as best I can remember, to select academics, professionals, etc., or pay walled., I was, as a mere taxpayer, not privileged and thus not happy.
I seem also to recall researching the matter, or perhaps just adding 2+2, and came up with the reason behind NCDC’s new policy being an attempt to prevent the burgeoning private weather sector from benefiting from public scientists hard work – and even threatening to supplant said scientists’ jobs or others’ depending on government contracts and grants. There were those, at the time( and no doubt still) calling for much of the federal government’s climate and weather functions to be privatized..In other words, making climate data hard to obtain may originally have been a mechanism to protect bureaucratic turf and livelihoods.
My personal hobby horse is that bureaucracy is, by its very nature, totalitarian and obstructive. That goes for governmental or corporate. Fascism used to be known by the left as corporatism (but by golly the trains ran on time! and folks dressed in fancy uniforms) Now we have left wing corporatism and its called environmentalism (trains are a little rickety and the uniform has allowed for tattoos, piercings, and odd hair styles).
When I was a kid in the fifties there was a truism kicking about that one could do things the “right way, the wrong way or the Army way” as the rebuke to mindless bureaucratic behavior that one learns to accept. I think it would be real important if this generation could get beyond democrat or republican and take back our government (of the people, by the people, and for the people) from the mindless bureaucracies and lawyers whose ultimate objective is, by accident or design, every bit as oppressive as any dictator to plague our planet in the past
That is probably a reasonable approximation of their cost to produce the erroneous data. They do not want honest debate. They do not want to learn. They already KNOW the truth. This is the nature of religion.
Anyone here with good contacts to the NSA? They -supposedly- know more about data and its retrieval than most people here. And, not improbable, they do have the requested data already in store. So ask them…
good point,:) Seriously, though I have to wonder at what point the NSA’s data will become the next Nixon tapes
They’ll be known as “The Barmy Records”.
I’ll do it in 1.3 years for $220,000. 🙂
It’s gonna take peasants with pitchforks for us to get our country back. Write your Congressman? Hah!
Yepper, like spreading the revisions around between different websites. I made a comparison between websites that give “local conditions” for public consumption. The computer changes the numbers back to the previous reading. It can be seen if you graph them in real time (shows up as a straight line). Nature isnt a straight line! They also leave out conspictious readings (another form of filtering).
I think it should be “Blarney Records”.
The NOAA FOI administrator did offer to ‘negotiate’ with Goddard over a reduced set of data which would therefore be much cheaper (or free) and much quicker to arrive.
I understand that in the UK, it is common, in complex FOI cases to ‘negotiate’ once all parties ‘understand’ the impact of each line in a FOI request.
Perhaps a rewrite or negotiation, to get some/most/nearly all data then consider what to do about the remainder?
A dir /s *.txt is cheaper than a dir /s *.* ?
May be the new senate and congress will sort them out?
Reblogged this on Wolsten and commented:
Hard to believe this isn’t made up. Either NOAA are completely incompetent at record handling, which ought to be a scandal in itself, or they want to hide the data, derived from publicly funded research, from the public. Either way, this stinks.
NOAA /sarc
40 days and nights of rain
FROM THEIR WEBSITE:
“NOAA’s dedicated scientists use cutting-edge research and high-tech instrumentation to provide citizens, planners, emergency managers and other decision makers with reliable information they need when they need it.”
I didn’t see a mention of: “For just $262,000”
LOL. Website “mission” statements, or statements of due diligence? I used to work for IBM (UK), Fujitsu (NZ), Honda (UK), Renault (UK), ANZ Bank (NZ and Aus), Bank of New Zealand, Commonwealth Bank of Australia, HP (Aus), did work for Ministory of Justice (NZ), Department of Conservation (NZ). All such similar statements were bullcarp! HAH! NIWA’s (NZ) fish “database” was, I dunno, one probably that the use of crayons to create something more reliable and workable would have been better!
NOAA had their data on Hillary Clinton’s mail server before it was wiped clean. The backup unfortunately was on Lois Lerner’s harddrive.
The 262,000 USD pays for a wholesale reconstruction from memory, and the necessary crayons.
If the files are too big, they can burn them to a CD or DVD and put it in the mail. Decidedly low-tech, yes, but a lot less effort than setting up a password-protected website.
Why not start a crowd sourced fund-raising effort? I’d pay a reasonable amount to have shared open access to all that information. My guess is thousands of other readers of this blog would as well. All we need is $10 each from 26,200 people (a small fraction of total readers of WUWT). I would really love to see NOAA’s bluff called on this.
I’d join that lovely party.
Yeah, give the bloodsuckers more money to be used against us. Great plan.
How about reminding them who is funding their freaking budget by sending NOAA an e-mail from all WUWT readers?
That will teach them.
Call in the expert: Chris Horner.
I’m fairly sure that if Steve Goddard got together with Chris Horner from CEI they should be able to get the info for free. I’ve read through some of Horner’s FOIA requests and they are incredibly comprehensive and to the point.
FREEDOM of information means just that and the courts have supported that attempting to charge exorbitant violates they act. NOAA is screwing with Goddard.
Whatever is cheaper: either sue NOAA, with the help of Chris Horner, or ‘buy’ the information requested. Anyway, it’s going to be a nice embarassment. Or a nauseating experience for NOAA when they have to admit in public that their files and data are anything else but orderly kept.
I’d join that lovely party.
Doing the math, they’re saying it will take around 6,000 man hours to do the search. What, are they doing it by hand?!
Oops! The previous comment is from Bob Shapiro, and I have the http://www.US-Issue.com Blog.
So why not just take NOAA up on their offer to scope the request more precisely?
I don’t understand what the problem is here.
Try the revised and trimmed request (eg using Horner’s skills), then if that doesn’t get NOAA attention or a climb down, go for the crowd funding mechanism to buy the info as NOAA outlined in their response……I’d be happy to contribute…..regularlurker
This shows how deep we have sunk. Our Governments and Government Institutions financed with taxpayer money no longer serve the public. Defunding them is the only solution. Fortunately they are on path of self destruction producing the biggest National debt in history. In the meantime civil servant got a whole different meaning.
They’re all parasites now kanabalising their host. It’s them or us. Any questions?
Having spent well over a half century as a scientist working very closely with NASA and NOAA on a great many programs I am extremely disappointed that those once respected scientific organizations have become totally politicalized. NOAA had a special problem, that problem was NASA. After the moon project NASA was running out of things to do so over the years, helped by their high public profile, they started moving in and taking over NOAA’s traditional programs (most of the public never heard of NOAA) leaving NOAA, so to speak, “sucking hind tit”, thus when the opportunity came knocking NOAA quickly jumped on the Obama bandwagon. Today Obama, NASA and NOAA are loyal members and strong supporters of the IPCC Anthropogenic Club; in fact if both NASA and NOAA stopped supporting IPCC that organization would quickly disappear from public view…
Without any comment:
http://www.zerohedge.com/news/2015-03-27/government-secrecy-all-time-high
and
http://www.zerohedge.com/news/2015-03-28/proof-positive-recovery-lie-and-fed-only-interested-tbtfs
Providing internal and external e-mails, letters, phone logs, memos, and other communications would be very difficult. As far as the cost, it is a government estimate so expect the final cost to be at least 4 times the cost if it ever actually completed.
On the other hand as public servants, NOAA could have responded with, “It would be problematic to provide external e-mails, letters, phone logs, memos, and other communications, however this is what we can provide. In addition to the raw data for the time period requested, we can provide the specifications that explains the rational and change to the raw data.”
When I worked in Information technology, nothing changed in the system, the programming as well as data such as look-up data (factors, rates, constants, etc) without a change request which included the reason for the change, and a specification that detailed the changes that would be made. This formal process insured a complete historical record of what, why and how of changes. Transactional data would go through an even more rigorous process, since it represented a legally binding transaction in point in time. If NOAA does not have a record of the what, why and how of the changes made and cannot provide this information easily and in a timely way then that organization is grossly incompetent; I wouldn’t put them in charge of keeping track of the money at a bake sale.
So it looks like Steve goofed in asking for the communications, however NOAAs ill-considered, non-responsive response leads to questions of gross incompetence at NOAA in managing the temperature record.
Here is a documentary film on Trey Gowdy tirelessly chasing down NOAA and Hillary’s missing e-mails:
http://youtu.be/iRlZaSWocQk
Steve Goddard appears to have built a straw man here – the response from NOAA seems reasonable given the scope of the FOI request.
Ever hear of the internet? Why is it public information isn’t already posted on a web server?
Straw man arguments are pretty weak. Who says all the info is public – why would an agency put their emails, telephone logs, meeting minutes on the internet?
why didnt goddard post a copy of his request and all correspondence?
what is he hiding?
harrytwinotter,
For a great example of a ‘strawman’ argument, see Steven Mosher’s post above.
The letter was typed on the 17th, and the quarter million dollars and change to pull some data was due a week later—or screw the FOI request!
Question: If it takes $262,000 every time you want to pull some data, how useful is that data?
Well it doesn’t matter. It’s obviously a coverup.
But whatever amount of money they’re asking for so they can bury their data, Monday? Seriously, by Monday?
Because science.
Has anyone yet considered the fact that simply calculating the estimation of those costs – $262,000, for pity’s sake! – in about a week is goddam remarkable?
Has anyone reading here ever dealt with a federal or state bureaucracy, trying to get anything like cost estimates out of them? Nobody working on the taxpayer’s dime ever responds with such alacrity to a taxpayer’s request with that kind of alleged precision.
What, has NOAA got some kind of fee schedule for responding to FOIA requests?
One schedule, of course, for members of Mike’s Hockey Team, and another for “deniers.”
Christoph Dollis says:
Question: If it takes $262,000 every time you want to pull some data, how useful is that data?
For that matter, if it costs $262,000 every time you want to pull some data, how useful is the FOI law?
If only they believed in AGW then that info would not only be free, but filtered and sorted. Being that’s it is 1. public info, and 2. the public has paid for it already. But the non believer???!! They must pay so that no erroneous conclusions can be reached that disagree with official dogma.
It’s nothing short of a simple attempt to avoid insight, access and transparency, hardly legal.
Just par for the course during the “most transparent administration ever”.
When I tried to request a copy or copies of one or more of the WBAN Form-10 reports I personally authored, I ran into the same runaround from one office to the next only to be told the cost of even one copy of a form was going to be non-specific and too expensive to contemplate. While talking to these various offices I learned that many of these original forms were being stored in a government office basement, and the forms I wanted may have already been destroyed as a result of waterlogging, mold, mildew, and being eaten by insects. It was suggested it may not be possible to retrieve my forms because they had been destroyed and lost as a result of such damage.
NOAA and EPA have been using this tactic to keep their fraud out of the public eye. A similar wall was met by itssd.org last year. The home page of their site has two “portals” down along the right side. One for EPA FOIAs and one for NOAA.
Goodard doesn’t understand how to do an FOIA request.
“As an IT expert with over 20 years of experience, my expert opinion on the claim by NOAA that it would require $262,000 to gather up a few computer files and send them to Steve is that it sounds like a complete crock. Even if some of the files are in printed form, they can just be run through a scanner – my automatic page feed scanner can process a page every few seconds, even cheap scanners can process thousands of pages per day. If the files are too big to put in an email (likely), for trivial cost NOAA could publish them on a password protected web page – it would take at most a day to set up such a web page, and add the files to it.”
The request has nothing to do with your IT experience. It involves more than gathering up a few computer files as Goddard apparently went on a fishing expedition with a broad request. That almost always fails.
“The National Environmental Information Center (which includes the former National Climatic
Data Center) has access to a subset of the requested records. Very few if any letters, phone logs,
memos, and other communications on this subject would be available. Historical internal and
external emails are archived, though they are expensive to access and analyze due to unsupported
technology.
As data stewardship – including homogeneity adjustments – has been central to NCDC’s mission
for decades, determining which records are responsive to this extremely broad request will
require significant resources. Responding to this part of the request would require retrieving,
reviewing, and packaging many tens of thousands of items in at least 29 years of
communications, if they can be located”
he has an option
“Another option based on your request is to narrow your scope or update the description of what
you are asking for. If you agree to narrow your scope or update your request, please contact me
and I will set up a teleconference between you and the lead on this task”
Complete nonsense. This data is being used to formulate policy that will have a dramatic impact on the economy of the US. If they have already accessed all of this previous data to use in recent studies, and then had reason to “adjust” the prior year data, then this should be easily available and retrievable, complete with explanations as to why the data needed to be “adjusted.” This is clearly nothing but stonewalling. Why do people stonewall? I’d love to see Steve Mosher tell the IRS that their audit request for his previous years financial data was to difficult and expensive to retrieve. I bet you would have great success with that. But with all the taxpayer funds being used to keep this agency operating, and supposedly being so scientific and data intensive, it is beyond belief that this kind of dissembling is tolerated. I think you are part of the problem.