Eric Worrall writes: It looks like NOAA have found a new way to stifle FOIA inquiries from the public. According to Steve Goddard, NOAA have just demanded a $262,000 administrative fee for zipping up a few raw data files.
Steve Goddard has published a scan of the outrageous fee demand he and fellow FOIA requestor Kent Clizbe received from NOAA administrator Maria S Williams. The letter, sent on March 17th, demands $262,000 by March 24th, or further communication – otherwise Maria says they will consider the matter closed.
https://stevengoddard.files.wordpress.com/2015/03/fee-notification-letter-2014-001602.pdf
The NOAA staff directory lists Maria Williams as the Chief of Staff Support Services Branch. https://nsd.rdc.noaa.gov/nsd/pubresult?LNAME=wil
For the full story read Steve’s post – https://stevengoddard.wordpress.com/2015/03/20/freedom-isnt-free/
As an IT expert with over 20 years of experience, my expert opinion on the claim by NOAA that it would require $262,000 to gather up a few computer files and send them to Steve is that it sounds like a complete crock. Even if some of the files are in printed form, they can just be run through a scanner – my automatic page feed scanner can process a page every few seconds, even cheap scanners can process thousands of pages per day. If the files are too big to put in an email (likely), for trivial cost NOAA could publish them on a password protected web page – it would take at most a day to set up such a web page, and add the files to it.
If NOAA’s data files really are so poorly catalogued that several man years of effort would be required to find them, this is something NOAA should be fixing on their own time. If this is the case, NOAA should not be attempting to charge FOIA petitioners outrageous fees to cover NOAA’s own incompetence.
Well ignoring the slightly faux outrage (they actually say by amending the request to identify more accurately specific material, the cost would be less, and are not demanding it, but saying thats their estimate of what it would cost, charged as a commercial request) may I suggest you crowd source it. It would only take 2000 people contributing 100 dollars each, and I certainly would contribute. Call their bluff on the matter…
None,
I’ll match your offer.
Typical public servants. You pay, we play!
The reason the search will take so long is to allow time to alter the data to fit the global warming fantasy.
Typical public servants. You pay, we play……
Tell the Senate to send someone there to copy every hard drive in their possession and we’ll look for what we want ourselves.
Problem solved.
Yep, free all the data. Be transparent….
I’m in the wrong business
hmmm this makes me wonder who at NOAA is getting the money from the “NOAA Weather” app for the iphone, Also like to know; if it was created, serviced or uses NOAA computers or time….. since there is a free and pay version to whom does the money go?? The free version has pop up “Ads” so money is generated from both versions.
Cheers!
Joe
“Indeed, experience suggests that agencies are most resistant to granting fee waivers when they suspect that the information sought may cast them in a less than flattering light or may lead to proposals to reform their practices. Yet that is precisely the type of information which the FOIA is supposed to disclose, and agencies should not be allowed to use fees as an offensive weapon against requesters seeking access to Government information….”
http://www.foiadvocates.com/fees.html
To me, it sure sounds like someone has something to hide…
1.9 years ! so are they lying or incompetent ?
Both, it appears.
If NOAA is so mismanaged and disorganized that they feel they have to charge these amounts to search for data that should be readily available, then perhaps NOAA needs to be disbanded. Its obvious from their response that further tax payer monies will only be wasted.
‘The employee’s conducting the search’s salary range from….’ Good grief, these jokers can’t even write English!
At that price they should know how to use apostrophe’s. 🙂
Maybe’s they’s hasing’s charge’s’ per each’s keystrokes’s?
For GISS personnel: Bob’s Quick guide to apostrophe use:
http://www.angryflower.com/aposter3.jpg
In re. Bob’s Quick Guide to the Apostrophe, You Idiots….
Is this the work of cartoonist Phil Foglio</a?? Certainly looks like his stuff….
Tucci,
Don’t know who did it, maybe this will tell you:
http://www.angryflower.com
[Contact info at bottom]
Thank’ee. Bob the Angry Flower is the creative work of one Stephen Notley, though he sure as hell uses stylistic tricks I’d long identified with Foglio’s cartooning.
My initial reaction was to side with NOAA. In an organization that large and diverse, a broadly worded data request could easily result in expenses that high. It is a LOT more complicated than just “doing a search”. You’ve got everything from email to realtime data collection to analytic systems and so on. Even if there was once search tool that could access all these systems (which there isn’t) you also have the problem of false positives. For example, a given keyword could well turn up correspondence regarding an HR invesitgation that they’d get sued for releasing details of.
That said, I wondered over to Goddard’s site to see what he asked for. According to his site:
Kent Clizbe and I have been working for almost a year to get them to release their published monthly temperature data over the past couple of decades, which they overwrite in place-
That changes everything, if that is all that was asked for. Over writing data in place isn’t necessarily a good IT practice, but neither is it wrong. One of the purposes of backup and recovery systems in an IT environment is to be able to reproduce the data exactly as it existed at any given point in time in the past. NOAA’s response doesn’t say they don’t have that data, only that it would be very expensive to produce, so I can only presume that such backup and recovery systems do in fact exist. Casting them as “documents” that might have to be “copied” is a red herring. They aren’t documents, they are data files as they existed at previous points in time. There are many different backup and recovery architectures out there, but the most common is “weekly full, daily incremental”. Producing a copy of the data as it existed at the end of every “weekly full” is technically trivial.
Now, that doesn’t mean NOAA’s estimate is unreasonable. First of all, not knowing anything about NOAA’s IT infrastructure, I can only surmise that they have proper backup and recovery systems. But even if they do, there is a massive difference in recovering data from say five years ago compared to twenty years ago. Computer systems have changed, operating systems have changed, file formats have changed. H*ll, one of the biggest problems with trying to recover data from a twenty year old tape cartridge may well be finding a 20 year old tape drive of the correct format that still works. Having had customers in absolute panic mode due to an unexpected requirement to do just that, I can advise that the challenge is greater than one might think, and not inexpensive.
You went with this pretty much where I did. My big question is, even if Goddard got the data gratis, how would he verify that it hasn’t been tampered with?
And if he paid the asking price for it, how would he verify that it hasn’t been tampered with? Money doesn’t buy integrity.
Tampering with data of this complexity without leaving traces of it being tampered with is a LOT harder to do than one might think. In this case, Goddard would have multiple versions of the data, and any tampering would have to be carried forward from one version to the next in a consistent fashion. Very large challenge and a LOT of ways to leave traces of what you did without meaning to, not to mention that you’d need a lot of compute power and a lot of people who know EXACTLY what they’re doing. In other words, do an amateur job of the tampering, you’ll be instantly exposed. Do a professional job of it, and a lot of people have to be involved, and one of them will eventually leak….
Yeah? Have you taken a look at what’s been passed off as Obozo’s 1980 draft registration card?
Datestamped “80” (when U.S. Postal Service datestamps always include all four digits in the year) in a manner which is obviously – even to the untrained eye – the result of taking a “2008” stamp element, slicing it in half, and inverting the “08” to look as if it reads “80.”
Incredibly “amateur job” of forgery, and yet nobody in the Democrat Party Audiovisual Club (formerly called “the mainstream media”) has yet given it any play whatsoever.
And yet the present environment is such that any changes at all, even when they’re fully documented with both the putative before and processed after data made publicly available, along with the source codes used to do the processing, are considered tampering.
Goddard “wins” and NOAA “loses” no matter what happens. Even so, I side with Goddard. He shouldn’t be charged for the cost of retrieval. I’m more than willing to pay my share of it, in taxes, from a general fund. I’d prefer FOIA not be the normal mechanism for this, or any ad hoc process, but damn, there are an awful lot of gummint data out there which simply cannot be easy to retrieve on a whim, nor put on public servers in bullk without a serious amount of expense.
There really ought to be a reasonable and happy medium here, but I know better.
So NOAA has not been doing their job?
If they cannot supply the information, how can they make any statements of trends?
Sure systems have changed, but that data is the foundation of the NOAA climate prognosis scam.
Maybe it is time to fire them all.
For how is the taxpayer served by such bureaucrats?
They can make statements about trends from the current version of the data, which would contain historical calculations. For example, suppose the current data says that the temperature at a particular location on March 23, 1975 was 56 degrees F. Well, we can draw a trend line with that as one data point. What Goddard is trying to find out is what did the data say the temperature was at that location on that date in 1980? 1985? 1990? etc
The trouble with such a surmise is that when the members of “the consensus” make reference to precisely such NOAA data (or claim in their precious “peer-reviewed” publications to having secured that data in comparison with recent measurements and projected trends), it means that the data had been accessed by those who were privileged – by way of their religious orthodoxy, perhaps? – to get that information out of “NOAA’s IT infrastructure.”
If it had been done once (for the benefit of those “Mike’s Hockey Team” members in good standing), what excuse do the NOAA bureaucrats have for claiming that it’s murderously expensive to re-do the data collection from those legacy files?
No back-ups on their e-mail servers?
Ooh, how Clintonian!
@davidmhoffer – agreed but,
I would expect an outfit such as NOAA would protect the sanctity of their data with backup to contemporary media (full time work that) and I would hope they have an IT crew with that responsibility. I am not diminishing the task, but that stuff shouldn’t be on antique tape drives.
Maybe 15 years ago I worked with NASA at Kennedy Center to devise ways for database integration. I was on a different mission, but I expected they had a larger task than I and so I went to them for help. I was amazed at the mess of ways and forms in which they had the data. They had Access and Excel stuff in Houston, parts lists on every imaginable proprietary legacy system, Banner . . . plain text files, mixed OS’s, you name it.
Anyway, that is IT-wise now a trivial task, assuming they spent the huge man hours to integrate and maintain their information, particularly if the FOI simply requires a careful query of a single table however large.
I think it may be reasonable to have a fee, even though they’ve already got our money, but this is outrageous and Brandon’s point is critical – can you believe it?
I was amazed at the mess of ways and forms in which they had the data. They had Access and Excel stuff in Houston, parts lists on every imaginable proprietary legacy system, Banner . . . plain text files, mixed OS’s, you name it.
Absolutely typical of a large research organization, public or private.
Anyway, that is IT-wise now a trivial task, assuming they spent the huge man hours to integrate and maintain their information,
I would be astonished if they had. IT staff tends to focus on the most pressing problems. Converting 20… or even 10, year old data from a legacy operating system with a legacy application with a legacy file system on a legacy media that has to be read by a legacy backup software package… well you get the idea. No urgent need for the data, tasks like that just drift to the bottom of the “to do” pile and rarely get attended to.
Smart IT guys do amazing things. As a test, we ran VAX ands DEC PDP-11 programs on a Victor 9000 in about 1982. The Victor 9000 preceded the IBM PC. VAX/PDP-11 share time was expensive even internally, so a skunk works developed that duplicated almost all of the functionality of our “mainframes” – much slower of course, but at costs that were virtually zero in comparison. We also discovered “distributed” computing and within a few years centralized computing systems disappeared, replaced with centralized storage and backup, and complete searchability. We even had some folks that had figured out how to “computerize” the old AES word processing machines. All on going files were moved to new platforms as they were implemented so data access was continuous.
Victor 9000’s, AES, PDP-11 and VAX machines are still available today and there are services that will “try” to restore 30 year old data. It is amazing what actually can be recovered; including written over information. I am no computer expert, but as a manager in an engineering company, I had several “opportunities” to recover old data for clients. Often as a “business development” exercise.
The requested information belongs to the people who paid for it: the American taxpaying public.
There would be zero impact on the original information if it was simply copied electroniocally, and handed over to those requesting it. What’s wrong with that??
Instead, they are stonewalling. All NOAA needs to do is make electronic copies of what they have. Simples, no? Send DVD’s of the info; ALL of it, and then some.
Because to an outside observer seeing their billing of hundreds of thousands of dollars on the one hand for things they can provide, and on the other hand, telling folks it’s too much trouble, the Occam’s Razor explanation seems to be that they are hiding plenty. You can bet if Obama or Harry Reid or Pelsoi asked for the info, it would be promptly provided, and at no cost.
NOAA should just make copies, dump it on Goddard and Clibze, and walk away…
But, NO-O-O-O-O-O-OOO…
Why not?
that is not what NOAA in the reply says he requested, so either NOAA is lying or Steve “forgot” to mention all the other stuff he requested.
You’re jumping to conclusions without knowing what the FOIA request was. Kent Clizbe gives some details in the comments (March 24, 2015 at 9:33 pm). The request was much broader than Goddard says in his blog post. The NOAA response is understandable, reasonable and polite. They suggest a narrower request might be possible, rather than laughing in Goddard’s general direction for his monumentally pretentious letter, which I believe is the more appropriate response. We do ourselves no favors when we do ridiculous things and expect others to simply accommodate us.
davidmhoffer, the NOAA FTP server always contains the most current version, which is computed every night. However, if you want an older version of GHCNv3 you just have to send an email. No need for a FOIA request for that.
The problem of this FOIA request is that the time is not limited and that thus also correspondence is asked from a time that everything was done on paper. That is a lot of work.
Just a month ago this blog was all up in arms because of the FOIA harassment of 7 of your friends by Democrat Raul M. Grijalva. Wouldn’t it be a good idea if all sides stop their FOIA harassment? This is not how science is done. This is targeting scientists for their politically inconvenient results.
@V. Venema,
It is a completely different situation, which you would know if you had followed it. Grijalva picked only his political enemies to harass. G&C are simply asking the custodians for scientific data. Big difference.
dbstealey,
https://stevengoddard.wordpress.com/2015/03/20/freedom-isnt-free/#comment-510634
There’s a bit of confusion about what we are asking for.
Wellllll …. that DOES tend to happen when the information isn’t in the head post.
Our goal, instead of endlessly debating the direction of temperature changes, is to get directly to the point–to report on the government, non-government, private, academic, and other individuals who colluded to begin “homogenizing” the data to support the man-caused runaway global warming scenario.
That’s why we’re asking for the communications about the changes to the data.
Let’s review your reply to Victor:
The big difference here is what you say was asked for, and what Kent Clizbe says was asked for. Which you would know had you been following this.
Gates,
What was that all about?
You need to get out more. It’s a nice day, go out and relax. You’re getting fixated on things that don’t matter.
dbstealey,
My mistake. It’s not that you haven’t been paying attention. The issue here is that you simply never learned how to read.
Irony. I’m not the one making noises about suing the gummint for (“only”) weather data available for the price of a simple email. Since you’ve flunked the literacy test, let’s see how you do on the numeracy portion. Do you think a successful lawsuit against the Feds would cost more, or less than $262,000 NOAA says they require to honor the law and fulfil Goddard and Clizbe’s FOIA request? Which would be most cost-effective from the taxpayer’s point of view?
Tell you what. Since I’m a big supporter of government and scientific transparency, and willing to put my money where my mouth is: If Anthony starts a coordinated pledge drive with Goddard and Clizbe to raise the $262,000 for the FOIA to go through, I’ll contribute $500.00 out of my own funds in support. If the drive reaches 50% of that amount, I’ll contribute another $500.00 as a further demonstration of my commitment to the effort. If, after 90 days from the opening bell of the pledge drive, the full $262,000 is not met, all collected funds are to be donated to: http://climatesciencedefensefund.org/
Please run my offer up the ladder. Thanks.
Gates, AKA: Victor Venema’s Nanny says:
Since you’ve flunked the literacy test…
Anyone who reads these posts knows that my grammar, spelling and general literacy is superior to Gates’. That’s why he just elected himself as putative judge. No one else would.
Dream on, Gates, and while you’re at it, get a life.
dbstealey,
I just made you a $1,000 bet, you’re still running your mouth. Logic dictates that you must be dictating, after all. So not only can you not read, nor write, you are exceptionally stupid when it comes to tactics. I called your bluff, you turned tail and folded in a fit of cowardly buffoonery. As usual. But my offer to Anthony still stands. Let’s see if he’s got more spine than you do.
Where is the original letter with the FOIA request? NOAA’s response sounds like much more than just a few data files were requested:
“Very few if any letters, phone logs, memos, and other communications on this subject would be available. Historical internal and external emails are archived, though they are expensive to access and analyze due to unsupported technology.”
Without the original request, it is not possible to judge NOAA’s response.
Go to the linked article on Goddard’s blog. Read comments by kentclizbe on March 24,2015 at 9:33 pm: “e-mails, letters, phone logs, memos and other communications between…” Ridiculous! You have to read it to believe it. Only the most bombastic of Congressmen could rise to that level of hubris.
It’s over at https://stevengoddard.wordpress.com/2015/03/20/freedom-isnt-free/#comment-508739
It merely asks for analyzing pretty much every piece of communication at the NCDC over at least a decade and maybe for the life of the agency.
Oh, it asks for data and software too. At least that’s feasible.
He obviously has no experience using FOIA. “from the beginning of the adjustments through today” is just sloppy.
Brought to you by the “Administtation of Transparency.”
Despicable.
“According to Steve Goddard, NOAA have just demanded a $262,000 administrative fee for zipping up a few raw data files.”
It would be helpful to know what “a few raw data files” means. Are we talking about data files for a local area or for the globe? Is it for one year or multiple years? Perhaps if Steve asked for “all” their raw data, it would save them having to search though it. They could just copy everything to a password protected area on their website and provide Steve access to it.
Goddard isn’t exactly being forthcoming about his original request, to put it politely. Go to the linked article on Goddard’s blog. Read comments by kentclizbe on March 24,2015 at 9:33 pm: “e-mails, letters, phone logs, memos and other communications between…” Yeah, a whole lot more than “a few raw data files”.
yep. That’s the comment I found interesting. The first part of the request is for all communications internal and external, phone logs, memos, etc. regarding adjustments. The 2nd part is a request for some data files. There was no time limit apparent in the request, so what the request boils down to is going back through every single scrap of paper, disks, and data storage looking for discussion a regarding adjustments.
The head post is very misleading.
One of the people I used to work with was a Contracting Officer for the VA. One day, he received a FOIA request. He called the people initiating the FOIA request, and told him he’d send them what they wanted if they dropped the FOIA request. They did, and he sent them the information. He said complying with everything required under FOIA is a royal PITA.
Tell them ‘the check is in the mail”.
I’m guessing “original data” in many federal agencies is so deeply buried under a mountain of revised revisions of that data that lots of heavy machinery would be required to dig it out. That quarter $ mil is probably just staff time required to locate the general vicinity where digging should begin. But first must come a series of staff meetings to reach consensus on a working definition for “original data”, that term being almost certainly unfamiliar to many federal employees involved in any endeavor related to “climate”. Then the real expenses begin.
As a former NOAA employee, there is just no question that NOAA is a disaster zone…filled with politically correct narcissists especially in the management and administrative ranks. I was forced to file several complaints against NOAA management over a 20 year period, some of which were successful, and all I can say is I’m so thankful I’m outta there. Several other of my former colleagues feel the same, so those in charge can’t just say it was me that was the bad guy, the cancer, etc. Many of the field people do good work and are conscientious, but the management is completely in another realm, and I think that they think they are in a higher and much-deserved elevated realm. NOAA management and administration run roughshod over well-meaning employees who are trying hard to do good jobs, but at the same time these subordinate employees are trying not only to keep their jobs, but to get promotions; they are completely at the mercy of management, and also of complicit employee unions. NWS, NOAA, and Department of Commerce management need to be turned out and completely overhauled. But, of course, that applies to most of government leadership as well. A pity.
I’ve seen this somewhere before.
Ah, yes.
http://www.nytimes.com/2014/04/28/opinion/verbatim-what-is-a-photocopier.html?_r=0
Jeff Cagle,
Good one. With no judge present, the witness can play those games.
cough Hillary Clinton’s emails cough.
1.9 years x 2080 hrs/years x $37/hr x 1.16 overhead = $169,620. $77,000 for a file clerk? No wonder they can’t figure out what the temperature is.
There’s a post on Goddard’s blog that shows the request, including:
For the NOAA/National Climactic Data Center: Please provide Internal and external e-mails, letters, phone logs, memos, and other communications, from, to, and between: government employees, external consultants, experts, advisors, or other parties regarding the rationale, methodology, and other issues concerning adjustments/homogenization or other changes to both the US and global temperature record data, from the beginning of the adjustments through today.
it is complete misrepresentation to say these are “a few computer files”.
It’s a fishing expedition. No wonder it is so labor intensive and expensive
Missed that, thanks for pointing it out.
If that’s what he asked for, it would be very hard to produce.
That’s a lot different than what he said in his blog post.
Kent Clizbe and I have been working for almost a year to get them to release their published monthly temperature data over the past couple of decades, which they overwrite in place in order to hide their ongoing manipulations.
If what he said above is what he is ACTUALLY after, then the request needs to be rewritten.
just saw this – yup that’s a different ball game, although I appreciate what they’re after . . .
David Sanger: I tried to verify your claim. I searched Goddard’s blog for FOIA, NOAA, Clizbe, and emails. I can’t seem to find the post you’re referring to.
I think you’re just talking smack. Put up or shut up. Provide a link to the post.
Not that hard to find
https://stevengoddard.wordpress.com/2015/03/20/freedom-isnt-free/#comment-508739
David Sanger,
The post immediately below that one makes the central point:
That should all be publicly available in a transparent government.
What’s the problem? If NOAA is halfway competently organized and supervised, it would surely take a low-level employee less than a day to produce everything requested to comply with the law.
You make it sound onerous, but with everything electronically recorded and backed up these days, the actual ‘work’ involved amounts to some browsing for files, and a few clicks of a mouse.
Contrast that with the multi-$millions funneled through NOAA every month. Do you think a little transparency is such a difficult thing?
NOAA is treating Clibze and Goddard exactly like they are ememies, instead of fellow taxpaying citizens. Someone is wrong here. Want to take a guess as to which one it is?
Louis couldn’t even find the source David quoted in few minutes yet dbstealy thinks NOAA should be able to comply with that lengthy request in a few minutes and email it.
This stuff just cracks me up some days.
David thanks for putting up, sure we will both get flamed for using logic and being critically minded.
For clarity ” This FOI request could be done literally in a few minutes and emailed” dbstealy March 27 3:22 PM. I am curious what the search terms dbstealy thanks would cover this and still be able to be filtered for personal info in a year no less a day.
David Sanger: I stand corrected. My apologies.
ironargonaut: you read this blog enough to know that alarmist trolls show up and make bizarre and often false claims. I asked David Sanger to put up or shut up, and to his credit, he did. I didn’t flame him, I challenged him (although I would have flamed him as a troll if he hadn’t).
Most readers of WUWT thrive on logic and critical thinking. As long as you continue to display both, you won’t get flamed here.
Try a little harder, Louis. Go to the linked article on Goddard’s blog. Read comments by kentclizbe on March 24,2015 at 9:33 pm: “e-mails, letters, phone logs, memos and other communications between…”
Louis, no offense intended towards you. Put up or shut is a perfectly acceptable request in most cases. I in fact stated Goddard should have done the same. I was merely illustrating that searching for even specific info is not always as easy as many claim. I know at my work I couldn’t search the companies entire database, I don’t have the correct permissions, I doubt any single person does. Even finding what a policy is on the company intranet is a hair pulling experience some days. David had a specific quote so asking for the link is OK and you apologized for the smack comment, which since you did try to find the quote before making the comment it was not over the top.
NOAA has finally lost all my respect. That cost is way out of line with the true cost, and is meant only to quash the FOIA. Isn’t that illegal?