Over at Climate Audit, Steve reports on the Update for the FOI for the Wahl Attachments
He’s wondering about the use of that mailserver and why there are inconsistencies, for example:
I have a quick question for the technically-inclined about backup protocols. I had asked UEA the following question:
4. You stated that the earliest backup of Briffa’s computer that the university located was on August 2, 2009. I must confess to being completely astonished at this information, particularly since the Climategate dossier included Briffa emails from 2006 that were said to have been deleted.
To provide reassurance on this point, can you explain whether this late date of earliest backup also applied to other CRU computers e.g. it is my understanding that CRUBACK3 contained backups of four of Phil Jones’ computers, with a total of 22 individual backups. Did any of these backups date prior to July 2009? What was this earliest date? If there were earlier backups for other computers, why was the earliest backup of Briffa’s computer so late? Is there perhaps another machine attributable to Briffa that needs to be searched?
UEA replied as follows:
It is right to say that the earliest backup that is held for Professor Briffa’s work PC is the 2 August 2009 backup. However, that is not to say that that backup does not store emails dating back to a period before 2 August 2009. It is merely to say that there are no earlier backups. UEA’s position is that the 2 August 2009 backup would have included copies of all emails and attachments stored on Professor Briffa’s PC as at 2 August 2009 and this could easily have included documents and emails dating back to 2005/2006. You should in any event note that the backup server had an automated function that operated so as to remove older backups on a rolling basis. It is possible that the hacker who obtained and disclosed the emails to which you refer had access to the server for a number of months and that he or she obtained the emails from a backup that is no longer on the server.
…
But some aspects of the backup don’t make obvious sense to me. (They appear to have used BackupPC). Is this common practice: “the backup server had an automated function that operated so as to remove older backups on a rolling basis”. Wouldn’t it be standard practice to periodically preserve some of the older backups?
I note that the police report indicated that access to the CRU backup was not established until September 2009 so that the presence of emails in the CLimategate dossier that cannot be located on the CRUBACK3 server would require a different explanation than the one proffered here by the UEA.
==============================================================
I left this comment at CA:
Backups of operating system active drives typically use rolling backups…because why would you need a backup from 3 years prior if your intent is simply to recover the operational state of the machine?
In my server room we keep current backups for operational recovery, but not old backups unless that old backup has some particular configuration of value, like only running on specific older hardware that me may have to revert to.
For a mailserver, one labeled CRUBACK3, the question then becomes, what is the purpose of that server?
1. Is it a server that acts as a failover for the main mail server?
…or…
2. Is it an archiving server?
If the latter, then there would be absolutely no reason to use a rolling backup, and in fact it would be contrary to the archival mission. The fact that that same server had emails on it from 2006 suggests its mission was archival.
Archival servers typically have removable storage, so that you can put years of data/correspondence on the shelf. The FOI request may be too narrow in stating that the specific server be searched. I would restate it to include removable storage, including media such as: magnetic tape, DVD’s, CD ROM’s, removable hard drives, and Network attached storage drives that were used on CRUBACk3.
You might also ask what happened to CRUBACK1 and CRUBACK2 servers.
==================================================================
I f any readers have anything valuable to add, leave a comment please.
Back in the Dark Ages (1992), I made the mistake of demonstrating to my boss I knew what DOS stood for. For my sins I was appointed the network manager (Novell 2.15; yes, that old). I had an automated tape backup routine that was incremental, Monday through Thursday, Friday was a complete backup. The complete backups were Week 1, Week 2, and Week 3. The fourth weekly backup was Month 1. At the end of the next cycle, Month 1 came home with me and sat in my desk there, Month 2 stayed on site. The following cycle, Month 2 came home and Month 1 went back in the lineup to be recycled. I never viewed the scheme as anything but a backup in case something went sidways on the server, or someone had a serious “oops!” and deleted the wrong file.
Sounds like the file system that was backed up in 2009 contained email archives dating back some time
@ur momisugly John Whitman
@ur momisugly rgbatduke
John, on reflection, I find rgbatduke’s highly persuasive, yours less so. I am reminded that the creation of the fundamental law of the land (U.S.) took place in utter secrecy. The founding fathers understood that free and uninhibited exchange could not take place under the whithering glare of political scrutiny. Once they had hammered out the finished product it could be vigorously debated in the coffee houses, newspapers and legislatures. Would it have informed the public discourse to find that, say, James Wilson of Pennsylvania thought William Patterson of New Jersey a scoundrel and buffoon (as an example only, not suggesting this was the case) during the Conventions’s deliberations?
On the other hand, it strikes me as more than a little valuable to preserve the work product after the fact for any government grant or government funded activity. This would correspond to the various diaries and notes kept by the Convention participants and published after the matters were settled. For e-mails pertaining directly to the work, it is the public’s property. How hard can this be? In my own work life, every project I work on has it’s own Outlook folder. If I have something nasty to say about someone, it doesn’t go in a project-related folder. At the end of the project I usually print the entire folder to a PDF document and store it with the other electronic copies of the project’s supporting documents. Then the folder gets archived on our server. Done and done.
In many private sector businesses, the backup policy may be customized to meet the regulatory policies that apply. It is not uncommon for all e-mails to be saved for company “insiders” (as defined by the SEC), but to delete e-mail after one-year for all other employees.
D. J. Hawkins says:
October 7, 2012 at 2:17 pm
@ur momisugly John Whitman
@ur momisugly rgbatduke
John, on reflection, I find rgbatduke’s highly persuasive, yours less so.
– – – – – – – –
D. J. Hawkins,
Thank you for your comment.
NOTE: I have the greatest respect for rgb’s scientific content comments at WUWT. I am a fan from his very first comment at WUWT. : )
Since I am a retired engineer and have not ever taken government money to do research, I am an outsider looking in to the gov’t use of money in scientific research in climate.
rgb, I appears to me, has a much more precise insider view since he appears to have performed government funded research, but apparently not in climate.
I expect him to be more persuasive therefore. I would be surprised if I was more persuasive in that regard. : )
I, as a non-scientist (like some of the public), am very negative to any suggestion that we must trust climate scientists ‘in-process’ with public money. I am absolutely against it. I will trust only after I am assured that everything, including ‘in-process’ is completely done with the highest intellectual integrity on the part of the scientist receiving government funding. No ‘a priori’ trust.
John
rgbatduke says:
October 7, 2012 at 10:53 am:
“There are some cures that are worse than the disease, and the current tendency to wield the FOIA as a political weapon or instrument of a kind of harassment is one of them.”
========================================================
I have never heard of such a “tendency”. You are the first one I hear to claim that such a tendency exists. At the moment I doubt that you or anyone else scientifically studied a sufficient number of FOIA requests and can prove your claim.
As far as understand, FOIA requests concerning climate have a purpose to expose climate liars.
I want to second what people are saying here. Email is backed up, but isn’t usually archived. The backups will contain a hodge-podge of old and new stuff – whatever random people happen to keep in their inboxes. Some people delete everything right away, and others hoard everything until you tell them they’re using too much server space. But once an email message has been deleted, you only have a limited amount of time (90 days, or 1 year, or whatever the backup overwrite policy is) before the message disappears from all backups.
Most people will save a few very old emails – if they’re important or personally meaningful – so you should see a few old messages in the backups. But just because there are a few old messages doesn’t mean all old messages will be there.
If you’re looking for a particular message, your best bet is to find it in the backup of an account of one of the email horders. Another possible source is the “outbox” of the person who sent the email.
Large disks are now very cheap. It was not always so. Going back a few years, backup servers were generally struggling for space so it was common to delete all but a small number of old backups.
For simple data recovery backups, you frequently kept just two or three backups. The more paranoid (a.k.a. sensible) would use some sort of “grandfathering” algorithm.
For permanent(-ish) archives, they were generally stored on some form of tape, or maybe a DVD or (recently) a Blue Ray disk.
For people without an (implemented) retention policy it would be common to keep email and working copies of files in just a small number of backups. Important files (data, work product, etc.) they might be archived by project at certain milestones (e.g. project release).
Obviously people implementing a retention policy would need a longer-term solution>
D. J. Hawkins says:
October 7, 2012 at 2:17 pm
@ur momisugly John Whitman
@ur momisugly rgbatduke
John, on reflection, I find rgbatduke’s highly persuasive, yours less so. I am reminded that the creation of the fundamental law of the land (U.S.) took place in utter secrecy. The founding fathers understood that free and uninhibited exchange could not take place under the whithering glare of political scrutiny.
It is extremely simple, if you do not wish to have the ‘withering glare of political scrutiny’ – then do not take government money. The government contract will state that everything that you do while claiming that government money belongs to the government and everything that you purchase (and sometimes use) belongs to the government. If you do not like those terms – do not take the research funding or job – or in some way alter the contract to remove the ‘withering glare of scrutiny’. To put it another way – the government needs you ‘to show your working’ as well as the results.
This is absolutely no different from a 7/11 checkout being videoed if you don’t want to be continually videoed then you can’t have the job at 7/11. It is part of the rules of the contract. If you want privacy in how you work on the research then you do not take _any_ government funding and you definitely do _not_ use any government funded systems to play those games on. Your reductio ad absurdum of continually videoing discussions is invalid. What is being discussed here is the conduct of research and discussions – on email – where the terms of the job state that there is ‘no expectation of privacy’, as mine do in my commercial job and as the woman at the 7/11 has in her terms of employment.
It may be unfair to those who work correctly – but if the sunshine is removed then some people are tempted to, and do, make use of the darkness. The person who decides where that sunshine falls is the funding agency – in the 7/11 case it is the cash till area and the fuel pumps, in the government research funding case email is covered.
.
Um; which fundamental law was that? If you mean the Constitution of the United States, you are incorrect about “utter secrecy”. Only the deliberations themselves were to considered secret and even this ‘secrecy’ meaning is loose. Remember, 55 delegates reported back to their delegations, state legislatures and others. Thomas Jefferson, who at the time was in France as United States minister kept in the loop via James Madison and was able to add his thinking/influence. This is before taking into account that the delegates often stayed at the same taverns/rooming houses. Secret wasn’t so secret, but what was kept quiet was any acrimony from badly reported ‘he said they said’ getting published and poisoning discussions. Basically, anyone could trundle on down and meet with one of their delegates and get the lowdown, just not the ugly arguments from the floor of the convention.
On ‘privacy’ and other outhouse stuff; if you don’t want it read, don’t write it!. The whole idea of writing something that clearly expresses your thoughts, is that you thought about it before you wrote it. If gutter mouth, envy, burn your bridges, hate, pompous, ego, and many other nasty personal traits are how you want to be remembered, then by all means communicate that way (sound like some infamous people we discuss?). Don’t expect any sympathy from me if it goes public, ever.
Having spent my career in US Government, mostly in IT, I do disagree with you rgb. A grant recipient takes Federal money and is expected to operate within ALL legal boundaries. Whether, EEO, FOIA, lack of prejudice, and so on you are considered a Federal representative. Claims of sex/gender/race discrimination against you and you will not be protected by claiming you are independent of the Federal government or that you don’t keep emails/records whatever… As the Manniacal one and UVA have discovered, grant recipients are NOT protected from FOIA requests for emails.
On the matter of a deliverable; you took money from the Federal government and left the government with some expectations of what they will get for their money. If you fall off of a cliff, are the Feds expected to ‘write off’ that money as a bad investment? Shame, the Feds don’t agree. When accepting a Federal (and most state) grants, the grantee agrees to comply with all grant conditions. http://www.whitehouse.gov/omb/grants_circulars/.
Steve:
Dale gives some great advice (amongst others who cover the backup/archive detail well) about seeking the server’s journal. If you want to get pissy technical, every server those emails passed through may have a copy; that store and forward notion goes a little astray when IT guys get really going on CYA. It’s one thing to not be able to restore some cubicle worker’s damaged emails, it is entirely different when the top boss in your building is missing his great auntie’s address in an email from years ago. When hard drives were expensive, tape was cheap coparatively. Tape is slow and very linear. Now hard drives are cheap and getting cheaper all the time, plus they’re easily searched. I used to have write an ROI (return on investment) to buy drives over a hundred Mb; now Tb RAIDs can be had easily even on small budgets. Even back in the early 2000s, it was very easy to punch out archivals. If someone wanted to they could use DVDs at a personal level and keep multiple whole copies on the same DVD and then keep the DVD in storage. As I type this, I have an external HD I use to copy (much easier than a backup) my work to. I also have a 32Gb flashdrive I use to keep ‘fresher’ copies of my work. Back in 2007, the flashdrives were smaller but still effective as were external HDs. And this is my personal laptop.
It may be a pain to re-install software, but it is murder to redo previous work, especially if that work is older stuff.
Other that that, I think you’re missing the concept. When you submit an FOIA request, you should not have to state all of the possibilities where information may reside. That is their job! When you’re left unsatisfied and dissatisfied, escalate. Lame excuses from CRU/UEA are irresponsible and escalating to the next level are your only true course. I would add as documentation that UEA and CRU are trying to make FOIA requests a joke as you are supposed to identify technical options where they can look.
Stick to the facts. Several have posted the retention policies and standards links. State the relevant policy (not the specific words, just the policy) and that you have no recourse since UEA/CRU (different every time just like them) because they’re uncooperative and perhaps are avoiding due diligence. Remember, when you made your original request all relevant items of your request are required to be maintained, no matter what the original expiration date was.
As an aside; I keep remembering that one ‘team member’ said he kept all of his emails on a thumb drive and I wonder if that might’ve been ‘advice’ circulated by the team at some time.
I’ll help but I get bored of this as I do it 8 hours a day. If you get confused can you ask close ended questions, cheers.
Oh and don’t think just because you hit delete the e-mail is deleted, although this is more technical and would not have anything to do with FOI.
Right I’m off to check the backups ran over the weekend.
Are you suggesting that in principle no investigation should be allowed by media, private citizen or enforcement agencies is possible wrt the in-process activities of a government funded scientific researcher? During the time they receive grant money and the time they produce the research product a publically funded research scientist, are you suggesting there should be exemption from investigation by private citizens, media and enforcement agencies? They get a free pass?
, and I came to believe that it was this value (it fits to between four and five significant figures). This was naturally very exciting — it suggested that a simple/tractable solution might exist where this result could be proven!
— in the fifth and sixth significant digit — within my own error bars. The seductive pretty story sadly gave way to actual computations that eventually falsified it.
.
for the critical Heisenberg ferromagnet, well, all I can say is get a life. Go away. Leave me and my email spool alone.
Not at all. Here’s what one can investigate. Is researcher A conducting the contracted research, spending a reasonable fraction of their time on it? If researcher A alleged that they spent salary money hiring postdoc B and grad student C, did they indeed do so? When postdoc B spent valuable grant moneys on a trip that was claimed to be to a workshop relevant to the ongoing research, did they in fact do so or did they instead use the money to purchase cocaine and hookers? Finally, when the contracted work is eventually published, did the author(s) of that work actually do the work alleged to have been done in the publications and obtain the results those publications assert? In the event that the work was done for an agency (such as NASA) that requires data and methods to be made available post-publication, are they in fact both available?
None of this implies carte blanche permission to open their underwear drawer and paw through it at home, or their email spool to paw through it at work. All of the first part of it is satisfied by what we in the business call an “audit” and is a routine part of actually doing work for a granting agency that is administered by e.g. a University. Every year I fill in a form (even though I’m not currently supported by any grants) certifying my “effort” distribution among activities such as teaching, doing research, and non-University work such as consulting or entrepreneurial stuff. This certification must be co-signed by the department administrative staff and correspond within reason with their observations and resolve any complaints of me doing or not doing my job. This goes to sponsored programs, which is responsible to all of the granting agencies supporting work done at Duke. At any time, a granting agency or sponsored program can and cheerfully will conduct a far more detailed audit if there is any reason to suspect abuse.
Nobody gets to sit in judgment of the work itself except my colleagues — which includes everybody in the whole world, but with weight given according to expertise and corresponding effort. Nothing constrains my results, which may or may not be what I expected them to be when writing the proposal, politically popular, or what everybody else gets when they study the same problem. There is no recourse for the granting agency to recover any part of the contracted funds should I or my work prove to be incompetent or for that matter mildly wasteful of their money at a level short of active embezzlement — their only recourse is to not fund me in the future if I waste their money now.
The single exception to this are as follows: If, after my work is published, it proves to be egregiously non-reproducible in a way that suggests that I faked my results and the work itself is broadly challenged as not, in fact, being an objective presentation of work that I actually did, a granting agency and sponsored programs can delve deeper and investigate the possibility that actual fraud was committed. They don’t even need a subpoena — I give them implicit permission to do this when I apply for and accept the grant. At that time I do indeed have to open up my “books” to them, show them the details of my intermediate work, show them the actual data if I’m an experimentalist (I personally am not) and/or work from data, show them my methods or code or algebra if I’m a theorist (which I personally am).
Only in the event that my work proves to be academically dishonest — to put it bluntly, I either faked the data or claimed to have done work I did not actually do — does my employer (the University) and the granting agency have to pursue civil remedies and of course the possibility of criminal prosecution also looms if my actions shade over towards embezzlement rather than mere sloth or academic dishonesty. The most likely outcome in this case would be me getting fired, for cause, end of career in academia on the spot, the University absorbing the loss (they were responsible for my hiring in the first place) and reimbursing the granting agency as a condition for continuing to receive grant money in the future. It is possible, but moderately unlikely, that the University would pursue a civil suit against me to recover some or all of this loss.
Sadly, the University is the home of a poster child for scientific misconduct, Anil Potti, see:
http://en.wikipedia.org/wiki/Scientific_misconduct#United_States
Crime: Faking data extensively, over years, to assert egregious claims in the general area of cancer research IIRC. Revealed by the utter failure of his work to be reproducible. Exposed and verified by interviews with his postdocs and research fellows, by delving into the details of his supposed “patient data”, by interviewing the patients. Punishment (so far): systematic withdrawal of all of his papers. Public humiliation. Termination of employment, with a near guarantee that his future career will be as a used car salesman or working in sanitary engineering. Exposure to lawsuits seeking to recover damages on the part of both the University and any patients who claim to have been hurt by the “therapies” that he claimed were fruitful on the basis of faked data. Possible criminal prosecution, although it will never happen — it doesn’t need to, he’s finished.
Note well that the issue isn’t whether or not his work was mistaken — there is a ton of stuff published in science that turns out to be wrong. It doesn’t have anything to do with whether or not he believed his own claims — I rather expect that he did, so strongly that he didn’t think he needed to actually verify them or to the point where he was willing to bend them by a few percentage points to make them statistically significant instead of insignificant. At issue was scientific misconduct — deliberately faking data, bending it by those percentage points, and publishing something that as a consequence was egregiously false.
Note well that at no time is the implicit right to investigate cases of possible scientific misconduct for cause by the granting agencies and the adminstrators of the grants to be construed as a carte blanche invitation to the general public to embark on a witch hunt at will anytime the results being published by some researcher don’t agree with their own personal and political biases and beliefs. Nor does it extend even to the granting agency and/or administrator complete freedom to open and read all correspondence electronic or otherwise.
Finally, only a tiny handful of even the climategate emails cross the line and provide evidence of “edgy” behavior, behavior shading on scientific misconduct. Incompetence yes. An egregious tendency towards confirmation bias absolutely. An entirely unhealthy superman save-the-world complex sure. But none of these are scientific misconduct; they are scientific incompetence.
To be explicit, Michael Mann’s promotion of bristlecone pine trees from a tiny fraction of the surface of the Earth into the dominant component in a flawed PCA method that produced only hockey stick shapes, and the subsequent promotion of his hockey stick into “the” proxy record of global temperature by political extremists is not scientific misconduct on Mann’s part, it is scientific incompetence on his part and political misconduct on the part of others. If he had faked the data it would have been scientific misconduct. Using handwritten PCA code tuned to produce a result that he expected to find, however, was just incompetent because if he were competent he would have doubted his own work and taken steps to verify that his own results were robust instead of waiting for M&M to do it for him. The one place where his activities edged over into scientific misconduct was his reluctance to release his data and code when M&M requested it, but back then granting agencies were a lot more lax with disclosure rules if they had them at all!
This reluctance was rather understandable from a human point of view. Even if you publish results obtained in the best of faith — and I have little doubt that Mann’s results were good-faith results and do not think to this very day that he doubts that they are correct — the public demonstration that they are crap because you made a number of mistakes and failed to exercise sufficient care checking them is rather bad for a career. To some extent we create this monster by supporting a research system that despises negative results and rewards positive ones. As Feynman points out in the cargo cult lecture, negative results are often more valuable (and honest!) than positive ones but nobody gets tenure and future grant money for negative results. Why should we then be surprised when young researchers with their entire future on the line drag their heels and seek to at least postpone the day of reckoning when they make a mistake that could cost them their entire future career?
The structure of the ivory tower itself created Potti, created Mann, and continues to create piles of poor results every day. Incompetent results. Results tainted with confirmation bias and cherrypicking of data, but still honest results in the sense that the researcher in question honestly believes their conclusions to be true and can rationalize their arguments for inclusion of the data in some way. The whole point of the scientific process is to correct for this nearly inevitable tendency by exposing it to the process of verification and falsification. It is, or should be, all right to make a mistake, as long as you don’t lie and create false data to support it.
I am far from immune to this myself. I spent a decade or so investigating the critical behavior of the classical Heisenberg ferromagnet via Monte Carlo computations. This is a model that looks like it “should” be analytically solvable, but in fact it has so far proven intractable (only one major magnetic model — the Ising ferromagnet — has proven solvable). In the course of my simulations I discovered that the dimensionless critical temperature/coupling of the model was very, very close to
If one had opened up my email records to scrutiny, I’m quite certain that they would have contained all sorts of discussions of this that I had with colleagues I was working with. I was quite convinced of it — too many digits to be a coincidence, surely, I thought. However other researchers got numbers that were very slightly different from mine — still within my error bars, but they claimed smaller error bars. I doubted their error bars were as small as they claimed they were, they agreed that my results were correct as far as they went but that I needed to make my error smaller.
They, as it turns out, were quite correct. Eventually I accumulated enough data to positively reject my own hypothesis of a “nice” analytic form for
At no time did I participate in scientific misconduct, mind you. I didn’t fake my results. I didn’t misreport them. I presented my hypothesis as a hypothesis, not a proven fact (one would have to have the derivation itself to report it as a proven fact). But I damn skippy sure believed, throughout a couple of years of work, that the result was analytic, and presented results that were very slightly erroneous by assuming (as a stated assumption) that this was
This is only one of several pretty stories I’ve been misled by over the years. The formation of a scientific hypothesis is the formation of a pretty story. It is by its nature seductive, the observation of a pattern and a proposed explanation for that pattern. It isn’t always easy to do work to accept or reject it, either — one’s results tend to reflect the bias of assumptions made in obtaining them, the bete noire of scientific research and the reason many eyes and hands looking at things and challenging one another are good.
In the end, there are checks and balances in the scientific process a plenty. Treating the email spools of every scientist in the world as open to public scrutiny is not, and should not, be one of them, not without far more cause than what is behind a typical FOI request. If someone wants my code — they are welcome to it. If someone wants my data (well, I personally don’t have any data, theorist) sure, why not. If somebody wants to read through my private emails, many of them discussing students with special needs and other things that would violate privacy laws if exposed, just so they can see whether or not there was a time I believed something that turned out to be incorrect, such as the value of
That’s a line that can and should only be crossed armed with the equivalent of a warrant issued on the basis of actual hard evidence of a committed crime, precisely like my real underwear drawer in my own house, precisely like my real, federally protected paper mail. Email and paper mail should absolutely enjoy the same protections under law, and those protections should completely block and actively punish its exposure by third parties armed with anything less than a court order, or even my employer (who technically “own” the spool file itself) acting as an agent of granting agencies that support me without just cause and due process.
I repeat: Without a reasonable expectation of privacy, frank discussions and speculations essential to the scientific process cannot occur. Freedom of scientific discourse includes the freedom to be wrong, the freedom to be misled, the freedom to pursue the wrong path to a fault, to stubbornly stay on it even in the teeth of contrary evidence. The system has checks and balances galore built into it, and is loose enough to tolerate the iconoclast in the short and intermediate run while still ending up quite rigorous in the long run. FOI is often being used as a weapon and tool for harassment in a political war, not a scientific war, and is just as repugnant to me as the gatekeeping and other over-the-edge activities revealed in climategate. It is one thing to ask for data and methods, quite another to ask for open access to email spools so one can see if there is a “conspiracy” of some sort that can be discovered there.
Whatever happened to ordinary civility in scientific discourse?
rgb
Anthony/mods;
I suggest that rgbatduke’s post at 07:56 be elevated to guest post.
———-
rgb
Again, appreciate your continued dialog in this important subject of the role of open transparent professionalism by climate scientists in the performance of the entire process of executing a US government contract for climate research.
We are from different philosophical foundations, apparently.
Private is private, but conduct of public research is inherently not private in any common sense way.
I see that a climate scientist receiving US government funding is not prevented from fully and creatively conducting activities via the scientific method especially if there is a clear public expectation of ‘in-process’ openness and transparency in conduct of government funded climate research. I really see no problem for said scientist. I see no dilemma, whereas you have expressed, at length, a critically severe concern over ‘privacy’ and ‘suppressing’ free thought’.
What scientist fears being seen, during the entire process of grant through final research product, as a highly professional and open scientist in public research? I see no suppression of free thought and no intimidation. I take that as civility in the highest possibly mode. Where has civility gone you ask? That is where civility should be.
In my long professional experience in corporate America, there was never the expectation of any ‘in-process’ privacy from the owning corporation’s eyes; it did not reduce creativity nor dampen open discussion of even our failed attempts at achieving original goals . . . on the contrary the openness stimulated new ideas and solutions. Likewise, that same expectation of no privacy from owning government (and public) eyes during the conduct of government funded research is not inherently inimical to creativity, false starts and failure to meet original goals; in fact it can and should enhance the creative ideation process and scientific objectives.
As to fundamental philosophical differences; I live in a benignly indifferent universe.
John
rgbatduke says:
October 8, 2012 at 7:56 am:
“That’s a line that can and should only be crossed armed with the equivalent of a warrant issued on the basis of actual hard evidence of a committed crime, precisely like my real underwear drawer in my own house, precisely like my real, federally protected paper mail.”
=======================================================
Look, talking about your real underwear, I’ll give you one example: going through Customs. They can touch your real underwear without a warrant. They can confiscate or arrest it under circumstances. You can not reasonably say they should not be allowed to go through your luggage, because you might have some real underwear there.
Back to the taxpayers funded climate liars, it looks like there are quite a few around and they do not just perpetrate their “research” alone, but in teams. In this context their emails are relevant and no way are their taxpayers funded working emails private. And then there is a law allowing taxpayers to look into that stuff. This is the way the liars can be caught and I hope they will.
I will not claim expertise on the precise manner in which the deliberations were held. I’ve skimmed some of the commentary by Madison and it seems clear you could NOT siddle up to a participant and “get the lowdown”, and they were NOT reporting back to their respective legislatures, particulary since many of the said legislatures would have recalled their delegations if they’d gotten wind of what they were up to. They weren’t supposed to be writing a new constitution, just fixing the Articles of Confederation.
On the broader issue, I fail to see the utility of what seems to be the general demand of “real time” open public monitoring of every e-mail under the sun possibly related to a government project. Too frequently it seem commenters here and at other blogs conflate “wrong” with “evil”. Therefore, the only result I can see is the premature and misguided pillorying of investigators as they go about the very difficult task of trying to understand the universe, or at least a small part of it.
Unless there is evidence of E. coli contamination, I don’t much care to wade through every particular of the sausage making process. And those who do care to, should have compelling reasons related to the public good, not just idle curiosity or a desire to tar a researcher’s efforts with the brush of a few ill-considered personal or political remarks dropped among the pedestrian messages regarding the real work at hand. This is simple voyeurism and serves no legitimate interest.
Steve McIntyre came to Mann’s defense (re the FOIA for his e-mails) on these grounds. But maybe there’s a middle way. For instance, maybe the judge (or an expert appointed by him) could look through the e-mails for evidence of skullduggery, if there were reason for suspicion, even without probable cause.
Look, you should just assume that they are trying to hide scientific misconduct from you. Actually getting proof of this is just a time consuming distraction. Just assume it.
Aye in some respects, but this discussion turns the FOIA back into a very real need. Madison, was in frequent contact with Thomas Jefferson. And yes, in Colonial America, it was possible to sit down at a table, buy the delegate a beer (or some of George Washington’s rye whiskey) and chat. No, the delgate would NOT tell you that they were overturning the present government (Articles of Confederation). However, if the delegate thought you could add something, he might discuss any number of ideas that were floating around the Convention.
If you think a peer review is difficult, imagine the result if 13 delegations returned to their states and suddenly dropped the bomb of an idea of throwing out the present government? As it was, only 39 of the delegates signed the constitution for presentation to the present Congress. Add to that sudden bombshell the idea that the present Articles of Confederation Congress did not have the authority to create a new national government. That was a State choice and could have only happened if the States themselves were leaning towards trashing the Articles. of Confederation.
There were a number of plans submitted to the Convention:
•The Annapolis Conference
http://www.usconstitution.net/consttop_ccon.html#annapolis
•Madison and the Virginia Plan
James Madison submitted a plan that encompassed ideas from others. Ideas that had been long thought over and discussed with many people.
•Sherman and the Connecticut (Great) Compromise
•Paterson and the New Jersey Plan
Paterson and other delegates of the smaller states formed a caucus to discuss/rebut Madison’s Virginia plan. Again, many of the ideas Paterson and other small state delegates favored were the results of the, in their day, overwhelming power of the big states and large population centers. Ideas generated back in their state legislatures from dissatisfaction arising from the Articles of Confederation (AoC). Surprisingly Paterson’s group ideas were closest to their original mandate of fixing the AoC.
•Hamilton and the British Plan
Hamilton proposed many features of government that are recognizable in our consitution today, yet he left the Convention early and was frequently outvoted by his fellow New York delegates. His frustration was not evident when the Constitution was announced and sent for ratification; as he very strongly campaigned for the new Constitution.
•Pinckney – step-father of the Constitution
Pinckney is considered a step-father to the Constitution, where Madison is considered a Father. Pinckney himself did not propose radical new ideas, intead he submitted many ideas coalesced from the ideas of others. The Constitutional Convention convened on May 25, 1787. Madison submitted the Virginia plan on May 29, 1787 and Pinckney submitted his plan also on May 29, 1787.
Both Madison’s and Pinckney’s plans had several similarities:
•A bicameral legislature
•The lower house, the House of Delegates, was elected by the people, with proportional representation
•The upper house, the Senate, (Pinckney’s proposal was for equal state representation, Madison’s proposal was for state proportional representation)
•An executive – elected by the legislature
•Ability for veto over bills (Pinckney – by executive council; Madison – executive and judiciary)
•National veto power over any state legislation
•A judiciary was established
For that matter Pinckney had submitted a similar plan to the AoC Congress before the Annapolis Conference.
Perhaps today people think that Madison cribbed ideaas from Pinckney and Hamilton cribbed ideas from Madison and King George…? The truth is that many of these ideas had been floated and discussed openly since before the AoC. With the appalling lack of AoC ability to act for the nation it was more and more apparent that changes were needed.
The colonies and then the states had operated on a central power concept, with that power controlled by the elite, along the lines of Hamilton’s ideas. Yet the end result was for “…Madison’s idea, certainly not an original one, but unique for the new United States, was to recreate the United States under an entirely different form of government – a republican model. In a republic, the people are the ultimate power, and the people transfer that power to representatives….” http://www.usconstitution.net/consttop_ccon.html#madison
The Delaware delegation were sent with explicit instructions to leave the Convention if equal state representation in the legislature was compromised.
So secrecy was kept, in a matter of speaking. No one let on that the Convention was dissolving the AoC or public confusion would be a problem. Especially how the the public over in England, Spain and France would view the situation. Diatribes and pernicious arguments were quashed by not exposing them outside the Convention. Think about how the legislature of New York thinks about Alexander Hamilton suddenly returning home from a Convention where his input is of high value to the state? Can you imagine telling your boss that you left an important meeting, because you felt like it… Oh yeah that’d go over well.
Back to the FOIA, the whole Constitutional Convention was a success, because neither it nor many of the ideas it proposed were a surprise. Many of the ideas were greatly enhanced by open discussion both in the Convention and in the years prior by anyone interested.
Most of our knowledge about the internal discussions of the Convention came from letters and notes of participants after their deaths. Even then, they were strongly flavored by the person’s attitudes who wrote them. I am not aware of any ‘orders’ to keep the notes private (secret) so I’m inclined to believe that the authors considered it ‘ungentlemanly’ to release them. Whatever their personal reasons, their papers eventually came under public scrutiny and this was in a time where fireplaces were commonplace.
Fast forward to today; not only are electronic systems in place, but many of them have multiple fail-safe capabilities. The Delete function at the user level may not mean what they think it means.
rgb stated “…Whatever happened to ordinary civility in scientific discourse?” and I paraphrased that as ordinary civility in discourse. Many of us were raised with the admonition of “If you can’t say anything nice, then don’t say anything.”. Civility in discussion should always be practiced, no matter the medium.
The example provided by the Constitutional Convention is that ideas exposed to open discussion usually are improved by that discussion. Coupled with the fact that publishing a concept or idea establishes ‘proof’ of ownership/authorship. Yeah, the world is full of smart people who can see a rough idea and then form their own expansion of that idea. Normally that’s called progress. Many people believe it’s an invasion of their ‘personal’ idea and they want to protect it from violation. They’re allowed that right and “research that hasn’t been published yet” is protected from FOIA. Published research is an assumption that everything is published sufficient for anyone to replicate the research. Retaining and ‘hiding’ some of the research that went into what is published so replication is impossible is anti-scientific in purpose and principal.
We’re back to, if you don’t want something read by everyone/everybody at some point in time, then do NOT write it. That is civility. When you refuse to release ALL information pertinent to published research; expect people, especially scientists, to question the research. Uncivil communications that support the research are poor reasons to claim privacy. So is feeling motherly about ‘invasive violation of personal space’ by others wanting the background information.
It’s a big world and there are billions of people who live on it. Privacy is relative and fleeting at all times. Civility and social behaviors are necessary in all societies. Hermits and others desiring hermit lifestyles should find their own mountaintops and cease being human if that is their desire. Refusing to share is something many people try to minimize in their children by teaching them to share. Privacy is protected where necessary and open to challenge when it is not.
Steve has legitimately FOIA documentation and some communications; let us help him obtain that information. What is posted above are supposed technical challenges to properly responding to Steve’s FOIA. Does everyone truly believe this important background information no longer exists? If so, what is proper procedure regarding the published research that these documents and communications resulted in? Where is replicability?
Frankly, I have very little sympathy for the Duke professor claiming that FOIA requests are “harassment.” He’s crazy if he wrote anything in an email while at Duke which in any way could lead to anything like a discrimination charge! 😉