In my opinion, this is a testament to Steve McIntyre’s tenacity.
Via the GWPF: At Last, The Right Lesson From Climategate Fiasco
A diverse group of academic research scientists from across the U.S. have written a policy paper which has been published in the journal Science, suggesting that the time has come for all science journals to begin requiring computer source code be made available as a condition of publication. Currently, they say, only three of the top twenty journals do so.
The group argues that because computer programs are now an integral part of research in almost every scientific field, it has become critical that researchers provide the source code for custom written applications in order for work to be peer reviewed or duplicated by other researchers attempting to verify results.
Not providing source code, they say, is now akin to withholding parts of the procedural process, which results in a “black box” approach to science, which is of course, not tolerated in virtually every other area of research in which results are published. It’s difficult to imagine any other realm of scientific research getting such a pass and the fact that code is not published in an open source forum detracts from the credibility of any study upon which it is based. Articles based on computer simulations, for example, such as many of those written about astrophysics or environmental predictions, tend to become meaningless when they are offered without also offering the source code of the simulations on which they are based.
The team acknowledges that many researchers are clearly reticent to reveal code that they feel is amateurish due to computer programming not being their profession and that some code may have commercial value, but suggest that such reasons should no longer be considered sufficient for withholding such code. They suggest that forcing researchers to reveal their code would likely result in cleaner more portable code and that open-source licensing could be made available for proprietary code.
They also point out that many researchers use public funds to conduct their research and suggest that entities that provide such funds should require that source code created as part of any research effort be made public, as is the case with other resource materials.
The group also points out that the use of computer code, both off the shelf and custom written will likely become ever more present in research endeavors, and thus as time passes, it becomes ever more crucial that such code is made available when results are published, otherwise, the very nature of peer review and reproducibility will cease to have meaning in the scientific context.
More information: Shining Light into Black Boxes, Science 13 April 2012: Vol. 336 no. 6078 pp. 159-160 DOI: 10.1126/science.1218263
Abstract
The publication and open exchange of knowledge and material form the backbone of scientific progress and reproducibility and are obligatory for publicly funded research. Despite increasing reliance on computing in every domain of scientific endeavor, the computer source code critical to understanding and evaluating computer programs is commonly withheld, effectively rendering these programs “black boxes” in the research work flow. Exempting from basic publication and disclosure standards such a ubiquitous category of research tool carries substantial negative consequences. Eliminating this disparity will require concerted policy action by funding agencies and journal publishers, as well as changes in the way research institutions receiving public funds manage their intellectual property (IP).
=========================================
More Soylent Green! (April 20, 2012 at 6:08 am) wrote:
“I am having trouble following you here, but I think you’re saying the code is trash and this concern with the code is unproductive.”
Might as well run stories on vomit entrails. This fascination is just plain sick. It will make WUWT look bad to sensible people.
The zealousness to rip through a heaping pile of steaming trash is particularly creepy. Interest in code is fine, but mouth watering over trash? We expect that of creatures that scurry around in the dark at night. Rodents, bugs, …
The community looks to be going off the rails here. I put in my 2 cents in the hopes of helping with sober checks & balances, but in the end if the community goes through with this ugliness, I’ll just step aside.
I’ve NEVER run into a scenario where I needed to reproduce results and could not. Even when others have made mistakes, I’ve always been able to reproduce their results – by figuring out EXACTLY what mistake they made. In fact, I always used that ability to design efficient marking keys when I taught stats & marked thousands upon thousands of exams. I would design marking keys that anticipated (based on intuition & experience) every possible mistake students would make, showed what final result they would arrive at, and had a fair mark ready. I didn’t require my students to show their work; quite the contrary, I required them to BE CONCISE.
I might VOLUNTEER code UNDER FAVORABLE CIRCUMSTANCES, but no one here has any leverage over me, so demands will meet (friendly) defiance. Favorable circumstances are far from existing at present. I don’t have enough time & resources to come even remotely close to meeting MY OWN standards for formal presentation. I can whip off informal research notes relatively fast and then handle precise technical questions informally, but it will be years – maybe decades, maybe never – before I have time to cosmetically & editorially engineer formal aesthetic structure tailored for a general audience. The ONLY reason to meet formal demands is a huge paycheck and pension.
The advantage few are recognizing here is that informal communications are ruthlessly EFFICIENT, whereas formality will consume 99% of a budget IF ONE ALLOWS SUCH INEFFICIENCY. Beware the sink of inefficiency. I would suggest that checks & balances on budget expenditures are not only sensible but also NECESSARY. Waste creates trash. Consuming trash is unhealthy. Suggestion: Be efficient friends.
Operating on the corollary of the Pareto Principle is a SEVERE drain that gives ALMOST NO RETURNS (aside from fluffy cosmetics). In sharp contrast, operating lean & mean on the Pareto Principle (rather than its corollary) one can accomplish A LOT WITH SURPRISINGLY LITTLE.
Is there no one here concerned with engineering efficient operations? Do we have access to some pile of infinite resources that no one is sharing with me? Is that why no one appears to be concerned with conserving precious time & resources by operating tactically & strategically – rather than squandering luxuriously?
Good fun watching this code comedy at least. We all enjoy a good laugh. Nature’s beauty is simple & compact. Her secrets aren’t wastefully encoded in a heaping pile of stinking CAGW code. Why waste time looking for something in a place where it’s guaranteed to not be? Paparazzi-style politics at it’s worst, perhaps.
But have your sickening fun if that’s all you aspire to do with your freedom. Maybe we’ll one day lose our freedom for the simple reason that we wasted it.
All the Best.
jimmi_the_dalek says:
April 19, 2012 at 11:38 pm
Thanks, Jimmi. It seems that you agree that publishing yet another reconstruction would NOT falsify Mann’s paper, which is what I was saying.
You say that if someone posted another paper, Mann would have to “put up or shut up” … why on earth do you think that would put even the slightest pressure on Mann? He was in the catbird seat, he had finagled his “Hockeystick” into the IPCC report and it had become the icon of the movement … why would he say anything?
The part you seem to be overlooking is that Mann is a fraud. He knew before he published the Hockeystick paper that the data didn’t support it, particularly the 15th century data. He knew about (and lied to Congress about) the abysmal failure of his work to pass simple statistical tests.
As a result, there was no way that simply publishing yet another reconstruction would have forced him to do a damn thing. He was not about to reveal that he was a fraud.
And this is the problem with your proposal that code not be required. You assume that the scientists are honest, which marks you as pretty clueless about climate science. Part of the reason I want to see the code required, as I said above in capital letters, is to prevent people like Michael Mann making a mockery of the scientific process by lying and cheating.
So your idea, that someone else publishing yet another reconstruction would somehow force Mann to admit that he is a liar and a fraud, is ridiculous. For example, my friend and co-author Craig Loehle published another reconstruction, and a good one, called A 2000-Year Global Temperature Reconstruction Based on Non-Treering Proxies.
You can read what Mann said about Craig’s reconstruction here … and if after you read that you still think that someone publishing another reconstruction will do anything other than lead Mann to slime the author’s name in any sleazy way he can, you’re not following the story …
w.
Windchaser says:
April 20, 2012 at 11:47 am
Thanks, Windchaser. The problem in climate science is twofold.
First, many of the top people in the field are not statisticians, but they are writing papers full of statistics … and as a result they make errors.
So Michael Mann says quite clearly that he is using principal components analysis.
Unfortunately, he’s made a newbie mistake in the analysis … he didn’t center the matrix first. No way to determine that from the outside.
You are right, the code for this is much, much longer than saying “I’m using principle components analysis” … which is why we program in code rather than English. The code contains all the details, including the errors of which the author is totally unaware.
Second, many of the top people in the field are serial liars who are victims of Noble Cause Corruption. They think nothing of bending the truth to and past the breaking point, because they are Saving The World!!! and are glad to inform you of that.
So Michael Mann says quite clearly that he never calculated the R^2 for his reconstruction … and that’s a total lie.
That’s why we need the code. Because finding the errors is part of science, and in many cases you simply cannot find the errors without the code.
In response, you say:
The issue is replication. With an experiment with beakers and glassware, we can duplicate the experiment.
But with an “experiment” which is nothing but a pile of data and some rules for mathematical transformation of the data, to duplicate it we need two things:
1. The data, and
2. The transformations. Not some english language claim about the transformations, that’s orders of magnitude too vague to be of use.
And that is why people like Michael Mann and Lonnie Thompson and far too many of the un-indicted co-conspirators hide their data and their code … and you keep claiming that hiding those is perfectly fine.
I don’t get it. I don’t see how on earth you could determine if say the Hockeystick is valid or not without having access to the data and the code. You can’t do it by trying it yourself using your data and your code, that doesn’t verify or falsify one damn thing about the Hockeystick.
So you tell me, Windchaser, or Jimmi, or anyone—how would you falsify the Hockeystick without access to the data and the code and when the authors don’t answer questions or emails about their work?
I await your method of falsification of the Hockeystick …
w.
PS—Here’s another way to explain it that might make sense. You would admit that a scientific experiment is invalid without a detailed description of the experimental setup, detailed enough to allow the setup to be duplicated.
But in climate science and some other fields, all we have is data and code, and the code IS the experimental setup.
Now, Michael Mann described his code … but not in anywhere near enough detail to allow it to be duplicated. In fact, for a computer program of any complexity, it is hugely difficult to describe it in enough detail that someone can duplicate it. You can say, for example, as you do above:
But that may or may not correspond to the actual code, and more to the point, you may be totally unaware that it doesn’t correspond to the actual code. Your code may have gotten slightly corrupted along the line.
So I come along, I follow your description exactly … and I get a slightly different answer. Not a lot different, this may be one of the cases where the error is trivially small.
So I post my results, and you say “no, you’re wrong, we got a different answer … it’s probably a rounding error.”
So we go away happy, thinking the scientific method is working, where in fact your code is flawed. Then in the next instance, the next time it comes up, your code and mine give widely differing answers …
… then what? We’ve already agreed that my code and your code were both good, but now they’re not giving the same answer.
Please tell me how that can be resolved without an examination of your code and my code, to see where they differ.
The problem is that in far too many cases, unless you are willing to show your code, your work simply cannot be falsified, and thus it is not science. That’s how Mann got away with the Hockeystick fraud—because he hid the data and the code, nobody could falsify it.
That is the path you are recommending, and that, my friend, is not science …
Paul Vaughan says:
April 20, 2012 at 12:23 pm
Hey, that’s great Paul. In that case, I’m sure you can figure out the method that Michael Mann used to set the weights for the proxies in his reconstruction. No one else has been able to do so, but you sound like just the man for the job. The puzzle is spelled out here, the issue is:
So how were the weights assigned, Paul? You implicitly claim you can reproduce these results of Mann’s … time to put your money where your mouth is. I await your report. Me, I think you’re just a braggart, hugely impressed with yourself because you can outguess most of your students, but I’m more than happy to have you prove me wrong …
w.
Windchaser says:
April 20, 2012 at 12:11 pm
Sure. I come from a family of actual, working scientists. My cousin (until his retirement) was a high-temperature physicist. My older brother is a gifted scientist, who was one of Discover Magazine’s scientists of the year, and who used to head up one of the two Hewlett Packard research labs until he was headhunted by Trimble.
My brother is a good example. He had a fistful of patents, both in the name of the company, and in his own name. He also was responsible for a number of discoveries that weren’t patented … because to patent them you have to disclose them, and either he or the company was unwilling to do that. That’s the nature of disclosure, it’s a choice, and it’s one that many “actual, working scientists” don’t want to take for very sound reasons. More power to them.
The same was true about the journals. My brother published very little in the journals, for the same reason—he didn’t want to disclose the nature of what he had discovered and done. And as an “actual, working scientist” he did not have the “publish or perish” requirements of college professors.
So I fear that I have little sympathy for your argument that requiring code will somehow cripple scientists. If there are advantages to keeping a discovery secret, those advantages exist whether or not the journals require code.
w.
But in climate science and some other fields, all we have is data and code, and the code IS the experimental setup.
Right. Good, we agree on that.
Now, Michael Mann described his code … but not in anywhere near enough detail to allow it to be duplicated. In fact, for a computer program of any complexity, it is hugely difficult to describe it in enough detail that someone can duplicate it.
Not usually true. As I mentioned before, we describe parts of a code like parts of a machine or an experiment: Over here we have part A, which does ___ and works by principles A1, A2, A3, and over here we have this other part, part B, which does..
You get the idea.
But that may or may not correspond to the actual code, and more to the point, you may be totally unaware that it doesn’t correspond to the actual code. Your code may have gotten slightly corrupted along the line.
Exactly! If the code is the experimental setup, and the code goes wrong somewhere along the line, then it’s like an experimental setup going wrong. Something like mislabeling your 1M hydrochloric acid as 10M, or maybe the daily humidity affects your experiment, or maybe you misweighed your reagants or mismeasured the temperature.
In which case, your experiment may not be repeatable even if you provide your methodology, right?
But if I understand you correctly, you’re not asking experimentalists to open up their labs. And understandably so; it’s even more of an imposition. But don’t say that what you’re asking is SOP for the scientific method.
So we go away happy, thinking the scientific method is working, where in fact your code is flawed. Then in the next instance, the next time it comes up, your code and mine give widely differing answers …
… then what? We’ve already agreed that my code and your code were both good, but now they’re not giving the same answer.
Please tell me how that can be resolved without an examination of your code and my code, to see where they differ.
Well, say you look at your code, and he looks at his, and you both say that yours is right. As I see it, you have two options:
1) You publish your code and let others inspect it. If no major flaws are found in your code, you win.
2) You try to get others to come along and make their own code, and you compare the results. This is the “normal scientific method” and comparable to a normal experimental reproduction: when you get a big result, other groups will often attempt to independently reproduce your work. Why? Two reasons:
a) They want to “get in” on the new findings. The first paper never covers all the interesting or important details of the find.
b) If you’re wrong, they get to be the first to publish it.
Inviting others into your lab and letting them use your equipment wouldn’t help much. If you still had an error in your equipment, they’d “reproduce” the results, but they’d still be reproducing the wrong results. Maybe they’d find the error, but then again, maybe they wouldn’t..
This is why independent verification is the standard.
The problem is that in far too many cases, unless you are willing to show your code, your work simply cannot be falsified, and thus it is not science.
This is completely wrong, I’m afraid. Say Pons and Fleischmann had never let anyone into their cold fusion lab, but they completely and fully described their setup to others. If one, then two, then dozens of other groups tried to reproduce their results, and couldn’t? Those results are gone, baby. Dead. They’re falsified.
In fact, Pons and Fleischmann did eventually let people into their lab, because they wanted to uphold their cold fusion claims. And yes, in their lab you could reproduce some of their results, but.. not very well, and the experimental setup was somewhat sloppy. So, although we never really found out what was wrong with the experimental setup, the lack of reproducibility by independent groups means that cold fusion was a no go.
That’s how Mann got away with the Hockeystick fraud—because he hid the data and the code, nobody could falsify it.
Note that withholding input data and withholding code are two very different things; input data is a necessary part of ‘methods’, just like a full description of the analysis performed on the data. But as you could write a code to perform the analysis yourself, they don’t have to give you their code.
Basically, the authors are obliged to make sure that all the information necessary to reproduce their work is either included in the paper or in the references. If that fails, if the information they provided is not enough, then their results are in doubt.
If you manage to find errors in an important paper, or that the results are significantly wrong, write it up and publish it. It’ll be good for your career.. well, assuming you’re the one who’s right.
So I fear that I have little sympathy for your argument that requiring code will somehow cripple scientists. If there are advantages to keeping a discovery secret, those advantages exist whether or not the journals require code.
It won’t “cripple” all scientists, just some.
Say you have one guy who spends 2 years developing and testing some amazing new code, and you have another guy who spends 2 years developing and testing an amazing new microscope. One of these guys will publish, publish, publish, keeping his advantage until someone else comes along and spends the time and money to reproduce the (now publicly available) microscope designs. The other one will publish once, and immediately others will start using his code to do all the research he didn’t have time to do. The first mover advantage becomes vastly different for these two scientists.
Yes, science as a whole would be helped by the code being available. But right now, the scientific funding model doesn’t recognize that scientist’s contributions as being significant. You get no kudos or grant money for writing code that others use; you get grant money for publishing.
Since there’s no money for writing code without publishing, there will soon be no professors who do this. If they try, they’ll lose their funding, and cease being professors.
Like “survival of the fittest”, it’s a tautology that there will be few professors who repeatedly do things that are unhealthy for their career. And research careers depend on money, which comes from publishing. There’s a reason we say “publish or perish”.
If you change the system to reward publication of code, this won’t be a problem. But don’t think that just requiring code publication will be enough: you need to actually reward it, in some way that gives continued funding to those scientists who are willing to develop and publish it.
And as an “actual, working scientist” he did not have the “publish or perish” requirements of college professors.
Yep. That makes your comparison rather moot, I think. Sorry.
Joel Shore up above said something that I’ve been mulling over. He said:
I said I didn’t care what it was called. It is important to be able to follow someone’s steps exactly.
I think that the misunderstanding revolves around around the idea of falsification. Here’s the problem.
If a claim cannot be falsified, it is not a scientific claim.
Take something like Mann’s Hockeystick. As long as he was successful in hiding his code from public examination, it was impossible to falsify his claim.
You can’t falsify his claim in the traditional manner, by trying to replicate his calculations with the same data. If your results are different from his, he can just say “you didn’t do it right”. That’s what happened with Craig Loehle, for example.
You see, Mann’s problem (inter alia) was bad math. As long as his code was hidden, it was a guessing game. Nobody could prove that he had done anything wrong. His work was not possible to falsify.
Once his code came to light, however, the code was quickly shown to contain a fatal math error that made it mine even a random dataset for hockeystick shapes … and there was no way to tell that without access to his code.
So let me be clear why it should be a requirement that computer code be published along with the study—without it, your work cannot be falsified.
If someone wants to milk their code for all it is worth, that’s fine. Yes, there could be a cost in that, they may delay publication.
But a late publication that is falsifiable is science. An early publication without the code is not science, because it is not falsifiable. Easy choice.
I understand the issues and the questions that you guys raise. Yes, there will be implementation issues. I do not deny that there will be costs for some people. If you’ve been publishing unverifiable stuff, sure, you’re gonna scream about being brought to account. On the other hand, if you want to work with your code until you get every possible advantage, you’ll just have to wait to publish until you’ve done that. That’s a cost/benefit equation you’ll have to work out for yourself.
The issue that has forced Science magazine to take action is that far too often, what are being published these days are unfalsifiable claims … and the cost in that publication of pseudo-science, to the public as well as to science itself, is huge.
Here’s the short version. Scientists who publish but won’t reveal their code are peddling unfalsifiable non-science … and some of you folks, who claim to be scientists, are defending that …
??
w.
@Willis Eschenbach (April 20, 2012 at 2:40 pm)
Necessity’s the mother of invention. (The key word: “need”.) You’d have to set me up with an absolutely permantently guaranteed $100+k/year with sweet pension to get me to comb through that ugliness. Even then it would be a waste of time & effort that could be tolerated only to maintain financial security. I’ve seen nothing that would lead me to believe nature’s secrets are encoded where you direct me. I’ll volunteer this much: You would NOT need code to figure that out. You would only need DEEP, SOUND conceptual foundations. Suggestion: Less luxurious attention to distracting sports and more strategic, tactical focus on the war of survival. I acknowledge that your interests may lie more in the area of climate politics whereas mine are firmly rooted in the area of nature exploration. All the best.
Willis,
As has been pointed out, a claim can be falsified without having access to all the code. Is it harder than if the author gives you access to all the code? Sure, but there are also advantages to having other scientists investigating the claims write their own code rather than just “auditing” the code. This concept of auditing other people’s code is not really what the whole replication notion was meant to entail.
I am not dead-set against requiring the release of code, but I think there are lots of significant questions that need to be addressed both regarding the necessity of such a requirement and how it can work the way science is currently practiced. From my work at Kodak, I can say that there was little enough internal incentives to publish externally as it was; it would become a lot worse if scientists had to convince their managers to allow them to release all of the computer code related to the publication!
I think that is a harsh thing to say about Spencer and Christy, but if you feel that way, then I guess you need to take it up with them.
Paul Vaughan says:
April 21, 2012 at 10:57 am
Gosh, Paul, I see that what I thought was just boasting was actually 100% factual. You said:
Now, having seen you run as fast as you can away from a challenge to reproduce some actual result, I’m sure your boast is true.
I’m now convinced that as you say, you’ve “NEVER run into a scenario” where you could not reproduce the results … because I’ve witnessed the speed at which you run away from such scenarios. It all makes sense now.
w.
joeldshore says:
April 21, 2012 at 1:48 pm
No, Joel, that hasn’t “been pointed out”. I have asked repeatedly in this thread how you could falsify the Hockeystick without access to the code. Neither you nor anyone has answered, so your claim is false.
w.
joeldshore says:
April 21, 2012 at 1:48 pm
RSS found an error in S&C’s code, so it is obvious that they have revealed their code, or RSS couldn’t have done that. Since RSS falsified part of the S&C code, the S&C code is OBVIOUSLY FALSIFIABLE … and since I pointed this out before, I do wish you’d pay closer attention.
As a result of a portion of their code being falsified, it is patently clear that S&C are not “peddling unfalsifiable non-science”.
w.
joeldshore says:
April 21, 2012 at 1:48 pm
So what? Seriously, so what if some Kodak guy can’t publish unfalsifiable claims? That just reduces the amount of anti-scientific, unsupported, unverifiable BS I have to wade through in the journals.
Kodak guys publishing claims without code are just stroking their egos. They are not adding to the world’s knowledge, because without the code their claims are nothing but anecdotes. I see that you think they should be free to publish such anecdotes as if they were actually science … me, not so much.
So I hope that you are correct, and that as a result the number of anecdotes published in the journals goes down as more and more of the journals require code.
w.
Willis “If a claim cannot be falsified, it is not a scientific claim.”
Oh dear , Popper’s rather simplistic view of science strikes again. However, let that pass for the moment.
OK, some of Mann’s work is not science (that would make a good quote if truncated there…), rather it is maths. The process of taking a lot of data and fitting a curve to it is not science, it is maths, and the computer program is the equivalent of all those scribbles on the back of the exam paper. The science comes in the interpretation of the curve. So since it is maths there are two ways , at least, of showing it is wrong. Imagine you are a maths teacher. You could sit down and go through the students scribbles line by line, or you could present him with your working instead, and say “Here, this is how it should be done, go find your mistake”. Both ways show the original answer to be in error.
Now what I have suggested, is the equivalent of the maths teacher writing the correct solution on the board. Get all the data, write your own code, publish the result. You say Mann may just ignore this? Well he can’t, not if it is done properly. A lot of people think that “peer review” stops when a paper is published, but it doesn’t. There is a mechanism for forcing revision. You write a paper titled “Comment on Mann 1999” and send it to the original journal. The authors you are criticising will have a right of reply, and both your comment and their reply will be published next to each other in the journal (of course it is way too late for that by now – it would have had to been done at the time). Then everyone can read them and make their own judgement. I have done this myself in the area I work in. This is what should have been done a decade ago.
Here is why the issue is so important right now. You guys seem to think that this is about honest scientists. It’s not. In large and increasing part it is about the age-old fight to keep humans from gaming the system.
For example, jimmi says above:
First, the reviewers of Mann’s paper were likely his friends, the climategate emails reveal extensive packing of the reviewer box. I say that because a real review would not have let his paper pass. As a result, it is unlikely they would let an opposing paper through. Second, Mann had also refused to release the data, although later he did let it go. Third, he didn’t publish his method in enough detail to allow anyone to replicate it. Fourth, journals don’t like to publish papers blowing previous papers out of the water, they favor new research that will garner them headlines, and disagreements over old papers don’t do that.
But suppose they did let it through. Suppose I write my own code and publish my own paper … then what? I can’t claim I’ve replicated Mann’s method, because he hasn’t left sufficient clues to do that. He’s left lots and lots of steps out of what he did, so I’d be lying if I said I was sure I’d replicated his actions. Then what?
Then Mann does nothing. There’s no reason for him to do anything. He’s already won, the IPCC has splashed his graph all over the planet, he’s been catapulted to fame from deserved obscurity … why say a word? At a maximum he just has to say, “No, Willis is doing something different from what I did.” And almost assuredly, given how vague his description was, he’s right … I would be doing something different, because I’m just guessing.
In that situation, why on earth would he say anything? Peer pressure? Get real, there’s no peer pressure in climate science, climategate proved that beyond a shadow of a doubt. He has absolutely no reason to say a single word. After all, he knows he’s a fraud, and that anything he says will not be to his advantage. All he has to do is keep his mouth shut, say he’s “moved on”, and publish a new paper. There is no pressure on him at all to discuss, much less expose, his previous work.
You are living in a dream world, jimmi, if you think Michael Mann would care in the slightest about your math teacher example. He’s not a student, he’s a crook … and you actually think peer pressure would cause him to admit his malfeasance and thereby throw away his career? Really?
My dear fellow, you truly haven’t spent enough time around climate science. You are advancing ideas that might, and I emphasize might, keep honest scientists in line if they were to wander a bit.
But here in climate science, we’re kinda short on the honest breed, the dishonest ones are ruling the roost, and tragically, the honest scientists are keeping their heads down and their mouths shut.
In such a situation, your idealistic pollyanna solutions involving stories of how it works in school are just a bitter joke … remember, this is a guy who flat-out lied to a Congressional committee, and you think your fairy tales about maths teachers are going to bring him into line?
I don’t know whether to laugh or cry about that level of naivete …
w.
jimmi_the_dalek says:
April 21, 2012 at 4:37 pm
No, let’s not let that snide attempt at falsification via contempt pass for the moment. You appear to be taking the position that a claim that cannot be falsified is still a scientific claim. You’ll have to back that up, you can’t just assert it and then walk away.
Perhaps you could provide us with say three samples of claims which cannot be falsified, but which in your view are still scientific statements?
w.
Willis Eschenbach says:
McIntyre found an error in Mann et al.’s code, so it is obvious that they have revealed their code, or McIntyre couldn’t have done that. Since McIntyre falsified part of the Mann code, the Mann code is OBVIOUSLY FALSIFIABLE … and since I pointed this out before, I do wish you’d pay closer attention.
As a result of a portion of their code being falsified, it is patently clear that Mann et al are not “peddling unfalsifiable non-science”.
Seriously, I don’t understand the difference, except for the following facts:
Mann has released the code for his 2008 paper so completely for all to see that even McIntyre, who has made his second career out of finding things to complain about, can’t find anything to complain about. (He has also publicly released his 1998 code, although there are apparently some things that McIntyre and you can find to complain about.) In contrast, S&C have never publicly released ANY of their code as far as I can tell…and while they did eventually share a small piece of it with RSS, what I have heard is that it involved quite a bit of work just to get that to happen.
Willis, I think that everyone can see the obvious double-standard here. It is really quite amusing.
“Perhaps you could provide us with say three samples of claims which cannot be falsified, but which in your view are still scientific statements?”
I have no beef with the concept of falsification per se, except that it leads people to think that the only way to progress is by the equivalent of going through a students maths line by line. There are other ways to make progress. Put it this way : does using an idea which technically speaking has already been falsified, mean you are not doing science?
jimmi_the_dalek says:
April 21, 2012 at 7:34 pm
So everything I said is true, but your beef is that it is not complex enough? Certainly there are other ways to make progress, lots of them … but that has nothing to do with falsification.
Depends on what you are doing with the idea, so your question is far too vague to answer.
w.
joeldshore says:
April 21, 2012 at 7:25 pm
Mann’s code was only falsifiable because he left it on an open server. Yes, once that great stroke of luck happened, it became falsifiable … but not by any action of Manns, and against his will.
S&C’s code, on the other hand, was given voluntarily by Spencer and Christy to their strongest opponents and nay-sayers, RSS.
If Mann had done that, we wouldn’t be discussing this. But Mann did everything he could to keep his code private and unfalsifiable, while S&C made their code falsifiable. If you can’t see the difference, I do wish you’d pay closer attention.
w.
joel shore says:
“McIntyre found an error in Mann et al.’s code, so it is obvious that they have revealed their code, or McIntyre couldn’t have done that.”
Shore apparently isn’t up to speed on any of this. The fact is that McIntyre and McKitrick spent years reverse engineering Mann’s code, and got it almost exactly right — with no help from Mann.
When Steve McIntyre himself states that Mann has fully cooperated in sharing his code, methodologies, data and metadata, I will accept that as definitive. But so far he has not stated that, AFAIK.
It is amazing to see the ease with which Willis cuts the heart out of Joel Shore’s weak arguments. Joel would be smart to quit digging. But no one has ever accused Joel Shore of being smart, as far as I know.
The scientific method absolutely requires transparency. Otherwise, it is extremely difficult, if not impossible, to replicate experiments and support or falsify hypotheses. That is why the alarmist clique hides their data, code and methods: they know that their conjectures would be easily falsified if they practiced transparency per the scientific method.
Really, all of Joel Shore’s arguments in suport of Mann’s obfuscation stem from the knowledge that CAGW is unsupportable. Planet earth is b!tch slapping Shore’s nonsense, but he endlessly repeats Mann’s deconstructed narrative, hoping to keep his pals riding the tax-sucking grant gravy train.
Willis: “Certainly there are other ways to make progress, lots of them”
Good so now you agree with me. There are other ways to make progress. In fact the main way to make progress does not use Popper’s philosophical theories, but instead progresses by the simple expedient of replacing partially incorrect ideas with less incorrect ideas. All science currently known is incorrect (or perhaps you would prefer ‘incomplete’) it is just that some of it is more incorrect than the rest. Even Mann99 must have some bits correct…..
“Depends on what you are doing with the idea, so your question is far too vague to answer.”
Ok, on using already falsified ideas, what if you were using some equations with were known to be incorrect to make a prediction about some real world phenomenon – go on, take a stab at it.
I didn’t read Willis’ comment before I posted, but it looks like we’re on the same page.
. . .
Jimmy the d says:
“…on using already falsified ideas, what if you were using some equations with were known to be incorrect to make a prediction about some real world phenomenon…”
I don’t understand that at all. Could you please re-phrase, with context? Thanks.
Smokey say:
(1) It must certainly be fun to be Steve McIntyre: All you have to do is feed people like Smokey what they want to hear and they will unquestionably believe you, while self-parodyingly calling themselves “skeptics”.
(2) In fact, McIntyre can’t find anything to complain about in regards to sharing of the data, methodologies, data, and metadata for Mann’s major work in this area that is not already over a decade old.
Meanwhile, I am still waiting for someone to show me where I can find Spencer and Christy’s code. Heck, forget their whole code, at this point, I might even settle for one or two subroutines!
Willis Eschenbach says:
This statement is pure spin. Even Christy has admitted that they only gave RSS a part of the code and it is unclear how much effort RSS had to expend to get even this much. I have given links to Mann’s 2008 code that is so completely public that even McIntyre can’t find a nit to pick in that regard. And, I have given links to the most, if not all, of Mann’s 1998 code. You in return haven’t been able to find one line of Spencer and Christy’s code to provide to me.