Quote of the Week – rubbishing peer review

Here I am, minding my own business, looking at my local newspaper, and just reading what is happening locally. Then, out pops this profound quote of the week that is just gobsmacking.

The first line of the story starts out like this:

CHICO — A graduate student researching local rattlesnakes has something to prove, but he isn’t yet sure of what.

Sounds like some climate science RFC’s I’ve read. But wait, there’s more. Get a load of the punchline:

Like I said, gobsmacking. I have no desire to quash any study of rattlesnakes, but the sheer arrogance of the statement about publishing a scientific paper “even if it’s garbage” was just mind blowing. I hope Mr. Woodruff uses better judgement in the future. For now, I’ll just assign it to the youthful indiscretion file that I’m sure every one of us has.

Full story here.

I am reminded though, of the Seinfeld episode.

Advertisements

  Subscribe  
newest oldest most voted
Notify of

Yup, shocking attitude.
Oops on the grammar or spelling or both: “For now, I’ll just assign it to the youthful indiscretion file that I’m sure very one of us has.”
[Fixed, thanx. ~dbs]

Is that what one calls a poisoned paper?
That could bite him unexpectedly.

OssQss

Incredible! Same goes for the video clip, thanks for that 🙂

If he keeps up that attitude he’ll likely encounter more snakes than ladders as he goes through his career setbacks.

Myron Mesecke

Seems he has learned from the best. Sadly.

Typo – change “very” to “every” [Done, thanks. ~dbs]
Love rattlesnakes! Always had to watch out for ’em in eastern Oregon near the Columbia River…

John David Galt

If he’s really smart (or, depending on your point of view, really Machiavellian), he’ll come up with a way to cast his findings to say Government has to Do Something Right Now! Then he’ll get media attention and wind up a rich and successful politician.
Or, he can be honest, and then no one will ever notice him again.

Doug in Seattle

Yup, the guy is now a bum!

wermet

This sounds a lot like the elementary school “experiments” that my friends I used to do in our parents’ backyards and sheds (minus the possibility of lethal snakebites). Let’s mix these chemicals together and see if something interesting happens. It was often followed by bad smells, smoke, flames, chlorine gas, explosions or some combination thereof.
“So no matter what we did, even if it’s garbage, it’s educational.”
Seems like Mr. Woodruff needed to get this type of “science” out of his system about 15 years ago.
On a more serious note: if, as Mr. Woodruff conjectures, that the venom of snakes within a single species varied with locality, wouldn’t that render our current snake anti-venom serums ineffective? I’m not an expert on the subject, but I have not heard about any major problems with anti-venoms, except for a lack of available supply and allergic reactions.

Brilliant!
Only thing missing now is a parody about the show ‘Seinfeld’, replacing Seinfeld and Alexander with Schmidt and Hansen, having Hansen call Schmidt a bum for using Mann’s data (garbage).
I keep hoping to find a image of the two of them sitting in Tom’s Restaurant, pointing fingers at each other.
That would be choice.

polistra

Well, he’s right.
Unfortunately, he has publicly expressed the universal private understanding of the Least Publishable Unit. And in science as in all other areas of modern life, telling the truth in public is the only sin.

Don’t worry. He’ll soon be a hero when he publishes this: “Rattlesnake venom in Northern Pacific more potent due to Man-made Global Warming.” I’m taking bets…Anyone want to bet against this?

He’s a kid. I’m pretty sure that if he thought about it, he didn’t mean exactly what the words said. In excitement I have mis-spoken before. What he may have meant was that this is a unique opportunity to study an isolated population. No matter what turns up, it should be interesting. If the venom is the same as all the other rattlesnakes then that’s interesting. If it’s different, then that’s interesting also.
Scientifically, that’s not the problem. The problem is that it’s not science. Most biology is not science. He’s just collecting and classifying.

Alan

I think you’re all being a tad too harsh. His choice of words was poor, but his study is testing an hypothesis. So even if he doesn’t prove his hypothesis true (i.e. “garbage”), it should be published to demonstrate that it’s been tried.
Just because an experiment fails doesn’t mean it shouldn’t be published. Science should be advancing with the help of failures — maybe someone will notice a glitch and improve the process for another go-round.
Just my “non-scientist” two cents.

joshua Corning

Huh?
He is analyzing the venom of an isolated population of snakes. If there is a difference then it might be interesting…if it is not well then it is garbage…yet should he withhold his results?
I am failing to see how or why this statement is gobsmacking.
Negative or uninteresting results (garbage) is just as important for science as positive or interesting results.
Furthermore he is a graduate student. He needs to study something in order to demonstrate skill so he can graduate and get a job.

Guy

He could measure the impact of global warming on the rattlesnake venom and be sure to be publishd. A scary scenario would be finding the local rattlesnakes to be unique and in need of being put on the endangered species list.

joshua Corning

Actually the more i think about this the more i think Anthony at least in this case is not understanding how science works.
Didn’t Mann produce some garbage results with a statistical test which showed that his hockey stick lacked significance and then withheld it from his study and it was only later when Mcintyre found that undocumented and unpublished result on an ftp site?
Wouldn’t we have a better understanding if Mann had simply published that garbage?

Wil

Perfect! This guy will go on the have a brilliant career in the Global Warming industry.

Christian Bultmann

I see a bright future in the IPCC for him.

philincalifornia

Alan says:
July 19, 2011 at 6:20 pm
joshua Corning says:
July 19, 2011 at 6:20 pm
=========================
Exactly.
Glenn, if you don’t advance science, it’s not a waste to publish it. Saves other people wasting their time. Crap results can go in crap journals. I’ve done it.
On the other hand, you might find something really interesting. Think yourself lucky that you’re doing something no one else has done before. You’re in a very small minority. Take the opportunity and build on it.

Doubting Thomas

R&D often depends on what someone once called a “perfect failure.” You absolutely fail to develop what you intended, but you find something better than you’d hoped for.
It might be that what is “wrong” with science-by-peer-reviewed-paper, is that known garbage doesn’t get published. We can’t learn from negative results if we don’t know about them.
Anyway, the days of American “science for the sake of science” seem to be numbered … we’re running up debt at a astonishing rate. We need the next generation nuclear plant, more than we need to know about subtle variations in the protein makeup of snake venom. Maybe more importantly, we need to break up the financial-scientific-medical-industrial super monopolies that suck the life blood out of our economy.
dT

Rather than assuming that the student is arrogant and boastful let us instead be charitable and hope that he is being a good scientist (in expecting to publish any results even if they prove any conjecture wrong) and a good critic (in correctly evaluating that the state of scientific publishing is sadly deficient).

Philip Peake

Long long ago, in a faraway land, I went to University to get my degree. A degree in Computer Science, when that was a new and wonderful thing.
I met a Geology postgrad, doing his PhD thesis.
Over a few beers, he told me what he was doing to earn his PhD. He was writing this HUGE program (in Fortran — fail right there as far as I am concerned), to simulate … a rock.
Yes, a rock.
Basically, he was taking a bunch of fluid mechanics equations and changing the coefficients so that the rock “flowed” over thousands/millions of years.
I could have knocked off that program in probably less than a week …
As far as I know, he got his PhD.

RockyRoad

Just shows Anthony has a nose for scientific news even when he’s supposed to be on vacation!

JDN

@Anthony. We have several students in the lab right now working on “garbage”. They don’t understand it, don’t know how it relates to other things and aren’t convinced its worth doing. Student research is sometimes a Hobson’s choice. So, give the kid a break.

If he performs accurate original observational work on a distinctly different species population, whether he demonstrates commonality with or divergence from the venom compositions of the populations outside the area of isolation, that’s real science, larrygeiger to the contrary.
We may now reliably infer that however larrygeiger is educated and trained, it is not in either biology, biochemistry, toxicology, pharmacology, immunology, neurology, anesthesiology, or emergency medicine. The work of herpetologists studying venomous critters has led to saving beaucoup human lives.
This particular grad student’s present work may be of low value, but it will serve to incrementally improve the knowledge base about this species’ venom characteristics, and – in the bargain – it is from such biological substances that we have derived (and will continue to derive) diagnostic and therapeutic principals of use in the medical management of both human beings and valuable domesticated animals.

This quote is what I call “the stench of a paper mill.” AND it has infested much of the academic world:
http://depriest-mpu.blogspot.com/2008/03/stench-of-paper-mill.html

kadaka (KD Knoebel)

Agriculture has isolated the rattlesnakes here from others in the state, meaning it’s possible the snakes have adapted their venom to the area. At least, that might be something he proves with this study, he said.
Wow, (C)AGW is making evolution progress faster! Is there nothing it can’t do?

Katherine

Heh. Isn’t a project supposed to have its objectives written out first, before data gathering starts? What kind of adviser would green light a project as vague as “studying the venom of Northern Pacific rattlesnakes”? After “21 2 months” into his study, he still can’t rattle off his objective off the top of his head?
What seems obvious is that Woodruff is going for the low-hanging fruit. Since that rattlesnake population has never been studied before, any new information is publishable.

JimboW

I must agree with the minority of posters here on this one, i.e. that there is absolutely nothing wrong with what this kid is doing (apart from saying “garbage” rather than “uninteresting” or “inconclusive”). As some have pointed out, would it be better if he made a point of burying results which don’t support his hypothesis (for which he clearly has at least one candidate)?
I bit more reflection by some posters before , joining the jeers squad, might be advisable. Otherwise people might get the impression that said posters really have no idea what transparency in science entails.

Chris Smith

It is difficult being a PhD student. He is under a lot of pressure to publish *something*… *anything*. It is a box that more or less must be ticked in order to get the PhD.
It is okay to tell someone, okay, here is some money, go and study those rattlesnakes and see what you can find out. Where it goes wrong is when he feels that he must publish *something*, *anything*, simply because that box must be ticked. It should be okay for him to write his thesis showing how he scientifically examined the snakes and found that there was nothing extraordinary about them. If he conducted the research to a high standard then that should be okay and he should be awarded the PhD.
Of course, if he can mention CO2 in a paper somewhere then no doubt he will get to travel around the world on jumbo jets presenting posters and short talks about it.

Rick

Isn’t that how James Hansen got the start to his career?

Gary Hladik

I agree with others on this thread that the student’s choice of words was unfortunate, i.e. by “garbage” he probably meant “negative results”. Nothing wrong with publishing negative results if they were rigorously (robustly?) attained (unless you want continued funding for the project, of course).
On the other hand the quote as published is quite striking and well worth mentioning on this blog as a good example of a bad scientific attitude (though not necessarily on the part of this particular student).
Liked the “Seinfeld” clip, too. 🙂

What he’s doing is the best kind of science, that of pure curiosity and discovery with no preconceptions of what the result should be. He just chose an unfortunate, if completely accurate, description of what the academic endgame requires him to do.

jorgekafkazar

If he finds no detectable difference in venom, he publishes that. If he finds a difference, he says so. Fine, either way. Garbage, like the man says, but it’s good garbage. I just don’t want to see:
(1) “We must hurry and spend millions on further research before the new super-snake spreads to all of the US.”
(2) “We must hurry and declare the super-snakes an endangered species before they are wiped out by motorcyclists.”
And I especially don’t want to see the usual PC shibboleth:
(3) “These results do not mean that AGW is not taking place.”

Alan is right: He probably meant (we hope he meant) the hypothesis maybe garbage but the paper will be published even if the venom proteins are identical. The data needs to have integrity (can’t be garbage). Maybe he chose his words carelessly – or maybe the reporter takes poor notes.

kadaka (KD Knoebel)

From Chris Smith on July 19, 2011 at 7:41 pm:

It is difficult being a PhD student. He is under a lot of pressure to publish *something*… *anything*. It is a box that more or less must be ticked in order to get the PhD.

Ack! Are you trying to scare this guy off from getting his next degree?
(bold added)

Glenn Woodruff, the 25-year-old student collecting data, is currently 21 2 months into his project as part of his master’s education in biology from Chico State University.

Going broke fast, practically zero funding, and nearly two years into a project towards just his Master’s degree. And you want to worry him with how bad it’ll be to go further and get a doctorate as well?

Jordan J. Phillips

I think maybe you are taking his words the wrong way, or at worst there isn’t enough context to judge either way.
He may have been speaking very off the cuff and casually. If it is an unusual and unexplored area, then any honest research that demonstrates something, even if it is disproving a hypothesis, is an honest attempt at establishing knowledge and can be publishable depending on the merit and what’s involved. He probably meant that, but there’s no telling.
A good quote I’ve heard is “if you knew it would work beforehand, then it’s not real research”.

dp

There was a time when this attitude would have been dealt with by peers. Either this guy has no peers in which case he is doomed to to this, his 15 minutes of fame, or his peers are imbeciles as is he and they are all doomed to ignominy. His will not be known as the greatest generation by any subsequent generations, and his efforts land him condemned as a fool by this one.
But what the hell, it’s a start.

Jordan J. Phillips

“It is difficult being a PhD student. He is under a lot of pressure to publish *something*… *anything*. It is a box that more or less must be ticked in order to get the PhD.”
At Chris: Yes, yes. There is a considerable amount of “paper inflation”, because collaborators stick their names on projects even if they had only the loosest of involvement in a paper. The result is this standard that judges quantity over quality in papers that get published, which puts an immense pressure on a young PhD student, often prompting then to forego juicy but more challenging projects for something that can more easily payout in terms of a paper.
An important point to remember is that just because research is done at the university instead of the private sector doesn’t mean the profit motive is no longer involved. It is instead though a different kind of “profit” the university researcher has in mind, as the goal is publication in a journal, not commercially selling something. But similar to the private world, a project is often pursued based on the likelihood that it can payout for the time invested into it.

MinB

As a graduate student, I wrote an inconsequential thesis that was probably only read by my thesis adviser, so I wasn’t too critical of Woodruff until I read, “The nature of the small exploratory study means Woodruff doesn’t have much funding, he said.” Funding? What irritates me is that funding has become an expectation for a mere masters thesis.

JimboW

Philip Peake says:
“Basically, he was taking a bunch of fluid mechanics equations and changing the coefficients so that the rock “flowed” over thousands/millions of years.
I could have knocked off that program in probably less than a week … ”
Big call there. He can also probably knock that program of relatively quickly (although not in a week, and I think anything dashed of by anyone in a week would be basically useless, much like a geomechanics model designed by a computer scientist with zero domain knowledge). I think you’ll find that most of that 3 years goes into researching what those pesky little coefficients are, and how they change over time with progressive deformation, in something modifying them to make them useful for something that flows like, well, a rock.

Kasuha

At least he’s honest, because that’s how things are. He must publish something whatever it is or he failed – and getting his grade is a priority, not scientific quality of the paper.
I’d blame the rules, not him.

Greg Cavanagh

quote: “So no matter what I do, even if it’s (garbage), it’s publishable”.
I see some here believe a non-conclusive result as meaning a “garbage” document.
I think he’s implying that he could “write” a garbage document.

janama

“It is difficult being a PhD student. He is under a lot of pressure to publish *something*… *anything*. It is a box that more or less must be ticked in order to get the PhD.”
yup! – my daughter is doing post grad medicine and she says it doesn’t really matter what she presents so long as she presents something!!

DJ

A perfect trio of thesis advisors would be Michael, Gavin, and Ben.
(Ben because he’d beat up anyone in a dark alley who questioned the thesis)
Now, all he needs is an NSF grant. He can then prove that the rattlesnakes are different because of climate change.

David Falkner

Well, it seems like there is no other research in the area, so he could technically publish anything and build on the knowledge base. His wording was terrible and sounded incredibly cocky, which is a problem in the arena of perception. You know, that is somewhat reminiscent of climate science.

John Tofflemire

He’s young, give him a break. I wish I could only have had his level of stupidity when I was 25. In the end, it’s just a masters thesis which, at this level, should only be publishable if it is exceptional.
The fact that this young man is taking two years of (uncompleted) research to earn a masters degree says volumes about his university and about his advisers.

Laurie

He just wants to play with (poisonous) snakes, second only to things that blow up.

nano pope

That’s the wonderful thing about being on a scientific frontier – even lack of results are meaningful. A poor choice of words, but he has the right spirit.