Margaret writes in tips and notes:
More about the failure of peer review— or more precisely its inconsistency in producing reliable assessments of the value of the submitted article
http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=6577844
Abstract
A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.
The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.
With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
I’m not the one saying peer review does not work. Besides, I am asking is there anything more up to date, now that we have more modern ways of communicating and publishing.
Did you miss the last update was 2011? The article is considered timely as of that date. We also know thanks to climate gate that the peer review system in climate science is fubar in totality and we have that from peer reviewers and those who work withing the peer review system.
@dp
I think the last update refers to the admin on the site (ie when did they last check the listing and its details, rather than an update of the paper. I’ve read the paper, it’s not pretty but it is a review article rather than new research. And it is commenting on the first paper’s findings (I’ve read that too and the comments that arose from it) so it would be unlikely to get an actual update but if you have a link or reference to an updated version of the paper then I would welcome the chance to read it.
Once again I would welcome links or sources for new research on peer review rather than assertions and anecdotes. Can anyone provide any?
Poor Ms. Hardman can’t get anyone in this unruly class to do her homework assignment.
Anyone else reminded of this?
John M
No, doesn’t remind me of anything. Perhaps you were unlucky enough to have a teacher like that. Perhaps you were unlucky enough to be the boy in the class wanting to learn surrounded by those who would rather disrupt the learning and come to regret it later. Not quite sure why you chose to search YouTube rather than for papers on peer review from this century but I can assure you there are some things out there and they aren’t hard to find. Haven’t checked them myself but I have found some titles. The “homework” is a bit of an accidental test to see how skeptical those who proclaim that epithet really are. Do you really want to find things out or do you just, as someone else commented, only want to know things that confirm your worldview? Not you personally, John, but the followers of this site in general. I didn’t plan it that way but the request has been largely ignored which may mean nothing but suggests at least that it hasn’t been done.
Since the lid of the box labelled “Personal Insults” has now been lifted, I expect more. That is the pattern, as on eristic sites dotted all over the Net.
Actually Margaret, I found this last night, but I wondered how prissy you wanted to get about your “homework assignment”.
Funny, it took me about 2 minutes of googling, but I guess that was too hard for you, an esteemed “educator”.
http://onlinelibrary.wiley.com/doi/10.1002/asi.22784/full
You can see that the preponderance of literature reviewed points out various types of bias. The Review authors try their best at “false balance”, but basicaly, it’s pretty clear that there is a large body of literature reporting on bias in the peer reviewed literature area.
I guess another experiment we’ve just done is confirmed an extreme deficiency among strident CAGWers.
You wouldn’t by any chance have had some trouble finding your way around southewestern Arizona? You seem to lack a sense of Yuma.
What an indictment of a system.
What it does imply is that any review at all is doing nothing but excluding work, work that may have some value.
Better no system than this system
BTW: This wouldn’t be possible in climate science, where everybody knows everybody.
Steve Garcia
Did you miss the climate gate mail where the gate keepers swore to do what was needed to prevent papers from being published? In their own words, by their own hand. I’m not doing your homework, either.
Had no trouble finding my way around Google or Arizona. I found the right Page.