I’ve been made aware of two different opinions on state of science publishing as it relates to peer review and the pressure to publish even faster due to the Internet and all of its “instalaunch” tools.
First, in Nature, a comment by Dr. Jerome Ravetz: Sociology of science: Keep standards high.
He argues for embracing the new medium, while maintaining quality:
As more people become involved in online debates, quality need not fall by the wayside. It is encouraging to see that well-conducted discussions of points of contention between the scientific mainstream and critics are emerging, as the Berkeley Earth Surface Temperature study demonstrates (see Nature 478, 428; 2011).
Ultimately, effective quality assurance depends on trust. And science relies on trust more than most institutions. As Steven Shapin, a historian of science at Harvard University in Cambridge, Massachusetts, showed in his 1994 book A Social History of Truth, trust is achieved and maintained only by mutual respect and civility of discourse. In a digital age, civility should be extended to, and reciprocated by, the extended peer community.
Scientists have a special responsibility, but also a special difficulty. When their training has been restricted to puzzles with just one right answer, scientists may find it hard to comprehend honest error, and may condemn those who persist in apparently wrong beliefs. But amid all the uncertainties of science in the digital age, if quality assurance is to be effective, this lesson of civility will need to be learned by us all.
Dr. Judith Curry has some thoughts on this here, she writes:
I am a fan of the concept of “extended peer community” put forth by Funtowicz and Ravetz. Also, Ravetz’s phrase “the radical implications of the blogosphere” has definitely stuck in my head. Re the civility issue, I agree some level of civility is needed. Some think that Climate Etc. is too raucous (a not infrequent complaint made at collide-a-scape). A fair place for an honest debate might not be especially courteous. But the blogosphere enables a range of different types of fora and moderation rules. The challenge is to extract signal from the noise. I am pleased that sociologists are studying this.
At the same time, we have an editorial in Nature Geoscience, Embargoes on the web stating that scientists are increasingly acting as reporters now, and as a result, sometimes run afoul of publication rules. I see this as a shot across the bow against such practice.
Now that researchers, too, are acting as reporters, the guideline for talking freely to scientists but not to journalists may sound contradictory. Who should count as a member of the media for the purpose of the Nature journals’ embargo policy? The same basic rule applies: if an author actively seeks media attention before publication, we consider this a breach of our embargo policy.
At the same time, it is important to Nature Geoscience and fellow Nature journals that the scientific debate does not stop while a paper is under consideration. This principle also remains: we want our authors to present and discuss their results at conferences and communicate them to their peers. So if someone in the audience — journalist or scientist — tweets or blogs about a talk, we will not consider it to be a breach of our pre-publication embargo (see also Nature 457, 1058; 2009).
Where they say:
…if an author actively seeks media attention before publication, we consider this a breach of our embargo policy.
This squarely applies to the pre-publication publicity stunts pulled by Dr. Richard Muller and his BEST team.
People wonder why I dropped my support for him (like the feckless Dr. Peter Glieck and his science B.S. of the year awards), the answer lies within the shenanigans he pulled after earning my trust to use my data. I had always expected my data to appear in a full peer reviewed publication, instead, Muller spewed it in Congress and in his own media blitz in releasing papers that hadn’t even run the peer review gauntlet.
It may take some time (and additional train wrecks like BEST) before scientists learn that they can be their own worst enemy with these sort of behaviors.
OTOH, I’ve been considering a web 2.0 peer review experiment of my own. WUWT now has the ability to offer a peer review service for articles and papers. It is a new feature I can activate into WordPress, and would allow comments by invited reviewers to be posted for authors prior to publication, so that articles can be evaluated by a broad base of techical readers prior to publication.
I welcome readers thoughts on this idea. – Anthony
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Anthony: If I understand your Web 2.0 peer-review model, it would allow a network of scientifically savvy readers (affiliated by at least their readership of WUWT) to read the draft paper and critique it (anonymously or not) before it’s published. (Of course, in the new online world, “publication” is a bit of a continuum, less of a distinct event. One doesn’t see bundles of newsprint with still-wet ink being dropped off at kiosks all over town at 2 AM).
I like it. Both because you have a terrific model here for intelligent and constructive discourse, and you should continue to build it, and this is a logical extension of what you’re (we’re?) already doing. And because the world really, really needs leadership on the process, to get us away from the corrupt and broken oligopolies that trace their power back to the physical print model. The same problem is present in medical journal publishing, and I think the same kinds of solutions are emerging. Go for it.
Anthony, did Guttenberg ask permission of the Church before printing the Bible? I don’t know, he might have, but would that have stopped him? He was in it for the money.
You web 2.0 idea will be done. By someone. Do you want to be part of that “someone.” Go for it.
It is time to take this necessary step. Professional journals have had a lucrative stranglehold on the peer review process. That in and of itself may not be so objectionable. But they have been co-opted, dominated and intimidated by a relativelt small clique of Mann/Jones and their followers. This clique has used its undue influence to turn peer review into pal review. We see the results every day, where authors insert the obligatory reference to climate change, global warming, etc., no matter how inappropriate or far-fetched the reference may be.
Providing an alternate venue that is open to true peer review by numerous experts in the relevant field is good for all concerned. Only the journals will hate it. But then, they brought it upon themselves.
You would be adding to science. So many of your readers are scientists. This would be an excellent forum for reviewing papers. Peer review has lost some of its authority and cache with the heinous behavior of people like Mann, Hansen, Jones, Gore (although, I list him with reservations as he is not a trained scientist), et al, I do not trust what I read from so many publications now due to their fraudulent behavior.
Web 2.0 peer review…..
Quis custodiet ipsos custodes?
AleaJactaEst,
Well, we know who watches the watchers now, don’t we? And we know pal review is corrupt; evidence is found throughout the Climategate emails. At least with Web 2.0, invited experts would be free to hash out any problems no matter what their point of view, and without interference from anonymous gatekeepers. Really, it’s all good.
Trust in peer review will only be gained if there is a high level of confidence in the standards of peer review and an acknowledged humility concerning the limits. The lack of trust is due to the highly partisan ways that peer review has been used to enforce consensus opinions, and the false/exaggerated claims made for peer review. In particular that:-
1) Peer review is the unambiguous demarcation between science and non-science
2) The peer-review process is sufficient to establish the thesis within a paper as the most advanced thoughts on a subject.
Lack of civility in discourse has the same routes as civil unrest within society. When a group in power has a tenuous grip on power, and excludes other groups by perceived heavy-handed and unfair means, then frictions will arise. Civility comes from recognizing other points of view, even those one feels firmly in error.
Peer reviewers are appointed by the publishing companies or by the scientific societies on the basis of their scientific credentials. They are under time pressure, don’t earn nothing in doing it, but the pride to be a reviewer. Some cronyism may take place but this is just human.
Peer review is a kind of quality assurance scheme:
a) Reviewers are asked to ascertain that the methodology presented in a paper is not absurd, doesn’t contain gross errors, to request more details when the description of the work looks fuzzy or incomplete, and to verify that the conclusions are plausible. They can recommend to refuse a bad paper. But let’s not assume that a peered review article is correct in everything because it has been reviewed.
b) It induces some quality concerns to the authors who know that peers will review their work, thus ensuring a kind of auto-censorship before publication (like knowing that radar controls on the highway will induce us to respect the speed limits).
I don’t see how a “web 2.0” review would replace or enhance the existing and needed peer review. How would be reviewers appointed? on the basis of which credentials? by whom?Would they do the same quality control as the current reviewers? Or would they engage in an early judgmental exercise without contribution to scientific quality but to an opinion debate. Would it result in an anti-peer review? Or just the same as a blog discussion?
I think it’s a great idea. The alternative news media has already had a big negative impact on monopolised global news outlets, particularly newspapers. Do the same with the published scientific journals, that’s what I say – make them irrelevant and, goodness knows, we’re well aware of how riddled by corruption they are after CG 1 & 2.
So WUWT has the credibility and wide global circulation (as evidenced by recent stats) to be the perfect forum for said reviewers.
This is going right up against the establishment though. They won’t like that. Might get SOPA on your back. All those clips from commenters will have to go, etc.. And look what happened to TB…
Hi Anthony
It seems a logical step forward despite the anticipated outcry from the existing Journals. Do you have thoughts about how reviewers would be nominated and chosen?
Anthony,
I know there were some wrinkles, but I see nothing wrong with Muller wanting the paper to have both regular peer review and open peer review. That way, they can address any major objections people come up with.
Whatever difficulties or dilemmas stand in the way, resources will be found to overcome them. This is an idea whose time has come.
Go for it!
Smokey,
Can we tar all of science with the same brush and do we go further and apply Web 2.0 to all scientific peer review?
What state is peer review in the hallowed halls of physics, chemistry, natural sciences etc.. & can we apply the precautionary principle that our detractors in CAGW are so beloved of? Be careful what you wish for.
On the other hand, it is said that human development moves in steps that are either evolution or revolution, the latter coming out of left field. Perhaps we ourselves inhabit left field?
peer – a person who is equal to another in abilities, qualifications, age, background, and social status.
….status quo
I’ve never understood why it would work in the first place….
……….I would want someone smarter than me to review my work
An open review (which can’t be done with even current peer review) with more ‘peers’, in an invite only setup, seems superior then the current system.
Even taking the ‘web’ out of it, the more reviewers the better, a more aggressive review system with more transparent comments, and various outlying experts involved would be better what any magazine or journal could do with 3 reviewers.
The idea as proposed could only be superior. There will be some things to iron out, but the wider the reviewer base, the wider the expertise, the better then end result. With the Internet, we could produce some very good science, the likes of that never has been as well reviewed.
Please, please, attempt it. Science could use the ‘Crowd source’ approach instead of the 3 experts in a dark room, who’s bias might not be apparent. Even if the first attempt fails, what will be learned from the process would make the next attempt even better.
Best of luck!
One of the reasons for lack of civility in scientific discourse is the deliberate exclusion of dissenting views, and making claims that are not sufficiently backed by data and method to enable replication. As an example, please see Steve McIntyre’s posting of yesterday on the partisan withholding of data by Prof Phil Jones.
http://climateaudit.org/2012/01/08/nature-and-the-inundation-legend/
In particular this quote from one of Phil Jones’s emails
“Years ago I did send much paleo data to McIntyre but have also had nothing but criticism on his blog ever since. As I said, this criticism on blog sites is not the way to do science. If they want to engage, they have to converse in civil tones, and if people don’t want to work with them, they have to respect that and live with it.”
In other words, to engage one has to refrain from criticism, or one should accept exclusion. Much like a Gentlemen’s club of a bygone era, but without the standards.
Since there’s a general tendency for warmist expertets to avoid this site, I think your idea for “peer review” would be just as much “pal review” as the current “Team” approach. I would be skeptical about whether the results would be of any higher quality than current methods.
From my researches in super-conductivity I know academics talk a lot of rubbish.
Open review of current scientific theories will almost certainly trigger advances across all disciplines.
And frankly this site is the perfect initial host (until it gets too big 😉 ).
Latitude says: January 9, 2012 at 11:03 am
And suppose there IS no one smarter?
Anthony,
Lean forward…
No new system will work unless there’s money pulling in the proper direction.
Quality control currently works well in areas where good science leads to salable products, or where good science leads to fewer lawsuits. Materials science, agricultural research, engineering, medicine.
Quality control fails in areas where bad science leads to bigger grants. Climatology, quantum physics, “social sciences”, cosmology.
And where money or lives are not involved, quality control frankly doesn’t matter. It’s just fiddling around anyway, so quality is moot. If it’s fascinating to somebody, it’s OK.
Latitude says:
January 9, 2012 at 11:03 am
……….I would want someone smarter than me to review my work
——————————————————————–
Well said. Most people here are smarter than me: that’s why I come here.
(Mods, I left a comment about this on the milestone thread but I think it may have been spam-binned – don’t mean to make a fuss but it’s just that it is relevant to this thread. Alternatively, I’m going mad. Cheers).
[REPLY: It’s not there. Resubmit, if you wish. -REP]
Do it.
The internet is far more open and so less corruptible and probably immune to the setting in of group-think.
Some publications have been found wanting in recent years and some have succumbed to lower standards. Whilst some (Nature ?) have taken worthwhile steps to improve their readers confidence in their standards, I feel the that the future lies with the people and the internet.
I am not a scientist and will probably never present a paper. But if I were to present a paper to the World, I would never dream of going to a print magazine. For me, publishing my paper on WUWT or Tallbloke’s Talkshop, would be far more satisfactory, allowing me to gain a wider readership and to interact with those readers and answer questions.
We see this sort of thing more and more, with major scientific figures presenting initial papers on WUWT and Tallbloke. The internet is without doubt the future of scientific publishing and peer review, prior to subsequent publishing on the main site.
Do it.
From the Ecological Society of America: Fighting back against open access.
A telling quote:
One way to make taxpayer funded research more visible and accessible to interested members of the public would be to require federally-funded grantees to provide a second version of the research summaries they already prepare, specifically for the lay reader.
I guess that we’re not smart enough to read the real thing. /sarcasm
http://www.esa.org/pao/policyStatements/Letters/ESAResponsetoPublicAccessRFI2011.pdf
I welcome readers thoughts on this idea. – Anthony
=====================================
Why shouldn’t peer review evolve like everything else?
The system that is in place right now, is simply because of the paper work (literally) involved.
We did away with the paper work years ago………