Dr. Roger Pielke Jr. recently submitted this paper to Geophysical Research Letters (GRL):
A homogeneous database of global landfalling tropical cyclones
Jessica Weinkle* and Roger Pielke, Jr.
Center for Science and Technology Policy Research, University of Colorado, 1333 Grandview Ave, Campus Box 488, Boulder, Colorado 80309
Abstract
In recent decades, economic damage from tropical cyclones (TCs) around the world has increased dramatically. Scientific literature published to date is strongly suggestive that the increase in losses can be explained entirely by increasing wealth in locations prone to tropical cyclone landfalls. However, no homogenized dataset of tropical cyclone landfalls has been created. We have constructed such a homogenized global landfall TC database. We find no long-term global trends in the frequency or intensity of landfalling TCs for the period with reliable data, providing very strong support for the conclusion that increasing damage around the world over the period(s) of record can be explained entirely by increasing wealth in locations prone to TC landfalls, and adding confidence in the fidelity of economic normalization analyses.
Seems straightforward enough. It came back with two reviews, both with some corrections, one reviewer suggesting publication without major caveats, the other grudgingly suggesting publication to the editor, Noah Diffenbaugh, and asking for revisions. So far so good (you’d think). But it starts getting weird from here. Pielke Jr. asks this set of questions:
As the editor what would you do?
A) Provisionally accept the paper pending a revision that meets the editor’s judgment of responsiveness
B) Provisionally accept the paper pending re-review by the two reviewers
C) Reject the paper
D) Reject the paper and tell the authors that any reconsideration of the paper would have to be accompanied by a detailed response to the two reviewers followed by selection of new reviewers and a restart of the review process
If you picked (D) then you too can be an editor at GRL.
Read the whole bizarre peer review story here.
KnR: From a scientific point of view just saying its wrong but not why is meaningless , his a journal editor not 3 year old refusing to share their toys and just screaming no I won’t
You do not seem to get the situation, as Leif Svalgaard noted above.
KnR says:
September 28, 2011 at 2:40 pm
you just like the editor simply can not state what this major revision is , you got an excuse they have not .
You are still not getting it. The proper action for the authors would have been to rewrite such as to meet the objections of the referee. If those objections are week, the rewrite is easy. Then simply resubmit. This is the way it works and we all must conform to that [if we want to publish in GRL].
Steve, none of what you say is wrong…GRL just operates differently. The ‘some caveats’ included what many perceive as, at minimum, indifference (“just publishable”)… To which the editor’s reply us basically: “ok, c’mon, give us more…how badly do you want it”. But with every opportunity to resubmit. They’re always looking for a reason to initially bounce a paper. It’s a broad filtering mechanism, different from the other AGU journals.
What’s different is Pielke’s response, presumably the result of inexperience. Or rather, what would normally occur as grumbling over beers, he’s chosen to escalate up to Calais and then publish on his blog. Different times, I guess.
It seems to me for the reviews that there are two ways to address the reviewers (mainly #2).
1. Justify the reviewers definition of “homogenous database” which would require a significant rework with significant new content — i.e. not a revision major or otherwise.
2. Dial back claims of “homogeneous database” and change title to reflect this. — It seems to me that this doesn’t fall under the GRL definition of major revision.
I’m not sure Pielke’s reaction was reasonable either. Pick #2, resubmit and see what happens. I’m also guessing that there was a little “even if we have to change what peer review means” going on here based on the editor’s seeming desire to ignore the reviewers recommendations. I’ll comment here that although review #2’s comments were less than glowing he gave it a 3A rating and the other review gave it a 1B.
DRE, A means it met the format requirements. 3 means the content needs (major) work;. See Walt Meiers post above. Others, who have reviewed for GRL, have confirmed his understanding. A 3 for science ain’t so hot.
re post by: Leif Svalgaard says: September 27, 2011 at 7:44 pm
Thanks for the reply Leif. I hadn’t see the rating category descriptions when I first posted to you (thanks to those who posted those). Even so, my understanding was that Peilke had no problem with the reviewer comments – he was thrown by the form letter stating “major revision” with no explanation of what aspect of the paper was being referred to, as the reviewer comments were minor in nature. It seems to me that the editor, on being queried, ought to have simply replied that it’s a form letter with outdated wording, and the only revision being required were as noted by the reviewers. If they had, I suspect Peilke would have just made the revisions and resubmitted.
So, why the stonewalling and ambiguous responses by GRL, rather than a simple explanation to an honest query from someone trying to meet their requirements?
re post by: Pamela Gray says: September 27, 2011 at 7:56 pm
I’m glad you can take such a positive view on such treatment – I doubt I’d be so charitable. Most people (including yourself I’d bet) at the point of working on a thesis are perfectly able to learn from general guidance and corrections – gross trial and error isn’t necessary. Most I suspect have already read large numbers of published papers, and would be perfectly willing to read another set while contemplating form and standards, and learn from that also – without having to go thru trees and time doing it incorrectly over and over. Frankly, by the time I started on my thesis, I’d already been taught reams about what was acceptable, what wasn’t, basic standards and form, etc. I’m rather surprised that you got to that point without having already had much of that pounded into you.
For everyone still complaining about this answer this question,
Is the phrase “Major Revision” subjective?
Pielke and everyone else complaining is hung up on a phrase that anyone can interpret however they feel like – including the editor.
re post by: Septic Matthew says: September 27, 2011 at 8:34 pm
Oh, I see. So by definition, all whistleblowers or anyone factually reporting how an agency, company, journal, government, policeman etc., treats them, are to be despised as whiners throwing temper tantrums. Interesting world you live in.
re post by: Septic Matthew says: September 27, 2011 at 8:45 pm
Your entire reply was a nice strawman that has nothing to do with the issue. Peilke stated explicitly that he had no problems making all of the corrections the reviewers specified. He asked for clarification of the editor’s comment which was contradictory to the reviewers.
No it was NOT! The second reviewers comments can be interpreted as requiring a “major revision”.
re post by: Richard Tol (@RichardTol) says: September 28, 2011 at 4:08 am
Thank you for the reply Richard. There really is no way to tell if there is publication bias simply by scanning published titles, or even reading a number of the published papers. One probably could pick out a problem that way if the journal covered a very limited scope, and there were some major well known controversy and one side was obviously excluded. But you know as well as I do that often it’s impossible to tell from paper titles just what ‘side’ the research may come down on – and the more diverse the scope (which your journal certainly appears to be), the more impossible it becomes, especially if that journal is covering areas that don’t happen to be your own speciality.
Even if it IS your speciality – it would still be almost impossible to pick out publication bias without knowing what papers are being rejected, and if there are any patterns over time to those rejections. Although I grant you that over time folks in a particular industry or academic field often pick up on this sort of thing and it becomes “known” around the coffee pot.
You mention no political bias in your journal… but of course the political spectrum isn’t the only area that can be involved in publication bias – leanings one way or the other with regard to competing scientific hypothesis about particular subjects can obviously be another, or a preference for articles about certain subjects vs. others, and so on can all be forms of publication bias. It is also entirely possible for well meaning people who believe they are not expressing any bias to actually be quite biased without realizing it.
Point is, that to really be able to tell if a journal has publication bias issues, one would have to put in a good bit of work researching it, AND have access to details about what gets rejected and why.
I’m curious if you’ve read any of Ioannidis’s work on these issues? Obviously he’s evaluating the medical research field primarily, but his work is applicable to all research and associated publication issues – it’s human nature issues that he’s really covering. Particularly since you are a journal editor, I would very highly recommend that you take a look at his work if you haven’t already. I think you will find much of it quite interesting.
Rational Debate says:
September 28, 2011 at 8:45 pm
It seems to me that the editor, on being queried, ought to have simply replied that it’s a form letter with outdated wording
This is not ‘outdated’ wording, but the newest policy of GRL. Anything more than ‘minor revision’ is automatically rejected [and possibly invited to submit]. This is standard procedure and is clearly explained in the link provided by the editor: http://www.agu.org/pubs/pdf/Editorial_GRL.pdf
re post by: populartechnology says: September 28, 2011 at 8:56 pm
And therein lies the problem – far too much in this issue is/was subjective, vague, or very ambiguous. The reviewer’s comments, the editor’s replies, the standards, etc., can all be interpreted quite differently when read by otherwise intelligent and reasonable individuals.
and @Poptech says: September 28, 2011 at 9:22 pm — there’s that darned “3” in the rating that caught me again I guess. So with that included, I would agree it could be said that reviewer 2 suggested a major revision – but even that is ambiguous, because his comments imply that simply downplaying the degree of homogeneity, and adding toward to the title would be sufficient. Things which, as Peilke said, could be done in a day’s turn around time which wouldn’t have slowed GRL’s “fast turn around” process at all.
Assuming, as Christoffer Bugge Harder says: September 28, 2011 at 2:32 am, that it is considered a major revision to make a simple word change in the title that doesn’t significantly change the content or import of the paper, as compared to a change that radically alters things or that requires significantly more work/research… then the whole thing is even more ambiguous.
Then, as others have already mentioned, there is the bugagoo of how Dressler’s paper/comment was treated in comparison. A comment which would have been more properly submitted to RS to begin with.
re post by: Leif Svalgaard says: September 28, 2011 at 9:45 pm
I stand corrected Leif. When I read the editor’s reply to Peilke, which stated “GRL has not been using “major revisions” for several years now…” I mistakenly took it to mean they no longer use the term – and, my bad, I didn’t go read the link to the actual policy.
From the beginning I’ve thought that were I in Peilke’s shoe’s, I’d’ve probably gone ahead with the easy changes and resubmitted (playing the devil’s advocate taking issue with flaws doesn’t negate that)… That said, I still feel that it would have been far more reasonable and productive – for everyone involved – if the editors had simply tried to provide a little more clarity once asked.
@Leif Svalgaard says: September 28, 2011 at 9:45 pm
p.s., Thanks also for supplying the GRL link in your reply!
Rational Debate says:
September 28, 2011 at 10:23 pm
if the editors had simply tried to provide a little more clarity once asked.
To my mind, the editor was crystal clear and provided a supporting link. But, then again, I don’t feel threatened by conspiracy thinking. Nor, am I a whiner.
Agreed, the problem here is everyone is reading this review in the same light as Spencer and Braswell. I don’t feel the review is being handled remotely like the treatment they received as the editor did not reject it but asked for a revision and a re-submission. Remember Pielke Jr. is NOT an “AGW skeptic”. Half the people jumping into this probably do not know this.
If you read the review carefully you can see the language that someone can interpret to mean more than simply changing the title,
“Reviewer 2: The work seems essentially sound and useful to the community but lacks in-depth analysis and illustration. It does confront the issue of continued misrepresentation by some of the impact of “climate change” on presently experienced insurance and other losses from tropical cyclones. For that reason it is perhaps (just) publishable but claims of a new homogeneous database (based on JTWC outside of the US) are grossly over-stated as there is much work needed before that can be genuinely claimed. This is especially so in regard to intensity, which the authors treat fairly simplistically in any case. I would like to see that aspect down-played and perhaps the title adjusted to read “Towards a homogeneous database …” or some such.”
What happened here is the editor read these as requiring a “major revision” or just quickly used the phrase not knowing Pielke would take such offense to the word “major”. Pielke took offense [he will argue this] as he felt that simply changing the title or such would resolve all these issues and did not feel it was “major” or whatever but this is irrelevant as he is essentially trying to tell someone else that they cannot subjectively view something the way they do – this is pointless.
These are two very different issues as this review should be judged independent of Dressler’s since Pielke had nothing to do with the other. Spencer should submit his comment to Dressler in RS, LOL. He probably has a better chance of getting it published.
Rational debate, DRE, KnR + others
“2. Dial back claims of “homogeneous database” and change title to reflect this. — It seems to me that this doesn’t fall under the GRL definition of major revision”.
Could any of you who insists that the objections from reviewer #2 do not constitute a “Major revision” please answer this question: Is there no major difference between
A. launching a new, truly homogenous database on tropical cyclones and presenting groundbreaking new results showing a clear trend (or a lack thereof), or
B. merely trying to move TOWARDS this desirable goal and presenting some preliminary results?
Again, to my mind this is not a difficult decision at all, and nor do I see how the verdict from the editor is in the slightest way subjective or unclear: Change the title and the manuscript so it reflects the fact that you are just taking a step on the way (rather than having reached the goalline) – then it is perhaps (just) publishable. Then resubmit following the universal instructions here (http://www.agu.org/pubs/pdf/Editorial_GRL.pdf).
I honestly do not see the problem, and seeing any kind of evidence of unfair and evil peer-review gatekeeping in this would certainly require conspiratorial thinking to a point which should embarass Alex Jones………..
@RationalDebate
I’m aware of that literature, even made a small contribution to it (Tol, 2009, JASIST).
Abstracts are in the public domain. If you read the abstracts of the six most recent papers on the EU ETS in Energy Economics, you will find a diversity of opinion that is in sharp contrast to the consensus among the political elite.
As I said, I am prepared to open the journal’s archives to scientometrists. (Not to all and sundry because there is a lot of confidential material, including accusations of plagiarism.)
Leif – your misreading of the words as there is much work needed betrays your bias and unreasonableness.
Of course reviewer 2, having just recommended “perhaps publication”, meant to say that Pielke Jr’s work was a step towards building the “homogeneous dataset” (and not that the authors were supposed to do all the “much work” by themselves). After all, if much work is needed it would make no sense to prevent publication to papers going in that direction: otherwise, there will never be anything approaching a “homogeneous dataset”.
Anyway, your refusal to list the major revisions needed is enough. Thank you.
Poptech – your claims would make sense had the editor used them to reply to Pielke Jr. For mysterious reasons, he did not. Since both reviewers implied publication and the editors refused it, it’s up to the editor to detail why. If there is language that someone can interpret to mean more than simply changing the title then that someone should explain their interpretation.
I can relate to what Pamela Gray said, except that my experiences were at work, not so much in college (except for my wonderful English composition professor from my freshman year).
I don’t know enough about how scientific journals work to comment directly one way or another on this situation, but my own experience with the fussiest bosses (who constantly drove me crazy) when it came to writing legal opinions, research papers, and the like, convinced me that it improved my writing to simply revise and resubmit, just as Leif Svaalgard said in his posts, and suffer through the process as best as one could.
Christoffer Bugge Harder : Change the title and the manuscript so it reflects the fact that you are just taking a step on the way
My last peer-reviewed paper is dated 2005 (no worries, nothing about climate) but I can’t recall having to go through any hit-and-miss guesswork with the reviewers and the editors. They told me what they wanted to see changed, and I changed it accordingly.
Vague pronouncements followed by an editorial refusal followed by the same editor failing to communicate the changes needed, well, that all sounds distinctly unprofessional.
That said, when I have sent a climate-related commentary to Nature a couple of years ago, and the reviewers agreed it was worth publishing, the editors stepped in and binned the paper claiming it was not interesting enough (against the explicit judgement of one of the reviewers). So I am not surprised if lack of editorial professionalism is widespread in climate science.
Maurizio Morabito (omnologos) says:
September 29, 2011 at 1:38 am
If there is language that someone can interpret to mean more than simply changing the title then that someone should explain their interpretation.
Reviewer #2 is very clear:
The work seems essentially sound and useful to the community but lacks in-depth analysis and illustration.
So ‘in-depth analysis and illustration’ is lacking. Adding that is part of the revision called for. GRL’s procedure states “manuscripts are routinely declined if the reviews point to a need for additional analyses…”.
I would like to see that aspect down-played and perhaps the title adjusted to read “Towards a homogeneous database …”
Here the reviewer calls for additional text to ‘down-play’ the finding, in addition to changing the title. This is the second place where revision is needed. The notion that ONLY changing the title would be enough is thus false, and GRL in following their standard procedure should [and did] reject with the option of resubmission.
So, in short: GRL is dealing fairly with the paper, applying the same rules to it as to all other papers. My latest GRL paper got the same treatment. The proper reaction is to revise and to resubmit. Not to whine about unfairness.
Your other comments on bias, unreasonableness, refusal, etc should be beneath serious discourse, but, then, I realize that some people have lower standards than I.
@Alfred Ledner. I see your point Ted and wonder if publications like GRL take the blogosphere into account (perhaps leaning away from submitters who are active bloggers)?
A different world for sure today and one where real contributions probably can’t stay hidden for long.