Dr. Roger Pielke Jr. recently submitted this paper to Geophysical Research Letters (GRL):
A homogeneous database of global landfalling tropical cyclones
Jessica Weinkle* and Roger Pielke, Jr.
Center for Science and Technology Policy Research, University of Colorado, 1333 Grandview Ave, Campus Box 488, Boulder, Colorado 80309
Abstract
In recent decades, economic damage from tropical cyclones (TCs) around the world has increased dramatically. Scientific literature published to date is strongly suggestive that the increase in losses can be explained entirely by increasing wealth in locations prone to tropical cyclone landfalls. However, no homogenized dataset of tropical cyclone landfalls has been created. We have constructed such a homogenized global landfall TC database. We find no long-term global trends in the frequency or intensity of landfalling TCs for the period with reliable data, providing very strong support for the conclusion that increasing damage around the world over the period(s) of record can be explained entirely by increasing wealth in locations prone to TC landfalls, and adding confidence in the fidelity of economic normalization analyses.
Seems straightforward enough. It came back with two reviews, both with some corrections, one reviewer suggesting publication without major caveats, the other grudgingly suggesting publication to the editor, Noah Diffenbaugh, and asking for revisions. So far so good (you’d think). But it starts getting weird from here. Pielke Jr. asks this set of questions:
As the editor what would you do?
A) Provisionally accept the paper pending a revision that meets the editor’s judgment of responsiveness
B) Provisionally accept the paper pending re-review by the two reviewers
C) Reject the paper
D) Reject the paper and tell the authors that any reconsideration of the paper would have to be accompanied by a detailed response to the two reviewers followed by selection of new reviewers and a restart of the review process
If you picked (D) then you too can be an editor at GRL.
Read the whole bizarre peer review story here.
GRL = Gatekeepers ‘R Loyal ™
John
[snip -uncalled for- Anthony]
James Sexton says:
September 27, 2011 at 6:45 am
Steve from rockwood says:
September 27, 2011 at 6:22 am
At what point in history was GRL run by a group of virgins dedicated to the advancement of science? And can someone point me in the direction of a journal that does not have it’s own internal mandate?
=========================================================
Then, don’t you think our scientists, and editors and reviewers (allegedly some the world’s greatest minds) would have the intellectual honesty to state as such? This has gotten to point of lunacy. Publish or perish? But forced to publish in some misanthropic journal or another? What ever…… we need to quit lending these organizations credence.
=========================================================
For many years now I have approached most of the scientific journals as an industry onto themselves. Yes there are some important papers by credible scientists doing original and ground-breaking work. The other 99% of the papers are by plodders adding to their CVs. To ask for intellectual honesty given the position that publishing scientists now find themselves (publish or perish) is not easily possible.
Another problem I have seen in a few journals (Geophysics being the one I am most familiar with) is the tendency to go off on tangents and over-represent a specific area of the discipline, for example three dimensional inversion (of magnetics, gravity and more recently electromagnetics). At first blush there are genuine ground-breaking advances (much it was the advent of high speed computing). But as the years roll by more and more of these papers (mostly of marginal value) to get published by the same group of authors, one or more often appearing on the editorial board of the journal. These later papers form the great bulk of the total publications and rarely offer anything new.
Tenure and grants are closely linked to the number of publications rather than quality which is not easily measured.
I’ve drifted back to text books, a handful of “ground-breaking” papers (many from the 1960s and 1970s) and the Internet. The only subscription I have these days is to a vintage car magazine and I am grateful that these people publish out of a love for their craft and a genuine belief they are helping others to learn.
Stunts like this just serve to make me more untrustworthy of climate science. I’m sure I’m not alone in this…
Steve from Rockwood
I know exactly where you are coming frombwith this comment – it is a simple fact of Academic existence today that you must get a lot of work published (because papers make for more research money and a better reputation for the department). What is the best way of getting lots of publications? Certainly isn’t to undertake speculative and potentially ground-breaking research (which could make you a big reputation, but where the chances of anything publishable are very small given the amount of time required for the research). Much more efficient to simply tinker at the edges of an established field.
One of my Masters professors had the simple method of using a pro forma as the basis for the majority of his papers that he could simply slot in newly obtained (geochemical) data from different sites or different phases of his fieldwork. OK, so it got some data into the public domain, but did it really advance the understanding of the science?
George E. Smith says (September 27, 2011 at 12:09 pm): “As for New Orleans; who but the French would build a city below sea level…”
The Dutch? 🙂
I’m a bit off topic with this post but with regard to:
Steve from Rockwood says: September 27, 2011 at 2:16 pm
and: Ian B says: September 27, 2011 at 3:20 pm
And there in a nutshell; “publish or perish” along with how little value teaching is accorded, is why I decided to stay out of academia even tho I loved the academic world, research, and teaching. I’m dating myself, but in grad school in the mid 80’s at a major/top university, I was in the professors’ office areas enough to hear about and see the frequent grief one particular professor was subjected to because he wasn’t bringing in ‘enough’ grant money. The man published a few excellent solid papers each year, but because his area of interest wasn’t ‘hot’ at the time, it was far more difficult for him to publish, there was far less grant money available and what was available, was far more difficult to get. Everyone (e.g., Dean, peers, students) liked him, and his research was generally thought to be very high quality as were his papers. But that wasn’t enough and he was always on the verge of being let go.
So even with multiple requests from my Dean to stay on for a PhD, I opted to stop with an M.S. an enter my field as a highly qualified individual (in my discipline, at the time, a PhD made you “too qualified” for positions anywhere but academia or National Labs).
Teaching, research, putting research together for presentation to peers & working the scientific method? AWESOME Chasing grants, dealing with highly biased publication issues, and huge problems if your area doesn’t happen to currently be ‘hot’ or your findings run counter to the current paradigm? BLECH
re post by: Leif Svalgaard says: September 27, 2011 at 5:51 am
Leif, did both of your reviewers recommend publication? From your description, it sounds like one did and one didn’t – if so, I would think that’s a different animal than Peilke’s, where both reviewer’s recommended publication.
There is a discrepancy in definitions of Category 3:
Roger quotes:
“Science Category 3: The paper is publishable in the refereed literature but is unlikely to become a Category 1 paper. For example:…”
I just went to the GRL site, and copied their definition of Category 3:
“Science Category 3. Science is sound and paper is publishable in the refereed literature but is unlikely to become a Category 1 paper for GRL. Possible reasons are:”
Perhaps the version Roger quotes is the ‘shorthand’ version sent in replies?
In any case, the version off the web site, IMO, makes it clear that Category 3 is publishable, somewhere – but probably not in GRL. That’s pretty much what the reviewer said, too ‘perhaps just publishable, but only because of the useful new content on adjusting loss data,’ and there are other major issues that he does’t like and that need to be dealt with.
And his category 3 ranking means “I don’t think a revised paper will become GRL category 1, but perhaps.”
BTW, I once had a paper rejected from J. Neuroscience, despite lukewarm ‘accept with minor revisions’ responses from 3 reviewers. Basically the editor thought the work sounded potentially new and interesting, but enough outside his field he was not able to judge the potential impact. The reviewers said the paper was sound, the work of high enough quality to be accept in J Neuro., of sufficient general interest, just barely – they made it clear this was a good paper but probably not at the impact level the journal wanted. We were then rejected by the editor, who wanted higher impact papers.
He was right. We published elsewhere, the paper has been cited a dozen times, exclusively for the data in it, and then dropped out of view.
re post by: Steinar Midtskogen says: September 27, 2011 at 5:03 am
I would think this probably happens often. If it were the case with Peilke’s submission, however, it seems to me that the rejection would state that problem, and that they’re not currently interested in publishing a paper with that particular content. Instead, they said it was promising and just needed some specified minor revision’s according to the reviewers, and unspecified major revision according to the editor – but then it would be of interest. So I don’t think the reasoning you’re mentioning fits this situation. Unless, of course, GRL just likes to lead researchers on and waste their time – which I seriously doubt.
re post by: Mike Jonas says: September 27, 2011 at 5:10 am
Thanks for digging out and posting that bit of history on Tol. It certainly may help explain his comments.
re post by: Pascvaks says: September 27, 2011 at 6:05 am
Except for your suggestion to actually work, one would also have to know how and manage to set up a peer review system, then have the time to actually manage the process including the actual website itself. I don’t believe the process is nearly as simple as you suggest, not once practical considerations are taken into account. I wish it were tho!
From climatereason on September 27, 2011 at 12:22 am:
If only it would be that simple. I’m speaking from still-flood-ravaged Central Pennsylvania, which is set to get some more flooding right now with an inch or so of rain today on top of the soaked ground, although nothing like the “Great Flood of 2011” three weeks ago that arguably surpassed our previous benchmark, Hurricane Agnes in 1972. Personally I’m on a hilltop, the flood wall for the “small town city” below along the Susquehanna River held, but it looked dicey for awhile and between the bridge and road closings we were nearly cut off. I know people who had flooding issues, some nearby roads are still closed.
1. Many people live where flooding is possible (if not flood-prone) because it is cheap. A common example around here is streams and creeks that “rarely” overflow their banks. We’ve seen a lot of that lately, and it’s a mess when they do. Nearby between Northumberland and Sunbury, which are on opposite banks of the Susquehanna (Norry being where the two main branches join on two sides of it, Sunbury *just* being past the intersection), there’s a strip of land called Packers (or Packer) Island. In 1972 you could only tell it was there because you could see the bridges going to where it should be. Down river, next to Selinsgrove, is the Isle of Que. Local joke, TV says it has flooding, when don’t they have flooding?
Also nearby Bloomsburg, home of Bloomsburg University but mainly the world-renowned annual Bloomsburg Fair held on the last week of September, got hit hard. Seasonal flooding of the Fairgrounds is almost normal, in Spring. They were hammered, first time in 157 years they canceled the Fair. And as I heard from someone who knows the area and the people, there’s wondering as to why they built low-income housing in one of the worst areas.
Common denominator: cheaper land, cheaper housing. Sure, there are those who’ll have their multi-million dollar beachfront homes, their trendy riverside vacation properties. But by and large, there’s just many people living where they can afford to live. And as populations have gone up over the decades, that means more people where flooding can *sometimes* be a problem. Sure, it doesn’t seem to make much sense, quite possibly cost them more in the long run. But when living paycheck to paycheck, the monthly costs alone define what is affordable and possible.
2&3. These go together. Standards have changed over the decades. It’s nowhere near as simple as mounting outlets higher. Homes are built tighter, with plasterboard, and many closed-off spaces which could have water-absorbing insulation (outside wall). Where our ancestors would’ve just cleaned off the plank walls and floors and waited for them to dry, now the walls are stripped down to the studs and exterior sheathing over fear of toxic mold, with much spraying of chlorine bleach. Plus the ripping out of the installed wall-to-wall carpet and padding. All that drives up the restoration costs considerably.
Then there are things that make less sense to me. In the many pictures I’ve seen of former home contents piled outside waiting for disposal, there’s a lot of wood furniture. Modern cheap particleboard and plywood construction aside, my grandparents would’ve just cleaned off those solid wood pieces and kept using them. There was also a lot of plastic and metal housewares that just needed cleaning. Etc.
The “professional restoration” people likewise said interesting things. Air ducts need replacing. If furnaces were in water, it’s no longer acceptable to just clean them up, Code (code office, building codes) says all the electrical parts need replaced. Likely the furnace will just be replaced. (Guess it’s a given that electrical panels, some wiring, maybe even the meter base will need replaced.) As something that makes even less sense to me, if that “toxic river water” even touches the base of a water heater then it must be replaced, as the potable water inside could get contaminated. Excuse me, but the only water-related thing on the bottom is the drain valve, which is by default closed thus can be cleaned without contamination, the water lines are fed in from the top. By their reasoning, if that “toxic river water” touched a clothes washer hookup, or a sink faucet, you’d have to re-plumb the entire house.
The increasing costs are a sign of increasing wealth, but not necessarily in a good way. People now have more costlier stuff to lose, like modern appliances and electronics. We have housing that costs more, but the standards for restoration have grown adding additional costs. And, we also have insurance, leading to items being replaced that could have been salvaged, perhaps easily saved, and also to having things that would have been considered too dear to own if the would-be owner knew they’d have to shoulder 100% of the replacement costs when inevitably replaced.
It’s not that simple at all.
This is a typical GRL reply when one of the reviews is negative or even indifferent. It’s no big deal. Just do the work and resubmit.
The “situation” is largely the result (judging by his publication list) of the author’s unfamiliarity with how GRL works.
re post by: Richard Tol (@RichardTol) says: September 27, 2011 at 6:22 am
I think most would agree that all papers should be treated equally. The simple fact is that they’re not, because we are human. Try reading some of Ioannidis’s published research on these issues, they are a real eye opener.
Your suggestion that we can check your journal for bias, however, is ludicrous almost to the point of being insulting. Obviously the only way we could really check for bias is if we were able to have ready access to full papers – of ALL the papers that were submitted – both those published, and those rejected, along with all of the associated peer review comments and reasons for rejections. Short of that, I suppose if a run thru published papers showed that many contradictory papers were published, it would give a small positive indication in your favor, but that’d be about the extent of it, and it certainly wouldn’t begin to show that the journal doesn’t have publication bias involved in it’s selection process.
re post by: Jeremy says: September 27, 2011 at 7:35 am
Well said!
re post by: Septic Matthew says: September 27, 2011 at 10:25 am
When did reasonably politely asking for clarification of requirements, especially when contradictory statements were used, become a temper tantrum? Or as I believed someone else catagorized it, ‘picking a fight?’ Particularly when the contradictory terms are so according to the published definitions of the organization involved?
If the issue was a simple one of miscommunication because of the stock letter terms, all the journal would have need to do was reply along the lines of ‘just make the changes requested by the reviewers and resubmit, that’s all we’re referring to.’ Presto, problem solved. Instead, they refused to explain what they acutally meant and what they actually required.
Rational Debate says:
September 27, 2011 at 5:50 pm
Leif, did both of your reviewers recommend publication? From your description, it sounds like one did and one didn’t – if so, I would think that’s a different animal than Peilke’s, where both reviewer’s recommended publication.
One reviewer gave a rating of 1A [the very best], and the other of 4C [the worst – meaning unpublishable in any form]. GRL does not use the category ‘major revision’ anymore, So if one reviewer asks for ‘major revision’, GRL will reject the paper, but encourage you to rework it and submit it as a new paper. I concur with other posters that have commented that Pielke, Jr, simply did not know how GRL works. He should have bit the bullet, reworked the paper a bit, and tried again. Whining to the editor will not help.
When I wrote my first research based thesis, from beginning to end, my prof did not tell me how to write it. Not the proposal and not the final article. I must have submitted 30 versions to him. He kept telling me to read other research articles that had been published and to completely rework mine to conform to the standard, without telling me how. It was infuriating. I called him every name in the book and wanted to chop up the tree I killed submitting the paper to him and make him eat it. In the end, it was the best possible thing he could have done for me. A major researcher read my final thesis and described it as a “gold mine”. I credit that damned professor for that result.
Rational Debate: When did reasonably politely asking for clarification of requirements, especially when contradictory statements were used, become a temper tantrum?
When Pielke Jr. whined about it on his blog.
Rational Debate: Instead, they refused to explain what they acutally meant and what they actually required.
Reviewers, working without pay, provided an itemized list of defects requiring corrections, as they have have (collectively, not just these 2) for thousands of papers. Then they went on to review the next papers on their worklist. The standard response, of which Pielke Jr. was reminded, is to revise, resubmit, and include a detailed list of the original objections and how they have been fixed. They couldn’t rewrite the paper for him.
Sorry, Leif Svalgaard says:
September 27, 2011 at 7:44 pm
Perhaps that is a rational choice in a normal world. That is not the case today, editors are all too frequently throwing up roadblocks where none appear warranted. In this case, both reviewers said publish. It was the individual editor, perhaps not wanting to be forced to resign, that pulled the black ace that said go somewhere else, don’t poop on my porch.
You can rationalize it, but the bottom line is that GRL did not want the product and any publicity it might generate. To me, its a degeneration of the Science that you live bye.
Defend it, but it’s rapidly going bad in front of our eyes. And your accommodations assist in that end result. Your attitude and apparent role is something called an enabler, in other halls of science.
I have to agree with Richard Tol’s analysis, “The verdict by Referee 2 is clearly negative: You oversell.” as the second review was clearly negative as the following wording was used,
“For that reason it is perhaps (just) publishable”
“claims of a new homogeneous database (based on JTWC outside of the US) are grossly over-stated“
“This is especially so in regard to intensity, which the authors treat fairly simplistically in any case”
Seriously, how can you not read those comments as negative? [I am not arguing whether these comments accurately reflect the contents of the paper]
The editor took these criticism upon himself to make a decision to ask for a “major revision”. The editor pointed to a definition that used the phrase “or other significant changes”. This phrase is subjective and thus it allows the editor to make a judgement call as to whether he felt the second reviewers comments met this criteria for a “major revision”. He felt it did. I don’t see anything outrageous here. Either resubmit (too late for that) or go to another journal.
I could not find anything published by Roger in GRL before but saw his dad has published multiple times there.
waltmeier says:
September 27, 2011 at 10:29 am
In his comments on his post, Roger says that the 2nd reviewer gave it a Science Category 3.
As someone who has reviewed several GRL submissions, I have always taken the Science Categories as a ranked order, i.e., “1″ is most publishable, “4″ is least publishable. When I give a manuscript a “3″ science category, I would expect major revisions. Essentially, GRL will only publish papers that can be Category 1 after revision. Category 3 states that it is publishable in some journals, but is not likely to become a Category 1 for GRL.
I agree that the reviewers comments are a bit muddled – pointing out many major issues, but then seeming to suggest that a change of title might address it. However, I can see where the editor would interpret those comments in a negative light, feel that a title change would not be sufficient, and suggest a “major revision” with a resubmission, particularly in light of the reviewer’s “3″ rating. Basically, as a reviewer if I give a paper a 3, I’m essentially saying “I think this this paper should probably be rejected, but if other reviewers and the editor disagrees, I’ll accept that my view is in a minority.” Category 4 is either very harsh – basically, “this paper is rubbish” – or it is not an appropriate topic for the journal.
I do agree that the editor should be responsive to Roger’s questions and provide useful feedback. Below is the full description of the Science Category 2 and Category 3….
Walt Meier
Professor Meier, I think you are missing the point Roger Pielke, Jr. is trying to make. He never said that the paper did not need revision, or even that he was unwilling to make revisions. He wrote to the editors to specifically find out what revisions he needed to make in order for the paper to be published, and the editors were unable to answer that question. He was willing to make all the revisions the reviewers suggested, and was willing to make more, if the editors would be specific about the revisions they wanted. The editors were unable to tell him what it would take to make his paper publishable in their forum.
Surely, they must have had some specific reason or aspect of the paper that they thought was what made it unpublishable. But they were not able to name it. That is more damning to their cause then just rejecting the paper. By just saying, “we reject your paper,” without giving cause, nothing could be conjectured about the reason. But by saying they would be willing to accept the paper if changes were made, then not being able to articulate those desired changes, they made themselves and their journal look foolish.
Leif Svalgaard says:
September 27, 2011 at 7:17 am
Of course, the process is not perfect, but like Democracy the best we have got.
Is it, or is this just what the research community believes?
I think it should become more acceptable to publish papers on the web (I don’t necessarily mean on personal blogs or the like, but usually through one’s organisation, research faculty, etc). Academic libraries could store copies and print as needed, while journals could be excellent places to announce and discuss new papers (and if they don’t, the paper is still out there). There are many practical reasons. This doesn’t stop anyone from going through a proper review process, though it’s less enforced. Researchers will still be interested to ensure the best quality of their papers, and to address possible criticism or problems. The community should also think less of getting published as a mark of quality. Too much prestige is involved. As you say, peer review isn’t perfect, and it can hardly be the task of the reviewer to replicate and verify the results, anyway. Good papers will be recognised (eventually). This could also reduce the need for adding “honorary authors” to a paper. I believe many “authors” in papers at best belong to an acknowledgement section instead.