Guest essay by Abzats.
The most exciting period in science was, arguably, 1895-1945. It was marked by discoveries that changed the foundations of modern science: X-rays, quantum mechanics, superconductivity, relativity theory and nuclear energy. Then, compare this with the next 50 years in science. Incomparable. Nothing of that scale or impact. Yes, technology has advanced, but fundamental science – has come to a crawl. Have you ever wondered why? What changed as the 20th century grew older? Among other things, research budgets and the number of PhDs increased exponentially. This cannot be bad.
Well, it can. All depends on the rules of the game. And they have changed. The change went largely unnoticed by the general public. In this article I will try to bring everyone up to speed. I will explain to non-scientists the “business model” of modern science. People may want to know. After all, scientists are burning public money, billions a year. And, I am quite sure, those who get my message will react with “you cannot be serious!” And leaders of organized crime will be pulling their hair out in despair: “why did not we think of this first?”
Single most important element of the modern science machinery is the peer review process. It was introduced a long time ago, but it took over the scientific community at about mid 20th century. Why is it important? Every scientist must publish his or her work. If you do not publish, you will not advance your career. This works the same way as it does, say, for a businessman – if you cannot close a single deal, you are finished. Most journals have adopted peer review policies. Peer review process is also standard for research grants competitions. It is also the foundation of the tenure and promotion process at universities.
Well then, what is it exactly? To save time, let me explain peer review of papers submitted for publication in scientific journals. Once a journal receives a manuscript the journal sends it to 2-3 reviewers, who are experts in the field. Each reviewer writes a report that includes a recommendation on whether or not the manuscript should be published and advice to the author on how the manuscript can be improved. So far so good. Nothing seems wrong. This should work wonderfully. Well, in theory only. In reality it does not. In reality it is more of a disaster.
Let me explain. All the reviewers are anonymous. That is, they know your name but you do not know theirs. This is the first red flag: unless you plan to do something really bad, why do you insists being anonymous? The second red flag is that none of them gets paid. Those who believe in Santa Claus will say, well, they are just nice people volunteering their time to help advance science. Those who work for a living will smell a rat. I can give you one reason: being a reviewer gives you power over other people. Some just enjoy it, others use it to advance their own agenda. Such as approve manuscripts that praise reviewer’s own research and reject those that criticize it.
The power reviewers have is enormous. Put yourself in author’s shoes. You worked hard for six months on a manuscript. Your work is brilliant, if you publish it, not only will you advance your career, it will make you a leader in the field. Then, the manuscript goes to a reviewer who just happens to be having a bad day. He browses through the manuscript for 20-30 minutes, does not like the name of the author (never heard of him, “wrong” ethnicity, or … whatever), and rejects the paper. Can you appeal? No. You can write an angry letter, but you cannot call your attorney. Because nobody is breaking the law. because there isn’t any.
They can ruin your career and drive research, often funded by the public, to a dead end, and they are not accountable to anyone. In such a system, for most scientists the best, or should I say the only, way to advance their careers is by kissing up to those in higher positions: in person, in manuscripts, and in the whole research strategy. This has been going on for decades. As a result of this “natural selection”, the scientific community has been consumed by cronyism. Parts of it are rotten to the core.
Let me give you one example. Last year I attended a Radiation Research Society meeting. It was held in Maui, Hawaii. Why? Obviously it is a great spot for a vacation. You will not find any major research centers in the neighborhood. If you are still thinking of defending this choice, get this – the conference was held at the Grand Wailea Resort. The thing about this place is that luxury here is obscene. It is a kind of place a bum would go to after winning a lottery. And, guess what, I believe I have seen a few. Never before had I seen an invited speaker at a major conference making bodily function jokes. Here I had seen more than one, including a recipient of a lifetime achievement award spelling a word for body waste and thinking it was funny. Do not get me wrong, I am not judging here. But, if he jokes at a preschooler level, would you trust him to be a reviewer of your work? Do I need to mention who paid for the event? Or, that it took place during the worst economic crisis in decades?
A couple of other problems. Reviewers have no real motivation to work fast. Here is what you would see when checking status of your manuscript on a journal’s web site: manuscript to referee, unable to report – sent to another referee, and so on, several times, for weeks and months. Nonsense. With all the technology available, a manuscript can be published within hours. But, no, it has to sit for weeks on somebody’s desk. Somebody who just does not care enough. Or, worse, someone who is interested in delaying the process. The reviewer may be working on exactly the same problem and wants his paper published first.
Another problem that extremely frustrates me as an author are suggestions reviewers make on how I should improve my manuscript. Originally, may be, it was a good idea – your peers offering you advice that will help you improve your work. But it all has gone very wrong. These days these are not suggestions or advice – these are demands. You change your manuscript exactly as you are told, or it will be rejected. I am a well established scientist, why do I have to take advice from someone who would not even reveal his identity or credentials? And, finally, this system is perfect for stealing ideas. After you submit your manuscript you have no control of who will access it. All you can hope for is human decency, and it is not always there.
This brings us to the root of the problem. People, including scientists, are flawed. Few will miss a chance to stab competition in the back and abuse whatever little power they may have. I am not the first to criticize the peer review process. But I am not. Criticizing implies it can be fixed. It cannot. It was a bad idea all along. Then, what can be done? There is no quick and easy solution.
But I know where to start – ban peer review. And I know this can be done, this nonsense can be dealt with. This is not brain surgery, this is all about leveling the playing field, making rules for fair and open competition. These problems have been solved in all other spheres. Only scientists for whatever bizarre reasons received a special treatment and the right to live in lawlessness. Which is so wrong, I cannot find words to describe. Science is one most important sphere of human activity.
Who will find cure from cancer? Who will prevent the planet from becoming uninhabitable? Scientists. Not those from the beaches of Grand Wailea. Real ones. I hope we can still find some and reverse, before it’s too late, the depletion of brains. Let’s get started. Ban peer review!
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Armchair sociology and anecdotes are inadequate arguments. Do some science, real science, and come back and report your findings.
I agree 100%. There is no recourse in peer review. Consider that Einstein had a very hard time, in the early days of his career, getting a position in a university, of any kind. Why? Surely not because of a lack of talent or effort. His papers were initially ignored by the scientific community. It took him seven years after publishing his first great paper to be recognized by the community as a leading scientist. This to the greatest mind of the century. This type of thing happens every day in the scientific world these days – great work is altered, sometimes rejected. All because of a system of authority called peer review. What is needed is a more open process.
Double-blind Peer Review won’t protect good researchers even if others don’t spot their work if the research is some radical like that crystallographer who won the Nobel Prize for Chemistry with his work on odd crystallographic symmetries. For years he couldn’t get published. For those who say journals should use paid Peer Review, no Pay Wall etc. That’s financially not possible- where would the money come from but deleting Peer Review is finanically possible- pay the small cost by Pay Ads. Peer Review is an inherently slow and faulty process. Apparently scientific papers don’t contain any maths so they don’t need to checked by an appropriate mathematician. No methods so no methodological analysis needed. Peer who aren’t strong in those area are that are needed. Why don’t universities or collectives of institutes have internal peer review- Peer Review wouldn’t needed at all then ?
CodeTech @ur momisugly 2.09
So you think you could explain how a computer worked in 5 mins to at scientist from say 1940?
OK you have 5 mins to explain, the LCD screen, the transistor, the laser, integrated circuits, and the polymers the casing is made from, and then (unless it is Turing you are talking to) the whole concept of an ‘automated calculating machine’ and the fundamentals of programming. I wish you luck.
Bob Kutz says: @ur momisugly June 25, 2013 at 8:41 am
…..(Picture Mike Mann in a full zulu-mask dancing around a bunch of bristle-cone pine cores while grad students holding fluorescent light bulbs recite an al-gorian chant.) If you think about it; the reading of entrails and tea leaves to predict the future is a really good analogy for the ‘science’ of GCM’s…..
>>>>>>>>>>>>>>>>>>>>>>>
Actually the reading of entrails was probably a bit more scientific. Entrails being animal innards.
This comment comes to mind when ever I hear that observation on entrails
Given internal parasites like liver flukes, barber pole worms, hookworms, tape worms… can weaken, or in the case of barber pole worms and strongyles kill your prey animals or can infect humans, checking out the entrails could easily give a ‘Shaman’ clues as to the future health of his tribe or their prey animals. Therefore ‘the reading of entrails’ could be considered more scientific than GCMs in the hands of a knowledgeable shaman just as ‘the reading of marmots’ was.
Cloa5132013 @ur momisugly 4:09
“Why don’t universities or collectives of institutes have internal peer review”
Many do – it is common for someone about to publish a paper to give a departmental seminar on the subject – that sorts out really basic mistakes.
buggs says:
June 25, 2013 at 12:37 pm
Long time lurker, first time poster.
I have to say this is probably one of the least reasonable articles I have come across ….
>>>>>>>>>>>>>>>>>
That maybe true but it certainly got an interesting discussion going.
This comment thread exposes and clarifies a VITAL problem of which the AGW scam is but a minor symptom. Thanks Abzats.
MFG, omb
jimmi_the_dalek says:
June 25, 2013 at 4:13 pm
CodeTech @ur momisugly 2.09
So you think you could explain how a computer worked in 5 mins to at scientist from say 1940?
>>>>>>>>>>>>>>>>>>>>>
From The Computer History Museum
It was the engineering that was the problem not the ‘Science’ Babbage then devoted time into improving the quality of machine tools. (As a computer illiterate why am I the one pointing this out?)
Gail
I am asking if CodeTech could explain the working of the computer on your desk – not a Babbage machine – for which you will need transistors, lasers, LCDs etc etc – you probably could not even explain the LED indicator lights to a scientist from the 1940s (not in 5 minutes anyway). The point is that all of those are based on principles discovered after 1945.
j_the_d, maybe you don’t understand enough of the history of computing and the internals of a modern computer.
As I said, I’m talking to intelligent people in 1940, right? So obviously they understand the concept of a switch, and a relay, then the electron valve (vacuum tube). Extend that to the transistor, which in a digital system operates as a switch. It’s pretty obvious that once you have that basic understanding the idea of printed circuits is simple, and improved technology making ICs doesn’t require a great leap of understanding. The idea of a machine state toggle every clock is pretty simple to grasp, what would be fascinating is their reaction to my explaining how we can do that a few billion times per second.
The concept of step programming has its origins in loom cards, Hollerith cards used in the census, and player pianos, I don’t really see how you think someone in 1940 wouldn’t understand the same concept being implemented in the computer instead of using physical cards. The way we’ve broken functions down into logical modules would make it simple to understand the separation between storage, processing, memory, display, etc, too.
Thing is, back in those days it was extremely improbable that these technologies, the beginnings of which were already in place, would ever become small and inexpensive enough for an individual to own. In the 80s I used to record in a million dollar recording studio, and it seemed pretty unlikely that I would be able to have equal or better quality equipment in my home for just a few thousand, but I do.
Storage, case construction, lasers, still not brain surgery, each has been an incremental improvement on earlier technology. For example, you can comprehend that at some point computers will be significantly faster and have what seems to us to be huge storage capability, but the details of the technology, and the applications that will be developed for them, are still a mystery to us today. If someone came back from 50 years in the future and showed you where technology has gone I doubt you’d be lost or incapable of understanding.
I’d be far more interested in their time travel system.
CodeTech says:
June 25, 2013 at 6:26 pm
==============
I almost called you on that “intelligent people” thing you posted awhile ago.
Never overestimate the intelligence of your audience, some of us are dumber than a box of rocks, which is to say, we appreciate your/or anyones explanations.
Gail
You miss the point completely. Read the whole thread. Many people have objected to the OP’s postulate that science has stagnated after 1945 and have given multiple examples of why this is wrong, from molecular biology to astrophysics. I pointed out that sitting in front of everyone who reads this thread is a device, a modern desktop computer, which relies on technologies whose basic principles were not even established in 1945 – it is not that they are evolutions of older technologies – they rely on things which were not even thought of, like transistors and lasers. Lasers are not an incremental improvement on earlier technology – they emerged in the 1950’s as a completely new invention.
Your attempt to describe a modern computer in terms of what was known in 1945 would end up falling foul of Arthur Clarke’s Law “Any sufficiently advanced technology is indistinguishable from magic”
The point of my comments were not related to the history of computers as such, but were intended as another illustration that the had started from a ridiculous assertion, and gone on to inevitably reach a ridiculous conclusion.
Oops – previous comment addressed to CodeTech, not Gail – sorry.
I barely ever comment on blog posts but this… this… made me want to stand and applaud!
I have been telling my colleagues that we need to ban peer review for soooo long. It is a failed and ego-driven model. I feel like it is based on the fairytale impression you pointed out: “scientists are magical selfless beasts living to discover and uncover. They have the morals of angels and are never petty”.
I can tell you from experience that egos are the hallmark… that people pull rank all the time and we have to do the equivalent of kiss rings if the name is well known enough. At the very least can reviewers be compensated and the process be double-blind?!? I mean come on!
I’ve been trying to think of another system and quite honestly the only real answer is peer-review by being rewarded for better results. Who gets to decide better results? Well, in markets thats done through profit and losses. So the better system must take advantage of such an incentive. Currently, there is only profit (if you play ball) and no real loss in research, and therefore no incentive to get the right answers for the right reasons.
thank you for having the chutzpah to just come out and say what everyone knows is true…
ps- we’ve all had our “Maui” conferences that we’ve attended. My maui was a beautiful mountain town in switzerland.
In Eisenhower’s farewell address, just after his warnings of the military industrial complex, he warns us of a state of scientific research that sounds all too familiar:
“Akin to, and largely responsible for the sweeping changes in our industrial-military posture, has been the technological revolution during recent decades. In this revolution, research has become central; it also becomes more formalized, complex, and costly. A steadily increasing share is conducted for, by, or at the direction of, the Federal government. Today, the solitary inventor, tinkering in his shop, has been overshadowed by task forces of scientists in laboratories and testing fields. In the same fashion, the free university, historically the fountainhead of free ideas and scientific discovery, has experienced a revolution in the conduct of research. Partly because of the huge costs involved, a government contract becomes virtually a substitute for intellectual curiosity. For every old blackboard there are now hundreds of new electronic computers. The prospect of domination of the nation’s scholars by Federal employment, project allocations, and the power of money is ever present — and is gravely to be regarded. Yet, in holding scientific research and discovery in respect, as we should, we must also be alert to the equal and opposite danger that public policy could itself become the captive of a scientific-technological elite. It is the task of statesmanship to mold, to balance, and to integrate these and other forces, new and old, within the principles of our democratic system — ever aiming toward the supreme goals of our free society.”
My experience with reviewers is that on the whole their comments have resulted in a considerably improved paper and I have learned a lot from them. A few clearly were out of their depth or read what they expected to see rather than what was written. My personal preference would be double blind review or completely open.
I look on reviewing tasks as an obligation for participating in science. It is my role to help good papers be better and poor papers are a learning experience for the authors. Poor papers take a lot more effort to review.
Besides, jimmi_the_dalek, I did clearly state:
Arthur C. Clarke observed that any sufficiently advanced technology is indistinguishable from magic. And just like magic, explaining the steps in between takes away the wonder and mystery.
so the reviewers are annonymous and there are no standards? what could possibly go wrong!
How can a review by an annonymous reviewer that is not to any standard have any meaning?. How do we know that the review wasn’t done by Wanda the Fish or Mad as a Hatter? Or even done at all?
rogerknights says:
June 25, 2013 at 2:56 am
It is very much like the way Holy Mother Church used to act back in the 15-17th centuries.
Right–Organized Science has taken over from Organized Religion.
Er, there’s a reason science (and universities!) originated and flourished in Christian Europe, rather than, say, China, the Roman Empire, Islam, or ancient Greece. The Church taught that a reasonable, law-giving God created a reasonable, law-following universe that human beings could understand with their God-given reason. That simple but profound concept was missing in other cultures and hence they never developed science, despite many technological innovations.
I don’t say this as a Catholic — I’m not — but just want to set the record straight.
Forgive me for mentioning melanin politics, but your point about anonymity and “funny names” makes me wonder why people like Jesse Jackson are not trying to sue.
If you want to see something funny, Talbot was addressing a bunch of diet researchers and none commented on the facts he presented. That makes sense, since they are probably scared to death to even been seen hearing a critic of their research in case their manuscripts get rejected.
Reviewers do answer to journal-editors, who know the reviewers’ identities. If, for example, they publish something which they had reviewed, or which they were in the process of reviewing, the editor is supposed to report this. After that, they do not get to publish on anything related, or at least not review for any journal again.
Also, we have had some pretty stunning advances. It looks, to me, like the basic laws of physics just got rewritten, to a significant degree, just last year. Check out the 2012 Nobel Prize in physics, look at the implications for application with entangled particles, look over the laws of relativity and causality, and then tell me we’re not looking at a viable mechanism for causality-violation.
Dumping Peer review makes sense: over time it locks in orthodoxy of thinking with entrenched, and quite possibly low level, performers given a power via the review process to rein-in new developments.
The alternative which makes sense is Support Review/Endorsement. Scientists should publish what they can, when they can – get it out there to be debated. The problem really rests with the Journals who must pick and choose what to publish (the current review system both helps them to decide what to publish, and provides a mechanism for them to decide for themselves – by choosing the reviewers – but not appear to be actually making the choices).
For a Positive Review/Endorsement approach: get your paper written, and get prominent people (scientists or whoever) to endorse it to persuade a Journal to publish. The writer may need to convince peers or peer-equivalents to endorse their paper to get it published, but if they can’t do that, likely it isn’t worth publishing. All the Journals need to do is decide on the basis of what papers they like, and/or appears credible, assisted by who is prepared to endorse the paper.
Not a perfect system, but less imperfect than the current peer review system.
Why cant peer review be double-blind with the peer review neither knowing the scientist involved or the institution behind it, they would have to be more careful then as they might be stepping on the wrong persons toes.
j_the_d, as you know, your comment was in moderation and I didn’t see it when I last posted. Interesting that we both brought up Clarke’s quote.
However, you are not correct. Transistors were only a breakthrough in how they worked and how small they could be. Vacuum tubes and relays perform similar tasks. A transistor is easily explained, it’s not complicated, especially to someone who understands even the basics of electricity. So is a laser, although the laboratory behemoths in the 60s can barely be compared with the tiny counterparts that read your DVD. In fact, for a while lasers were a solution looking for a problem.
Can you actually compare the step-by-step-by-step progress of the computer industry since WW2 with the breakthrough research done during the Manhattan Project, or by the Curies, or by Einstein or Planck or… well, as the point of the post, any of the greats before the current epoch?
I bet only a very few people can even NAME a modern Scientist, with the possible exception of Hawking. Einstein’s picture is instantly recognized by pretty much everyone on the planet.
And no, I’m not saying progress has stagnated, although I posted the Big Bang Theory clip because it’s hilarious. Things have changed, a lot. I suspect a lot of the change in how Science is done was because the US needed to keep the “secrets” of The Bomb from becoming too widely known.
Any reasonably intelligent person extracted from the 1940s to today, or even the 20s or earlier, would integrate just fine, although they might not like the level of communication we’ve achieved. Text messages? Phone calls in the car? Discussing deep topics with a literally world-wide audience, all participating in real time? To my parents’ generation, it’s too much, but it’s not voodoo.