Short Circuiting The Scientific Process – A Serious Problem In The Climate Science Community

Guest post from Roger Pielke Sr., originally posted on Climate Science

There has been a development over the last 10-15 years or so in the scientific peer reviewed literature that is short circuiting the scientific method.

The scientific method involves developing a hypothesis and then seeking to refute it. If all attempts to discredit the hypothesis fails, we start to accept the proposed theory as being an accurate description of how the real world works.

A useful summary of the scientific method is given on the website sciencebuddies.org.where they list six steps

  • Ask a Question
  • Do Background Research
  • Construct a Hypothesis
  • Test Your Hypothesis by Doing an Experiment
  • Analyze Your Data and Draw a Conclusion
  • Communicate Your Results

Unfortunately, in recent years papers have been published in the peer reviewed literature that fail to follow these proper steps of scientific investigation. These papers are short circuiting the scientific method.

Specifically, papers that present predictions of the climate decades into the future have proliferated. Just a two recent examples (and there are many others) are

Hu, A., G. A. Meehl, W. Han, and J. Yin (2009), Transient response of the MOC and climate to potential melting of the Greenland Ice Sheet in the 21st century, Geophys. Res. Lett., 36, L10707, doi:10.1029/2009GL037998.

Solomon, S. 2009: Irreversible climate change due to carbon dioxide emissions. The Proceedings of the National Academy of Sciences. Published online before print January 28, 2009, doi: 10.1073/pnas.0812721106

Such studies are even reported in the media before the peer reviewed process is completed; e.g. see in the article by Hannad Hoag in the May 27 2009 issue of Nature News Hot times ahead for the Wild West.

These studies are based on models, of which only a portion of which represent basic physics (e.g. the pressure gradient force, advection and the universal gravitational constant), with the remainder of the physics parameterized with tuned engineering code (e.g see).

When I served as Chief Editor of the Monthly Weather Reviews (1981-1985), The Co-Chief Editor of the Journal of Atmospheric Sciences (1996-2000), and as Editor-in-Chief of the US National Science Report to the IUGG  for the American Geophysical Union (1993-1996), such papers would never have been accepted.

What the current publication process has evolved into, at the detriment of proper scientific investigation, are the publication of untested (and often untestable) hypotheses.  The fourth step in the scientific method “Test Your Hypothesis by Doing an Experiment” is bypassed.

This is a main reason that the policy community is being significantly misinformed about the actual status of our understanding of the climate system and the role of humans within it.

The climate data they don't want you to find — free, to your inbox.
Join readers who get 5–8 new articles daily — no algorithms, no shadow bans.
0 0 votes
Article Rating
305 Comments
Inline Feedbacks
View all comments
Paul Vaughan
June 9, 2009 6:35 pm

Leif Svalgaard (17:58:35) “it is clear from the context what is what.”
not to those who ‘ski’ (skim & skip)

Paul Vaughan
June 9, 2009 6:36 pm

Gilbert (17:56:57) “My apologies.”
appreciated – thank you

Jim Arndt
June 9, 2009 7:18 pm

” Leif Svalgaard (17:18:00) :
Mike Bryant (17:00:42) :
The reporter you linked to for the Texas school Board is probably a product of the type of education that some here are talking about.
The reporter is not important, the text books that the school boards mandate are. I had lived in the Great State of Texas for many years and know the issues well. I say again: people get what they want.”
Leif you are completely right. I you want a certain type of people in office to represent then you elect them. They will hopefully follow the will of the majority. If we don’t like what they do then we throw them out. Very simple and if you don’t like the region your in then move if you can’t move then do something to change it. I keep referring to a Monty Python skit were the guy goes into a room to buy an argument.

Evan Jones
Editor
June 9, 2009 7:30 pm

Leif, I’m sure you’d be amused by the irony of one side pointing out the faith the other side has that certain highly complex biological processes and structures have evolved naturalistically despite the absence of a mechanism for that evolution.
I can’t speak to mechanism, but I have had a “birdseye view” of evolution by process of natural selection:
I live in NYC. When I was a kid (before the Clean Air Act), all the buildings were dingy and gray (or became so a few years after construction). After the clean air act became effective, there was a massive sandblasting that went on for a decade. You couldn’t walk down the street without seeing some building getting sandblasted. All sorts of colors emerged from under the grime.
In the succeeding years, the pigeons have evolved from standard grays (with a few black or heavily spotty whites) to a whole collection of colors, including (but not restricted to) lovely brick-red variety, “brownstone”, and “sandstone” varieties. It’s an ongoing process, but only around half the NYC pigeons nowadays are standard grays and most of those show partial traits from the other strains. They are also a lot healthier, cleaner, and plumper.
In contrast, I visited Essen a few years back. The pigeons there all identical, with tight gray ringlet markings over their wings, with no variety whatsoever. (Must be a more fixed strain.)

Gilbert
June 9, 2009 7:32 pm

Leif Svalgaard (18:17:59) :
Gilbert (17:51:47) :
I have a lot of respect for your abilities as a Solar Physicist, but maybe you should stop short of starting a childish rock fight
Takes two to fight…I’m not starting a fight, you are…

It only takes one to start a fight, and that’s exactly what you did when you impugned the integrity of everyone in the states of Kansas and Texas.
When you dump on me along with everyone else, then you’re dead wrong. Kansas has more than its’ share of rednecks and religious fanatics, but not all Kansans are among them.
“And my point stands: people get what they want, so don’t blame the system.”
Only the majority gets what it wants. The minority goes along for the ride.
Note also that the curriculum in Kansas is set by the State in compliance with federal guidelines. So no, the locals don’t always get what they want.
Let’s please get back to the issue at hand.
I grew up and went to school in the state of Kansas and was well versed in the scientific method by junior high school.
I also learned about the theory of evolution as was commonplace at the time. Apparently, the times have changed.

June 9, 2009 8:33 pm

Gilbert (19:32:01) :
It only takes one to start a fight, and that’s exactly what you did when you impugned the integrity of everyone in the states of Kansas and Texas.
Everyone still have their integrity intact, as integrity has nothing to do with misguided or faith-based crippling of the coming generation(s). This is not about integrity [as these folks are sincere – we assume], but about malfeasance towards their (and others who may not share their beliefs) children.

June 9, 2009 8:43 pm

… the pigeons have evolved from standard grays (with a few black or heavily spotty whites) to a whole collection of colors…
… I also learned about the theory of evolution as was commonplace at the time…
Sorry, boys, but NYC pigeons have not “evolved”. They are the same species they always were. Evidently the liberal education you received, so pumped to promote evolution, failed to teach you just what that phenomenon actually is. And that’s a big part of the problem: liberal theology masquerading as education.

Gilbert
June 9, 2009 9:00 pm

Anthony,
Thanks for your forbearance. I will end this now.

Glenn
June 9, 2009 9:43 pm

evanmjones (19:30:41) :
“In contrast, I visited Essen a few years back. The pigeons there all identical, with tight gray ringlet markings over their wings, with no variety whatsoever. (Must be a more fixed strain.)”
Quite hard to believe, Evan.

Allan M R MacRae
June 9, 2009 10:06 pm

Seems like I’ve started another controversy, this time about the Law of Gravity.
Did not think that gravity would be contentious.
I appreciate the fact that it keeps me grounded.
Cannot help much regarding how gravity works:
Something about gravity waves or something; which can be overcome by sprinklings of pixey dust.

anna v
June 9, 2009 11:09 pm

Allan M R MacRae (22:06:56) :
Seems like I’ve started another controversy, this time about the Law of Gravity.
I do not think it is controversy, just a load of sophistry on semantics of what gravity is, a theory or a model.
from webster:
Model:
1 obsolete : a set of plans for a building
2 dialect British : copy, image
3: structural design
4: a usually miniature representation of something ; also : a pattern of something to be made
5: an example for imitation or emulation
6: a person or thing that serves as a pattern for an artist ; especially : one who poses for an artist
7: archetype
8: an organism whose appearance a mimic imitates
9: one who is employed to display clothes or other merchandise
10 a: a type or design of clothing b: a type or design of product (as a car)
11: a description or analogy used to help visualize something (as an atom) that cannot be directly observed
12: a system of postulates, data, and inferences presented as a mathematical description of an entity or state of affairs ; also : a computer simulation based on such a system
13: version 3
14: animal model
theory:
1: the analysis of a set of facts in their relation to one another
2: abstract thought : speculation
3: the general or abstract principles of a body of fact, a science, or an art
4 a: a belief, policy, or procedure proposed or followed as the basis of action b: an ideal or hypothetical set of facts, principles, or circumstances —often used in the phrase in theory
5: a plausible or scientifically acceptable general principle or body of principles offered to explain phenomena
6 a: a hypothesis assumed for the sake of argument or investigation b: an unproved assumption : conjecture c: a body of theorems presenting a concise systematic view of a subject
This should settle in which category gravity equations belong.

anna v
June 9, 2009 11:14 pm

Benjamin P. (16:48:33) :
Nobody here questions gravity, but nobody here knows how gravity works.
Speak for yourself, John.

Paul Vaughan
June 10, 2009 12:22 am

Re: Mike D. (20:43:45)
It sounds like what population geneticists call ‘directional selection’. (I say ‘sounds like’ because I am not familiar with this particular population of pigeons.)

Paul Vaughan
June 10, 2009 12:52 am

One of the great things about WUWT conversations is their multi-disciplinary nature.
Since gravity has come up as a modeling – or not-modeling (depending on one’s perspective/beliefs/…whatever) – example, this might be as-good-a-time-as-any to ask for opinions on the following articles:
1.
Omerbashich, M. (2008). Scale invariablity.
http://arxiv.org/ftp/arxiv/papers/0801/0801.0876.pdf
[For those with a casual interest, taking a good look at Figure 1 (including the caption) should give you a general sense of a theme in some of the papers of M. Omerbashich.]
2.
Omerbashich, M. (2006) Springtide-induced magnification of Earth mantle’s resonance causes tectonics and conceals universality of physics at all scales.
http://arxiv.org/ftp/physics/papers/0608/0608026.pdf
Thanks for comments.

June 10, 2009 4:26 am

anna v (23:14:45) :
Benjamin P. (16:48:33) :
Nobody here questions gravity, but nobody here knows how gravity works.
Speak for yourself, John.

Of course they do Ben. It’s an action at a distance mediated by an assumed particle no-one has ever detected.

anna v
June 10, 2009 5:12 am

Paul Vaughan (00:52:08) :
I cannot presume to be able to review these papers with respect to the accepted GR and gravitational theories, i.e. whether they are a new way of inventing the wheel, are taking off on a tangent or are offering something new. This is what peer review is about, and this needs seismologists and GR guys to be able to gauge it.
Usually when somebody takes a hit at both Newton and Einstein I tend to not take them seriously. Ether to boot. I thought that the Michelson Morley experiment killed ether, but who knows.
I’ll take superstrings after all.

cba
June 10, 2009 5:34 am

I looked through some 7th & 8th grade TX science textbooks last year. While I avoided the fraction dedicated to biology, health, and enviro guff, I must say that the physics, astronomy tidbit, and chemistry actually looked pretty good, much better than I was expecting.
It turns out that the intellectual black hole I was visiting didn’t issue the books to the students but were kept only as in class references to be brought out on rare occaisions. What passed for teaching was something totally different, centered around canned lcd slides that apparently did not meet the state mandated curiculum requirements as they had to spend a great deal of time near the standardized tests to quickly get the students to memorize the actual sort of material that would be on those test. I have suspicions even worse about how bad it is there.
In short, their textbook – if they used it – would have made an unbelievable difference for the better.

cba
June 10, 2009 5:39 am

So Anna,
what’s the office pool odds & winnings size running on the potential discovery of the Higgs (or its unfortunate demise via exclusion limits) happening this summer at the lhc these days?

anna v
June 10, 2009 5:57 am

tallbloke (04:26:14) :
It’s an action at a distance mediated by an assumed particle no-one has ever detected.
Let us be clear on this. Classical Newtonian gravity is action at a distance. The same holds for classical electricity and magnetism. No medium is necessary for the equations to work and give accurate predictions for time and space experiments. Nor are any particles necessary to visualize mediation. Knowing how gravity works is knowing how to predict its effects by using the equations.
It is while studying further electricity and magnetism that gave rise to electromagnetism and the electromagnetic waves that finally the positing of a photon as mediator became necessary in the quantum framework. The graviton is an extension of this to gravity but yes, it has not been observed nor is gravity quantized.
General relativity is a different story, because it manipulates the very space time, it does not work with “forces” and “particles”. We have to wait and see whether a unified framework can be made, as super strings are trying to make, or something entirely new will appear.

Arthur Glass
June 10, 2009 7:19 am

Calling Dr Popper! Calling Dr Karl Popper!
Ludwig, put down that poker!
At the risk of opening an old (what, four days ago?) argument, philosphy departments, before they fell into Derrida-da-ism, used to offer a course called Logic and the Scientific Method. In a rigorous liberal arts and sciences curriculum, this would be a required course for the Bachelor’s degree regardless of major.
I suppose that using as one of the texts Newton’s __Principia__ in Latin would be a little too rigorous.
But really, instilling in children from K-12 a genuine appreciation for the nature of scientific knowledge and for the method of scientific inquiry is a teaching task even more crucial. Education is about the preservation and passing-on, from one generation to the next, of ‘the best that has been thought and said,’ to quote Matthew Arnold. The perfection of scientific method is as much a part of the cultural patrimony of the West as is the poetry of Dante and Shakespeare or the music of Bach, Mozart and Beethoven.
On some recent thread, there is a post of mine, off-topic in the context, in which I indulge in reminiscent appreciation of learning a little about meteorology in 7th grade science class (I was already a long-time weather- weenie and storm-lover at the age of twelve). I also suggested to Anthony that a continuing thread on science education in elementary and secondary schools might make for a pleasant and illuminating discussion. Perhaps this is it; it would be appropriate for such a thread to derive from an essay by Roger Pielke, Sr, who, for my money at least, is a paradigm of scientific integrity.

anna v
June 10, 2009 7:43 am

cba (05:39:03) :
So Anna,
what’s the office pool odds & winnings size running on the potential discovery of the Higgs (or its unfortunate demise via exclusion limits) happening this summer at the lhc these days?

50/50 ? The Higgs has been tooted too much, part of the PR. If it is not found at the current energies, a Higgs mechanism will be postulated, not to worry :).
Lets get the LHC working first, and we shall see.

Benjamin P.
June 10, 2009 8:11 am

anna v (23:09:16) :
Pay close attention to this part of your definition of model…
“12: a system of postulates, data, and inferences presented as a mathematical description of an entity or state of affairs ; also : a computer simulation based on such a system”
…especially that part that says ‘…as a mathematical description…’
“This should settle in which category gravity equations belong.”
Indeed it should.
Gravity is a Theory, to make predictions about the way gravity manifests itself we use mathematical models.
Again though, we still do not have a mechanism to explain how one mass feels another mass.
So we don’t know how it works, but we can predict exactly how it behaves because of accurate models!

Arthur Glass
June 10, 2009 8:11 am

” Communicate Your Results”
Communicate them to your peers and communicate them, ultimately, to the public at large.
This latter obligation, if that is not too strong a word, may be, and usually is, delegated to science writers, e.g. Tom Siegfried, and to reporters and journalists, e.g. Andy Revkin. This is no doubt an oversimplification; in many cases there may be several intermediaries between the researcher and the popularizer. This, of course, increases the possibility of miscommunication of the original message, as in the party-game ‘Telephone’.
I do not believe it an exaggeration to say that the intelligence, curiosity and good faith of these intermediaries between researchers and the general public have a critical role to play if we continue to value the idea of a democratic polity even in a society where technical expertise, or at least pretentions thereto, has an accelerating impact on public policy and private life.
Blogs such as this one are thus of vital importance. I, for one, do not wish to be governed by a Sanhedrin of anointed ‘experts’.

Giles Winterbourne
June 10, 2009 8:17 am

“Few realize like I do that wilderness is a myth.”
Not so much. Well observed and written about in 18th and 19th Cent. Ex: Thoreau and Lewis & Clark, T. Jefferson. Earlier refs by Chaucer, Milton. Also see Romanticists, Ruskin.
The myth you are referring to is mostly a religious and political construct to justify subjective and hegemonic behaviors.

Giles Winterbourne
June 10, 2009 8:44 am

“The most recent findings by Dr. Svalgaard et al. are based on only the first of three years of data during the current decline of solar cycle 23. At least two more years of data (through the solar minimum) are required to provide a more accurate prediction.”
The Next Solar Maximum the Smallest in 100 Years? Space Weather News : Research News 10 June, 2009. http://www.spacew.com/news/05Mar2005/index.php

1 6 7 8 9 10 13