Fernbach et al.: Advantages of Pre-Registration of Study Design

Guest Essay by Kip Hansen

Untitled-2We recently saw on this blog a  guest essay from Joel O’Bryan regarding a paper from Fernbach et al. titled “Extreme opponents of genetically modified foods know the least but think they know the most” which appears in the form of a “Letter” in the journal Nature – Human Behavior.    The paper itself is yet another social science study of “How can anyone fail to support/believe a proffered scientific consensus?”

My interest was piqued when I read the abstract of the study quoted in Joel O’Bryan’s essay.  As it has been a couple of weeks, here’s the abstract to refresh your memory:

“There is widespread agreement among scientists that genetically modified foods are safe to consume and have the potential to provide substantial benefits to humankind.  However, many people still harbour concerns about them or oppose their use. In a nationally representative sample of US adults, we find that as extremity of opposition to and concern about genetically modified foods increases, objective knowledge about science and genetics decreases, but perceived understanding of genetically modified foods increases.  Extreme opponents know the least, but think they know the most. Moreover, the relationship between self-assessed and objective knowledge shifts from positive to negative at high levels of opposition. Similar results were obtained in a parallel study with representative samples from the United States, France and Germany, and in a study testing attitudes about a medical application of genetic engineering technology (gene  therapy). This pattern did not emerge, however, for attitudes and beliefs about climate change.”  [bold emphasis mine: kh]

Like Joel, I was surprised by the last sentence in the Abstract — it seemed a total non-sequitur.   The abstract is about attitudes, beliefs and knowledge regarding GMOs  (and mentions other studies that found a similar result concerning gene therapies)….and there at the end is this:  “This pattern did not emerge, however, for attitudes and beliefs about climate change.” 

 Yes, I’m sure you see the same thing I do:  The paper’s title and the rest of the abstract are all about GMOs yet the last sentence of the abstract tells us that they studied attitudes, beliefs and knowledge about Climate Change and the results were different than all-the-above found for GMOs.  That’s all they give us in the abstract on the Climate Change part of the study.

So what’s the story here?

 First, kudos go to Philip Fernbach and his team for seeing that all the study data are available online at a data depository, the Center for Open Science.  Here is the link to the full paper  and here is a link to the Supplemental Information.   Even the code used in the analyses is available.

Reading the full paper, one discovers that they have carried out a two-forked study:  one fork concerning GMOs and one fork concerning Climate Change.  But the majority of the paper as written focuses on and discusses the findings as related to GMOs — and the Climate Change portion of the study is given short shrift.

Why?   Quite simply, the Climate Change part of the study had a null result.

There has been  a lot written recently about biases in scientific literature and one of these biases is Publication Bias which is brought about by the fact that researchers and journals are much less likely to publish a paper that shows a study had a null result —  “In science, a null result is a result without the expected content: that is, the proposed result is absent.”

To verify that this finding about Climate Change is a case of a null result (one half of the study having had a null result in this case), it is necessary to look at the Study Design.  Yes, that all important document that is prepared before any data is collected.  Joe Lief Uri explains the what, why and how of pre-registering study designs here. The study design lays out carefully how the study is to be conducted, and contains all of the information below:

study_design_800

[ image from http://datacolada.org/64 —  click here for larger image]

We don’t very often see such documents and, because of that, studies often not only suffer from p-hacking and post hoc analysis but the authors get away with it.  I can’t remember seeing a single published study design document for any climate change research but this just may be a personal knowledge deficit.  [If any reader knows of published, pre-registered study designs in climate change, please give me a link in comments.]

The pre-registering of a study design is becoming required by such agencies as the National Institutes of Health.  An example of an NIH pre-registered study design can be found here for “Melatonin Use for Sleep Problems in Alcohol Dependent Patients”.

Again, Fernbach et al. are to be commended for pre-registering their research for this study.   It has been registered at the site AsPredicted .  For this particular study, the study design is found here (pdf).  From the study design document we find:

“2) What’s the main question being asked or hypothesis being tested in this study?

The key prediction is that the gap between objective and subjective knowledge widens as extremity of anti-scientific consensus beliefs increase. We further predict that as extremity increases, subjective knowledge increases, but objective knowledge, as measured by a battery of scientific literacy questions, decreases. We will test this in two domains: Genetically modified foods and climate change.

 What are they expecting to find?

They are expecting:  “We will conduct all analyses separately for each question and after averaging the two questions for each issue. We expect the pattern to be similar across all analyses.

So, they do the study — in the United States, France and Germany.  Lo and behold, they find that on the topic of GMOs, they are spot on:  “opponents of genetically modified foods know the least but think they know the most” — and this pattern is the strongest in the United States.  The paper goes into a lot of detail on how they arrive at this conclusion (which turns out to be exactly what they expected to find).

Things did not go so well with the Climate Change arm of the study.  Fernbach et al. don’t give us the nice graphs of the data for Climate Change — because for Climate Change, their expected pattern did not appear — they had a null result.

The researchers summarize the Climate Change finding thus: “Unlike beliefs about GM foods, climate change beliefs were highly polarized by political identification, with conservatives much more likely to oppose the scientific consensus than liberals.”     “For climate change, the direction of the effects was the same [as was found for GMOs], but the results were not statistically significant. The lack of a relationship between scientific literacy and extremity of anti-scientific-consensus climate change beliefs is consistent with previous findings and we believe that this is attributable to the polarized nature of the climate change issue.”

Unfortunately, the authors do just what pre-registration of a study design is meant to help prevent:  they perform an analysis that was not called for (meaning not designed into the original study) and they engaged in some light HARKing and/or JARKing –the two italicized sentences above.

hark

To their credit, they admit to the null result in the abstract — the last sentence which was quoted earlier: “This pattern did not emerge, however, for attitudes and beliefs about climate change.”  A good, clear statement of the actual results on Climate Change.

But what they don’t do is do the same exact analysis on this half of the study as they did on the GMO half of the study  (or, if they did, it does not appear in the published paper or its SI.)   It would have been nice to see the Climate Change statistics illustrated in the same manner as those for GMOs and the Overall (combined) findings.  This, simply put, is publication bias writ-small. Perhaps this is the result of the simple need to keep the paper within the length bounds of the publisher, and thus maybe we shouldn’t make much of this lack.  But then I would have expected at least to see these missing graphics in the SI.

What the authors do is try to explain [explain away?] why their hypothesis did not hold for the Climate Science half of the study.

Remember, their pre-registered hypothesis was basically as bluntly stated in the abstract “Extreme opponents know the least, but think they know the  most.“ (The full wording was previously quoted.)

The authors preform an analysis that is not called for in the Study Design [We will also collect the following demographic variables for each participant: age, gender, income, education, political ideology, and political party. We do this for completeness, but don’t plan to analyze any of these variables with respect to our hypotheses.] — they analyze the climate change results by political leaning (liberal → conservative) and build a just-so-story explanation based on the findings of others [not their own study] when they say “The lack of a relationship between scientific literacy and extremity of anti-scientific-consensus climate change beliefs is consistent with previous findings and we believe that this is attributable to the polarized nature of the climate change issue.”

They say when writing this paper “we believe…”  but that was not what they believed before the study was done. 

That’s the value of pre-registration of study designs:  researchers get caught out in these little  attempts to trick or fool themselves.  What they believed before collecting the data was: “We expect the pattern to be similar across all analyses.

When the results were in, the authors were stuck with a null result for the Climate Change half of the study.  They then fall back on the work of others to try to explain or justify why their hypothesis did not hold for Climate Change.

They did not include in this study anything about polarization of issues. There is no data generated in this study showing political polarization of this issue (there is no doubt that Climate Science is polarized along political lines, especially in the United States but the authors knew this prior to the study and still thought they would find the same pattern in both halves of the study.)  The authors present a new hypothesis after the results are in: “… attributable to the polarized nature of the climate change issue” and base their new post hoc hypothesis on the work of others — not on data generated by their own work.

My purpose in discussing this paper is not to protest or celebrate their findings but simply to point out the value of Pre-Registering study designs and placing all the data and code in a public repository.  Doing so allows for a much more nuanced evaluation by others of the findings and exposes where authors have stepped outside of the proper protocols of research and engaged in a wee bit of HARKing, JARKing and/or p-hacking (not seen in this case).

protocol_800

[larger image here].

Overall, Philip Fernbach, Nicholas Light and their 3 co-authors have done the right things — they have Pre-Registered their study design and subsequently posted all data and code to publicly accessible repositories.  Further, they have included blunt recognition of achieving a null result for a full half of their originally planned study.   With that many pluses, I am happy to give them a free pass on slipping off the track a little by “explaining”  in the discussion section of their paper.

# # # # #

It would be a major improvement in all the sciences, helping to curtail the Irreproducibility Crisis,  if researchers worked up carefully-crafted, precise Study Designs for every proposed research effort, posted them to a registration site for their scientific specialty, and then followed the design with scientific precision, utilizing the standard protocol illustrated above.   This emerging trend is called for by the National Academies of Science;  The Center for Open Science; the American Psychological Association and others.

# # # # #

Author’s Comment Policy:

Lots to talk about, both in regards to this particular paper and the steps that will be necessary to begin to end the Irreproducibility Crisis.  Imagine a paper on historical surface temperature that pre-defined all of its steps, methods, and proposed analyses in a pre-registered study design … with no post hoc selection or rejection of data sets, no “let’s try some other analysis”, and no fudging allowed with everyone able to look over their shoulders seeing exactly what the researchers are doing at every step.

I am not particularly concerned with the actual findings of the featured study … as for climate science it was a null.  Note that it would be a mistake to claim a pro-skeptic result from the paper as the study design was only adequate to test the original hypothesis which failed.

Happy to answer any questions.

# # # # #

0 0 votes
Article Rating
117 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Lewis Buckingham
February 12, 2019 12:26 pm

Was the null hypotheses tested
‘There is no difference between the extent of scientific understanding between opposite to extreme views of the cause[s] of climate change’?
The result being ‘We cannot disprove this at the 5% confidence level.

Just trying to formulate the analysis.

Editor
Reply to  Lewis Buckingham
February 12, 2019 12:37 pm

Lewis ==> I am guessing at your meaning a bit here — their hypothesis is that the surveys of subjective/objective scientific knowledge and anti-consensus views would display a “Extreme opponents know the least, but think they know the most. ” pattern for both topics. That was not shown to be the case in their data — it was only found for the data on GMOs.

Let me know if I mis-read your question.

commieBob
Reply to  Kip Hansen
February 12, 2019 2:45 pm

My guess is that they didn’t do their homework properly when they designed the study. It’s been known for a while that skeptics have somewhat more climate science knowledge than CAGW believers. example They seem to have become aware of that late in the process.

The lack of a relationship between scientific literacy and extremity of anti-scientific-consensus climate change beliefs is consistent with previous findings …

Well, if they knew that in the first place they wouldn’t have formulated their hypothesis the way they did, would they?

Time after time we see folks who do an extensive literature search and still manage to miss the blindingly obvious.

commieBob
Reply to  commieBob
February 12, 2019 2:48 pm

Aargh!

The lack of a relationship between scientific literacy and extremity of anti-scientific-consensus climate change beliefs is consistent with previous findings …

Well, if they knew that in the first place they wouldn’t have formulated their hypothesis the way they did, would they?

Time after time we see folks who do an extensive literature search and still manage to miss the blindingly obvious.

Editor
Reply to  commieBob
February 12, 2019 3:10 pm

Commie ==> It wasn’t ignorance of the previous studies and their results….they just had a new and different hypothesis concerning anti-scientific-consensus beliefs and subjective-objective knowledge of science. Their hypothesis was supported concerning GMOs, but rejected concerning AGW.

We only know this because the authors followed proper science protocols, developed and pre-registered their Study Design, and then posted all the data and code on a publicly accessible web depository.

Previously, with less forthright authors, we might have seen this study written up just in regards to GMO, with the authors crowing about how right their hypothesis was (and simply not mentioning the null result on Climate).

LdB
Reply to  commieBob
February 12, 2019 6:12 pm

Yep agree with Kip and you having lost your control you can’t go back an post fix-it … I guess if you are into your statistics you could go homogenize it 🙂

Lewis Buckingham
Reply to  Kip Hansen
February 12, 2019 7:12 pm

Thanks for clarification.
I was wondering if they tested extreme ‘, we are all going to burn up’, advocates as well.

Curious George
February 12, 2019 12:31 pm

Another 97% opinion poll?

Lance Wallace
February 12, 2019 12:32 pm

In fact, their finding might be characterized as the anti-null (i.e., significant disproof of the hypothesis). This in fact was already found in 2012 by Kahane in a letter to Nature common Change NATURE CLIMATE CHANGE j VOL 2 j OCTOBER 2012 j http://www.nature.com/natureclimatechange. “As respondents’
science-literacy scores increased, concern with climate change
decreased (r D 􀀀0:05, P D 0:05). There was also a negative
correlation between numeracy and climate change risk (r D􀀀0:09,
P <0:01)"
Unfortunately, Kahane, faced with this clear result, fell back on an alternate explanation as did Fernbach–the perceived effect of the polarization by politics. They simply cannot allow themselves to consider the possibility that the climate change consensus is wrong.

Editor
Reply to  Lance Wallace
February 12, 2019 12:40 pm

Larry ==> Yes, you are absolutely right. In fact, Fernbach et al. cite the Kahan paper in support of their hypothesis-after-results-are-known.

Reply to  Kip Hansen
February 12, 2019 5:43 pm

That was my poorly made point in my original guest essay-post on this a paper. Where the authors here want to Post Hoc explain their climate change attitude result due to “political polarization,” (which is blindingly obvious as you point out in today’s world Kip) the simpler explanation they avoid is that climate change consensus really is 97% junk science. A junk science based on highly manipulated surface data sets, treemometers with really bad math, and hand-puppet climate models.
It is just that the Liberal mind loves the lie if it supports their internal bias.

Hugs
Reply to  Joel O'Bryan
February 13, 2019 3:20 am

I think now you’re too harsh. The consensus is not complete junk, the irrational alarmism we see daily in newspapers, tv, and more important, the Internet, is the thing that should be questioned.

The right-thinking people have a difficulty of placing a concern on those who express extreme alarmism. They seem to frame it like this:

People who’re right (AGW is real and dangerous and must be mitigated with energy policy)
People who have ideas (like above, but they tell about the policy we should follow)
People who deny AGW (anybody who doesn’t immediately accept ideas presented by the second group)

In this simple model, there are no optimists, only denialists. Also note that there are no extreme alarmists, because they’re renamed as people who have ideas. Like AO-C and Greta Thunberg, for example. Hail them as saviours.

How I would frame this? I think we have optimists, pessimists and alarmists. And if you look carefully, in some blogs you see people who are just denialists, but they have an insignificant public visibility. I’m not sure what Trump is. The Chinese hoax thing suggests he’s not a pessimist or worse. I neither believe he’s a real denialist, but he might as well play one to polarize, and then win the mudwrestling.

“It is just that the Liberal mind loves the lie if it supports their internal bias.”

We all do. It is the first thing a scientist must stop doing. Fooling themselves.

john
Reply to  Hugs
February 13, 2019 6:24 am

The “consensus” is worse than junk. It is a conspiracy of self interest amongst activist/scientists and Socialist politicos. It is garbage-which smells worse than junk for a reason. It’s rotten!

Lewis Buckingham
Reply to  Lance Wallace
February 12, 2019 7:09 pm

‘the perceived effect of the polarization by politics.’
Another more credible explanation is that where a political party advocates something that is logically correct and the voter agrees, then that voter becomes a supporter of that party.
When mainstream parties all adopt policies that are not logically scientific, the scientific voter changes to the least damaging of them.

Editor
February 12, 2019 12:33 pm

Preemptively, let me point out the larger error in these psychological/social science studies on “anti-scientific consensus beliefs”.

From the discussion section of the study:

d the relationship between scientific literacy, political affiliation
and extremity of counter-scientific-consensus beliefs across a range
of issues such as GM foods, climate change, evolution and the big
bang.. For most issues, including GM foods, there was an overall
negative relationship between scientific literacy and opposition to
the consensus, consistent with the idea that knowledge plays a role.
However, half of the issues also demonstrated an interaction with
political identification; the relationship between knowledge and
attitudes was weaker for the political group holding the counter-
consensus position, suggesting that ideology can diminish effects
of knowledge. This interaction effect was strongest for the climate
change issue, where conservatives actually showed an opposite
effect: an increase in extremity with scientific literacy.

Nowhere in their discussion, and I assume nowhere in their range of acceptable ideas, is there any mention of the fact that there may be something different about the subject consensus itself…that there may be a very strong knowledge-based anti-consensus position in Climate Science.

The similar problem appears for “evolution” — which is a quagmire of differing scientific and religious opinions and “The Big Bang” — which is a bit of still-theoretical physics about which Popular Science agrees, but physicists do not.

ladylifegrows
Reply to  Kip Hansen
February 12, 2019 6:08 pm

The actual SCIENTIFIC consensus on climate change is that it has always occurred and most of the 20th-21st century temperature changes were within natural variability and nothing to get upset about. Mankind probably increased temperatures slightly (hard to tell with all the adjusted data), and definitely raised temperatures a few degrees in cities and towns.

The 97% nonsense was from squeezing the data until it screamed, which has been done more than once and reported here.

IPCC stands for Inter[b]governmental[/b] Panel on Climate Change which means it is politics, not science. The world’s presses are not exactly sophisticated in matters of science and have been falsifying this subject for years. It is hard to get past them, but we do have the facts.

sycomputing
Reply to  ladylifegrows
February 12, 2019 8:42 pm

The 97% nonsense was from squeezing the data until it screamed, which has been done more than once and reported here.

Bravo!

Lance Wallace
February 12, 2019 12:36 pm

That should be r = -0.05, p = 0.05; and r = -0.09, p<0.01

Marty
February 12, 2019 12:41 pm

Just to state the obvious – the study lumps together apples and oranges. Believing that genetically engineered food is hazardous is irrational. Believing that global warming is mostly junk science is rational and supported by evidence.

Editor
Reply to  Marty
February 12, 2019 12:47 pm

Marty ==> In essence, you are quite right. None of these social science studies on anti-scientific-consensus beliefs allows even the slightest chance that the “consensus” may just be a majority opinion, or prevailing opinion, within a field, and not really a “consensus” at all. Nor do they allow the possibility that even when there is a true consensus, there may still be a scientifically valid counter-consensus — something which has happen time and again in science over the last century.

When these types of studies have found differences in public reactions to consensuses, the researchers seem unable to look to see if there are differences in the nature of the consensuses themselves that account for the findings.

Keith
Reply to  Marty
February 12, 2019 4:40 pm

Marty, completely agree. It’s a shame the researchers chose not to explore the AGW part of the study as it would have lead to a very interesting analysis. “If lack of scientific knowledge corresponds to anti-science viewpoints for GMO but not AGW, then what is the difference between those two subject areas?”

ozspeaksup
Reply to  Marty
February 13, 2019 3:34 am

well thats your opinion.
being skeptical of industry funded closely held data and results- if any-available to public scrutiny
a lot like mann and buddies really..
making it hard to get the GMO organisms/seed or other to test in independant labs by non industry scientists stinks to high heaven
claims of CRISPR being so precise…hmm? just like the older gene inserts were claimed and found NOT so,
the honest CRISPR researchers ARE admitting it isnt as precise OR controlled as claimed.
deaths serious adverse events etc already documented.
I suspect theyd hoped to follow lewpapers ravings that all skeptics on AGW were also anti gmo flatearther whatever conspiracy theory labels they can find to discredit anyone who is NOT a sheep and dares to hold a non concensus thought in their heads!
for clarity Im skeptical on vaccines after personal events, actually reading data, and the actual prevention rates compared to the small likelihood of serious harm from NOT using them, some are useful
but in far less enforced volumes and frequencies than pushed presently.
I wouldnt trust any Big Agri as far as I could throw them prior lies and events show they lie and apply duress and legal threats rather than be open.
and again from personal and friends serious issues including death, Im very skeptical of Pharmas drugs also
some are ok, in serious need like antibiotics and some heart or other meds but the majority of problems have safer options inc doing nothing. again its personal but heart meds I was taking didnt really resolve much but they sure gave me some extra effects to worry about. stopping them made me feel so much better;-)

fretslider
February 12, 2019 12:52 pm

we believe

Surely a scientist would posit an hypothesis.

Editor
Reply to  fretslider
February 12, 2019 1:01 pm

fretslider ==> I can allow the use of the word “believe” to imply an opinion held that is not entirely based on data — which is the case of its usage in this paper. They could have equally said: “it is our opinion” or “based on Kahan…we think..”
The use of the word believe indicates that it is not an idea derived from the data found in the study.

Reply to  fretslider
February 12, 2019 2:14 pm

Their comment:

we believe that this is attributable to the polarized nature of the climate change issue.”

is perhaps better re-phrased as:

“…. we hypothesise that this is attributable to the polarized nature of the climate change issue.”

Gary
February 12, 2019 12:54 pm

Kip

[If any reader knows of published, pre-registered study designs in climate change, please give me a link in comments.]

Federally funded climate research requires a grant proposal that should address most of the study design issues. Whether these are publicly available and, if so, are they archived and for how long are open questions.

Editor
Reply to  Gary
February 12, 2019 1:06 pm

Gary ==> A grant proposal or request is not the same as a Study Design, as far as I know. Grants may cover many studies over a wide period of time. Surely Grant Proposals contain some idea of what will be done in a general sense, but not the details required of a precise Study Design.

From the “funding” sections at the ends of many many papers I have read over the last year, studies may have partial funding from differing grants to different authors and institutions.

Some readers here write Grant Proposals and do studies, perhaps one will enlighten us on the difference.

Rick C PE
Reply to  Kip Hansen
February 12, 2019 6:29 pm

I’ve done a few proposals. In my field most proposals are responses to Request for Proposals (RFPs) written by bureaucrats from a government agency with some regulatory authority. They tend to be quite specific in their “statement of work”, “procedures” and “deliverables” sections. They also tend to be fairly transparent in what results they expect – i.e. justification for more regulation.

One RFP I reviewed and declined to answer was laughable in terms of sample size, lack of defined measurement procedures, replicates, statistical analytical methods, etc. A proper study to answer the question being investigated would have cost 10-20 times the budget allowed. Of course someone did submit a proposal and got the grant, but I’ve never been able to find a report of the results.

OweninGA
Reply to  Rick C PE
February 12, 2019 6:47 pm

I have seen far too many RFPs that basically tell the researcher what they will find. My response has always been “why pay for a study then? You already know what you are going to accept as valid.”

Paul Penrose
Reply to  OweninGA
February 13, 2019 9:52 am

Because they must spend the money in order to get more during the next budget cycle.

Quilter52
Reply to  OweninGA
February 14, 2019 11:23 pm

OweninGA

this is called FARKing – Falsifying after the results are in !

Crispin in Waterloo
Reply to  Kip Hansen
February 13, 2019 9:20 am

A study design is a detailed description of the experiment to be conducted. The reason to do it is to allow someone else to replicate the experiment in another location or with a different material (etc).

An experiment, following a Standard Operating Procedure, is usually stored at the lab and numbered. Using an SOP would have solved Michael Mann’s math errors before he started on his temperature proxy work.

If you use a non-standard experimental procedure, or intend to, then the Study Design option is there. Later, if the experiment proves useful, it can be published as an SOP.

Examples of SOP’s include how to write a Standard, and how to calibrate equipment, how to calculate a result and how to archive data.

Anything “new” is suspect until examined, validated and shared. All climate models are “new”.

Crispin in Waterloo
Reply to  Kip Hansen
February 13, 2019 4:15 pm

I am in the wonderful position of working on “blue sky” methods and procedures. From the “results “that work” I get to define the SOP’s that others follow. The reviews come after, not before.

Part of the work involves testing the SOP’s of others to see if they are correctly constructed, for example perform a conceptual analysis and uncertainty propagation calculation. It is true fun.

OweninGA
February 12, 2019 12:55 pm

I have had this argument with many – it isn’t lack of scientific knowledge that keeps me against the CAGW/CACC bandwagon. It is because I know science when I see it and what I see doesn’t pass the smell test. If they seemed to be in the least bit aware of the limitations of their very poor data and looked seriously at their experimental designs to try to find the glaring holes in the logic, I might be more inclined to listen to them.

This paper seems to do well, but then doesn’t present the null result in any detail or even recommend how to improve the study to attempt to tease out better cause/effect relationships. I wonder what scientific questions they asked? I have seen surveys put out by social scientists that use the conclusions of the global warming meme as the input for scientific knowledge, when the conclusions are not science. That would seem an especially foul practice if pro/con CAGW was the thing you were testing against. They should ask broad questions about biology, chemistry, and physics staying away from the two areas (GMO and CAGW) they are testing for and then measure. If they really want to quantify “scientific knowledge,” they would use the progressive testing strategy that gives the person harder questions as they get them correct to see how deep their understanding is in each area. You could then show that ideation was associated with a weakness in a particular area of science.

This works wonders if all your answers are correct, but I saw one psychologist’s survey that got the basic science questions wrong and then claimed it was the respondents who were messed up. It was stopped in publication process thankfully*, as that would have been an institutional black eye if it had seen the light of day. *(wasn’t my doing – the peer reviewers actually caught that one – I just saw the survey and key afterward.)

Editor
Reply to  OweninGA
February 12, 2019 1:13 pm

OweninGA ==> Part of the beauty of this particular study is that the authors post all the important bits on a public access data portal — found here. So we can read the questions used in each of the surveys.

In the larger sense, you are right — these surveys of scientific knowledge seem a little wonky sometimes.

Take a look at the link above and let me know what you think about those used in this study. I seem to recall that they used some pre-existing generally-accepted-in-the-social-sciences set of questions.

OweninGA
Reply to  Kip Hansen
February 12, 2019 2:18 pm

Those questions in study 1 would indicate to me whether or not the subject was awake in high school science class. Not that they ever actually cracked a book mind you, just awake for the lectures. They really were extremely basic. Of course, 60 years ago they would have gotten an argument on the plate tectonics question from some quarters. If the people complaining about GMOs did poorly on that survey, the world is in much worse shape than I thought! (oh dear, did I just use the AGW meme’s favorite line…oh bother.)

Looking at Study 2’s questions, they were taking a much deeper dive into the controversies surrounding GMOs and attitudes concerning food and life in general. They didn’t go into CC at all. It makes me wonder if they didn’t do the deep dive on attitudes on CC, or if they just didn’t post them. It also makes me wonder if they devised study 2 after doing study 1 and discovering their hypothesis on CC was a bust, which would indicate some post hoc bias going in.

I wonder about the grading though. A 7 point scale on a true/false answer key? Looking at the csv files, there were a large number of people who answered very poorly and got half the questions on the wrong end of the 7 point scale. It didn’t seem to correlate with education level either. A college assistant professor from the USA (who was anti-GMO) got over half of them wrong, while one of the folks who put 1 down for education (either less than high school or high school depending on start at 0 or 1) got all of them right.

Editor
Reply to  OweninGA
February 12, 2019 3:19 pm

Owen ==> Welcome to the uncertain and slippery world of the social sciences!

What should astonish and terrify us is the number of survey responders who are shown to know next to nothing about science (or worse, believe the reverse of the true answers.)

Read Hans Rosling’s FACTFULNESS : Ten Reasons We’re Wrong About the World–and Why Things Are Better Than You Think and take his test in the front of the book. (The link is to my book review.)

OweninGA
Reply to  Kip Hansen
February 12, 2019 6:57 pm

You know, Ronald Reagan had a great statement on that:

It isn’t so much that liberals are ignorant. It’s just that they know so many things that aren’t so.

Around the physics department we would substitute social sciences for liberal though.

Editor
February 12, 2019 12:55 pm

Thanks, Kip,

Regards,
Bob

troe
February 12, 2019 1:06 pm

To a layperson it is striking that this is not only suggested but mandatory:

“It would be a major improvement in all the sciences, helping to curtail the Irreproducibility Crisis, if researchers worked up carefully-crafted, precise Study Designs for every proposed research effort, posted them to a registration site for their scientific specialty, and then followed the design with scientific precision, utilizing the standard protocol illustrated above. This emerging trend is called for by the National Academies of Science; The Center for Open Science; the American Psychological Association and others.”

Seems obvious but then again if I’m a political type that might not achieve my goals. Call me a skeptic.

Editor
Reply to  troe
February 12, 2019 1:18 pm

troe ==> It is educational to read some of the links provided on WHY it is suggested and what the pre-registration movement hopes to achieve.

In this case, the authors performed an analysis not called for in their study design to try and recover from a null result in 1/2 of the study. Without the pre-registered study design, we would not have known this fact, and the authors might have been tempted to re-write their original hypothesis to include the unexpected findings.

troe
Reply to  Kip Hansen
February 12, 2019 1:29 pm

Thank You Kip. Unfortunately for honest scientists this is where things have gotten off track. These folks seem to have gone part of the way but we know here how difficult it can be to overcome preconception, peer pressure, and pride.

OweninGA
Reply to  troe
February 12, 2019 6:44 pm

From the Late Great Richard Feynman:

The first principle is that you must not fool yourself – and you are the easiest person to fool

In the lab, if I get very unexpected results, I always have someone outside the project take a look to see what I did wrong. I do not want to fool myself and make a fool of myself in public.

February 12, 2019 1:11 pm

The failure to find a statistically significant result for climate change must have been a huge disappointment for the authors. Their hopes for TV appearances, speaking engagements, a real journal article in Nature rather just a Letter, early promotion and a pay raise, a position in a more desirable university, etc.–all dashed.

Editor
Reply to  Ralph Dave Westfall
February 12, 2019 1:31 pm

Dave ==> The lead author, Phillip Fernbach, is well recognized in his field. His bio reads:
“I am an assistant professor of marketing in the Leeds School of Business at the University of Colorado, Boulder. I study how people think and make decisions. Much of my work is inspired by causal model theory, the idea that people’s judgments and decisions are based on knowledge of the world, knowledge that is represented in terms of causal structure.” He can’t be all bad, as it continues “When I’m not busy working on research I spend most of my time playing with my son and daughter, taking scenic walks with my wife, flatpicking my Martin HD28 at local bluegrass jams”.

(My Martin is a 0017 built in 1949 )

Fernbach has other interesting opinions.

troe
Reply to  Kip Hansen
February 12, 2019 1:49 pm

well there it was in the first sentence. He would be thrown clean out of the bluegrass club if he was honest on climate.

Brett Keane
Reply to  troe
February 13, 2019 2:43 pm

Uh, troe, it’s the sort of folk who mine coal, who invented Bluegrass. Folk like me…… Brett

Bruce Cobb
February 12, 2019 1:28 pm

Wait; “Extreme opponents of genetically modified foods fossil fuels, especially coal know the least but think they know the most”. It works!

Editor
Reply to  Bruce Cobb
February 12, 2019 1:35 pm

Bruce ==> Certainly the instantly famous AO-C erroneously believes she knows the most about this country’s energy needs but truly knows almost nothing.

D. J. Hawkins
Reply to  Kip Hansen
February 12, 2019 2:01 pm

Kip;
I would suggest that AO-C in fact knows less than nothing. Her understanding constitutes negative knowledge.

Editor
Reply to  D. J. Hawkins
February 12, 2019 2:18 pm

D.J. ==> I too believe that every time that you “know” something that is Not True, you become “dumber”, less knowledgeable, less able to make rational decisions about your life and the world around you. To me, this is not only a cognitive effect, but a spiritual effect as well — it is spiritually harmful to know/believe things that are not true.

Ignorance is better than false knowledge. Knowing that you don’t know something is almost as valuable as knowing.

Hugs
Reply to  Bruce Cobb
February 13, 2019 3:30 am

It seems we have a consensus on that.

HD Hoese
February 12, 2019 1:50 pm

More than a few papers nowadays seem to have circular reasoning. Deniers are conservative because conservatives (or less known synonyms) are deniers. It may be a coincidence but I started teaching advanced students about logical errors some three decades ago when a lot of this nonsense started. Science students seemed to need it.

These subjects are always more complex. I had a flu shot, but I am concerned about vaccines for a good reason.
I taught evolution, the fit survive because the survivors are the fittest. Null hypotheses are fun as are tests of significance. Surveys of beliefs are beyond me. Imprecise!

Editor
Reply to  HD Hoese
February 12, 2019 1:59 pm

HD Hoese ==> If you get a chance, volunteer to judge a local high school science fair — they nearly always need more educated volunteers.

I did so a few times while living in Florida — and discovered some very interesting holes in our science educational system and also some very bright students who far exceeded their instructors.

I judged a biology project that tested the effect of FM frequency radio waves on seedlings — by playing an FM radio (tuned to NPR) near the seedlings as the exposure. The students science teachers and advisors approved the project and it advanced to a regional science Fair….

And, yes Yes YES, so many of these “social science” studies fail before they begin — because of STUDY DESIGN incapable of testing the posited hypothesis.

Global Cooling
Reply to  Kip Hansen
February 12, 2019 2:43 pm

Do you mean that everything that happens to the seedlings must be due to the FM radio?
🙂

Editor
Reply to  Global Cooling
February 12, 2019 2:49 pm

Global Cooling ==> I engaged in a little HARKing on this one, figured the poor seedling performance under FM exposure was due to being forced to listen to NPR 24 hours a day.

old engineer
Reply to  Kip Hansen
February 12, 2019 6:54 pm

Kip
It’s hard to believe the teachers and advisors really didn’t understand that the radio was just a receiver of radio waves and not an instrument to measure the intensity of radio waves. But then they were biology teachers. Not sure if high school science teachers in Florida are required to major in the subject they teach or not. So they may just have a teaching degree, with some knowledge of biology, but no knowledge of physics. Still it is very disheartening!

OweninGA
Reply to  old engineer
February 13, 2019 5:26 am

In most states in the South East US, once you have your secondary teaching certificate, you just need to pass a basic knowledge test in the subject to be certified to teach it. That doesn’t mean there haven’t been teachers pressed into teaching subjects they aren’t really comfortable with due to shortages particularly in the math and science areas.

Editor
Reply to  old engineer
February 13, 2019 8:14 am

old engineer ==> Truthfully, I was flabbergasted that the project had made it to the regionals….so many teachers and advisors that should have seen the absurdity in less than ten seconds. But there it was….

It is worse than you think — the project stated that the radio emitted FM Frequency RF because it was tuned to an FM station…..and that the seedlings not “exposed” to the FM radio playing NPR were not exposed to FM Frequency RF.

Zig Zag Wanderer
Reply to  Kip Hansen
February 13, 2019 3:47 pm

To be absolutely fair, I believe that an FM receiver is aslo a transmitter, although of much lower power (and probably irrelevant). I believe tgats how tge signal is received, by interference with the transmission of the portable radio (at least in the antenna, I assume it gets ‘out’ too).

They pretended to have detector vans to detect unlicensed TVs in the UK using this principle.

Reply to  HD Hoese
February 12, 2019 9:29 pm

HD Hoese:

“More than a few papers nowadays seem to have circular reasoning.”

There seems to be a lot of that going around.

“the fit survive because the survivors are the fittest”

Isn’t that another circular argument, HD?

John F. Hultquist
February 12, 2019 3:02 pm

From the paper:
. . . , with conservatives much more likely to oppose the scientific consensus than liberals.

Insofar as there is NOT a scientific consensus regarding climate change, the authors go off the rails before they get started.

Editor
Reply to  John F. Hultquist
February 12, 2019 3:28 pm

John ==> Yes, see mine far above. So, there we have it — the researchers never think that maybe there isn’t a real consensus if the more knowledgeable one is the less he aligns with it. Climate Science has a consensus on two very narrow points (warming, human activity contributes) and that’s all. Everything else about CliSci and its call to action on policies are not only debatable but are actively being fought out in the journals and in legislative branches of governments.

The failure of social scientists to be aware of this is a big problem.

troe
February 12, 2019 3:04 pm

“Ken Stern is President of Palisades Media Ventures and the author of With Charity for All and Republican Like Me: How I Left the Liberal Bubble and Learned to Love the Right. He is a former Chief Executive Officer of National Public Radio.” NY Post 2017

February 12, 2019 3:09 pm

There are at least 4 areas in today’s modern society whereby there is a clear pattern overlapping pattern, a nexus of emotions, regardless of political ideology.

1. Genetically modified organisms (GMO) foods.
2. Vaccination attitudes (belief that vaccinations cause everything from autism in children to cancer or Alzheimer’s in adults.
3. Climate Change alarmism and near-paralysis (think Eric Holthus).
4. Personal cell phone radiation (which is about to get a whole lot worse with the high mm-wave RF and power levels that 5G will need).
The common ingredient in all 4 is Fear — an irrational fear that is based on zero observational evidence.

The late and great Dr. Michael Crichton explains this irrational fear very well in this presentation.
The full 100 minutes of his States of Fear talk:
https://www.youtube.com/watch?v=NvYPmADfZRI

or for those who want only the brief < 3 minutes beginning of his talk:

Editor
Reply to  Joel O'Bryan
February 12, 2019 3:36 pm

Joel ==> The disasterists and the apocalypse-mongers have been having a heyday during my entire lifetime. They only hope that I see is that there is some movement to teaching critical thinking to high-school and uni students.

Unfortunately, the internet has been a plus and a minus in this battle — with falsehoods being promoted by celebrities (think Gwyneth Paltrow) , Bill Nye, Al Gore, McKibben, etc — not only on CliSci but on nearly everything. Teaching discernment is a tough task, especially on health and science issue where to discern one needs a solid knowledge base of the basics of a multitude of topics.

That’s why I write.

Reply to  Joel O'Bryan
February 12, 2019 4:26 pm

Joel O’Bryan

Re: your point 4.

I had to laugh the other day when there was a discussion on BBC Radio 2 about Cellphone mast sites.

The discussion finished with two callers, one who recited what is probably an urban myth; a woman objecting to the erection of a Cellphone mast close to her house which she was convinced was a cancer risk. She was seen the next day standing outside a pub smoking a cigarette.

The next caller was genuine, and a cancer victim; she also blamed the nearby Cellphone mast for her cancer. Sadly, being the BBC, the presenter (Jeremy Vine) didn’t bother to ask her how many more cancer victims were living nearby.

The BBC make a habit of this, especially related to Climate Change. Unsubstantiated claims made by callers who are rarely challenged by the presenter are broadcast to the world (bearing in mind the BBC’s hitherto impeccable reputation for impartiality) and this subtle means of propaganda goes unchallenged because, well, what can you do about the public’s opinions?

However a leaked BBC document recently instructed broadcasting staff (presenters, producers etc.) that it was not necessary to entertain the opinions of Climate Sceptics, furthermore there was a days training on offer to help ‘deal’ with them.

Thanks to doing a lot of reading on WUWT and Notalotofpeopleknowthat, and engaging in discussion with many on these sites I am now far more critical about what is said on any subject where evidence is a prerequisite to substantiating a statement.

The BBC is also amongst the very worst at propagating the Catastrophic Climate Change myth both overtly and covertly. It’s ‘science’ advisor’s aren’t fit to wear a lab coat far less objectively present the science to the general public.

Hugs
Reply to  Joel O'Bryan
February 13, 2019 3:00 am

It seems I agree with you on all points, but the point three is what I pounded on in my comment below. GMO fears go well together with extreme fact-free climate alarmism. That is why it is so difficult to get the results. They were holding the ruler upside down, and were searching for the ecoloonie position from the denialist / AGW optimist end.

ozspeaksup
Reply to  Joel O'Bryan
February 13, 2019 3:44 am

ever heard of SV40?
go look
and ever read the ingredients list of a vaccine?
Im sorry but porcine/bovine/canine dna or human foetal cells as well as a slew of other stuff that would be banned in food, being injected into a babies system especially is just seriously questionable for safety and unknown issues in later life

Brett Keane
Reply to  ozspeaksup
February 13, 2019 8:22 pm

Oz: You should be convinced by the relative disease figures with and without population immunities, and the increase of 30yrs average life expectancy overall in 100yrs. If not, your problem until such time as you spread diseases to us……. But you are a true believer who should also refuse to use bridges. Brett

Reply to  Joel O'Bryan
February 13, 2019 10:13 pm

What is “overlapping pattern” in scientific terms? Have you left out the fear of asbestosis and mesothelioma by accident? Is fear of rabies irrational? How about fear of ineffective rabies vaccines? Is fear of radon rational? If so, at what threshhold?

More and more I get the impression here that anyone who raises an alarm about a potential health threat is simply crying “wolf!”. How rational is that?

All this study has proven is that it had no clear scientific purpose, only the aim to denigrate the intelligence and education of people who distrust orthodox science . And even in that it failed miserably. True scientists would have struggled to resolve the contradiction between the GMO and ACGW surveys instead of dismissing it carelessly.

michael hart
February 12, 2019 3:36 pm

Note that it would be a mistake to claim a pro-skeptic result from the paper as the study design was only adequate to test the original hypothesis which failed.

Many of us are skeptics on the basis that much of climate alarmism doesn’t get past the ‘null result’. So it is very much pro-skeptic in the sense that we would like to see real science, not environmental activism shot through with politics.

Editor
Reply to  michael hart
February 12, 2019 3:39 pm

Michael ==> I agree — see my two-part series on Why I Don’t Deny.

Chris Hanley
February 12, 2019 3:51 pm

“Unlike beliefs about GM foods, climate change beliefs were highly polarized by political identification, with conservatives much more likely to oppose the scientific consensus than liberals …”.
========================================================
The authors use a climate change™ “scientific consensus” that doesn’t exist as a distraction, the point is that anti-GMO and anti-fossil fuels are based on the Appeal to Nature Fallacy that usually is the province of the Left.
The precautionary principle is fine for individuals to employ as they see fit, everyone has the right to exercise their informed choice to consume GMOs or fossil fuels or not.
But it never ends there, the innate authoritarian statists amongst us always hatch some negative externality that supposedly justifies imposing their phobias through legislation.

Keith
February 12, 2019 4:46 pm

It’s a shame the researchers chose not to explore the AGW part of the study as it would have lead to a very interesting analysis. “If lack of scientific knowledge corresponds to anti-science viewpoints for GMO but not AGW, then what is the difference between those two subject areas?”

Keith
Reply to  Kip Hansen
February 12, 2019 8:12 pm

Kip: of course, didn’t mean to imply there wouldn’t be new studies for new questions. The researchers chose to bury the lead that would bring attention to such questions.

Gary Pearse
February 12, 2019 6:15 pm

Mods let me remove the last sentences and resubmit

Gary Pearse
February 12, 2019 6:20 pm

Some of the verbiage is awkwardly phrased and ambiguous, to wit ‘lack of relationship between “scientific literacy” and extreme anti-scientific consensus’. Scientific literacy has a an aspect of degree, does it not? Also, isnt anti-scientific consensus a virtue to a real scientist? ‘Extreme’ could be considered redundant from this viewpoint. I guess relying on a sociologist’s understanding of what science is all about is the problem.

So what about the survey? The sociologist’s misunderstandings (and the biased structure of the dicipline) makes climate science an ‘outlier’ in the company of GMO and gene therapy. These are working sciences and the products are real. No anti-GMO or anti-gene therapy extremist has the least doubt these sciences they just fear and oppose their application.

The scientifically literate opponents of what one could legitimately call extreme climate science (we’re toast in 12yrs!) don’t argue human activities don’t have an affect. They argue that the effect so far seems minor at most and benign – modest T rise and CO2 do have considerable benefits (greening of the planet, bumper crops) and in terms of the magnitude of the effect of higher temperatures in recent decades, perhaps most of it is natural variation (The Big Pause). There is yet no evidence for there being a serious problem and draconian remedies that will impoverish and kill a large percentage of the people of the world is not justified. Moreover the obvious jiggering of data to create a crisis and to try to correct predictions is an indictment from within!

Scientifically literate sceptics are certainly not inferior to scientific proponents of crisis warming so they would have to be considered strongly knowledge based. Even climate scientists reveal their fear of such sceptics in the climategate emails (gatekeeping peer review, intimidating editors and even forcing their resignation for publishing a sceptical paper, destroying data and other acts to keep their work from being scrutinized). Consensus science papers have had to be withdrawn, retracted and heavily revised following McIntyre’s critiques. Their fear of debating shows their own lack of confidence in the theory

Finally, the political aspect. Both sides have layperson support on political grounds and proponents deliberately conflate this to brand sceptics because they know to acknowledge the prodigious skill of the best of them would be throwing in the towel. Regarding the politucally motivated supporters of consensus, they have a much larger contingent of supporters because the ‘humanities making up 85% of university population are all-in on the crisis science side as are most from the media, government bureaucracy, learned societies, etc worldwide.

Global Cooling
February 12, 2019 9:59 pm

Excellent post altogether. Thank you. I saved the image of pnas.org to my archives.

Activists use media to distort people’s perception of the world. Instead of misreporting the methods, they often just ignore it. Appeal to authority is enough. Hypothesis becomes fact when a scientist says it. I immediately noticed these points of misreporting the results and misinterpretation when I read the IPCC’s reports a decade ago. The famous trick is only the top of the iceberg.

DaveW
February 12, 2019 11:11 pm

“The key prediction is that the gap between objective and subjective knowledge widens as extremity of anti-scientific consensus beliefs increase…”

Kip – as I understand Fernbach et al.’s hypothesis and experimental design, their results falsify the key predictions. That is, the study was replicated over two domains (GMO, AGM) but was supported in only GMO. Therefore, there is no generality in their prediction and the yawning gap between objective and subjective knowledge in those with GMO anxiety would appear to be a special case.

Just to do my own bit of Harking, and as someone commented above, I think they have an apples and oranges problem. Maybe they would have gotten similar results if they’d tested anti-vaxers, but GMO hysteria is largely fact-free and Climate Skepticism is largely fact-based. So, it appears to me that their biases led them to made an inappropriate comparison, even though there was pre-existing data indicating that climate skeptics tended to be knowledgeable.

Hugs
Reply to  DaveW
February 13, 2019 2:44 am

May I express my complete disagreement or something. The statement was

The key prediction is that the gap between objective and subjective knowledge widens as extremity of anti-scientific consensus beliefs increase.

I think this statement is not false but trivially true, if carefully investigated. Extreme anti-scientific beliefs are necessarily supported by subjective knowledge in absence of objective knowledge. What the word consensus above adds, I don’t really follow, other than that it appears the question above is framed with assumptions based on the consensus, whatever it might be (continents can’t move, space we live in is fully Euclidean).

They continued

We further predict that as extremity increases, subjective knowledge increases, but objective knowledge, as measured by a battery of scientific literacy questions, decreases.

This is all right, the problem is – and please correct me if you think I’m wrong – that the good researchers appeared to think any AGW optimism is an extreme position. There are denialist positions, such that clearly can’t be justified based on a scientific worldview and broad scientific knowledge. There are also positions in the global warming discussion that can be rightly grouped with GMO conspiracy thinking. Such thinking includes irrational fears of quick Greenland glacier melting, irrational energy policy goals, thinking that the Earth comes Venus-like in business-as-usual development, or thinking that the population bomb is gonna explode any second now.

The good researchers (and I really think they are good) were holding the ruler upside down. What they should have investigated is extreme climate alarmism, and they would probably have noticed that extreme anti-scientific alarmism goes and rhymes well with GMO fears in the sense that extreme fears, extreme certainty and lack of objective knowledge go well together.

They could also see this pattern among people who aggressively deny GHE, but I guess the other end would have a much larger gravity.

I should probably now read the effing paper before I make more a fool of myself.

DaveW
Reply to  Hugs
February 13, 2019 2:46 pm

Hi Hugs,

Let me know what you find out. I read the paper, but not as critically as Kip. I agree that the authors got AGW upside-down, but can you imagine a university ethics panel, government funding body, or high profile journal approving of a study whose hypothesis was ‘the more people fear CAGW, the less they actually know about climate’? Never happen. I think they included Climate Skepticism because (a) they assume the alleged consensus is correct and (b) that increased their chances of funding (this study was funded by several grants including one from NSF – the authors would have had to describe their basic hypothesis in the applications) and publication. I may be giving the authors short shrift here: maybe they also thought Climate Skepticism a strong challenge to their hypotheses because of the existing data that seemed to contradict it. That would be a good thing to do and these authors do seem to have tried to design a good study. More the pity they descending into handwaving in their discussion.

I’m only guessing if I try to understand what was in the authors’ minds, but it seems to me that they assume scientific consensuses are true and that any deviation from consensus is wrong; and therefore, the more one deviates from a consensus the less objective knowledge one must have (because how could one not agree with obvious truths?). So, if they can demonstrate that this is a general phenomenon, then they have something that could be published in a high profile journal.

I think they are wrong at a much deeper level – a ‘scientific consensus’ is a social construct, not an aspect of the scientific method. So, I see no a priori reason why knowledge of relevant facts should be limited to those who agree with a consensus. As Kip notes below, he has studied the same facts as the IPCC, but he doesn’t draw the same conclusions they do. This is a pretty normal aspect of science – the same facts give rise to different interpretations.

Editor
Reply to  DaveW
February 13, 2019 8:33 am

Dave W ==> I have given this point some real thought — before writing about it. There is nothing about the two topics — GMOs and AGW — that would disqualify examining attitudes etc about the two of them. They both have strong scientific consensus positions (in their respective fields) and have a continuing strong opposition as a minority view which does not disappear despite massive public propaganda/information campaigns.

But when they get different results in this study, the authors fail to suggest that there might be something that differs about the consensus topic…instead, they attempt to justify or re-hypothesize about the what is in the minds of the people surveyed — they refuse to see if they made a mistake considering their prior assumptions about the consensuses — that all anti-scientific-consensus positions are equally some sort of faulty cognition.

Judith Curry featured an essay about a paper that posited that the difference in climate science may be that the highly educated/knowledgeable anti-consensus position derives from the consensus position based on the value which each extreme places on the differing types of evidence and differing knowledgeable interpretation of the same evidence.

My two essays on Why I Don’t Deny: Confessions of a Climate Skeptic are a demonstration of this — I accept almost all the evidence supplied by the IPCC but draw opposite conclusions from it.

DaveW
Reply to  Kip Hansen
February 13, 2019 3:51 pm

Hi Kip,

I think we are in essential agreement. The authors weren’t expecting asymmetrical results and should have simply admitted that, based on their results, their primary and secondary hypotheses were not general explanations for disagreement with a politically dominant ‘scientific consensus’. Where we differ seems to be that I think one reason they got different results is that Climate Skepticism and fear of GMOs etc. are not the same, but apples and oranges.

Everyone has some experience with climate, has to live with its manifestations daily, and can’t help but notice that the predictions of doom are hyperbolic, constantly changing, and never seem to eventuate. Few have much experience of genetics (or nanotechnology) and the fears are of the unknown. Also, GMOs, gene therapy, nanotechnology, etc. are not subject to the same intensity of negative publicity and malign political pressure as is climate skepticism. You can be fact-free anti-GMO and tell your friends without them cutting you off. So, I think there are reasons not to consider anti-GMO etc. and Climate Skepticism members of the same class for this comparison.

Cheers,

Cheers

Gary Pearse
Reply to  DaveW
February 13, 2019 9:01 pm

Dave and Kip: my read on this is climate consensus doesnt belong in the same grouping as GMO and gene therapy simply because the latter two aren’t in dispute as solid science. They produce tangible products. They are eminently replicable and both sides agree on this The “anti” folk just don’t think we should be doing it. In alarmist climate science, the argument of skeptics concerns the fundamental legitimacy of consensus science itself, their methods and interpretations. So far, the only unequivocal climate change from added CO2 is the very beneficial Great Greening and bumper crops that none of them like to mention.. There is a better case for a preponderant net benefit as it stands to date.

BillP
February 13, 2019 2:47 am

A bit off topic, but I have always thought that it is stupid to regard all GMO as good or bad.

It seems to me that GM is simple a tool, and like most tools it can be used to produce good or bad results.

So, we should be sceptical about each GMO, but we should not allow fear of bad GM to deprive us of the benefits of good GM.

john
Reply to  BillP
February 13, 2019 6:35 am

I agree completely with this. The idea that all GMO’s are fine is ridiculous. That’s why the field is regulated. So the question becomes, do we trust the regulators? Since the developers of these organisms are mainly motivated by profit it is certainly not prudent to assume they wouldn’t push the boundaries of what’s safe. Additionally, we cannot assume that all the actors have infinite foresight as to what the permutations may be of adding new organisms to the world.

Editor
Reply to  john
February 13, 2019 10:01 am

Bill and John ==> We have all been eating GMO foods for many years, without knowing it for many. Corn and soy and cotton (thus, cottonseed oil):

“Currently, up to 92% of U.S. corn is genetically engineered (GE), as are 94% of soybeans and 94% of cotton (cottonseed oil is often used in food products).” (LINK)

Luther Burbank (my childhood hero) engineered fruit after fruit to improve types and produce “new” fruits. He also developed the Russet Burbank potato that was somewhat resistant to potato blight for the starving masses in Ireland. He did it all the old fashioned plant breeding way.

BillP
Reply to  Kip Hansen
February 13, 2019 10:44 am

As I said, I am as much against regarding all GM as bad as I am against regarding it as all good.

I just think we need caution, which regulation should be providing.

If you include traditional plant breeding; all our food is GM.

Gary Pearse
Reply to  Kip Hansen
February 13, 2019 9:17 pm

Kip, in a gross overview, GMO and original are both made out of Carbohydrates, the usual accessory minerals and trace elements. We probably change the actual chemistry more by cooking than by genetically modifying food. Early efforts did produce some muscular bland tomatoes and fishy cucumbers but they are getting hetter all the time.

Kip, I greatly appreciate your analytical essays and keeping the record straight. I originally bought into the “continent” of plastic in the Pacific although I always assume exaggeration. Your reporting on the issue I have shared widely.

Editor
Reply to  Johann Wundersamer
February 13, 2019 8:41 am

Johann ==> Not quite sure what the purpose of this google search link is….no signs of pre-registered study designs for climate science that I could see.

Doug Huffman
February 13, 2019 4:44 am

Two books have greatly influenced how I think of such.

First was E. T. Jaynes’ masterwork, Probability Theory: The Logic of Science (Cambridge 2003), where I believe occurred his first caution against ad-hockery and to maximize the entropy of the subjective naive prior probability (admit ignorance).

Then, and I have just finished a re-read, James Franklin’s The Science of Conjecture: Evidence and Probability Before Pascal (JHU 2001).

Editor
February 13, 2019 8:56 am

Wm. Briggs has posted on this paper as well, from a statistician’s point of view.

DaveW
Reply to  Kip Hansen
February 13, 2019 6:06 pm

Kip – this link goes to Word Press log-in when I try it.

brent
Reply to  DaveW
February 13, 2019 9:16 pm

On Over-Certainty In GMO Beliefs
http://wmbriggs.com/post/26368/

DaveW
Reply to  brent
February 14, 2019 3:15 am

Thanks Brent. With such a great header photo, from ‘Detour’ – possibly the best of the worst film noirs (although I lean towards ‘Gun Crazy’ because of the stronger femme fatale) – how could one not like the ‘Statistician to the Stars!’ and his take is an exceptionally logical take down – and I especially like his poking fun at Taleb. I have read of potential disasters like celery so replete with furanocoumarins that anyone brushing against a stalk came away with blisters once the sun hit them and potatoes so high in solanine they were toxic – although I think both of these pre-date GMOs. Anyway, best not to be the first person to try a GMO, and trusting government bodies to do due diligence is perhaps over-optimistic, but pretty much everything we eat has been genetically modified so far from its wild state that being worried about the technology used to produce new crops is not an informed position.

brent
February 13, 2019 4:30 pm

“How can anyone fail to support/believe a proffered scientific consensus?”

Richard Lindzen

And it’s part of NSF’s big mobilization. They are spending quite a lot of money to find out why people aren’t buying the alarm. And this harkens back to my personal attitude. Ordinary people have sense: Academics Don’t.
http://tinyurl.com/pkd7w7q
11 min

Alarming Global Warming: What Happens to Science in the Public Square. Richard S. Lindzen, Ph.D
Richard Lindzen does an excellent job in explaining what happens to Science in The Public square, when science is touted as “An Appeal to Authority” rather than science simply being viewed as a wonderful investigative process/methodology

brent
February 13, 2019 5:14 pm

GM corn set to stop man spreading his seed
Scientists have created the ultimate GM crop: contraceptive corn. Waiving fields of maize may one day save the world from overpopulation.
https://www.theguardian.com/science/2001/sep/09/gm.food

Wouldn’t this be a dream come true for the Ehrlich/Holdren crowd ??
Didn’t they muse about adulterating water or food supplies to achieve their goals?

February 13, 2019 9:43 pm

I couldn’t find the “15 true− false questions adapted from the National Science Foundation’s Science and Engineering Indicators survey”. Can anyone tell me what they were?

Reply to  Kip Hansen
February 14, 2019 10:34 pm

I found that sample, although the link footnote “25” gave the same list of “factual” questions as “Table 7-3”. I could scarcely believe that anyone claiming to be a scientist would consider yes or no answers to such questions to be a valid test of scientific knowledge.

In fact, they are all “received knowledge”. You could call them dogmas as well. A slightly more rational way of using such crude questions would be to offer a third option – “don’t know” and maybe even “not sure”. But in the end, no matter how you cut this nonsensical survey, the results are unquantifiable, and quantification is surely an essential element of all scientific research.

Just to add a personal note – I found that I knew the orthodox answer to all of these questions, so presumably I’m knowledgeable by the study’s criteria. But in fact, most of it is just rote learning for me. Some of these “facts” are nothing more than definitions (lasers, antibiotics, electrons). Others seems to make sense, such as the heat of the earth’s core. And yet others, like the Big Bang theory, require acceptance on faith. But if I’m willing to accept the Big Bang theory on faith, but not String theory, does that make me less factually knowledgeable?

My scientific stance is that if something can’t be communicated in the common language of society, it’s not credible, and deserves to be viewed skeptically and accepted only tentatively, if at all.

But I digress. The report refers to “questions ADAPTED from the National Science Foundation’s Science and Engineering Indicators survey”. And I asked what those ADAPTED questions are.

Must we conclude from your evasion that these critical questions were not published by the researchers?

brent
February 13, 2019 9:45 pm

Bioengineering Humans To Combat Climate Change
Aren’t bioethicists a fun bunch? When last we met this speculative crew, they were wondering whether we should “let baby live“, and they were developing a pill to eliminate racism.
And now they have figured the Final Solution for climate change. Bioengineering people. Yes: so say academic philosophers Matt Liao of NYU, and Anders Sandberg and Rebecca Roache from Oxford in their forthcoming peer reviewed “Human Engineering and Climate Change” in the journal Ethics, Policy and the Environment
http://wmbriggs.com/post/5392/

A Pill To Eliminate Racism? What’ll Science Think Of Next!
http://wmbriggs.com/post/5356/

Study Finds 9-Month-Old Babies Are Racist
http://wmbriggs.com/post/5595

Genetic Engineering To Create New Super Moral Race: Savulescu Strikes Again
The appalling Julian Savulescu1 is back, this time staining the pages of the British edition of Reader’s Digest2 with the piece The Maverick: “It’s Our Duty to Have Designer Babies”.
Savulescu is a bio-ethicist and is editor-in-chief of the Journal of Medical Ethics. The credential for being a bio-ethicist appears to be the ability to claim that one is a bio-ethicist. I thus am a fully qualified bio-ethicist. So today I offer these peer criticisms
http://wmbriggs.com/post/6047

Editor
February 14, 2019 6:52 am

Epilogue:

Science advances through an iterative process of successes and failures, right guesses and wrong guesses and a lot of almost right and almost wrong guesses. The key to the success of this method depends on the ability of scientists to replicate (repeat the same experiment exactly) in order to confirm the findings, hopefully before the flow of research has been sent down a dead-end path.

Help yourself to an internet search on the Irreproducibility Crisis or read the National Association of Scholars report.

Pre-registering a Study Design, and preferably having it peer reviewed at this stage, is one of the steps to correcting this situation.

Thanks for participating and

Thanks for Reading.

# # # # #

Shaun Cruickshank
February 16, 2019 12:17 pm

I expect I fit their GMO result fairly well, in that I know not much about it, but have a general distrust of the statement “scientists say they are safe”. To me this is “GMOs are not harmful”- so how exactly would one prove this (very general) negative?
You would need very long experience with each iteration of genetic modification to be sure it is OK.