More on Kaufmann's weather begets climate belief claims – 'junk methodology' exposed

We covered another press release on this subject December 20th, and I normally wouldn’t repeat it, but they have a new map that’s worth looking at because it bolsters the claim I made then that the issue is more about political bent and media reporting in highly democratic voter states than it is about experience with weather.

From Boston University.

By Caitlin Bird

In early 2014, freezing air from the polar vortex at the North Pole swept into the United States. The resulting record cold temperatures and snowfall grounded planes, knocked out power, and caused unusual weather across the country. Even states with normally balmy weather, like Georgia and Louisiana, saw freezing rain and declared states of emergency to get through the storms.

But the weather doesn’t have to be that extreme to leave a mark. In a study published in the Proceedings of the National Academy of Sciences in December 2016, scientists have found that experiencing record high or low temperatures affects people’s stated belief in climate change. The study was funded by the Robertson Foundation and the British Academy.

Robert Kaufmann, Boston University professor of Earth and environment, is lead author of the study. The research began when he and PhD candidate Xiaojing Tang (GRS’12,’17) wanted to develop a new measure of local climate change based on record high and low temperatures in the US. This index, called TMax, rises as the number of recent record high temperatures increases relative to the number of recent record low temperatures. After Tang calculated TMax using data from weather stations across the United States, he presented a map to Kaufmann, who was surprised to see a pattern.

“It clicked in my mind,” says Kaufmann. “It looked like where people do and do not believe in climate change.” Specifically, Tang’s map of TMax resembled maps that climate change researcher and study coauthor Peter Howe, assistant professor of environment and society at Utah State University, had compiled showing the percentage of state and county residents who, in 2013, answered yes to the question: “Is global warming happening?”

The climate picture is complicated because the US is both warming and cooling. If climate in the country remained stable, only about 5 percent of weather stations would, simply by chance, show local warming or cooling. Instead, Kaufmann and his team saw that nearly 50 percent of weather stations had high values for TMax, indicating local warming over time. Unexpectedly, about 10 percent of weather stations showed local cooling, with more frequent recent record cold temperatures. Looking at the map, warming areas are located at the coasts, cooling areas in the middle of the country, near the Ohio and Mississippi rivers.

When Kaufmann and his collaborators compared the map of TMax directly to Howe’s, they found a correlation: in counties where recent weather was dominated by record low temperatures, a smaller percentage of people were likely to agree that global warming was happening.

Why might this be? Study coauthor Jacqueline Liederman, BU professor of psychological and brain sciences and director of the University’s Cognitive Neurophysiology Lab, believes it’s because humans are prone to learn from their own experiences. What they hear from leading scientists doesn’t dissuade them from what they see for themselves. “We know people have certain biases,” says Liederman. One of these biases is something called “recency weighting,” the tendency for people to assign more value to events that have happened recently, even if they don’t fit a long pattern over time. This was especially true for counties that experienced recent low temperatures. Even if the data showed that record high temperatures were more recent over the past 30 to 50 years, people in counties where there had been many record lows since 2005 were more skeptical of global warming.

Liederman explains that the research also reflected an effect called “confirmation bias.” Essentially, it isn’t easy for anyone to accept information that goes against entrenched beliefs, so conflicting evidence is ignored. The effect of confirmation bias was one-sided in the study, found only in areas where there were recent cold temperatures. If you are more likely to trust your personal experience, and it’s been cold recently, you can discount a rogue record high temperature as just a hot day, rather than evidence of global warming.

Areas with more record low temperatures tended to be in traditionally conservative areas of the country, counties where belief in global warming was already low. Climate change is a politically charged topic in the United States and stark divisions exist along party lines regarding how serious the problem is and what the consequences will be, according to the Pew Research Center. But Kaufmann’s study found that local climate affected people’s willingness to believe in global warming beyond what party affiliation would suggest.

Kaufmann’s group is planning a future project to better determine whether political allegiances affect how people learn from experience. In the meantime, he hopes that what he and his colleagues have learned from this study will help change the way that scientists communicate with the public about climate change. “I think climate scientists have to step back and rethink…and use very different sorts of evidence to convince people that climate change is real,” he says.

###

The paper: http://dx.doi.org/10.1073/pnas.1607032113

Have a look at the map included with this press release:

Now, if you imagine reversing the colors, so that the map looks like the traditional red/blue Republican/Democratic states, it is easy to see the “leftcoast” of CA, OR, WA, and some other left-leaning hotspots, such as Austin, Minneapolis, Chicago, Washington, DC, Miami, and the traditionally democratic voting NE USA states, NY, VT, NH, CT, RI, MA, ME.

I made the same comparison before in commenting on the previous press release:

Personallly, this study looks more like a political bias coupled with a media reporting bias (left-leaning counties tend to have left leaning newspapers that will report the talking point belief of weather is a symptom of climate change) than it is anything else..

Kaufmann notes in the BU PR:

Kaufmann’s group is planning a future project to better determine whether political allegiances affect how people learn from experience.

I think he should have done that first, because that’s just good science, to weed out anything that biases the study. As it stands, I don’t think much of this study, and I’m not the only one:

Dr Richard Tol notes in comments:

The problems with this paper are more fundamental than Anthony suggests.

Kaufmann et al. extensively discuss their right-hand variable of interest, temperature, but they are almost completely silent about the left-hand side, belief. This variable is taken from an earlier paper by Howe et al. It is based on 12 nationally representative samples of about 1,000 observations each. The samples are for different years, of course, something that Kaufmann and co overlook. More seriously, there are 3,000 counties in the USA, so that a 1,000 strong sample will leave many counties unobserved. Howe et al. interpolate their data to obtain coverage for all counties. Kaufmann et al. do the same: They interpolate weather station data to counties.

In the end, therefore, Kaufmann regresses imputed, spatial data on imputed, spatial data. Chances are they found that the spatial imputation schemes are similar.

Rud Istvan adds to Tol’s comment:

My degee in econometrics taught that when you regress madeup data on madeup data, the result is highly statistically likely to be made up, err… imputated. Nice catch in the junk methodology.

Yep, “junk methodology”. Looks like fodder for Retraction Watch.

RomanM, a statistician, comments and does his own analysis:

Richard is correct on the fact that the belief data comes from the Howe paper that he refers to. If anyone wishes to look further at this aspect, the paper can be found at http://www.nature.com/nclimate/journal/v5/n6/full/nclimate2583.html

Although the paper itself is behind a paywall, the Supplementary material provides good information along with Excel files containing their results not only for the question of whether it is warming but also for a number of questions including whether humans are causing it, scientific consensus, etc. A pre-publication working copy of the paper itself is available at http://papers.ssrn.com/abstract=2515649

The ‘global warming question as used in Howe et al reads:

Question: Recently, you may have noticed that global warming has been getting some attention in the news. Global warming refers to the idea that the world‘s average temperature has been increasing over the past 150 years, may be increasing more in the future, and that the world‘s climate may change as a result. What do you think: Do you think that global warming is happening?

[Response Scale: Yes, No, Don’t Know]

Here is how the cobbled-up beliefs by county fare against the vote for Obama in 2012 (red lines are placed at the medians of the two variables):

https://statpad.files.wordpress.com/2016/12/gwarm_demvote.jpeg

But Kaufmann got his headlines. Anyone care to bet his followup study on political bias won’t happen?

And a personal note, Dr. Kaufmann left a comment on the previous thread that was pretty ugly, and I snipped it it because, well, it played that stupid “smoking causes cancer denial” that the left likes to hurl in the context of climate change, he didn’t give his full name, which is typical for people that throw insults. The email address he used and the IP address check out as being from bu.edu You’d think Kaufmann would stand behind his own words.

In retrospect, I should have let it stand to demonstrate how petty he is. Based on the stat experts that commented, plus my observation of the maps, I think it is safe to say that the Kaufmann study is little more than a statistical mishmash using some very bad assumptions. It should probably be retracted.