Guest essay by Dr. Richard Tol
In their eagerness to discredit a colleague[1] Harvey et al. (2017) got ahead of themselves. The write-up shows signs of haste – typographical errors (“principle component analysis”, “refereces cited”) and nonsensical statements (“95% normal probability”) escaped the attention of the 14 authors, 3 referees and editor – but so does the analysis. The paper does three things: It creates a database, it classifies subjects, and it conducts a principal component analysis. Details have not been shared on the database construction or the classification (Lewandowsky and Bishop 2016), so I focus on the principal component analysis.
Principal component analysis (PCA) aims to reduce the dimension of a dataset by a linear transformation of its variables into orthogonal components and limiting the attention to those components that are principal in explaining the variation in the variables. Harvey et al. (2017) reduce seven variables to two. One variable denotes citation of Susan Crockford. This is recorded as a binary variable, that is, no distinction is made between mentioning her work in passing, criticizing her work, and uncritically adopting her position. The remaining six variables denote agreement with the following statements:
1. sea-ice extent is on average declining rapidly in the Arctic;
2. sea-ice extent is decreasing only marginally, is not decreasing significantly, or is currently recovering in the Arctic;
3. changes in sea-ice extent in the Arctic are due to natural variability, and it is impossible to predict future conditions;
4. polar bears are threatened with extinction by present and future anthropogenic global warming;
5. polar bears are not threatened with extinction by present and future anthropogenic global warming; and
6. polar bears will adapt to any future changes in Arctic ice extent whether because of anthropogenic global warming or natural variability.
Agreement is measured on a binary scale, even though nuances are clearly possible. Taking the first statement, “extent” could refer to sea-ice area or volume, both of which vary over space and seasons so that “average” obtains different meanings, while “rapidly” could mean different things to different people. Similar objections can be raised against the other five statements.
The data released by Harvey et al. (2017) contains only zeroes and ones, suggesting that either agreement was recorded by a single coder or, less plausibly, that all coders agreed on all 6 statements by all 182 subjects.
Intriguingly, there are four subjects who appear to argue that sea-ice is neither shrinking nor stable or growing. One subject seems to argue that polar bears are both threatened by climate change and not threatened, and another is recorded as arguing that polar bears are neither threatened nor not threatened. This corroborates the above assertion that statements were coded by a single coder.
Statements 1 and 2, and 4 and 5 are mutually exclusive, and statements 3 and 6 are close to statements 2 and 4, respectively. A PCA is redundant in a case like this. The analysts artificially inflated the dimensionality of the data, before using PCA to reduce it again.
Figure 2 in Harvey et al. (2017) plays another statistical sleight of hand. The figure shows many observations taking many different positions. Seven binary observations can take at most 128 positions, rather than the 182 suggested in Figure 2. In fact, there are only 19 different positions in the underlying data. The jitter applied by Harvey et al. (2017) suggests that the “majority view papers” all take a slightly different position on sea-ice and polar bears – as you would expect had you not known about the binary coding. Actually, the 86 papers fully agree with each other, and with 27 of the “science-based blogs”. Figure 1 shows a more faithful depiction, with the first and second principal component on the axes and the size of the circles reflecting the number of observations.
Figure 1. Principal component analysis. The horizontal axis shows the first principal component, the vertical axis the second one. Colours denote the four classes (green = majority view papers; blue = science-based blogs; red = controversial papers; orange = denier blogs). The size of the circle denotes the number of subjects, with the smallest circle representing 1 subject and the largest 86.
Figure 2 in Harvey et al. (2017) shows that different classes of respondent[2] differ strongly on the first principal component, but that there are no significant difference on the second principal component. Table 1 confirms this.
Table 1. The average and standard deviation by observation class for the first and second principal component with seven or six variables per observation.
| P | majority | science | controversial | denier | |
| N | 86 | 45 | 6 | 45 | |
| PC1 | 7 | -1.50 | -1.25 | 1.40 | 3.92 |
| (0.16) | (0.41) | (1.39) | (0.91) | ||
| PC1 | 6 | -1.40 | -1.33 | 1.12 | 3.85 |
| (0.15) | (0.31) | (1.38) | (0.90) | ||
| PC2 | 7 | -0.26 | 0.51 | 1.30 | -0.18 |
| (0.03) | (1.04) | (1.06) | (1.20) | ||
| PC2 | 6 | -0.05 | -0.04 | 1.39 | -0.05 |
| (0.01) | 0.11) | (0.87) | (0.97) |
The second principal component largely reflects statement 6, whether polar bears can adapt to future climate change. Polar bears appear to have survived the onset of two interglacials and one ice age (Lindqvist et al. 2010). Unfortunately, 114 of the 182 subjects do not take a position on this. Harvey et al. (2017) replaced these missing observations with zeroes (after standardization). Omitting this column makes the first principal component more important (explained variance rises from 80% to 89%) and the second principal component less important (explained variance falls from 11% to 5%). This does not affect the qualitative results: The first principal component explains the differences between the types of observations. The second principal component does not have discriminatory power. See Table 1.
Harvey et al. (2017) thus really show that there are people who worry about sea-ice and polar bears, and those who do not and cite Dr Crockford.
But Harvey et al. (2017) do not just show that there are two camps. They take sides. Unfortunately, they count noses and argue from authority, rather than assess the strength of the evidence. It is well-known that like-minded blogs often copy or paraphrase material from one another. Similarly, academic papers often repeat a salient conclusion from previous research. Counting noses is a poor method.
The argument from authority is weakened by examining the 92 learned papers. Of the 86 “majority view” papers, 39 were authored by Steven Amstrup, Rascha Nuijten or Ian Stirling, who are among the alii in Harvey et al. (2017). Another 13 were authored by Andrew Derocher, a frequent (n=10) co-author of Amstrup.
The paper does not specify how these 92 papers were identified, beyond a “broad keyword search” on the “ISI Web of Science”. The Web of Science returns 179 articles for a query on “polar bear” and “sea ice”. No information is given how the larger sample of relevant papers was reduced to the smaller one used by Harvey et al. (2017). Comparing the relative contribution of the ten most prolific authors according to the Web of Science to their relative presence in Harvey’s sample reveals that the latter is not representative of the former (chi^2(9)=17.4, p=0.04). Research by co-authors Amstrup and Stirling is overrepresented in Harvey et al. (2017), and work by Jon Aars and Oystein Wiig underrepresented.[3] The sample used by Harvey et al. (2017) appears to be a sample of convenience, and unrepresentative.
In sum, Harvey et al. (2017) play a statistical game of smoke and mirrors. They validate their data, collected by an unclear process, by comparing it to data of unknown provenance. They artificially inflate the dimensionality of their data only to reduce that dimensionality using a principal component analysis. They pretend their results are two dimensional where there is only one dimension.
They suggest that there are many nuanced positions where there are only a few stark ones – at least, in their data. On a topic as complex as this, there are of course many nuanced positions; the jitter applied conceals the poor quality of Harvey’s data.
They show that there is disagreement on the vulnerability of polar bears to climate change, but offer no new evidence who is right or wrong – apart from a fallacious argument from authority, with a “majority view” taken from an unrepresentative sample.
Once the substandard statistical application to poor data is removed, what remains is a not-so-veiled attempt at a colleague’s reputation.
Acknowledgements
Peter Roessingh and Bart Verheggen gracefully shared data and code. I borrowed freely from comments by Roman Mureika and Shub Niggurath. Marco NN had useful comments on an earlier version.
References
Barta, J. L., C. Monroe, S. J. Crockford, and B. M. Kemp. 2014. “Mitochondrial DNA preservation across 3000-year-old northern fur seal ribs is not related to bone density: Implications for forensic investigations.” Forensic Science International 239:11-18. doi: 10.1016/j.forsciint.2014.02.029.
Crockford, S., G. Frederick, and R. Wigen. 1997. “A Humerus Story: Albatross Element Distribution from Two Northwest Coast Sites, North America.” International Journal of Osteoarchaeology 7 (4):287-291.
Crockford, S. J. 1997. “Archeological evidence of large northern bluefin tuna, Thunnus thynnus, in coastal waters of British Columbia and orthern Washington.” Fishery Bulletin 95 (1):11-24.
Crockford, S. J. 2003. “Thyroid rhythm phenotypes and hominid evolution: A new paradigm implicates pulsatile hormone secretion in speciation and adaptation changes.” Comparative Biochemistry and Physiology – A Molecular and Integrative Physiology 135 (1):105-129. doi: 10.1016/S1095-6433(02)00259-3.
Crockford, S. J. 2009. “Evolutionary roots of iodine and thyroid hormones in cellcell signaling.” Integrative and Comparative Biology 49 (2):155-166. doi: 10.1093/icb/icp053.
Crockford, S. J. 2016. “Prehistoric Mountain Goat (Oreamnos americanus) Mother Lode Near Prince Rupert, British Columbia and Implications for the Manufacture of High-Status Ceremonial Goods.” Journal of Island and Coastal Archaeology:1-22. doi: 10.1080/15564894.2016.1256357.
Crockford, S. J., and S. G. Frederick. 2007. “Sea ice expansion in the Bering Sea during the Neoglacial: Evidence from archaeozoology.” Holocene 17 (6):699-706. doi: 10.1177/0959683607080507.
Crockford, S. J., and S. G. Frederick. 2011. “Neoglacial sea ice and life history flexibility in ringed and fur seals.” In Human Impacts on Seals, Sea Lions, and Sea Otters: Integrating Archaeology and Ecology in the Northeast Pacific, 65-91.
Harvey, Jeffrey A., Daphne van den Berg, Jacintha Ellers, Remko Kampen, Thomas W. Crowther, Peter Roessingh, Bart Verheggen, Rascha J. M. Nuijten, Eric Post, Stephan Lewandowsky, Ian Stirling, Meena Balgopal, Steven C. Amstrup, and Michael E. Mann. 2017. “Internet Blogs, Polar Bears, and Climate-Change Denial by Proxy.” BioScience:bix133-bix133. doi: 10.1093/biosci/bix133.
Hatfield, V., K. Bruner, D. West, A. Savinetsky, O. Krylovich, B. Khasanov, D. Vasyukov, Z. Antipushina, M. Okuno, S. Crockford, K. Nicolaysen, B. Mac, L. Persico, P. Izbekov, C. Neal, T. Bartlett, L. Loopesko, and A. Fulton. 2016. “At the foot of the smoking mountains: The 2014 scientific investigations in the Islands of the Four Mountains.” Arctic Anthropology 53 (2):141-159. doi: 10.3368/aa.53.2.141.
Lewandowsky, S., and D. Bishop. 2016. “Research integrity: Don’t let transparency damage science.” Nature 529 (7587):459-461. doi: 10.1038/529459a.
Lindqvist, Charlotte, Stephan C. Schuster, Yazhou Sun, Sandra L. Talbot, Ji Qi, Aakrosh Ratan, Lynn P. Tomsho, Lindsay Kasson, Eve Zeyl, Jon Aars, Webb Miller, Ólafur Ingólfsson, Lutz Bachmann, and Øystein Wiig. 2010. “Complete mitochondrial genome of a Pleistocene jawbone unveils the origin of polar bear.” Proceedings of the National Academy of Sciences 107 (11):5053-5057.
Martinsson-Wallin, H., and S. J. Crockford. 2001. “Early settlement of Rapa Nui (Easter Island).” Asian Perspectives 40 (2):244-278.
Ovodov, N. D., S. J. Crockford, Y. V. Kuzmin, T. F. G. Higham, G. W. L. Hodgins, and J. van der Plicht. 2011. “A 33,000-Year-Old incipient dog from the Altai Mountains of Siberia: Evidence of the earliest domestication disrupted by the last Glacial Maximum.” PLoS ONE 6 (7). doi: 10.1371/journal.pone.0022821.
Tollit, D. J., A. D. Schulze, A. W. Trites, P. F. Olesiuk, S. J. Crockford, T. S. Gelatt, R. R. Ream, and K. M. Miller. 2009. “Development and application of DNA techniques for validating and improving pinniped diet estimates.” Ecological Applications 19 (4):889-905. doi: 10.1890/07-1701.1.
West, D., C. Lefèvre, D. Corbett, and S. Crockford. 2003. “A burial cave in the Western Aleutian Islands, Alaska.” Arctic Anthropology 40 (1):70-86.
Wilson, B. J., S. J. Crockford, J. W. Johnson, R. S. Malhi, and B. M. Kemp. 2011. “Genetic and archaeological evidence for a former breeding population of Aleutian cackling goose (Branta hutchinsii leucopareia) on Adak Island, central Aleutians, Alaska.” Canadian Journal of Zoology 89 (8):732-743. doi: 10.1139/z11-027.
Notes
[1] Susan Crockford has a decent publication record (Wilson et al. 2011, West et al. 2003, Tollit et al. 2009, Ovodov et al. 2011, Martinsson-Wallin and Crockford 2001, Hatfield et al. 2016, Crockford and Frederick 2011, 2007, Crockford 2016, 2009, 2003, 1997, Crockford, Frederick, and Wigen 1997, Barta et al. 2014).
[2] Respondent classes were generated by an unknown process. Blogs were classified on “their positions taken relative to those drawn by the IPCC”. The Working Groups of the Intergovernmental Panel have published 15 assessment reports and many special reports, each one hundreds if not thousands of pages long. Some blogs are small, others very large. It is not known which blog posts were examined for the classification. Harvey et al. (2017) performed two cluster analyses that show that their polar bear and sea ice data can be classified in the same manner as their blogs, but as their original classification is of unknown provenance, these validation tests are meaningless. The current author has repeatedly requested the data, but in vain, in direct contravention of the lead author’s employer’s data sharing policy – seehttps://www.knaw.nl/en/topics/openscience/open-access-and-digital-preservation/open-access/policy. Note that the journal does not have such a policy – seehttps://www.aibs.org/public-programs/biological_data_initiative.html.
[3] Scopus returns 216 articles for the same query. Harvey’s sample is closer to the Scopus sample (chi^2(9)=13.2, p=0.15).
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Unfortunately, they count noses and argue from authority, rather than assess the strength of the evidence.
A popular and well-known warmist tactic that works well when arguments to the contrary are conveniently left-out or misclassified.
” One subject seems to argue that polar bears are both threatened by climate change and not threatened”
… Schrödinger’s Polae Bear perchance? 😉
Haha, good response Eugene!
As an aside, I note that they treat inconsistent responses by implying that those responding have inconsistent and/or unexplainable views while failing to consider that this is an artifact poorly constructed questions.
But there is a glimmer of good news. From the BBC no less. They broadcast on boxing day a rather charming film about polar bears, called ‘the snow bears’. Aimed at a youthful audience, judging from the commentary, but marvelously filmed and stunning scenery. And guess what. Melting ice flows were mentioned and shown but not a single word about antropogenic climate change or similar nonsense. Rejoice!
Unlike many warmist-tinged nature documentaries, it’ll still have value as a rebroadcast, if the climate cools.
Is it possible to prepare a cookbook of abuses of statistics that would help constrain the free rangeing employment of the tool in self-serving exercises like the Harvey, Mann, etc. types. Analyses of such abuses reveals a shopping around for significance of bloodless data: processes that give the desired answers, manipulative weightings, and dubious data selection practices.
Indeed, evolution of statistical practice would appear to have been driven by efforts to squeeze significance out of more and more refractory data sets. How can much of this have an honest place in support of reality. If such manifold processes are necessary to obtain a trend, I suspect that there is a good chance the reverse trend is just as likely.
Obviously, use of statistical analysis tends to confer legitimacy of findings in the minds of most readers and so dishonesty will be rife in the lightweight work were becoming used to. This stuff definitely needs policing case by case to put a stop to abuses. We should be seeing more of the kind of work done by Tol, McIntyre….
I’ll offer this entry: the Law of Large Numbers applies neither if one takes measurements of one place at the rate of one measurement per day over a year, nor if one takes one measurement per day at thousands of separate locations.
There have been two papers attacking Dr. Crockford’s work, one underminmg her credentials (ad hominem) and another attempting to show that her views are inconsistent with ‘accepted science and the scientific consensus’ ( argumentum ad verecundium).
The papers, based on argumentative fallacies, not only lack substance but are flawed by inconsistencies and errors. It would seem that not only are they incapable of presenting a factual rebuttal to Dr. Crockford, they are too incompetent to even present a fallacious rebuttal.
Withp regard to Dr. Crockford, you know you are over the target when you start taking flak and the more flak you receive, they greater the threat you apparently pose.
Do not be discouraged Dr Crockford, stay on target as your aim is true.
I must say, as a scientist, I find the entire premise of the study deeply troubling.
What is the goal? To show that people who believe in some studies are a minority over scientists who believe in other studies? To ask the obvious question – when is that not true? There are always people who have alternate theories explaining the evidence. That is, in fact how science functions. It is how we progress. Science is not a poll, it is not opinion. Once a majority of scientists believe something, it doesn’t suddenly become true. It’s truth is independent of our beliefs.
It is setting up a classic “4 out of 5 dentists recommend Crest to their patients” scenario. Such a statement is, by definition, not science at all. It is done to look like science, to steal its credibility. It is classic pseudo-science, and the practitioners should have their reputations obliterated for signing their names to such garbage.
The mistake is to think there is scientific aim in the first place, much of climate ‘science’ is about PR and dogma hence why you get such poor practice, And when you get ‘leaders ‘ like Mann and Lew paper is really shows what not is merely acceptable but rewarded when it comes to ‘standards’ in the area ,
The paper not science at all. It’s illogic argument dressed up with some pretty statistics.
Over on Bart’s site he quotes Peter as saying, ““All piecharts and graphs, including those made by the critics of Harvey et all. tell the same story: the positions taken by the contrarian blogs is very different from that taken by the science based blogs and the peer review literature. That is the main conclusion from the paper.”
This “conclusion” tells us nothing of value. It only states that groups of people, when separated into groups based on their understanding of climate science, have different understandings. Which is absurdly obvious.
The conclusion appears to take on value only if one commits the logical fallacy of Appeal to Authority to conclude that the science-based blogs are correct. But doing so is illogical and, therefore, unscientific.
The conclusion could also appear to have some value if one accepts the implication that the contrarian blogs are wrong. But then we have circular reasoning; “the science-based blogs are correct because they hold an opposite view of the contrarian blogs,” which simplifies to, “the science-based blogs are correct because the science-based blogs are correct,” which is also illogical argument.
So, if Peter correctly defined the conclusion of the paper, it should be withdrawn because the conclusion is illogical handwaving that has no value. It attempts, in an underhanded and illogical way, to discredit people who hold different beliefs.
If the scientist authors wish to show that the contrarian blogs are incorrect, they have to actually produce scientific evidence that indicates they are incorrect.
Otherwise they are not doing good science, they are doing bad politics.
““the science-based blogs are correct because the science-based blogs are correct,””
Exactly. I was thinking that very thing early this morning that a circularity exists; the conclusion is fed back into the data to arrive at the conclusion.
Would that “the science-based blogs” were based on science :¬(