Guest essay by Dr. Richard Tol
In their eagerness to discredit a colleague Harvey et al. (2017) got ahead of themselves. The write-up shows signs of haste – typographical errors (“principle component analysis”, “refereces cited”) and nonsensical statements (“95% normal probability”) escaped the attention of the 14 authors, 3 referees and editor – but so does the analysis. The paper does three things: It creates a database, it classifies subjects, and it conducts a principal component analysis. Details have not been shared on the database construction or the classification (Lewandowsky and Bishop 2016), so I focus on the principal component analysis.
Principal component analysis (PCA) aims to reduce the dimension of a dataset by a linear transformation of its variables into orthogonal components and limiting the attention to those components that are principal in explaining the variation in the variables. Harvey et al. (2017) reduce seven variables to two. One variable denotes citation of Susan Crockford. This is recorded as a binary variable, that is, no distinction is made between mentioning her work in passing, criticizing her work, and uncritically adopting her position. The remaining six variables denote agreement with the following statements:
1. sea-ice extent is on average declining rapidly in the Arctic;
2. sea-ice extent is decreasing only marginally, is not decreasing significantly, or is currently recovering in the Arctic;
3. changes in sea-ice extent in the Arctic are due to natural variability, and it is impossible to predict future conditions;
4. polar bears are threatened with extinction by present and future anthropogenic global warming;
5. polar bears are not threatened with extinction by present and future anthropogenic global warming; and
6. polar bears will adapt to any future changes in Arctic ice extent whether because of anthropogenic global warming or natural variability.
Agreement is measured on a binary scale, even though nuances are clearly possible. Taking the first statement, “extent” could refer to sea-ice area or volume, both of which vary over space and seasons so that “average” obtains different meanings, while “rapidly” could mean different things to different people. Similar objections can be raised against the other five statements.
The data released by Harvey et al. (2017) contains only zeroes and ones, suggesting that either agreement was recorded by a single coder or, less plausibly, that all coders agreed on all 6 statements by all 182 subjects.
Intriguingly, there are four subjects who appear to argue that sea-ice is neither shrinking nor stable or growing. One subject seems to argue that polar bears are both threatened by climate change and not threatened, and another is recorded as arguing that polar bears are neither threatened nor not threatened. This corroborates the above assertion that statements were coded by a single coder.
Statements 1 and 2, and 4 and 5 are mutually exclusive, and statements 3 and 6 are close to statements 2 and 4, respectively. A PCA is redundant in a case like this. The analysts artificially inflated the dimensionality of the data, before using PCA to reduce it again.
Figure 2 in Harvey et al. (2017) plays another statistical sleight of hand. The figure shows many observations taking many different positions. Seven binary observations can take at most 128 positions, rather than the 182 suggested in Figure 2. In fact, there are only 19 different positions in the underlying data. The jitter applied by Harvey et al. (2017) suggests that the “majority view papers” all take a slightly different position on sea-ice and polar bears – as you would expect had you not known about the binary coding. Actually, the 86 papers fully agree with each other, and with 27 of the “science-based blogs”. Figure 1 shows a more faithful depiction, with the first and second principal component on the axes and the size of the circles reflecting the number of observations.
Figure 1. Principal component analysis. The horizontal axis shows the first principal component, the vertical axis the second one. Colours denote the four classes (green = majority view papers; blue = science-based blogs; red = controversial papers; orange = denier blogs). The size of the circle denotes the number of subjects, with the smallest circle representing 1 subject and the largest 86.
Figure 2 in Harvey et al. (2017) shows that different classes of respondent differ strongly on the first principal component, but that there are no significant difference on the second principal component. Table 1 confirms this.
Table 1. The average and standard deviation by observation class for the first and second principal component with seven or six variables per observation.
The second principal component largely reflects statement 6, whether polar bears can adapt to future climate change. Polar bears appear to have survived the onset of two interglacials and one ice age (Lindqvist et al. 2010). Unfortunately, 114 of the 182 subjects do not take a position on this. Harvey et al. (2017) replaced these missing observations with zeroes (after standardization). Omitting this column makes the first principal component more important (explained variance rises from 80% to 89%) and the second principal component less important (explained variance falls from 11% to 5%). This does not affect the qualitative results: The first principal component explains the differences between the types of observations. The second principal component does not have discriminatory power. See Table 1.
Harvey et al. (2017) thus really show that there are people who worry about sea-ice and polar bears, and those who do not and cite Dr Crockford.
But Harvey et al. (2017) do not just show that there are two camps. They take sides. Unfortunately, they count noses and argue from authority, rather than assess the strength of the evidence. It is well-known that like-minded blogs often copy or paraphrase material from one another. Similarly, academic papers often repeat a salient conclusion from previous research. Counting noses is a poor method.
The argument from authority is weakened by examining the 92 learned papers. Of the 86 “majority view” papers, 39 were authored by Steven Amstrup, Rascha Nuijten or Ian Stirling, who are among the alii in Harvey et al. (2017). Another 13 were authored by Andrew Derocher, a frequent (n=10) co-author of Amstrup.
The paper does not specify how these 92 papers were identified, beyond a “broad keyword search” on the “ISI Web of Science”. The Web of Science returns 179 articles for a query on “polar bear” and “sea ice”. No information is given how the larger sample of relevant papers was reduced to the smaller one used by Harvey et al. (2017). Comparing the relative contribution of the ten most prolific authors according to the Web of Science to their relative presence in Harvey’s sample reveals that the latter is not representative of the former (chi^2(9)=17.4, p=0.04). Research by co-authors Amstrup and Stirling is overrepresented in Harvey et al. (2017), and work by Jon Aars and Oystein Wiig underrepresented. The sample used by Harvey et al. (2017) appears to be a sample of convenience, and unrepresentative.
In sum, Harvey et al. (2017) play a statistical game of smoke and mirrors. They validate their data, collected by an unclear process, by comparing it to data of unknown provenance. They artificially inflate the dimensionality of their data only to reduce that dimensionality using a principal component analysis. They pretend their results are two dimensional where there is only one dimension.
They suggest that there are many nuanced positions where there are only a few stark ones – at least, in their data. On a topic as complex as this, there are of course many nuanced positions; the jitter applied conceals the poor quality of Harvey’s data.
They show that there is disagreement on the vulnerability of polar bears to climate change, but offer no new evidence who is right or wrong – apart from a fallacious argument from authority, with a “majority view” taken from an unrepresentative sample.
Once the substandard statistical application to poor data is removed, what remains is a not-so-veiled attempt at a colleague’s reputation.
Peter Roessingh and Bart Verheggen gracefully shared data and code. I borrowed freely from comments by Roman Mureika and Shub Niggurath. Marco NN had useful comments on an earlier version.
Barta, J. L., C. Monroe, S. J. Crockford, and B. M. Kemp. 2014. “Mitochondrial DNA preservation across 3000-year-old northern fur seal ribs is not related to bone density: Implications for forensic investigations.” Forensic Science International 239:11-18. doi: 10.1016/j.forsciint.2014.02.029.
Crockford, S., G. Frederick, and R. Wigen. 1997. “A Humerus Story: Albatross Element Distribution from Two Northwest Coast Sites, North America.” International Journal of Osteoarchaeology 7 (4):287-291.
Crockford, S. J. 1997. “Archeological evidence of large northern bluefin tuna, Thunnus thynnus, in coastal waters of British Columbia and orthern Washington.” Fishery Bulletin 95 (1):11-24.
Crockford, S. J. 2003. “Thyroid rhythm phenotypes and hominid evolution: A new paradigm implicates pulsatile hormone secretion in speciation and adaptation changes.” Comparative Biochemistry and Physiology – A Molecular and Integrative Physiology 135 (1):105-129. doi: 10.1016/S1095-6433(02)00259-3.
Crockford, S. J. 2009. “Evolutionary roots of iodine and thyroid hormones in cellcell signaling.” Integrative and Comparative Biology 49 (2):155-166. doi: 10.1093/icb/icp053.
Crockford, S. J. 2016. “Prehistoric Mountain Goat (Oreamnos americanus) Mother Lode Near Prince Rupert, British Columbia and Implications for the Manufacture of High-Status Ceremonial Goods.” Journal of Island and Coastal Archaeology:1-22. doi: 10.1080/15564894.2016.1256357.
Crockford, S. J., and S. G. Frederick. 2007. “Sea ice expansion in the Bering Sea during the Neoglacial: Evidence from archaeozoology.” Holocene 17 (6):699-706. doi: 10.1177/0959683607080507.
Crockford, S. J., and S. G. Frederick. 2011. “Neoglacial sea ice and life history flexibility in ringed and fur seals.” In Human Impacts on Seals, Sea Lions, and Sea Otters: Integrating Archaeology and Ecology in the Northeast Pacific, 65-91.
Harvey, Jeffrey A., Daphne van den Berg, Jacintha Ellers, Remko Kampen, Thomas W. Crowther, Peter Roessingh, Bart Verheggen, Rascha J. M. Nuijten, Eric Post, Stephan Lewandowsky, Ian Stirling, Meena Balgopal, Steven C. Amstrup, and Michael E. Mann. 2017. “Internet Blogs, Polar Bears, and Climate-Change Denial by Proxy.” BioScience:bix133-bix133. doi: 10.1093/biosci/bix133.
Hatfield, V., K. Bruner, D. West, A. Savinetsky, O. Krylovich, B. Khasanov, D. Vasyukov, Z. Antipushina, M. Okuno, S. Crockford, K. Nicolaysen, B. Mac, L. Persico, P. Izbekov, C. Neal, T. Bartlett, L. Loopesko, and A. Fulton. 2016. “At the foot of the smoking mountains: The 2014 scientific investigations in the Islands of the Four Mountains.” Arctic Anthropology 53 (2):141-159. doi: 10.3368/aa.53.2.141.
Lewandowsky, S., and D. Bishop. 2016. “Research integrity: Don’t let transparency damage science.” Nature 529 (7587):459-461. doi: 10.1038/529459a.
Lindqvist, Charlotte, Stephan C. Schuster, Yazhou Sun, Sandra L. Talbot, Ji Qi, Aakrosh Ratan, Lynn P. Tomsho, Lindsay Kasson, Eve Zeyl, Jon Aars, Webb Miller, Ólafur Ingólfsson, Lutz Bachmann, and Øystein Wiig. 2010. “Complete mitochondrial genome of a Pleistocene jawbone unveils the origin of polar bear.” Proceedings of the National Academy of Sciences 107 (11):5053-5057.
Martinsson-Wallin, H., and S. J. Crockford. 2001. “Early settlement of Rapa Nui (Easter Island).” Asian Perspectives 40 (2):244-278.
Ovodov, N. D., S. J. Crockford, Y. V. Kuzmin, T. F. G. Higham, G. W. L. Hodgins, and J. van der Plicht. 2011. “A 33,000-Year-Old incipient dog from the Altai Mountains of Siberia: Evidence of the earliest domestication disrupted by the last Glacial Maximum.” PLoS ONE 6 (7). doi: 10.1371/journal.pone.0022821.
Tollit, D. J., A. D. Schulze, A. W. Trites, P. F. Olesiuk, S. J. Crockford, T. S. Gelatt, R. R. Ream, and K. M. Miller. 2009. “Development and application of DNA techniques for validating and improving pinniped diet estimates.” Ecological Applications 19 (4):889-905. doi: 10.1890/07-1701.1.
West, D., C. Lefèvre, D. Corbett, and S. Crockford. 2003. “A burial cave in the Western Aleutian Islands, Alaska.” Arctic Anthropology 40 (1):70-86.
Wilson, B. J., S. J. Crockford, J. W. Johnson, R. S. Malhi, and B. M. Kemp. 2011. “Genetic and archaeological evidence for a former breeding population of Aleutian cackling goose (Branta hutchinsii leucopareia) on Adak Island, central Aleutians, Alaska.” Canadian Journal of Zoology 89 (8):732-743. doi: 10.1139/z11-027.
 Susan Crockford has a decent publication record (Wilson et al. 2011, West et al. 2003, Tollit et al. 2009, Ovodov et al. 2011, Martinsson-Wallin and Crockford 2001, Hatfield et al. 2016, Crockford and Frederick 2011, 2007, Crockford 2016, 2009, 2003, 1997, Crockford, Frederick, and Wigen 1997, Barta et al. 2014).
 Respondent classes were generated by an unknown process. Blogs were classified on “their positions taken relative to those drawn by the IPCC”. The Working Groups of the Intergovernmental Panel have published 15 assessment reports and many special reports, each one hundreds if not thousands of pages long. Some blogs are small, others very large. It is not known which blog posts were examined for the classification. Harvey et al. (2017) performed two cluster analyses that show that their polar bear and sea ice data can be classified in the same manner as their blogs, but as their original classification is of unknown provenance, these validation tests are meaningless. The current author has repeatedly requested the data, but in vain, in direct contravention of the lead author’s employer’s data sharing policy – seehttps://www.knaw.nl/en/topics/openscience/open-access-and-digital-preservation/open-access/policy. Note that the journal does not have such a policy – seehttps://www.aibs.org/public-programs/biological_data_initiative.html.
 Scopus returns 216 articles for the same query. Harvey’s sample is closer to the Scopus sample (chi^2(9)=13.2, p=0.15).