Butterfly study: a case study in confirmation bias

Guest post by Marc Hendrickx

A little over a month ago reports appeared in the press (eg. Butterflies ‘fly early as planet warms’) that the common Brown Butterfly (Heteronympha merope) was emerging 10 days earlier than it was 60 years ago all due to global warming attributed solely to CO2 emissions. The report was based on a paper published in Biology Letters. The article was titled “Early emergence in a butterfly causally linked to anthropogenic warming” by Michael R. Kearney, Natalie J. Briscoe, David J. Karoly, Warren P. Porter, Melanie Norgate and Paul Sunnucks was published online on 17 March 2010. The abstract can be accessed HERE.

The basis of the study was opportunistically collected observational data of butterfly emergence based on museum records and private data collected between 1941 and 2005 in an area centred around Melbourne, Australia, a city of about 4 million people. No links to the original data or location information of observations were provided in the published article.

The authors gauged the temperature dependence of Heteronympha merope under laboratory conditions and used historical weather data for 1945–2007 (Bureau of Meteorology, Australia) from Laverton (37.868 S, 144.768 E), a “rural” site close to Melbourne, to model the physiological response of H. merope to temperature. The authors claim that this weather station is a ‘high-quality’ site, unaffected by changes in exposure, urbanization, instrumentation, etc., during the study period. Weather records (mean monthly maximum and minimum air temperature, wind speed and cloud cover) were translated into microclimates experienced by immature H. merope using biophysical modelling software (NICHE MAPPER, http://www.zoology.wisc.edu/faculty/Por/Por.html#niche).

The observed temperature trends at Laverton were compared to output from extended climate model simulations for the single-model grid box overlying Melbourne and Laverton. Anthropogenic climate forcing included observed increases in greenhouse gases and estimated variations of anthropogenic aerosols, whereas natural external climate forcing included estimated changes in solar irradiance and volcanic aerosols.

The results are summarised in Figure 1 from the paper

I found a number of issues with this paper that pointed to strong confirmation bias and quickly put together a comment that I submitted to Biology Letters on 19 March 2010, just two days after the article was published on line. A copy of the manuscript appears below. I received notification this week that the manuscript was rejected. The reviewer comments make interesting reading (see below) and I thought I would share them with WUWT readers, with a view that the collective brain of WUWT readers would help find the necessary references such that I might be able to re-submit the comment to Biology Letters sometime over the next few weeks. I’d also be interested in hearing the views of the authors and invite them to add their comments.

Comment on Kearney et al., 2010: Early emergence in a butterfly causally linked to anthropogenic warming.

Kearney et al. (2010) examine phenological change in Heteronympha merope (Nymphalidae) to test whether (i) the phenological shift could be explained by air temperature change, and (ii) that the associated change could be attributed to human influences. Kearney et al., contend their results support:

1. a shift in the mean emergence date for H. merope of 1.6 days per decade over a 65 year period over 12,000 km2,
2. an increase in local air temperature of 0.14ºC over the same period, and
3. attribution of the phonological and temperature change to anthropogenic warming, due to greenhouse gas emissions.

There are significant issues with the study outlined below that negate the conclusions:

1. Observed emergence times for H. merope were based on opportunistically collected data over an area of about 12,000 km2 (geographic area-37.60-38.54 Lat, 144.17 to145.48 Long.) centred on the Melbourne CBD. The location of individual observation locations is not provided and there potential for location bias is not discussed. Nor is there a discussion of the potential effect of confounding influences that may affect emergence times. These influences include: human impact on habitat (Kobayashi et al., 2009), pollution, coincidence in emergence of H. merope  with changing emergence patterns of its food stock, food availability and variation over time. These factors may have provided adaptive stresses favouring earlier emergence.

2.The methodology for determining thermal dependence of development rate for eggs, larvae and pupae did not account for other variables that might be a factor in emergence such as: atmospheric CO2 content or affect of atmospheric pollutants such as CO, and ozone common in urban environments. There is a considerable body of evidence demonstrating that effects of elevated CO2 on plants can influence insect herbivore performance (Watt et al. 1995, Bezemer and Jones 1998). Changes in leaf chemistry for instance, such as decreased leaf nitrogen and increased carbohydrate and polyphenolic concentrations at elevated CO2 (Cotrufo et al. 1998, Penuelas and Estiarte 1998), might affect insect development (Slansky 1993) and potentially effect emergence timing. These factors were not taken into consideration and as such the link between emergence timing and temperature cannot be conclusively stated.

3.To assess whether the observed change in climate could be attributed to human influence, the observed April-October mean temperature trend for 1944-2007 for the weather station at Laverton (Bureau of Meteorology-BOM ID 87031) was compared to climate model simulations. Laverton is affected by urbanisation effects from significant changes in land use over the period of observations. Australian Bureau of Statistics (ABS 2008, ABS 2008a) data show an increase in population in the area from 7854 in 1933 to 132793 in 2008 (ABS, 2008, 2008a). Hence to define the station as “rural” is a misrepresentation. NASA GISTEMP defines the station as “Urban” with a population of 2.7 million (GISTEMP, 2010). A station at the western edge of the study area with records spanning the period 1903 to 1998 shows no substantial warming (Figure 1). This station, Durdidwarrah BOM ID 87021, is located in the Brisbane Ranges National Park in an area that has not experienced significant land use change since the 1870s when dams were constructed (Catrice, 1997). A comparison between Durdidwarrah, Laverton and the Melbourne CBD station (BOM ID 86071) indicates substantial warming over the Melbourne Region. The disparity between the rural station and the two urban stations suggest this warming is due to urbanization, rather than increases in greenhouse gases. The temperature increases due to urbanization are similar to those reported in China (Jones et al., 2008).

References

ABS 2008. Australian Bureau of Statistics 3105.0.65.001 – Australian Historical Population Statistics. www.abs.gov.au (accessed 18 March 2010).

ABS 2008a. Australian Bureau of Statistics 3218.0 Regional Population Growth, Australia. www.abs.gov.au (accessed 18 March 2010).

Bezemer, T. M., & Jones, T. H. 1998 Plant–insect herbivore interactions in elevated atmospheric CO2: quantitative analyses and guild effects. Oikos 82, 212–222.

Catrice D. 1997 Brisbane Ranges National Park. Parks Victoria. Department of Natural Resources and Environment, Melbourne Victoria (accessed 18 March 2010)

Cotrufo, M. F., Ineson, P. and Scott A. 1998 Elevated CO2 reduces the nitrogen concentration of plant tissues. Global Change Biology 4, 43–54

GISTEMP 2010. NASA GISS Surface Temperature Analysis – Station Data ‘Laverton’ GISTEMP ID 501948650000 (http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?id=501948650000&data_set=0&num_neighbors=1) (accessed 18 March 2010).

Goverde, M., Erhardt, A., & Niklaus P. A. (2002) In situ development of a satyrid butterfly on calcareous grassland exposed to elevated carbon dioxide. Ecology 83(5), 1399-1411

Jones, P. D., Lister, D. H., and Li Q. (2008), Urbanization effects in large-scale temperature records, with an emphasis on China, J. Geophys. Res., 113, D16122, doi:10.1029/2008JD009916.

Kearney, M. R., Briscoe, N. J., Karoly,  D. J., Porter, W. P., Norgate M. and Sunnucks P. 2010 Early emergence in a butterfly causally linked to anthropogenic warming. Biology Letters (doi: 10.1098/rsbl.2010.0053)

Kobayashi, T., Kitahara, M.,  Suzuki, Y. and Tachikawa, S. 2009. Assessment of the habitat quality of the threatened butterfly, Zizina emelina (Lepidoptera, Lycaenidae) in the agro-ecosystem of Japan and implications for conservation. Transactions of the Lepidopterological Society of Japan 60(1), 25-36.

Penuelas, J., & Estiarte M. 1998 Can elevated CO2 affect secondary metabolism and ecosystem function? Trends in Ecology and Evolution 13, 20–24.

Slansky, F. 1993 Nutritional ecology: the fundamental quest of nutrients. Pages 29–91 in N. E. Stamp and T. M. Casey, editors. Caterpillars: ecological and evolutionary constraints on foraging. Chapman and Hall, New York, New York, USA.

Watt, A. D., Whittaker, J. B. , Docherty, M., Brooks, G., Lindsay, E. and Salt D. T. 1995 The impact of elevated atmospheric CO2 on insect herbivores. Pages 197–217 in R. Harrington and N. E. Stork, editors. Insects in a changing environment. Academic Press, London, UK.

=================================

Rejection Letter received April 20 , 2010. Dear Mr Hendrickx

I am writing to inform you that we have now obtained responses from referees on manuscript RSBL-2010-0263 entitled “Comment on Kearney et al., 2010: Early emergence in a butterfly causally linked to anthropogenic warming.” which you submitted to Biology Letters.

Unfortunately, your manuscript has been rejected following full peer review. Competition for space in Biology Letters is currently very severe, as many more manuscripts are submitted to us than we have space to print. We are therefore only able to publish those that are exceptional and present significant advances of broad interest, and must reject many good manuscripts.

Please find below the comments received from referees concerning your manuscript, not including confidential reports to the Editor. I hope you may find these useful should you wish to submit your manuscript elsewhere.

We are sorry that your manuscript has had an unfavourable outcome, but would like to thank you for offering your work to Biology Letters.

Yours sincerely

Publishing Editor

Editor’s comments:

I am rejecting this in view of the strong criticisms by refs. 1 and 3. If the author can deal with these comments, we could consider this for e-letters.

Reviewer(s)’ Comments to Author:

(MH-I have added comments in italics)

Referee: 1

Comments to the Author(s)

The ms is a critique of a recent publication by Kearney et al in Biology Letters. But I am not convinced by any of the author’s three criticisms of the paper.

The first criticism is that the data presented in Kearney et al does not support evidence of a change in emergence times over the study period. Kearney et al note in their paper that while “the opportunistically collected data probably adds considerable noise to any signal of phenological shift, there is no reason to expect such data to be chronologically biased”. To me, this proviso seems sufficient (MH-this seems difficult to justify as no actual data is presented). For the criticisms in the current ms to be supported, the author should present some evidence that this species or others are shifting their phenology related to some of the other factors suggested, or some evidence that in fact the data does not support a shift in phenology. (MH-Can WUWT readers help out with suggestions?) I also do not know where the author has extracted the “area of 12000 km2” data from (MH-this was based on the geographic coordinates provided in the paper) , or that the data were drawn from “disparate, genetically diverse groups” (MH-This was an assumption I made that there would be significant genetic variation over a large geographic area-the area covered by the study contains a range of geographies and sub-climates that may provide local variation in emergence timing. The absence of location data for observations makes it impossibel to judge the potential affect of geographic bias).

The second criticism is that the physiological model did not account for other possible variables. No, but the fit of observed phenology to that modelled based on climate was extremely close. For this criticism to be justified the author should again present some empirical evidence that the other variables listed influence emergence times in this species or similar species. (MH-Can WUWT readers help out with suggestions?)

I am most concerned about the third criticism levelled by the author, that the temperature increase noted for the meteorological station in the Kearney et al paper is dependent on urbanisation effects. The author here presents data from a rural met station and argues that it has shown no increase in temperature over the same period of time. However, the comparison is not valid, because the regression of temperature against year in Fig 1 for the Durdidwarrah station is run from 1903 to 1998, rather than 1944 to 2007, as in the Kearney et al paper. Examination of the figure shows that had data for the approximate 1940 to 2000 period been analysed for Durdidwarrah, there would have probably been a significant increase in temperature, comparable to that reported for the Laverton station by Kearney et al. In this case it is essential to compare like with like, as the Kearney et al paper is not looking at changes to butterfly phenology since 1903, but from the 1940s. (MH-Durdidwarrah is a good station but suffers from a number of breaks in reporting. The reviewer is correct in arguing that a trend through Durdidwarrah from 1940 through 2000 would yield a decadal trend similar to Laverton, however virtually all this warming occurred in the late 1940s, the trend since 1950 has been flat).

There are a few presentational errors: various spellings of “phenological” and “effect” and “affect”, “Nymphalidae” spelt incorrectly, Fig 1 could be presented more clearly.

Referee: 2

Comments to the Author(s)

In the short intro, the author writes twice “phonological changes”. I guess that would be “phenological changes”? (MH based on this I take it that Ref 2 was generally happy with the manuscript)

Referee: 3

Comments to the Author(s)

The author makes some relevant and potentially relevant points in his comment on Kearney et al., (MH-my bold) but this manuscript does not bring this criticism in a sound way, as it stands. It needs major revision before it may become acceptable for publication.

1)      Point 1 – Hendrickx is criticizing the use of opportunistically collected data. Kearny et al have made the assumption that there is no obvious bias in these data. So, here the author should more convincingly show that there is indeed bias that may impact on the conclusions. It is not enough mentioning the opportunistic nature of the data. This point needs more work. (MH-again any references that demonstrate effect of other influences on emergence appreciated)

2)      Point 2 –CO2: that may be a valid issue that has not been considered as an alternative (or interaction) effect by Kearney. Another relevant paper would be Mevi-Schultz et al. 2003. behave Ecol Sociobiol 54: 36-43 (MH-this appears to be generally supportive of my point 2).

3)      Point 3: I don’t get this point. How can you distinguish between urbanization and an increase in greenhouse gasses per se? What would be the direct and the indirect effects of urbanization for the system considered. Again, the author is not making his point in a clear way (MH-I would have thought the comparison between the three stations clearly demonstrates a UHI effect over the Melbourne region).