Zhao and Running's plant productivity claim rebuked in Science: modeling errors, use of corrupted satellite data and statistically insignificant trends cited

UPDATE: Here’s the press release from  Boston University:

From Boston University

NASA study refutes claims of drought-driven declines in plant productivity, global food security

BU researchers find that modeling errors produced exaggerated claims

A new, comprehensive study by an international team of scientists, including scientists at Boston University in the US and the Universities of Viçosa and Campinas in Brazil, has been published in the current issue of Science (August 26, 2011) refuting earlier alarmist claims that drought has induced a decline in global plant productivity during the past decade and posed a threat to global food security.

Those earlier findings published by Zhao and Running in the August 2010 issue of Science (Vol. 329, p. 940) also warned of potentially serious consequences for biofuel production and the global carbon cycle. The two new technical comments in Science contest these claims on the basis of new evidence from NASA satellite data, which indicates that Zhao and Running’s findings resulted from several modeling errors, use of corrupted satellite data and statistically insignificant trends.

The main premise of Zhao and Running’s model-based study was an expectation of increased global plant productivity during the 2000s based on previously observed increases during the 1980s and 1990s under supposedly similar, favorable climatic conditions. Instead, Zhao and Running were surprised to see a decline, which they attributed it to large-scale droughts in the Southern Hemisphere.

“Their model has been tuned to predict lower productivity even for very small increases in temperature. Not surprisingly, their results were preordained,” said Arindam Samanta, the study’s lead author. (Samanta, now at Atmospheric and Environmental Research Inc., Lexington, MA, worked on the study as a graduate student at Boston University’s Department of Geography and Environment.)

Zhao and Running’s predictions of trends and year-to-year variability were largely based on simulated changes in the productivity of tropical forests, especially the Amazonian rainforests. However, according to the new study, their model failed miserably when tested against comparable ground measurements collected in these forests.

“The large (28%) disagreement between the model’s predictions and ground truth imbues very little confidence in Zhao and Running’s results,” said Marcos Costa, coauthor, Professor of Agricultural Engineering at the Federal University of Viçosa and Coordinator of Global Change Research at the Ministry of Science and Technology, Brazil.

This new study also found that the model actually predicted increased productivity during droughts, compared to field measurements, and decreased productivity in non-drought years 2006 and 2007 in the Amazon, in contradiction to the main finding of the previous report. “Such erratic behavior is typical of their poorly formulated model, which lacks explicit soil moisture dynamics,” said Edson Nunes, coauthor and researcher at the Federal University of Viçosa, Brazil.

The new study indicates that Zhao and Running used NASA’s MODIS satellite data products, such as vegetation leaf area, without paying caution to data corruption by clouds and aerosols. “Analyzing the same satellite data products after carefully filtering out cloud and aerosol-corrupted data, we could not reproduce the patterns published by Zhao and Running. Moreover, none of their reported productivity trends are statistically significant,” said Liang Xu, coauthor and graduate student at Boston University.

In any case, the trends in plant productivity reported by Zhao and Running are miniscule—a 0.34% reduction in the Southern Hemisphere offset by a 0.24% gain in the Northern Hemisphere for a net decline of 0.1% over a ten-year period from 2000 to 2009. “This is the proverbial needle in a haystack,” said Simone Vieira, coauthor and researcher at the State University of Campinas, Brazil. “There is no model accurate enough to predict such minute changes over such short time intervals, even at hemispheric scales.”

Any investigation of trends in plant growth requires not only consistent and accurate climate and satellite data but also a model suitable for such purposes. “The Zhao and Running study does not even come close,” said Ranga Myneni, senior author and Professor of Geography, Boston University. “Their analysis of satellite data is flawed because they included poor quality data and do not bother to test trends for statistically significance. Our analyses of four different higher-quality MODIS satellite vegetation products that have been carefully filtered for data corruption show no statistically significant trends over 85% of the global vegetated lands.”

###

This study was funded through a research grant by the NASA MODIS project to Prof. Myneni for investigation on the use of MODIS satellite data to study vegetation on our planet.

Graphics and more information at: http://cliveg.bu.edu/

Contact information for the authors:

Ranga Myneni: ranga.myneni@gmail.com, +1-617-470-7065

Arindam Samanta: arindam.sam@gmail.com, +1-617-852-5256

Marcos Costa: mhcosta@ufv.br, +55-31-8727-1899

Experts to comment on this story:

Prof. Ian Colin Prentice, Macquarie University, colin.prentice@mq.edu.au, +61-425-040669

Dr. Compton J. Tucker, NASA, compton.j.tucker@nasa.gov, +1-301-614-6644

Prof. Inez Fung, UC-Berkeley, ifung@berkeley.edu, +1-510-643-9367

About Boston University—Founded in 1839, Boston University is an internationally recognized private research university with more than 30,000 students participating in undergraduate, graduate, and professional programs. As Boston University’s largest academic division, the College and Graduate School of Arts & Sciences is the heart of the BU experience with a global reach that enhances the University’s reputation for teaching and research.

===============================================================

Original post:

I don’t have a lot of time to write this up at work today, so I’ll just post what I have and let WUWT readers fill in the blanks. Here’s what I have so far submitted from reader Peter Verhof:

Two Technical Comments will appear in “Science” this week on Friday (tomorrow). Note that all information is under an embargo till today (August 25th, 2011) 2 pm (USA Eastern Daylight Saving Time).

An article by Zhao and Running appeared a year back in “Science” claiming that droughts reduced plant productivity during the period 2000 to 2009 based on model analysis that used MODIS vegetation products and climate data.

Jeff Id called that work “Bovine Scatology” at that time http://noconsensus.wordpress.com/2010/08/21/charlatans-of-science-zhao-and-running/

The two Technical Comments argue that Zhao and Running’s results are indeed “bovine scatology”:

Of note in the technical comment:

These observations do not show any large-scale declines in the greenness level of the land surface (Fig. 1); 86% of all vegetated land south of 70°N shows no trends.

More information below with links.

From: http://cliveg.bu.edu/research/science-tc-2011/science-tc-2011.html

All Material on this Page is Under Strict Embargo Till 2:00 PM US ET Thursday, August-25-2011

Technical Comments and Response and Critique of Response

“Drought-Induced Reduction in Global Terrestrial Net Primary Production from 2000 Through 2009”

Technical Comment by Samanta et al., Science (2011)

Technical Comment by Medlyn Science (2011)

Boston University Press Release

Critique of Zhao and Running’s Technical Response to our Technical Comments

 

0 0 votes
Article Rating
22 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
August 25, 2011 11:25 am

Science magazine has morphed into all AGW propaganda, all the time. It’s name reminds me of the old Soviet Pravda [“Truth”]. This article exposes the hucksters at Science: click

Tamara
August 25, 2011 11:49 am

From Medlyn’s comment:
“otherwise we run the risk of mistaking model outcomes for reality.”
Priceless.

August 25, 2011 12:49 pm

That was a long time ago. In classic Id fashion, I lost my temper with the stupidity and wrote on it. I’ve taken the time to read one technical comment so far and it looks like the ‘science’ took notice of the paper for the same reasons I did.
What still bothers me is that it was so blatantly obvious and it got through review. I shouldn’t let these things bother me so much.

James Sexton
August 25, 2011 1:42 pm

Jeff Id says:
August 25, 2011 at 12:49 pm
…………..
What still bothers me is that it was so blatantly obvious and it got through review. I shouldn’t let these things bother me so much.
==================================================================
I’m beyond that, too. What used to make me livid was that people would attach validity to a paper solely on the basis of passing a peer review process. I think most rational people have come to realize the words “peer review” hold no meaning.

timetochooseagain
August 25, 2011 1:52 pm

Looking back, they were claiming that productivity increased substantially from 1980 to 1989, and from 1990 to 1999, but declined slightly from 2000 to 2009. Uh, hold on a second now, they are arguing that warming caused the increase to stop. Problem. There was a significant warming trend from 1980-1999, but during the period they attribute a decline in productivity to AGW, there was no warming. It sounds to me like what they found would indicate that productivity is positively correlated with temperatures, but apparently unfamiliar with the temperature record they concluded that there was actually a negative relationship. And now we find that their study was actually just bunk.

Allen
August 25, 2011 1:59 pm

In some areas claiming to be science peer-review is really pal-review (I’m blocking on the reference – is that plagiarism?).

Shevva
August 25, 2011 3:05 pm

‘Such erratic behavior is typical of their poorly formulated model’ – they must of put her make up on wrong.

Brian H
August 25, 2011 3:42 pm

The original study should have been rejected on sheer logical incoherence grounds!

Latitude
August 25, 2011 5:10 pm

a net decline of 0.1% over a ten-year period from 2000 to 2009. “This is the proverbial needle in a haystack,” said Simone Vieira, coauthor and researcher at the State University of Campinas, Brazil.
“There is no model accurate enough to predict such minute changes over such short time intervals, even at hemispheric scales.”
““Their analysis of satellite data is flawed because they included poor quality data and do not bother to test trends for statistically significance.”
=======================================================================
Tell me again how much temperatures are supposed to rise……………………..
Speal chex is a better peer review………………..

Pamela Gray
August 25, 2011 6:04 pm

Latitude re: Speal chex. Thanx. Sticky diet Pepsie snorted all over puter screen.

Rational Debate
August 25, 2011 8:53 pm

Id says: August 25, 2011 at 12:49 pm
That was a long time ago. In classic Id fashion, I lost my temper with the stupidity and wrote on it. I’ve taken the time to read one technical comment so far and it looks like the ‘science’ took notice of the paper for the same reasons I did.
What still bothers me is that it was so blatantly obvious and it got through review. I shouldn’t let these things bother me so much.

What still bothers me is that far too few scientists in academia – and professorships or teaching positions – have the exact reaction you did when it comes to rotten science wherever they come across it AND are verbal about it. If more did, there would be a lot less “Bovine Scatology” being produced out there, a lot less of what did still exist would get thru peer review, and we would all be vastly better off and lot further ahead scientifically to boot.

Rhoda Ramirez
August 25, 2011 10:05 pm

The original study would be a good one to point to when someone begins bleating about “peer reviewed” gold standard.

Jordan
August 25, 2011 11:52 pm

James Sexton “people would attach validity to a paper solely on the basis of passing a peer review process”
I’ll add to that – people read the “discussion” which usually goes into the published articles, and they assume validity because it is in one of the journals. In climatology, the “discussion” is where the presumed AGW orthodoxy gets a leg-up from pure supposition and speculation with no relationship to the article’s data, analysis or results.
A good example is Caillon et al and the CO2 lag at the last glacial termination. The discussion in that paper is something to behold.

vigilantfish
August 26, 2011 1:09 am

I am currently collaborating with some scientists in producing a history of their scientific institution over its first century, and am acting as editor of their articles. They have been somewhat distressed to see me reformatting their articles into a history format, and jettisoning the abstract and discussion sections. One of them told me that the abstract and discussion sections were usually the only sections ever read by scientists, so they felt that readers would not get the point if these sections were missing. This goes a long way to explain how CAGW is kept alive, because it means few scientists bother to read about the details as to how results were achieved. And here I’ve been diligently reading entire articles: silly me!

John Marshall
August 26, 2011 2:07 am

This report confirms my antipithy towards these type of models. It also shows the problems associated with lazy and poorly planned research. It also shows what can be achieved with a bit of thought and good planning. Good to see.
I hope to see more reports of this type of research to shower confusion on the alarmist enemy.

Nuke
August 26, 2011 7:09 am

…modeling errors, use of corrupted satellite data and statistically insignificant trends cited

On the other hand, the NSF completely vindicated them!
BTW: /sarc!

Hector M.
August 26, 2011 7:35 am

A related paper in Science by Yude Pan et al is worth glancing at.
Reference:
Yude Pan et al, A Large and Persistent Carbon Sink in the World’s Forests. Science 333, 988 (2011); DOI: 10.1126/science.1201609
Available at http://www.globalcarbonproject.org/global/pdf/pep/Pan.etal.science.Forest_Sink.pdf.

Hector M.
August 26, 2011 7:58 am

Abstract of Yude Pan et al:
The terrestrial carbon sink has been large in recent decades, but its size and location remain uncertain. Using forest inventory data and long-term ecosystem carbon studies, we estimate a total forest sink of 2.4 ± 0.4 petagrams of carbon per year (Pg C year–1) globally for 1990 to 2007. We also estimate a source of 1.3 ± 0.7 Pg C year–1 from tropical land-use change, consisting of a gross tropical deforestation emission of 2.9 ± 0.5 Pg C year–1 partially compensated by a carbon sink in tropical forest regrowth of 1.6 ± 0.5 Pg C year–1. Together, the fluxes comprise a net global forest sink of 1.1 ± 0.8 Pg C year–1, with tropical estimates having the largest uncertainties. Our total forest sink estimate is equivalent in magnitude to the terrestrial sink deduced from fossil fuel emissions and land-use change sources minus ocean and atmospheric sinks.

Mike
August 26, 2011 8:19 am

The significance of ZR’s claim is that plant growth (global terrestrial net primary production) increased in the previous two decades but deceased slightly in the last decade. The new paper seems to say that plant growth merely levelled off this past decade. But the new paper does not seem (I have only skimmed through it) to comment on comparisons with the previous two decades. The question is did plant growth change from previous decades?

Hector M.
August 26, 2011 10:07 am

The Y.Pan paper does not refer to total plant growth, but to FOREST carbon fluxes and sinks. It distinguishes between 1990-99 and 2000-07. The annual rate of forest NET carbon sink INCREASED from 1.04 Pt/yr in the 1990s to 1.20 Pg/yr in the 2000s. This balance is the results of various tendencies in deforestation, regrowth etc.

Hector M.
August 26, 2011 11:05 am

Comparing with previous decades (i.e. before 1990 or so) is difficult because good satellite data are scarce or absent. Hard data refer mostly to the 1990s and 2000s. The most complete series available is FAO series of forest area, which starts in 1961, but it has been widely noted that it relies on faulty national statistics and “expert opinion”, and serious departures from satellite data have been observed in the 1990s. There may presumably be larger error farther back because the quality of the FAO series is supposed to be worse as one goes back in time.

Sun Spot
August 26, 2011 12:10 pm

Mod
The provided link does not take you to any press release ??
UPDATE: Here’s the press release from Boston University:
From Boston University
[No link ?? Robt]