Truth(?) in testimony and convincing policy makers

From Dr. Judith Curry’s Climate Etc.

Posted on June 27, 2019 by curryja |

by Judith Curry

Some reflections, stimulated by yesterday’s Congressional Hearing, on the different strategies of presenting Congressional testimony.

Yesterday’s Hearing provided an ‘interesting’ contrast in approaches to presenting testimony, when comparing my testimony with Michael Mann’s.

What are the purposes of expert testimony?

There is an interesting document entitled A Guide to Expert Testimony for Climate Scientists, funded by the US National Science Foundation.  Most of this is related to court room hearings, but some is relevant for Congressional Hearings.  Excerpts:

<begin quote>

Experts may do one or more of the following:

• Provide the decision-maker with factual information and background to provide the decision-maker with an adequate context for the decision.
• Apply expert knowledge to the facts of a case and render an opinion about the facts, such as whether certain conditions actually caused an effect.
• Explain scientific principles and theories to the decision-maker.
• Extrapolate from the actual facts or hypothetical facts and rendering an opinion regarding the likelihood of an event or occurrence. Experts may speculate on events or occurrences because of their special knowledge or training.
• Provide an opinion that contradicts or undermines the opinions or conclusions of an expert who testified for the opposing party.

If you are assigned to cross-examine an expert, you should prepare questions that test and challenge the witness on the following subjects :

• Lack of thoroughness in investigating the facts or data;
• Insufficient testing of the facts or data;
• Lack of validity and reliability in testing of facts or data;
• Existence of other causes or explanations for conclusions or outcomes;
• Show differences of opinion among experts

<end quote>

‘Opinion’ or  ‘evidence’ for Hearings?

In my own testimony, I referenced (and even quoted) the IPCC AR5 and the US National Assessment report about a dozen times.  I also provided my (forthcoming) Report on Hurricanes and Climate Change, which includes about 100 references (nearly all are refereed journal publications) plus links to other review articles that provides further references.

Whatever happened to climate scientists using the IPCC and National Assessment Reports in their analyses, either to support their arguments or otherwise refuting specific statements in these Reports?  It seems that only scientists of the non-alarmist persuasion are citing these Reports any more.

Congressional testimony is not the place for scientists to present new, primary research.  Rather, it is an opportunity for scientists to present analyses of relevance to the topic at hand, related to their personal expertise.  This may take the form of an opinion piece (op-ed) or an analysis supported by evidence.

Mann took a different approach from mine.  His testimony reads like an op-ed, and he even cites his op-eds as supporting evidence.  Yes, it is readable, but it is not well documented.

Mann did not provide a bibliography for his testimony or any footnotes; rather he included hyperlinks.  I clicked on each of these, to see what sources he was using.

His links include 3 references to his own journal publications, plus two links to publications by other authors.  One link is provided to a NOAA statement. Several links are made to the StonyBrook University blog, describing unpublished analyses.  This selection was criticized by Andreas Schmittner on twitter:

All of the other links (~20) are to news articles, some of which are op-eds written by Michael Mann himself or articles that interviewed Michael Mann. The list of sources used by Mann in his written testimony:

Climate Central, PBS, Time, Slate, LiveScience, PennLive, The Guardian,  Scientific American, New Observer, Washington Post, NYTimes, ScienceNews, National Geographic, RollingStone, NewsWeek.

I understand the difficulty that policy makers have in wading through peer reviewed journal articles.  This is why Assessment Reports are useful for policy makers (although I am not a fan of the oversimplified, cherry picked conclusions in the Summary for Policymakers).  News articles are much more easily read by policy makers, but many of them are misleading at best.  And it is hard for me to imagine any of these articles being seriously considered as ‘evidence.’

Now several of these news organizations generally do a credible job in reporting on science, although they invariably suffer from single study syndrome in their individual articles.  But such articles are hardly a substitute for published primary journal articles or carefully considered assessments — or better yet, national or international assessment reports.

Although Mann’s testimony extensively referred to hurricanes, there was not a single reference in Mann’s testimony to the IPCC, the U.S. National Assessment Reports or the numerous review articles on hurricanes and climate change that have been written by teams of experts.

Yes, the published literature is sufficiently broad and diverse to support numerous narratives about climate change, and there are many reasons that rational scientists disagree:  insufficient data and disagreements about its quality; relative weighting of different types of evidence; and different logics for linking the evidence.

But when you open this up to include in a dominant way news articles and op-eds, then anything goes.

As summarized in the recent review by Knutson et al. on the issues of hurricanes and global warming (discussed in the Extremes blog post), there is a very substantial range of perspectives among scientists who have primary expertise in the climate dynamics of hurricanes.

Is it appropriate in Congressional testimony to present only your own perspective, without acknowledging other perspectives, disagreement, uncertainty?  Including both myself and Mann in the Hearing provides ‘dueling’ perspectives, but this hardly represents the range or distribution of perspectives in the community. Unlike a court case, there is insufficient time to probe all this.

Authority

So, which of the dueling experts is the Congressional Committee to believe?  Well that is almost certainly predisposed by their political party, to the extent that I have to wonder why we were even invited to this Hearing.

In the follow up to yesterdays Hearing, there has been some discussion on twitter related to Mann’s extensive emphasis on his own credentials in both his written and verbal testimony.

His verbal testimony spent almost a  minute listing his own credentials, out of an alotted 5 minutes (the Chair allowed Mann’s testimony to go over the time allotment).  Mann defended this by saying

I’m not interested in playing Mann’s little game re expertise.  But it is a tough argument to convince anyone that he has greater expertise than I on hurricanes.

Apart from someone’s political bias, that leaves the substance of our written testimony as a basis for being convinced by one versus the other.

I continue to have this naive, idealistic view that carefully crafted  and communicated analyses with credible documentation is what policy makers want and need.

So does Mann’s focus on his own credentials and publications  trump my analyses, documentation and references to the US National Climate Assessment, etc.?  At that Hearing and with that Committee, maybe it did.

Truth(?) in testimony

When testifying before Congress, each Witness signs a Truth in Testimony statement.  At yesterday’s Hearing, the witnesses were asked to stand and verbally agree to this (first time I recall doing this in a Congressional Hearing).

What does ‘Truth in Testimony’ actually mean regarding a controversial topic in science?  Yes, there is much disagreement about aspects of climate science, that is not what I am concerned about here.

In yesterday’s Hearing, Mann made a factually incorrect statement in response to a question:

90:13 “I want to correct a number of fallacies that we’ve heard here today when it comes to the connection between climate change and extreme weather events.  First of all, you sometimes hear this myth about there having been a supposed hurricane drought and there’s some sleight-of-hand going there because what’s going on Superstorm Sandy was a strong category 3 and then weakened to a category 2 hurricane off the coast of the US east Coast now it did go as they say extra-tropical it was technically no longer a hurricane when it made landfall but it was spinning off the East Coast for several days as a strong hurricane building up a very large storm surge and as we know it was this storm surge that was so devastating to the Jersey coast into New York City so its extremely misleading when you hear statements like that”

Mann’s statement misled the Committee with his statements about the drought in major hurricanes, Hurricane Sandy, and about my testimony being fallacious.  There was no opportunity for me to speak up in the Hearing.  I was shaking my head no, this was noticed by a Republican member, who asked for an opportunity for me to reply, but the Chair gave me no opportunity to respond.  Below is my response to Mann’s statement about my testimony.

My written testimony included the following statement:

“However, it was rarely mentioned that 2017 broke a drought in U.S. major hurricane landfalls since the end of 2005 — a major hurricane drought that is unprecedented in the historical record.”

This one is simple to fact check.  Go to the NOAA website and count the number of major hurricanes (Cat 3+) between Hurricane Wilma (2005) and Hurricane Harvey (2017).  Zilch.  Here is a graph of the data from the National Hurricane Center that was included in my written testimony:

With regards to Hurricane Sandy as an alleged ‘drought buster.’ Hurricane Sandy (2012) is included in the list of U.S. landfalling hurricanes with an * since technically it wasn’t a hurricane at landfall.  Sandy’s max wind speed at landfall is listed at 65 knots (Cat 1 territory).  As stated in my testimony, the  large storm surge associated with Sandy was caused by her transition to a horizontally large extra-tropical storm, not by her brief resurgence to a Cat 2.

I remember the details of Hurricane Sandy in excruciating detail, since my company CFAN was forecasting hurricanes (our Sandy forecast was exceptionally accurate relative to government provided forecasts).

In any event, even apart from the classification of Sandy as a hurricane or not, the terms ‘landfalling hurricane’ and ‘major hurricane’ (Cat 3+) have very clear and specific meanings, and Hurricane Sandy wasn’t a major hurricane at landfall, and only briefly reached low-end Cat 3 status near Cuba.  See the NHC’s Summary Report on Hurricane Sandy

There is no question that Hurricane Sandy was catastrophic for New Jersey and New York City.  Sandy illustrates how unprepared these cities were for even a Cat 1 hurricane with a significant storm surge.  Sandy is not a good poster child for manmade global warming, but rather supports the arguments made in my testimony about not being prepared for current or historical hurricanes.

JC verdict on Mann’s statement:  Five Pinocchios

Other rhetorically effective but misleading strategies used by Mann’s testimony were to cherry pick a single study and to imply that speculation about a linkage of some storm with global warming is actually a well accepted conclusion.  I will give one example here, that arose in the questioning, which is related to the high-profile issue of whether Category 4/5 hurricanes have been increasing:

“I actually co-authored an article in the journal Nature about 10 years ago where we use geological information from sedimentary deposits left behind by ancient hurricanes so we can actually reconstruct the history of landfalling hurricanes along the U.S. East coast along the Caribbean and so we have this rich archive of information that tells us in fact the increase in intensity that we’re seeing today does appear to be without precedent as far back as we can go.”

The paper that Mann refers to is [here].  Perhaps Mann hasn’t kept up with the literature on paleotempestology, which I summarized here.  Here is a summary paragraph from my Report on Hurricanes and Climate Change:

“Summary. There has not been a timeline or synthesis of the Atlantic hurricane paleotempestology results for the past five thousand years, either regionally or for the entire coastal region. However, it is clear from these analyses that significant variability of landfall probabilities occurs on century to millennial time scales. There appears to have been a broad hyperactive period from 3400 to 1000 years B.P. High activity persisted in the Gulf of Mexico until 1400 AD, with a shift to more frequent severe hurricane strikes from the Bahamas to New England occurring between 1400 and 1675 AD. Since 1760, there was a gradual decline in activity until the 1990’s.”

So, by cherry picking one paper (his own) that examines geologic data at only one location, Mann misled the committee regarding whether or not the intensity of Atlantic hurricanes has been increasing relative to the geological record.

JC verdict: two pinocchios

In Mann’s Congressional testimony two years ago [blog post], he made two statements in the questioning period that contradicted what was in his written testimony and his c.v.; for documentation of this see the links at  WUWT, Warmist Michael Mann tells whopper at Congressional Science hearing.

It was much more difficult for Mann to get away with  factually incorrect statements in a Hearing chaired by the Republicans than in a Hearing chaired by the Democrats.

JC reflections

I have often criticized the Congressional testimonies of other climate scientists as being normative, in the sense of advocating for specific policies related to climate change.

In hindsight, normative testimony seems pretty tame when compared with ‘assertion from authority’ testimony from scientists.  This style of testimony extensively establishes the witness’ expertise, and then makes a series of assertions with little or no documentation.  In short — appealing to their own authority.  This strategy is often accompanied by attempts to tear down the credibility of opposing witnesses.

If such testimony by assertion was presented in a legal trial, it would receive a severe grilling on cross-examination.  In a Congressional Hearing where the witness supports the majority’s perspective, the witness pretty much gets a pass, even by the opposing party. The minority members tend to focus their limited time on questioning  the witnesses invited by their own party.

This Hearing is certainly making me rethink my participation in future Hearings. I very much enjoy the challenge and opportunity of preparing written testimony and communicating my analyses of the issue at hand to policy makers.  However, I am not cut out to be a politician. I have a bad habit of answering any question as accurately and honestly as I can, rather than using my 90 seconds to refute my opponent or to emphasize my own point.

This makes me wonder what the Democrats are really trying to accomplish with these hearings on climate change.  If they are so convinced the science is completely settled, why do they bother with these Hearings?  Do they think they are going to convince the Republicans with a witness such as Michael Mann? The politics surrounding climate change make little sense to me.

Acknowledgements.  I would like to thank Larry Kummer for providing a transcript of the Hearing and for providing comments on earlier drafts of this post.