MCID: Minimal Climatically Important Difference

Guest Essay by Kip Hansen

Warning: This essay is about a recently emerging concept being applied in medical research. In the course of the essay, it will become clear how this concept might cross over to the field of climatology. Those readers only interested in climate issues or the Climate Wars should move on to the next offering here at WUWT – this essay is not for you.

There is a new concept slowly emerging in the field of clinical medical research. It is called Minimal Clinically Important Difference.

Here’s the definition:

“When assessing the clinical utility of therapies intended to improve subjective outcomes, the amount of improvement that is important to patients must be determined. The smallest benefit of value to patients is called the minimal clinically important difference (MCID). The MCID is a patient-centered concept, capturing both the magnitude of the improvement and also the value patients place on the change. Using patient-centered MCIDs is important for studies involving patient-reported outcomes, for which the clinical importance of a given change may not be obvious to clinicians selecting treatments. The MCID defines the smallest amount an outcome must change to be meaningful to patients.” — Minimal Clinically Important Difference — Defining What Really Matters to Patients by Anna E. McGlothlin, PhD and Roger J. Lewis, MD, PhD

This concept can also be applied to outcomes [that] are not patient-reported but which are simply measured numerically – such as blood pressure, body temperature, blood cholesterol levels and body weight or BMI.

Note: The first literature instance easily accessible is Measurement of health status: Ascertaining the minimal clinically important difference by Jaeschke, Singer and Guyatt (1989) doi:10.1016/0197-2456(89)90005-6 — after the turn of the century one finds more and more frequent references to his concept.

Let’s look at some examples:

Body Temperature:

We take our “temperature” to determine if we are ill or not. When we were children, our parents may have used an oral thermometer to determine whether we were too ill to attend school – if we had a fever, we stayed home – if not, off to school. It is generally accepted that normal body temperature ranges between 36.1°C (97°F) to 37.2°C (99.7°F) – alternately: the typical oral (under the tongue) measurement is slightly cooler, at 36.8° ± 0.4 °C (97.5°F to 99.5°F), with normal generally considered to be 98.6°F (and so marked on your mother’s glass oral thermometer). Well, we see right away that the term normal body temperature is not quite agreed upon. Nonetheless, there are things about body temperature that are clinically well understood: oral or core body temperatures above 100.4°F or so (for persons teenaged or older) is considered a fever and an indication of something wrong – such as an infection — high-grade fevers range from about 103°F-104°F — dangerous temperatures are high-grade fevers that range from over 104 F to 107 F or higher. Fevers are not only indicators of health problems but they can be dangerous in and of themselves. Temperatures below 95°F are considered dangerously low, verging on life threatening as core body temperature continues to fall lower.

Example: Let’s say we were researchers and wanted to test a fever-reducing drug. Various treatments that are supposed to affect body temperature, to treat fevers, would be judged by their action of changing the body temperatures of the patients in a clinical study – measured by thermometer. [Aspirin, for instance, is considered a fever-reducing drug and is often prescribed for just that purpose — it is one of many non-steroidal anti-inflammatory drugs.]

This concept of Minimal Clinically Important Difference calls for the researchers to determine in advance what numerical amount of change will represent the Minimal Clinically Important Difference – the numerical amount that must be surpassed for the results to actually mean that patients’ health has been improved in a way that is important to them – to the patients, to the patients’ health, not just to the doctors, not just numerical/statistical way.

In our body temperature example, it is apparent that lowering a raging fever of 105°F by 0.5°F to 104.5°F (but no lower) – even though it is a 7% decrease in the above-normal anomaly temperature – may not pass the Minimal Clinically Important Difference test – 104.5°F is still a raging fever and the patient may not be clinically better off. If the treatment does not bring it down by what was pre-determined to be the MCID (in this case, the MCID might be to reduce the fever to a certain level) then the treatment is less than successful or the effect of the treatment is not significant. It does not matter that the result – lowered average temperatures of patients in the study by 0.5°F – was found to be statistically significant if the result itself does not meet the pre-determined, and correctly identified, MCID.

Likewise, one could look at various treatments that lower a study cohort’s average of some metric – like blood pressure. This is often an approach used by epidemiologists looking at large health databases.

High Blood Pressure:

Note: Blood Pressure numbers represent another science war – though a quieter one. For many years, doctors considered 120/80 to be ideal and anything under 140 to be OK but now the threshold for hypertension has been dropped first from 160/100 to 140/90 and now, with prehypertension defined as 120/80 to 139/89, to where maximums of normal have been set at 120/80 – what used to be ideal is now the maximum acceptable before treatment is recommended. In the UK, however, blood pressures are usually categorized as: low (90/60 or lower), normal (between 90/60 and 139/80), and high (140/90 or higher). A recent study says doctors have been over-treating high blood pressure in seniors and that seniors should not be treated until their BP exceeds 150. And on it goes, with advocacy groups (and drug companies) pushing for the setting of official limits low and evidence-based-medicine doctors pushing back.

Example: Doctors wish to test a new blood-pressure-reducing drug. The doctors carrying out the study will be testing the drug on men with High Blood Pressure (US standard) >140/>90. Before anything else, they should define the MCID for blood pressure for this cohort – I don’t know what it would be, but they would have to set it before the study is carried out.

Let’s just guess that the average systolic blood pressure (the higher number) for the cohort is 150. In a cohort of 1000 men receiving treatment for high blood pressure (HBP) the study finds average reduction in systolic pressure to be 5 mm Hg, lowering the average to 145. Is this a MCID?

Well, I don’t know – the study’s authors would determine this by comparing their result with their pre-determined MCID. (In my opinion, 5 mm Hg with HBPs of 150 or so would not bring any one patient relief from the symptoms of HBP nor improve their health in any important way.) However, the result may have been found statistically significant with a very small P-value – it may just not be clinically important.

[ An aside: There is one more thing about many of today’s medical studies that is amusing – there might have been some effort in the above hypothetical blood pressure study to say that while that 5 mm Hg might not be much for individual patients it would be significant in public health setting (and I know this sounds crazy – but this really is claimed quite often in epidemiological studies) because 5 mm HG times 1,000,000 people is a huge reduction in public blood pressure! Epidemiological madness. ]

We have established that the Minimal Clinically Important Difference must be determined before the study is carried out, as part of the study’s planning and design. This is logical as the researchers must know in a general way what measurement tools to use – glass-and-alcohol oral thermometers or high accuracy digital thermometers, for instance. Or whether blood pressures should be determined my nurse-operated sphygmometers or mechanical digital equipment. What measurement scales to use: tenths of a degree C or thousandths of a degree C. How many subjects they must have to determine results to that accuracy or scale. There is no need for high accuracy thermometers if the MCID is “reduces body temperatures to within 1°F of 98.6°F”. No need for high-tech/high-accuracy blood pressure measurement tools if the MCID is “reduction of systolic blood pressure by 20 or more mm Hg.” Researchers comparing actual results with a pre-determined MCID will then know if their results will mean anything important for patients – to know if they really have a meaningful, clinically important, result.

So why is this concept being presented here at WUWT?

· First, because a pre-determined MCID is a very interesting addition to the scientific method in general. Requiring researchers to define what magnitude or type of result is clinically important (or in other fields: physically important, culturally important, militarily important, environmentally important, etc.) [Note that MCID may not apply in basic research in which one is simply trying to find out what there is to find out.]

· MCIDs become part of the study’s findings — just like methods and study design – and thus can be peer-reviewed. They can be taken into consideration when colleagues review (critique or praise) the resultant paper.

· MCIDs make it much more difficult to inflate the value of a research finding by disguising its insignificance. MCIDs can greatly reduce the bloat of “nothing-found” research papers being published – and the endless follow-on studies to those “nothing-found” studies.

Once you get the concept, you’ll be able to spot studies that should have been held to a MCID standard – that might better have been returned to the authors by peer-reviewers with a note: “What was the MCID for this study? Do you think that your result passed your MCID test?”

What does this have to do with climate science? Could or should such a concept be applied to climate science studies? If so, which ones and under what circumstances?

I will give just one example, and then we can all discuss this in the Comments.

Take a quick re-look (or first look) at my recent essay “Baked Alaska?…”. This essay includes some results that might have benefitted from the MCID approach. The basic hypothesis presented about the climate in Alaska is represented by Mike MacCracken’s, chief scientist for the Climate Institute, statement along the lines of ”how long-term trends will play out in Alaska” which are claimed to have been dangerously warming.

This one image gives us all the data in one go (enough for purposes of this discussion):

Box (a), labelled 1920-2012, shows a long-term warming trends in 12 of 13 climate regions of Alaska, with the state-as-a-whole warming 0.9°C over the 92 year period. Box (d), labelled 1981-2012, however shows the entire state warming trend of only 0.1°C over the latest 31 years. The North Slope (at the top of the state) has a warming trend of 1.9°C over the same 31-year period. The climate region that includes Fairbanks, Alaska is shown with a 31-year cooling trend of -0.1°C.

Do you see where this leads? What is the Minimal Climatically Important Difference that we should be looking for in this data? Does Fairbanks’ -0.1°C 31-year trend meet a logical and scientifically supportable MCID for average temperature? Does the North Slope’s +1.9°C? Does it make sense to look at MCIDs for statewide annual temperature averages? Can a MCID even be set for annual average temperature alone?

Are there such things as Minimal Climatically Important Differences in climatology? Well, for Fairbanks, Alaska we find that despite 31 years of cooling trend their “# of frost free days” has increased by 50% since the 1920s (see the link Page 29). Now that’s a climatically important difference! The “fact” that the temperature has been trending down for a period long enough to be called climate disguises the climatically important difference of a significantly longer growing season – extremely important to this agricultural area – “one of Alaska’s premier agricultural regions and produces one third of Alaska’s agricultural products.”

Maybe climate science is spinning its wheels and wasting its time looking at all the wrong metrics….metrics for which one cannot define a logically and scientifically valid Minimal Climatically Important Difference. Instead, producing what Dick Feyman would have called “cargo cult science” – science aimed at supporting or proving one’s pet hypothesis instead of trying one’s darnedest to disprove it; spinning the narrative and the data to make one’s desired result look more probable, to make one’s itty-bitty “leans-my-way” result look strikingly significant.

Of a few things I am certain – one of them is that a difference in a couple of hundredths (“hottest year on record”) or tenths of a degree Centigrade of annual difference in the derived “apples-and-oranges” metric we know as “Annual Global Mean Temperature Anomaly Over Land & Sea” does not, and probably cannot, pass any possible scientifically valid Minimal Climatically Important Difference test.

In the same way, I would argue that nearly every political-boundary-based climate data set does not present any information that could lead to a valid Minimal Climatically Important Difference, based on the concept that political boundaries almost always comprise more than one climatic region (as in our Alaska example – for which 13 are proposed).

It does not help water-starved farmers in the Midwest to know that the farmers in the Central Hudson Valley of New York have adequate rain for their crops, thus keeping the CONUS Average Precipitation at an anomaly of zero against the 30-year average. Nor does it benefit the victims of Mississippi River flooding to know that their sorrow is not caused by climate change but rather just by a perfectly-historically-normal warm and rapid snow melt season hundreds of miles away. It is unlikely that small changes in annual average temperatures, measured and reported in single-digit tenths of a degree, are Minimal Climatically Important Differences but the differences within the same data set that reflect annual minimum temperatures, measured and reported in tens of degrees, might make the difference between wide-spread insect infestation with its resultant damage and freedom from serious insect damage – certainly a Minimal Climatically Important Difference. In this last case, the MCID might be whether or not it has remained continuously below -20°F for seven or more days.

I’ll start the comments off with this question: In all the issues that are considered here at WUWT – in all the numerical and statistical presentations of findings here, on other climate science sites and in the climate science journals-of-record – what type of results might inform science better if the concept of Minimal Climatically Important Difference was used in study design and results evaluation?

# # # # #

Macro or Micro? While this photo looks like it could be some crystalline growth on a flat surface, looking a little chaotic with its resemblance to the Mandelbrot Set, it is actually: “Macro: Cumulus clouds over the South Pacific Ocean (Image created by Jacques Descloitres, MODIS Land Rapid Response Team, NASA/GSFC, additional image processing by Stephen Young.)”

# # # # #

Author’s Reply Policy: My father was a doctor, an M.D., a pediatrician – I am not. The medical examples are written from a general public perspective. If there are medical errors – please inform, but do not blame. I use them only to make a point about MCID.

I am not particularly interested in climate science or the Climate Wars – my focus is more on science methodology, scientific ethics and science journalism, its use and misuse. This essay is about the concept of pre-determined MCIDs and their importance as an additional research method or tool – bridging the concept from its current use in clinical medicine to other fields – in this case, climate science. I’d like to hear your views.

# # # # #