Splicing Clouds

Guest Post by Willis Eschenbach

So once again, I have donned my Don Quixote armor and continued my quest for a ~11-year sunspot-related solar signal in some surface weather dataset. My plan for the quest has been simple. It is based on the fact that all of the phenomena commonly credited with affecting the temperature, such as cosmic rays, the solar wind, changes in heliomagnetism, changes in extreme ultraviolet (EUV), or changes in total solar irradiation (TSI), all vary in phase with the sunspots. As a result, if there is no sunspot cycle visible in the terrestrial surface weather datasets, then we can assume that none of those phenomena are affecting the dataset.

To date I’ve looked for an 11-year cycle in local sea level datasets, river flow datasets, beryllium datasets, global surface temperatures, lower tropospheric temperatures, global sea levels, Nile River levels, and a bunch more surface datasets. I have looked at literally dozens of measures of the weather, and I’ve found … well … nothing.

Now, does this mean that the sunspot cycles have no effect on the weather? Absolutely not, you can’t prove a negative in any case. It just means, I haven’t found the evidence to support the claims that some weather phenomena somewhere follows the sunspots. It may still be there, and so I continue looking.

I freely admit that I entered upon this quest with a preconception of what I’d find … but not the preconception I’m often accused of having. What many people may not realize is that I thought this quest would be easy. I believed that the evidence of a sunspot-climate connection wouldn’t be hard to find. Back in 2008 I wrote:

There are strong suggestions that cloud cover is influenced by the 22-year solar Hale magnetic cycle.

And given the number of places I’d read the claim that sunspots influence the climate, I figured it would be no problem to find surface weather datasets that showed some correlation with the 11 or the 22-year sunspot cycle. But to my great surprise, I’ve not been able to come up with a single surface weather dataset of any kind which shows such effects.

In this quest, I have relied mostly on a couple of methods to determine if the sunspot cycles are present. One is the cross-correlation analysis. This shows how well correlated two time series are at a variety of time lags. It helps in distinguishing real from random results—for example, if the correlations are as great for sunspots lagging the climate as they are for sunspots leading the climate, you’ve got problems.

Note that there is an inherent problem when looking for correlations with a signal like sunspots, which has a strong cyclical nature and high autocorrelation. This is that you will get what look like real correlations when you correlate sunspots with a wide range of random data. In addition, again because of the strong cyclical nature of sunspots, whatever correlations you find you’ll also find around eleven years later. As a result, you have to include an adjustment for autocorrelation when you calculate the statistical significance of your results, or you need to do a Monte Carlo analysis of the question.

The other method that I rely on is the periodogram. This is an analysis of the underlying cycles in the data itself. For example, here are the sunspots since 1749:

Figure 1. Monthly sunspots since 1749. Pre-1947 values have been increased by 20% to bring them into line with the modern counts, as has been repeated discussed here at WUWT. It makes little difference for the analysis.

And here is the periodogram of the sunspots shown above:

Figure 2. Periodogram of monthly sunspots. A periodogram shows the strength of each of the component cycles. The units are the percentage of the range of the data (range of individual cycle divided by range of the data. Due to popular demand, the data is windowed with a Hanning window.

Note that there is a fairly broad peak with the largest value at 11.2 years, and a secondary narrow peak at 10 years. So if we find these kinds of signals in any surface weather dataset, that would be good evidence that there is a connection to either the sunspots, or to some phenomenon that varies in phase with the sunspot cycle, such as cosmic rays or TSI.

With that as prologue, let me show the latest dataset I’ve been investigating. This is US station records showing the percentage of clouds. The cloud data are from 192 locations scattered around the US. As is my custom, I started out by looking at each and every one of the 192 records in the dataset. Here is a sample of ten of them:

Figure 3. Ten of the individual cloud records in the dataset. 

When I looked at that, I was surprised by the trends. To be sure, the scale is different for each panel, which emphasizes the trends. But the same results are visible in the average of the dataset.

Figure 4. Average of all of the 192 cloud datasets. Unfortunately, the data ends in 1987. The data is calculated as the cumulative sum of the means of the first differences (monthly changes) of all available datasets. Note that in this case, this first-difference method shows little variation in the period 1900-1987 from a simple mean of the data (not shown).

Now this is what I call an interesting result. I would never have guessed that the cloud cover in the US was about 10% greater in the 1980s compared with the 1920s. I do find it interesting that in response to the rapid US surface temperature rise during the 1930s and 1940s, the cloud cover acted to minimize the change by increasing quite rapidly.

However, that’s just evidence in support of my hypothesis that the clouds (and other emergent phenomena) act to minimize changes in temperature. It says nothing about whether there is a solar signal there or not. To determine that, I calculated the periodogram for the average cloud percentages shown in Figure 4.

Figure 5. Periodogram, average of US station data.

As you can see, there is very little in the way of power in the range from five to twenty-five years … no sign of a solar signal.

The situation with the individual datasets is much the same. There are a variety of signals, but again nothing in the range from five to twenty-five years exceeds about 5% of the total signal. Here is a typical set of four:

Figure 6. Periodograms showing the strength of the various cycle lengths, for four individual stations.

As you can see, there is quite a bit of variation in the individual records. Some have strong annual cycles, while others have equal-sized annual and six-month cycles. And some show only a six-month cycle, with no clear annual cycle. This is interesting, because it allows us to identify those stations which have strong thermally-driven summer thunderstorms. They show strong six-month cycles.

But to return to the sunspots, none of the stations shows any significant strength in the range from five to twenty-five years, it’s all down in the weeds, and each of them is different.

So I had to conclude once again that the solar signal is simply not visible in these cloud datasets.

Once I was done looking at the solar question, I got to thinking about the fact that the ground station cloud dataset ends in 1987. I realized that the ISCCP satellite dataset started in 1983 … giving a 53-month overlap. So I decided to see what the ISCCP dataset had to say about the US clouds. Since the ISCCP dataset is gridded, it is a matter of extracting just the land data in the area covered by the US stations. I started by looking at the two datasets during the period of the overlap:

Figure 7. Overlap of the two cloud coverage datasets, one being the average of 184 ground station records (green), and one from the ISCCP satellite data (blue). In the raw form, the two are offset by only about one percent. As a result, I have rebaselined the ground station data upwards by about one percent to match the ISCCP data during the period of the overlap.

Now, I have to say that I was very impressed by the agreement between the two datasets. I had expected to find much poorer correlation. But in fact, by and large the two datasets are in surprisingly good agreement.

As a result, I think that we are justified in splicing the two datasets together, so that we can get a long-term record of the US clouds. Here is that record.

Figure 8. Spliced data, with the early data from the ground stations and the late data from the ISCCP satellite. I used the average of the two datasets during the period of the overlap.

Now, this is a fascinating graph containing lots of unanswered question. It puts the ISCCP data (black and red) into a most interesting long-term context. When looking at just the ISPPC data it seems as if the cloud coverage was falling. But in the longer context, it can be seen that this kind of variation has been going on since about 1960.

I draw no great conclusions from this, except that overall, as the US warmed post-1930, the US cloud coverage rose as well. Go figure …

One reason I’m reluctant to draw many conclusions about this graph of the clouds is a curious one. In science, it is very difficult to demonstrate causality. However, a very smart man named Granger realized that we can demonstrate a weaker form of causality, called “Granger causality”.

The essence of the idea of Granger causality is that if having historical information about phenomenon A allows us to significantly improve our predictions of phenomenon B, then A is said to “Granger-cause” B. Makes sense to me. If knowing historical cloud cover allows us to improve our predictions of temperature, then cloud cover would be said to “Granger-cause” temperature.

However, there’s an oddity of Granger causation, which is that there are not just three, but four possible states, viz:

1) Neither variable A nor variable B Granger-causes the other, or

2) Variable A Granger-causes variable B, or

3) Variable B Granger-causes variable A, or finally

4) Variable A Granger-causes variable B AND variable B Granger-causes variable A.

So … care to guess the Granger relationship between temperature and clouds? Yep, you’re right, it’s choice 4) … each one Granger-causes the other one. Clouds affect the temperature, and temperature affects the clouds. Knowing the history of either one helps us to predict the next step of the other one. It’s just another demonstration of Murphy’s Corollary, which is that “Nature always sides with the hidden flaw.” Which definitely makes it hard to draw lots of conclusions.

In any case, that’s all the fun I’m legally allowed to have in one twenty-four-hour period. So to keep the future full of interest, let me put out a request for people to send me a link to a dataset. What dataset? Why, of course, the surface weather dataset that you think most clearly shows the sunspot-related solar influence.

A few notes about the kinds of datasets I’m looking for.

First, I’m asking what you think is the very best dataset, the dataset that actually contains the clearest evidence for the solar signal. Please don’t send me links to ten datasets. I don’t have graduate students or assistants and I do have a day job pounding nails. So please, choose your best evidence to make your best case.

Note that I’m not asking for studies, I’m asking for observational data. I’m not interested in any of the hundreds of studies out there that claim to find a solar signal in some surface climate dataset, like my favorite study. That was the one that found the solar signal … in the ring widths of one single tree in Chile. Seriously, one single tree, the study is out there, and somebody sent it to me claiming it was evidence.

Here’s the issue. Every study that I’ve looked at has had serious and often horrendous problems, and I’m sick of digging through the trash looking for a diamond. In any case, no matter what the study may claim, I’m gonna have to do the analysis myself. And to do that, I don’t need the study—all I need is the link to the dataset.

Next, please provide a link to the exact dataset itself. I don’t want to guess which dataset you are referring to, or choose from among a dozen datasets on some web page, or have to google some obscure name.

Next, please no “reanalysis data” from NCAR or NCEP or anywhere. Let me explain the problem with using what is laughably called “reanalysis data” from NCAR or NCEP in any analysis of solar effects. The “reanalysis data” is not data at all. Instead, it is the output of a computer model. Now, I’m not an opponent of computer models, I’ve both written and also used lots of computer models. The problem is that the current generation of climate models are quite linear … but nature is antilinear.

Now, for some kinds of analyses this is not a big problem, but this flaw makes model output useless for some specific applications. In particular, since the model outputs are some kind of lagged linear transformation of the inputs, in general whatever you put in the front end of a climate model will be visible in the output … and downward solar radiation flux is most assuredly an input to the reanalysis models. Since the models are linear, this means that there is a very good chance that we will find a solar signal in the output of the reanalysis models, that we’ll see a sunspot-related wiggle in the so-called “reanalysis data” … but unfortunately finding that signal in the computer output means absolutely nothing about the extremely non-linear real world. So please, if it says NCEP or NCAR or “reanalysis” anywhere, don’t bother sending it.

Finally, as a radio amateur I’m well aware of the solar effect on the ionosphere. Up there where there is one molecule per cubic ohmmeter or somesuch miniscule number, and it’s not atmosphere as we know it but a plasma of ionized gases, yes, I agree that the sun has effects. However, I can find no evidence that those effects make it down to affect the weather where we live, at the surface. Might do so, but I haven’t seen it yet. This is why I’ve asked for surface datasets, because I’ve never found the claimed 11-year cycles down here at the bottom of the gravity well.

So with those exceptions, you’re free to send any of a host of surface datasets that you think have a sunspot signal—sea level, rainfall, river flow, lake levels, temperature, atmospheric pressure, river flow and many more, the choice is up to you.

Now, there is a commenter who objects strongly whenever I ask people to send me their best evidence. He claims I’m asking him to do the research that I should be doing. But all I’m asking for is a dataset—I am the person who has to do the research. The problem is, there are hundreds and hundreds of datasets out there, and my time is short. I don’t have time to analyze them all. So I’m asking for your assistance in winnowing the wheat from the chaff. If you don’t like my asking for assistance in this process, please, just ignore the post. Raising the same objections once again won’t burnish your reputation.

In addition, I see this as a chance for you guys to definitively show everyone that there actually is a sunspot-related signal that makes it down here to the surface where we live. So I invite you—bring out your best dataset. I’ve just looked at the US cloud record and found nothing … your turn.

Regarding the clouds, I walked this evening with the gorgeous ex-fiancee along the western edge of Bodega Bay. The cold front that brought us welcome rain earlier in the day was just past, and so we got to look east across the Bay at the back side of the cold front. Mostly it was kind of a wiggly stratus, an unusual formation. Out to the west, there was clear sky, and the late evening sun from behind us lit up the clouds above and the hills below. Looking at that blue, clean-washed piece of sky, I was reminded of the scientifically accurate words of the poet  about clouds:

 I am the daughter of Earth and Water,

And the nursling of the Sky;

I pass through the pores of the ocean and shores;

I change, but I cannot die.

For after the rain when with never a stain

The pavilion of Heaven is bare,

And the winds and sunbeams with their convex gleams

Build up the blue dome of air,

I silently laugh at my own cenotaph,

And out of the caverns of rain,

Like a child from the womb, like a ghost from the tomb,

I arise and unbuild it again.

Best regards to everyone, keep on unbuilding,

w.

MY REQUEST: If you disagree with someone (highly unlikely, I know, but work with me here), please QUOTE THE EXACT WORDS YOU DISAGREE WITH. That will let everyone know just what you think is incorrect.

MY POSITION: Note that I am NOT saying that the sun doesn’t affect the earth. It does affect the earth in a host of ways, and we can see that every dawn.

What I AM saying is that I have never found a significant sunspot-related ~11 year signal in any surface weather-related dataset. Doesn’t mean it’s not there somewhere, just means I haven’t found it yet. If you think it’s there … this is your chance to identify the dataset which contains the signal.

DATA: the ISCCP D2 cloud coverage data is here, the US ground station cloud coverage data is here, and the sunspot data (unadjusted) is here.

CATALOG: all of the ISCCP data is indexed here. Took me a while to find it, because it’s on the NOAA website.

CODE: R computer code, along with some of the collated data, is here as a zipped folder. The code is all there. However, I’d describe it as user-aggressive … and like all of my code, it’s not written to run from top to bottom. It also contains a lot of other things that don’t make it into the final report, various analyses and graphs. I believe that all of the code and functions are there to produce all of the graphics and analyses above. If not, let me know.