Sunspots and Norwegian Child Mortality

Guest Post by Willis Eschenbach

In January there was a study published by The Royal Society entitled “Solar activity at birth predicted infant survival and women’s fertility in historical Norway”, available here. It claimed that in Norway in the 1700s and 1800s the solar activity at birth affected a child’s survival chances. As you might imagine, this attracted much interest, particularly among the credulati. So I wanted to take a look at their data.

Unfortunately, the authors did not archive the data and code. So I got in touch with said authors, and they kindly provided part of the data. This is a dataset of some 5,167 individuals showing birth years, sex, solar “MAX/MIN” category, and lifespan. In particular, my thanks to Professor Eivin Røskaft who has been most responsive to requests and queries from myself and others regarding the data and code.

Now, before I do any detailed analysis of a given dataset, I first want to see what all of the data looks like. It’s a graphical game, because we can’t gain anything by looking at a huge pile of numbers. We have to convert the numbers into patterns in order to grasp their nature, distribution, and significance. Before doing any detailed analysis I like to start by looking at the big picture, all of the data. So here was my first plot of the Norwegian data. It’s a plot of the lifespans of the individuals (vertical axis) versus their year of birth.

Figure 1. Lifespan by Birth Year, Norwegian data. In order to show the underlying structures, I use partly transparent dots. In that way, it’s darker where the data piles up. 

Mmmm … you can see why the first step should always be to look at the whole dataset. It’s obvious that we are looking at some curious combination of information which has led to two very different subsets of data. There are a bunch of people who died before they were twenty-one … and then there are a bunch of people who mostly lived a full lifespan, sixty to ninety years or so.

I have no idea why the dataset is divided in that manner. I thought it might be data from two different sources, but that turns out not to be the case. Professor Røskaft was kind enough to provide us with the following answer to my question:

We see your point about variation in sample sizes and age classes across years, and there are certainly many caveats when analysing such historical records. However, the data are all gathered from the same source, and the “two-datasets” impression is an inevitable consequence of the data collection. These data were collected with tracking individuals across generations in mind. Starting with the first church record, we could find the first adult generation, which we then used to track their offspring, which of course only were born ca. 20 years later. If you do a simple exercise of reading a church record, you will find births, marriages and deaths. Let’s say that we start with the first book in 1700. Then we will know the number of births that year, the number of marriages and number of deaths. For the married couples, we could even go back in time, since we know their age at marriage (let’s say they were born in 1675). Hence, the different distribution of the “two-datsets” is the difference between the generations. However, there are an equal NUMBER of generations in each group.

The lack of deaths around 20 years of age is probably caused by the fact that we could only track married couples staying in the same village as they were born. As stated in the note under Table S1, unmarried and emigrated individuals could not be included.

While I appreciate his reply, I still don’t see why any of those would make the sharp cutoff at exactly twenty years seen in Figure 1. I certainly may be missing something in his explanation. But let’s set that aside, and continue with the data.

As is common in my life, I’m looking at some kind of dataset that I’ve never analyzed before. So the diagonal lines in Figure 1 puzzled me for a bit. But then I realized that these were mass mortality events, likely epidemics or crop failures. And in fact there are records of a bad epidemic in Norway in 1772-3. This is the darkest of the diagonal lines in Figure 1. It starts at the bottom of the graph in 1773 and goes up to the left, as indicated by the red arrow. It’s described in one church record as a “blistering fever”, so perhaps measles, cowpox, or smallpox.

I bring this epidemic up to highlight a problem with the data. Look at how many infants, children and young adults died as a result of the 1773 epidemic, lots of deaths right up to the age of 20. And there are also deaths among older individuals, in their thirties and forties and more … but we have no recorded deaths of 21- or 22- or 23-year-old individuals. The recorded deaths cut off abruptly at exactly 20 years. So obviously the dataset is not an accurate representation of the deaths of the times … worrisome.

In any case, here’s my first cut at comparing the sunspot numbers at birth with the lifespans for the full dataset. Figure 2 shows a scatterplot of lifespan versus sunspots during the birth year.

Figure 2. Scatterplot, lifespan versus sunspots during the birth year.

At least for the full dataset, there is no significant relationship between sunspots and lifespan.

Now, regarding the appearance of there being two datasets, Professor Røskaft was good enough to explain to us that it doesn’t matter because their work is based on infant and child mortality rates, viz:

As I am sure you know from reading our paper, we did not do any statistical analyses of life expectancy. For the survival analyses, we only used the subset of reaching adulthood or not (20 years of age). The life expectancies reported for each group in Table S1 (online) are merely means +/- SD of our complete dataset, as stated in the legend. As evolutionary biologists, our focus is fitness (reproductive success) and then you basically need to reach adulthood and produce many children. If you look at the survival curves (Fig S1), the major difference lies within the first two years of life. As we argue in our paper, the reduced lifespan is solely due to a high infant mortality.

So it appears that they are only using the lower section of the data shown in Figure 1, the under-21 data. OK, fair enough, that’s legit. There is a remaining problem, however. This is that nature is naturally clumpy. As the poet had it:

Shake and shake

The ketchup bottle

None’ll come and

Then a lot’ll.

The deaths of infants and children in Norway are a great example of this clumpy nature of nature. Figure 1 shows that at the longer term, there were lots of epidemics and/or crop failures from 1750 to 1800. Look at all of the slanted lines in the lower part of Figure 1 during that time. And then there were few mass mortality events for the next fifty years. But then from 1850 to 1900, the death rates jump again. Clumpy.

And the same is true on all time scales, from daily to annual to centennial. No deaths for a few years, or decades, and then a bunch. And this leads to a big problem when you want to relate and compare a clumped dataset to a highly cyclical phenomenon like say sunspots. The odds go way up that what looks significant is actually just random. When you are comparing a strongly cyclical signal like sunspots to a “clumpy” natural variable, it only takes a couple of clumps in the right spots to give the appearance of causality and correlation.

As a result of all of these considerations about just the dataset itself, I’d be very cautious about any interpretation of results from this dataset without a good hard look under the hood at whether they are significant. And with those caveats, let’s look at the under-21 lifespans. (I have omitted the final 20 years of the under-21 dataset, because we need at least 20 years of data to see the full lifespan distribution up to 20 years of age.)

Now the authors have divided the dataset into “MAX” and “MIN” based on solar activity during the birth year. I disapprove of this procedure for two reasons. First, I dislike it because it is arbitrary and subjective. For example, in this case the division ends up 70/30, with 70% of the individuals being counted in the “MIN” category, and 30% in the “MAX” category. But some other arbitrary threshold could reverse those numbers. So the binary “MAX/MIN” conversion is highly dependent on the threshold chosen, and with different thresholds you can get very different outcomes.

Second, it doesn’t make sense to me to replace a continuous variable (average annual sunspots) with a categorical variable, particularly a binary categorical variable (e.g. MAX/MIN). Seems to me like you’re throwing away valuable information when you do that. In particular you lose all dosage/response information. And when you are looking to establish causation, dosage/response is an important part of the evidence.

But that’s what they did, so let me do the same. Figure 3 shows two histograms of the lifespans of the under-21 part of the Norwegian dataset. The blue histogram is of the individuals listed as “MAX” for the sunspots during their birth year, and the hatched red histogram is of the “MIN” individuals. I’m using the authors’ MAX/MIN division as specified in the data they sent me.

Figure 3. Histograms, Norwegian under-21 lifespans during “minimum” and “maximum” solar activity. The solid blue columns show the individuals born during solar “maximum” (n = 502 individuals). The red hatched columns show the solar “minimum” individuals (n = 1,321 individuals).

Figure 3 shows what percentage of the individuals died at 0 years, 1 year, 2 years, and so on. For example, in both MAX and MIN groups about 38% of the individuals died before their first birthday.

In fact, there is very little difference at any age between the MAX and the MIN groups. I’m sorry, but I’m just not seeing any evidence. There’s no statistical difference between the survival rates based on the level of solar (sunspot) activity at birth.

The authors say that they have “controlled” for various other factors. According to their paper these factors are sex, maternal effects, socioeconomic status, cohort and ecology. I haven’t been given access to that data, so I can’t comment on their claim.

But when you start splitting things up into sub-sub-sub categories, you need to adjust your statistics to allow for that, and I can’t find any evidence that they’ve done so, or that they are even aware of the issue.

The problem is that if you look long enough you’ll find unusual things. Here’s an example. If a person takes six coins and throws them in the air at once, there’s only one chance in sixty-four that all six of them will fall heads up. That would be unusual. That result would be called “statistically significant”.

But if she picks all six coins back up and tosses them again, and does the same again, and again, sooner or later they’ll come up six heads. And when they do so, the person can’t claim at that point that six heads is statistically significant. It’s not. Six heads is unusual, but it’s only unusual if the coins are just tossed one time. It’s not unusual if the six coins are flipped twenty times. And each subdivision of your data, say first into male/female and then into max/min, is the same as flipping the coins twice as many times …

Note how far the subdivision has gone in their analysis. Above, I show all individuals divided into two groups by solar activity, MAX and MIN. The next step would be to split them up by sex, so then we have four groups (max and min male, max and min female). Divide them further into high and low socioeconomic status and we have eight groups. Then dividing by ecology gives us sixteen groups, and an additional division by cohort gives us thirty-two groups …

With 32 groups, are we surprised that we find things that look significant but aren’t? With thirty-two tosses of six coins, you have about a 40% chance of finding a toss that’s six heads … with that fine a division it’s not significant at all to find six heads.

As a result, I fear they have not established their claims. They haven’t allowed for the number of trials in their calculation of statistical significance.

Let me show one final look at just the under-21 data. Figure 4 shows a scatterplot of lifespan versus sunspot count for just the under-21 part of the Norwegian dataset. It’s the same as Figure 2, but for the under-21 individuals. This avoids the whole “MIN/MAX” problem, and relates the two variables directly:

Figure 4. Scatterplot of the lifespan of the individuals in the under-21 Norwegian dataset, versus the number of sunspots during the birth year. Red line is the trend line of lifespan with sunspots.

Well, there you have it. When the trend line is horizontal as in Figure 4, that means that there’s no relationship between under-21 lifespan and sunspots. Nor does this surprise me. If sunspots had that much effect on human gestation and survival, we’d see the same effects all over the map in all kinds of living organisms … but we don’t.

So are the findings of the authors statistically significant? I don’t know, because the authors haven’t provided me with the data to test their findings, and because the authors aren’t aware of the problems with their significance calculations.

But given my findings in Figures 4 & 5 regarding the full under-21 dataset, I greatly doubt the significance of their findings regarding sub-sub-subsets of that data. Even if we were to find a putative effect on say boys, who were high-status, from a particular cohort, born during the MIN, and raised in a certain ecology, it would have to be a very, very strong effect to be significant.

In closing, I have to give a mixed report regarding the authors. In their great favor is the fact that they were responsive to requests from myself and from others for data and code. Professor Røskaft in particular was most forthcoming, answering questions and providing further data. So all of the authors have my thanks for that.

However, at the end of the day I still don’t have their full dataset with all of the status and ecology and other information … it’s another example of why providing the data as used and the code as used is so important.

Best to all,

w.

My Usual Request: If you disagree with someone, please have the courtesy to quote their exact words that you disagree with. In that way, we can all understand the exact nature of what you object to.

Further Reading: After writing much of the above, I found that in some aspects I’d been anticipated by the irrepressible Matt Briggs, Statistician To The Stars.

Data: The Norwegian Data provided by Professor Røskaft is here as a text file.