I recently wrote three posts (first, second, and third), regarding climate sensitivity. I wanted to compare my results to another dataset. Continued digging has led me to the CERES monthly global albedo dataset from the Terra satellite. It’s an outstanding set, in that it contains downwelling solar (shortwave) radiation (DSR), upwelling solar radiation (USR), and most importantly for my purposes, upwelling longwave radiation (ULR). Upwelling solar radiation (USR) is the solar energy that is reflected by the earth rather than entering the climate system. It is in 1°x1° gridded format, so that each month’s data has almost 200,000 individual measurements, or over 64,000 measurements for each of those three separate phenomena. Unfortunately, it’s only just under five years of data, but there is lots of it and it is internally consistent. As climate datasets go, it is remarkable.
Now, my initial interest in the CERES dataset is in the response of the longwave radiation to the surface heating. I wanted to see what happens to the longwave coming up from the earth when the incoming energy is changing.
To do this, rather than look at the raw data, I need to look at the month-to-month change in the data. This is called the “first difference” of the data. It is the monthly change in the item of interest, with the “change” indicated by the Greek letter delta ( ∆ ).
When I look at a new dataset like this one, I want to see the big picture first. I’m a graphic artist, and I grasp the data graphically. So my first step was to graph the change in upwelling longwave radiation (∆ULR) against the change in net solar radiation (∆NSR). The net solar radiation (NSR) is downwelling solar minus upwelling solar (DSR – USR). It is the amount of solar energy that is actually entering the climate system.
Figure 1 shows the changes in longwave that accompany changes in net solar radiation.
Figure 1. Scatterplot of the change in upwelling longwave radiation (∆ ULR, vertical scale) with regards to the change in net solar radiation entering the system. Dotted line shows the linear trend. Colors indicate latitude, with red being the South Pole, yellow is the Equator, and blue is the North Pole. Data covers 90° N/S.
This illustrates why I use color in my graphs. I first did this scatterplot without the color, in black and white. I could see there was underlying structure, and I guessed it had to do with latitude, but I couldn’t tell if my guess were true. With the added color, it is easy to see that in the tropics the increase in upwelling longwave for a given change in solar energy is greater than at the poles. So my next move was to calculate the trend for each 1° band of latitude. Figure 2 shows that result, with colors indicating latitude to match with Figure 1.
Figure 2. Linear trend by latitude of the change in upwelling longwave with respect to a 1 W/m2 change in net solar radiation. “Net downwelling” is downwelling solar radiation DSR minus upwelling solar radiation USR. Colors are by latitude to match Figure 1. Values are area-adjusted, with the Equatorial values having an adjustment factor of 1.0.
Now, this is a very interesting result. Bear in mind that the sun is what is driving these changes. The way that I read this is that near the Equator, whenever the sun is stronger there is an increase in thunderstorms. The deep upwelling caused by the thunderstorms is moving huge amounts of energy through the core of the thunderstorms, slipping it past the majority of the CO2, to the upper atmosphere where it is much freer to radiate to space. This is one of the mechanisms that I discussed in my post “The Thermostat Hypothesis“. Note in Figure 2 that at the peak, which occurs in the Intertropical Convergence Zone (ITCZ) just north of the Equator, this upwelling radiation counteracts a full 60% of the incoming solar energy, and this is on average. This means that the peak response must be even larger.
Finally, I took a look at what I’d started out to investigate, which was the relationship between incoming energy and the surface temperature. I may be mistaken, but I think that this is the first observational analysis of the relationship between the actual top-of-atmosphere (TOA) imbalance (downwelling minus upwelling radiation, or DLR – USR -ULR) and the corresponding change in temperature.
As before, I have used a lagged calculation, to emulate the slow thermal response of the planet. This model has two variables, the climate sensitivity “lambda” and the time constant “tau”. The climate sensitivity is how much the temperature changes for a given change in TOA forcing. The time constant “tau” is a measure of how long it takes the system to adjust to a certain level.
Figure 3 shows the new results in graphic form:
Figure 3. Upper panel shows the Northern Hemisphere (NH) and Southern Hemisphere (SH) temperatures, and the calculation of those temperatures using the top of atmosphere (TOA) imbalance (downwelling – upwelling). Bottom panel shows the residuals from that calculation for the two hemispheres.
In my previous analysis, I calculated that climate sensitivity and the time constant for the Northern Hemisphere and the Southern Hemisphere were slightly different. Here are my previous results:
SH NH lambda 0.05 0.10°C per W/m2 tau 2.4 1.9 months RMS residual error 0.17 0.26 °C
Using this entirely new dataset, and including the upwelling longwave to give the full TOA imbalance, I now get the following results:
SH NH lambda 0.05 0.13°C per W/m2 tau 2.5 2.2 months RMS residual error 0.18 0.17 °C
(Due to the short length of the data, there is no statistically significant trend in either the actual or calculated datasets.)
These are very encouraging results, because they are very close to my prior calculations, despite using an entirely different albedo dataset. This indicates that we are looking at a real phenomenon, rather than the first result being specific to a certain dataset.
Now, is it possible that there is a second much longer time constant at work in the system? In theory, yes, but a couple of things militate against it. First, I have found no way to add a longer time constant to make it a “two-box” model without the sensitivity being only about a tenth of that shown above, and believe me, I’ve tried a host of possible ways. If someone can do it, more power to you, please show me how.
Second, I looked at what is happening when we remove the monthly average values (climatology) from both the TOA variations and the temperatures. Once I remove the monthly average values from both datasets, there is no relationship between the two remaining datasets, lagged or not.
However, absence of evidence is not evidence of absence, meaning that there may well be a second, longer time constant with a larger sensitivity going on in the system. However, before you claim that such a constant exists, please do the work to come up with a way to calculate such a constant (and associated sensitivity), and show us the actual results. It’s easy to say “There must be a longer time delay”, but I haven’t found any way to include one that works mathematically. I can put in a longer time constant, but it ends up with a sensitivity for the second time lag of only about a tenth of what I calculate for a single-box model … which doesn’t help.
All the best, and if you disagree with something I’ve written, please QUOTE MY WORDS that you disagree with. That way we can avoid misunderstandings.
w.
DATA: The Excel worksheet containing the hemispheric monthly averages and my calculations is here. The 1° x 1° gridded data is here as an R “save” file. WARNING: 70 Mbyte file!. The R data is contained in four 180 row x 360 column by 58 layer arrays. They start at 89.5N and -179.5W, with the first month being January 2001. There is an array for the albedo, for the upwelling and downwelling solar, and for the upwelling longwave. In addition, there are four corresponding 180 row x 360 column by 57 layer arrays, which contain the first differences of the actual data.
Philip Bradley says:
June 12, 2012 at 6:29 pm
Not really. What I’ve shown is that I personally haven’t been able to find a way to do it that leads to a combination of large sensitivity and longer time constant … but as events have shown me more than once, whether I can do it != whether it can be done.
w.
P. Solar says:
June 12, 2012 at 6:54 pm
OK, my full code is here, but you may regret asking. The code, far from being user-friendly, is overtly user aggressive. In addition, it is NOT designed to be run as a single block of code.
First you need to run the “load” statement to bring in the data, which is assumed to be in your workspace. Then you can start to play. Various chunks of code do various things. I have 8 GBytes of memory in my machine, these blocks of data are large.
Best of luck,
w.
[UPDATE: Well, we’re all out of luck … upon examination, I find that the computer crash I had earlier today wiped out my copy of the program. The R code I posted will open up the data, but that’s all, it’s a previously saved and very early version … GRRRR!. Sorry, P. Solar, and sorry for me too, now I have to start over. I have updated the data file (CERES_albedo_data.tab), and I’m working on rebuilding the program. -w.]
Sorry about my rant earlier about the non Microsoft browser message popping up at that NASA site. You can follow my rant at the following link where I post as Proton2:
http://channel9.msdn.com/Forums/Coffeehouse/Why-is-NASA-scolding-me-for-using-Microsoft/
The solution NASA should have used, in the words of Charles on the MSDN forum is:
“@Proton2: Well, given the error message, the problem has to do more with their browser detection strategy than it does their scientists or some anarchistic philo$ophy.
Send the site admins a note informing them that the way in which they are determining browser version is too limiting. Best to test for modern browser capabilities, not parse user agent strings. Further, tests for browser version are better served with the greater than approach versus the equality one.
This is what the spirits tell me.
C
“
Ed_B says:
June 12, 2012 at 8:12 pm
Good question, Ed_B. It depends on what you are looking for. If you are just looking for the raw slope, you don’t area adjust. If you want the results to be proportional to total watts affected by a given slope, you do. I’ll likely re-do it so it shows total watts (same shape, just different units).
w.
Leonard Lane says:
June 12, 2012 at 10:44 pm
I pound nails for a living, and I do science on my breaks and in the evening and the night, I’m a night owl. During the day, I let the ideas roll around in my head while I’m building, I don’t really work on them, just sort of play with them and let them knock up against each other.
w.
Carpentry is a noble trade.
P. Solar, after my crash mentioned above, here’s the reconstructed code for Figure 1. You’ll need to reload the data, it now contains the net downwelling solar radiation (nsrarray) along with the first difference of that array (nsrdiff). Given those, the code for Figure 1 is:
# ====================================================Figure 1 par(mgp=c(2.2,1,0)) # set the location of the labels color.palette = colorRampPalette(c("blue", "yellow", "red")) # make color palette mycolors=color.palette(180) # break the palette into 180 sections to match latitudes colorlist=array(rep(row(ulrdiff[,,1]),58),c(180,360,58)) # get the latitudes in the form 1:180 plot(ulrdiff~nsrdiff,pch=".",cex=.3,col=mycolors[colorlist]) # plot the resultNote that this program needs to plot some 3,693,600 data points, so it takes a few minutes to produce the graph.
w.
Willis, you may find Roy Spencer’s simple model interesting (in fact it’s not “his” model as he points out) , I mean his work on such a model. It’s done in Excel , so you’ll love it 😉
http://www.drroyspencer.com/research-articles/
Depth of ocean involved is a key factor.
Willis:
perhaps you could post the function definition to keep things in sync.
BTW, ignore the warning about slope fitting at your peril. On data with a wide spread like the tropical parts in yellow , it will be quite significant. This is not a knit-picking purist detail. I would not be surprised to see factor of two difference between the two estimates depending upon which way round you plot it.
Clearly the choice of axis can not affect the physics, so you have to look at the two extremes and see what that suggests about where the correct slope may lie.
I find the human eye can quite often see that either OLS fit is not “right” .
I once sat through a 2h meeting of all the PhD’s of an entire maths department in a UK university (about 15 egg heads) while they discussed why just such a scatter plot was producing an obviously wrong slope. Not one of them was aware of this issue !
The next day, I quietly approached the PhD student who was in the final stages of preparing her thesis and handed her two pages of maths showing the derivation of OLS and how it depended upon having negligible errors in x and thus why her OLS slope was so obviously wrong.
She was rather taken aback then said it was too late to redo all here work since she was close to submitting her thesis.
She added a paragraph of hand waving excuses and the need for more study, avoided mentioning the true reason and got herself a PhD in mathematics.
The incident taught me a lot about academia and the value of published science.
Much of Dessler’s criticism of Spencer’s recent papers comes from _exactly_ this error. His climate sensitivity is too high because he does OLS fits to scatter plots. These guys command budgets of millions of dollars but can’t even fit a straight line correctly. Sad but true.
I’m sure you are made of better stuff.
And here is my reconstructed code for Figure 2:
# ======================================================== Figure 2 radians=function(x) x/360*(2*pi) # function to convert degrees to radians lat1=seq(89.5,-89.5,-1) # latitude by row, cosinelat=cos(radians(lat1)) # cosine for weighting answervec=vector(length=180) # vector to hold answers for (i in 1:180) { answervec[i]=summary(lm(as.vector(ulrdiff[i,,])~as.vector(nsrdiff[i,,])))$coefficients[2,1] } # take slope of linear model plot(0~0,xlim=c(-90,90),xaxp=c(-90,90,18),ylim=c(0,1),col="white",xlab="Latitude",ylab="Slope") #setup plot lines(answervec~lat1,type="p",pch=20,col=mycolors) # unweighted lines(answervec*cosinelat~lat1,type="p",pch=21,bg=mycolors,cex=1.5) # weightedw.
Jim Cripwell asked:
“Are there, in fact, two different types of forcing, one where the lapse rate changes, and one where it does not change? And if so, what is the difference between the two types of forcing?”
I think that internal redistribution of energy does change the lapse rate whereas changes to the total system energy content do not change the lapse rate.The atmospheric volume changes in both cases.
Thus as regards CO2 I think that the tendency to absorb more energy is offset by the air circulation response for no gain in system energy content but instead a small change in the air circulation and the lapse rate with it.
As regards a change in solar input at TOA or an increase in atmospheric mass then the total system energy content does change and the lapse rate does not change.
P. Solar says:
June 13, 2012 at 1:22 am
Here you go …
Function averageif(r, s, t) averageif = Application.SumIf(r, s, t) / Application.CountIf(r, s) End Function Function trendse(r) As Double Dim x, c, tn As Double tn = truen(r) c = r.Cells.Count trendse = Application.Index(Application.LinEst(r, , , True), 2, 1) * Sqr(c - 1) / Sqr(tn - 1) End Function Function truen(r) 'calculates the "true" N adjusted for autocorrelation Dim cc, ac As Double cc = r.Cells.Count ac = acdet(r) truen = cc * (1 - ac - 0.68 / Sqr(cc)) / (1 + ac + 0.68 / Sqr(cc)) End Function Function acdet(r) 'detrends and takes the lag 1 autocorrelation Dim cc, m, b, n, d, sumxy, olddata, sumsqr As Double cc = r.Cells.Count m = Application.Index(Application.LinEst(r, , , True), 1, 1) b = Application.Index(Application.LinEst(r, , , True), 1, 2) For n = 1 To cc olddata = d d = r.Cells(n) - n * m - b sumsqr = sumsqr + d ^ 2 If n > 1 Then sumxy = sumxy + d * olddata End If Next acdet = sumxy / sumsqr End FunctionThe trendse function uses a number of samples “N” which is adjusted for autocorrelation using the method of Nychka.
w.
“the longer term sensitivity can not be more than 10% of the shorter term sensitivity”
On a per annum basis maybe.
But how about a large, slow secondary climate response as against a small fast initial climate response.
You could have a multicentennial process going on in the background at the rate of only one tenth of the short term process each year but building up to a much larger effect over the full multicentennial term.
That would produce the observed upward temperature stepping at approximately 60 year intervals too wouldn’t it ?
P. Solar says:
June 12, 2012 at 3:57 pm
I don’t follow this at all. Let me take figure 1 as an example. The dotted line shows the slope of ∆ longwave (vertical axis) with respect to ∆ net shortwave (horizontal axis).
I agree that there are errors in both the x and y axes … but I don’t see why that’s an issue. As long as the error is roughly symmetrical, with nearly four million data points in each of the two datasets that error comes out in the wash.
This is quite bizarre. Invert the axes to get boundaries on a slope? One is the slope of X with respect to Y, and the other (reversed axes) is the slope of Y with respect to X. They are very different things, and the slope of one is simply 1/slope(the other). In no way are those two the “bounding extremes” for one of the slopes.
Near as I can tell, you have just made this up, but in any case it’s like nothing I’ve ever heard.
w.
Dear Myrrh,
Please put your hand in front of a working blue laser for a few minutes, and then tell me that shortwave cannot heat. That’s absolutely ridiculous.
P. Solar says:
June 12, 2012 at 6:24 pm
Myrrh, before you go on too much, please go and work out what happens to all the energy in the rest of the spectrum once it get past the ocean surface. Assuming it does not come out in Australia, there is the principal of conservation of energy that may give you a clue.
I’m sorry, but I’ve never understood this repost. What has that got to do with the actual direct heat from the Sun missing from the equation? When you’re standing in the Sun and feeling the heat it is longwave, thermal infrared, you’re feeling. It may well be true that this is only 1% of the Light and Heat energy we get from the Sun at the surface, I find that difficult to think correct, but the 99% Light energy doesn’t directly heat the surface, which is the claim here, “shortwave in longwave out”. So that immediately strikes as BS conservation of energy or not.
Unlike the majority rest of you here I’m not a scientist and I don’t have the time to familiarise myself with the language of maths which you’re all so comfortable with, so I read these posts with interest for the concepts through simple arithmetic. That, I find illogical in these scenarios. Shortwave Light will not give Willis the heat necessary for his thunder clouds, for example, so what figures was he relating to this? But in his post here, is the upwelling thermal being measured a true relation to the shortwave coming in, because, as any thermal camera shows, the secondary heating from photosynthesis, the production of heat energy in life systems, is very small in its outgoing as actual thermal longwave because life actually uses it for growth and survival rather than expending it by radiating it away – (a camera will pick up thermal heat energy radiated out from that being generated by say a man in a forest, it’s not very much and the trees around even less).
So, the majority then of the upwelling must be from that directly generated by the thermal infrared beam heating land and oceans, that’s a lot of energy from 1% (if that one percent is correct). I’m just interested is all here..
Please also bear in mind that while NASA has lots of really smart people , some of its sections have people that are seriously challenged with basic physics. No names, just saying.
It’s not the seriously challenged with basic physics that bother me, it’s those who aren’t but know exactly what they’re doing in manipulating data.
Before NASA and all the other once great science bodies began spouting AGW memes and producing doctored figures and explanations, and teaching these at universities, it was standard well-known physics that it was the direct heat from the Sun which heated up the Earth. Standard physics divided this into categories, Heat which is thermal infrared and did the heating of Earth and Light which is mainly visible which didn’t. The first was, and still is, fully part of the division category thermodynamics and the second went into optics and biology. And biology is not just photosynthesis, which will contribute to the thermal upwelling indirectly, from Life, but such things as uv in the production of vitamin D, for example which will also indirectly contribute to life’s release of heat in living. But, these arguments, and Willis’s considerable and interesting work, don’t take any of this into consideration, so it ultimately is without conceptual meaning for me as there are all these logical disjuncts with the figures. If no one else is bothered by this, fine, I’m just pointing it out because it bothers me.
But also, this classic division into Heat and Light has been lost, which means that older work which would refer to heat and mean thermal infrared and not shortwave can’t now be properly grasped. Also, radiant referred to heat not light in thermodynamics. This AGW produced meme of “shortwave in longwave out” has confused the history of these sciences, besides being physically incorrect because shortwave direct from the Sun does not physically heat the Earth, it’s incapable of physically moving atoms and molecules into vibrational states which is what it takes to heat something up. It cannot heat up the land and oceans to give the great energy of the weather system, for example. It’s only the direct heat from the Sun which can do this and it’s only this which can account for the majority of the thermal upwelling being measured.
The long wave IR radiated from earth is not all necessarily reflected from the surface. Some will be from the reduction of energy states caused by the interaction with the GHG’s. Multiple interactions will lower the energy enough to get into the LW bracket.
P. Solar
I have to agree entirely with w. here. Even your point about minimal error in the independant seems a little forced – as error should, if unbiased, reduces “correlation” not increase it. For a start, in statistics you start with the assumption that any relationship is casual not causal. But I think that in a system (such as this) where we know the input and likely output, without having to state the bleeding obvious, it is fair to assume that with some degree you’re assuming the independant is controlling the output. The argument being made here is by how much and can we quantify it? Autocorrelation within each set makes it difficult to leave it just at that, but cross correlation between the pair should show that they come into phase at a given lag – but this is blindingly obvious from the plots. So we can continue with our assumption of causality – it looks good.
Finally, I’ve always been advised that when defining relationships between variables, and defining causal relationships In particular, that you take the function of XvsY and YvsX in order to remove the issue of drift (data non-stationary with repsect to one another). In short, you cannot play around with the notion of independant and dependant as if they’re an intrinsic property of the subjects you’re acquiring data on.
Willis,
Interesting data and analysis – good presentation – congratulations!
But this“The way that I read this is that near the Equator, whenever the sun is stronger there is an increase in thunderstorms. The deep upwelling caused by the thunderstorms is moving huge amounts of energy through the core of the thunderstorms, slipping it past the majority of the CO2, to the upper atmosphere where it is much freer to radiate to space. “
I’d interpret differently. Higher nett DSR is more likely to mean lower albedo than higher insolation (which doesn’t vary much). And that probably means less cloud, not thunderstorms.
And the related higher ULR could well be that with less clouds, more of the ULR (in frequency band) is able to come directly from the surface via the open atmospheric window. It’s coming from the warmest source.
Sorry last point should be:
Finally, I’ve always been advised that when defining relationships between variables, and defining causal relationships in particular, that you take the average function of XvsY and YvsX in order to remove the issue of drift (data non-stationary about each of the regression lines).
Sorry P. Solar and w.
I see your point now P.:
w. wrote:
“They are very different things, and the slope of one is simply 1/slope(the other)”
That’s not the case if your applying regression to scatter data:
s1 = cov(X,Y)/var(X)
s2 = cov(X,Y)/var(Y)
And unless you have a prefect fit:
s1 != s2
So P you’re right on your last point. Although it is a round-about way of saying it.
My thanks to Charlie and Chris. I am totally unconvinced. I have read all I can about the hypothetical ways that the proponents of CAGW have used to try and convert change of forcings into change of surface temperatures. So far as I can see, there is very little empirical data to support the idea that this process can be performed with any sort of reliability. I simply do not believe the numbers that are suppoosed to show that as we add more CO2 to the atmosphere, surface temperatures are going to rise catastrophically.
‘
What little empirical data we have, all tends to show that adding CO2 to the atmosphere causes a negligible rise in temperature. This work by Willis is simply further confirmation that no-one needs to worry about rising temperatures as a result of the burning of fossil fuels.
Willis Eschenbach: “Near as I can tell, you have just made this up, but in any case it’s like nothing I’ve ever heard.”
Google “total least squares.”
Nylo says:
June 13, 2012 at 2:27 am
Dear Myrrh,
Please put your hand in front of a working blue laser for a few minutes, and then tell me that shortwave cannot heat. That’s absolutely ridiculous.
==========
What’s ridiculous is the mindless unthinking regurgitation of AGWScience Fiction memes from their meme producing department. The Sun is not a laser, duh.
But maybe you can’t see it because you’ve been blinded by all that blue light in the sky?
Which, by the way, is visible light refracted/reflected by being bounced around by electrons of the molecules of nitrogen and oxygen – electronic transition level which is the electrons absorbing visible light.
Thank you Willis, absolutely elegant.