I recently wrote three posts (first, second, and third), regarding climate sensitivity. I wanted to compare my results to another dataset. Continued digging has led me to the CERES monthly global albedo dataset from the Terra satellite. It’s an outstanding set, in that it contains downwelling solar (shortwave) radiation (DSR), upwelling solar radiation (USR), and most importantly for my purposes, upwelling longwave radiation (ULR). Upwelling solar radiation (USR) is the solar energy that is reflected by the earth rather than entering the climate system. It is in 1°x1° gridded format, so that each month’s data has almost 200,000 individual measurements, or over 64,000 measurements for each of those three separate phenomena. Unfortunately, it’s only just under five years of data, but there is lots of it and it is internally consistent. As climate datasets go, it is remarkable.
Now, my initial interest in the CERES dataset is in the response of the longwave radiation to the surface heating. I wanted to see what happens to the longwave coming up from the earth when the incoming energy is changing.
To do this, rather than look at the raw data, I need to look at the month-to-month change in the data. This is called the “first difference” of the data. It is the monthly change in the item of interest, with the “change” indicated by the Greek letter delta ( ∆ ).
When I look at a new dataset like this one, I want to see the big picture first. I’m a graphic artist, and I grasp the data graphically. So my first step was to graph the change in upwelling longwave radiation (∆ULR) against the change in net solar radiation (∆NSR). The net solar radiation (NSR) is downwelling solar minus upwelling solar (DSR – USR). It is the amount of solar energy that is actually entering the climate system.
Figure 1 shows the changes in longwave that accompany changes in net solar radiation.
Figure 1. Scatterplot of the change in upwelling longwave radiation (∆ ULR, vertical scale) with regards to the change in net solar radiation entering the system. Dotted line shows the linear trend. Colors indicate latitude, with red being the South Pole, yellow is the Equator, and blue is the North Pole. Data covers 90° N/S.
This illustrates why I use color in my graphs. I first did this scatterplot without the color, in black and white. I could see there was underlying structure, and I guessed it had to do with latitude, but I couldn’t tell if my guess were true. With the added color, it is easy to see that in the tropics the increase in upwelling longwave for a given change in solar energy is greater than at the poles. So my next move was to calculate the trend for each 1° band of latitude. Figure 2 shows that result, with colors indicating latitude to match with Figure 1.
Figure 2. Linear trend by latitude of the change in upwelling longwave with respect to a 1 W/m2 change in net solar radiation. “Net downwelling” is downwelling solar radiation DSR minus upwelling solar radiation USR. Colors are by latitude to match Figure 1. Values are area-adjusted, with the Equatorial values having an adjustment factor of 1.0.
Now, this is a very interesting result. Bear in mind that the sun is what is driving these changes. The way that I read this is that near the Equator, whenever the sun is stronger there is an increase in thunderstorms. The deep upwelling caused by the thunderstorms is moving huge amounts of energy through the core of the thunderstorms, slipping it past the majority of the CO2, to the upper atmosphere where it is much freer to radiate to space. This is one of the mechanisms that I discussed in my post “The Thermostat Hypothesis“. Note in Figure 2 that at the peak, which occurs in the Intertropical Convergence Zone (ITCZ) just north of the Equator, this upwelling radiation counteracts a full 60% of the incoming solar energy, and this is on average. This means that the peak response must be even larger.
Finally, I took a look at what I’d started out to investigate, which was the relationship between incoming energy and the surface temperature. I may be mistaken, but I think that this is the first observational analysis of the relationship between the actual top-of-atmosphere (TOA) imbalance (downwelling minus upwelling radiation, or DLR – USR -ULR) and the corresponding change in temperature.
As before, I have used a lagged calculation, to emulate the slow thermal response of the planet. This model has two variables, the climate sensitivity “lambda” and the time constant “tau”. The climate sensitivity is how much the temperature changes for a given change in TOA forcing. The time constant “tau” is a measure of how long it takes the system to adjust to a certain level.
Figure 3 shows the new results in graphic form:
Figure 3. Upper panel shows the Northern Hemisphere (NH) and Southern Hemisphere (SH) temperatures, and the calculation of those temperatures using the top of atmosphere (TOA) imbalance (downwelling – upwelling). Bottom panel shows the residuals from that calculation for the two hemispheres.
In my previous analysis, I calculated that climate sensitivity and the time constant for the Northern Hemisphere and the Southern Hemisphere were slightly different. Here are my previous results:
SH NH lambda 0.05 0.10°C per W/m2 tau 2.4 1.9 months RMS residual error 0.17 0.26 °C
Using this entirely new dataset, and including the upwelling longwave to give the full TOA imbalance, I now get the following results:
SH NH lambda 0.05 0.13°C per W/m2 tau 2.5 2.2 months RMS residual error 0.18 0.17 °C
(Due to the short length of the data, there is no statistically significant trend in either the actual or calculated datasets.)
These are very encouraging results, because they are very close to my prior calculations, despite using an entirely different albedo dataset. This indicates that we are looking at a real phenomenon, rather than the first result being specific to a certain dataset.
Now, is it possible that there is a second much longer time constant at work in the system? In theory, yes, but a couple of things militate against it. First, I have found no way to add a longer time constant to make it a “two-box” model without the sensitivity being only about a tenth of that shown above, and believe me, I’ve tried a host of possible ways. If someone can do it, more power to you, please show me how.
Second, I looked at what is happening when we remove the monthly average values (climatology) from both the TOA variations and the temperatures. Once I remove the monthly average values from both datasets, there is no relationship between the two remaining datasets, lagged or not.
However, absence of evidence is not evidence of absence, meaning that there may well be a second, longer time constant with a larger sensitivity going on in the system. However, before you claim that such a constant exists, please do the work to come up with a way to calculate such a constant (and associated sensitivity), and show us the actual results. It’s easy to say “There must be a longer time delay”, but I haven’t found any way to include one that works mathematically. I can put in a longer time constant, but it ends up with a sensitivity for the second time lag of only about a tenth of what I calculate for a single-box model … which doesn’t help.
All the best, and if you disagree with something I’ve written, please QUOTE MY WORDS that you disagree with. That way we can avoid misunderstandings.
w.
DATA: The Excel worksheet containing the hemispheric monthly averages and my calculations is here. The 1° x 1° gridded data is here as an R “save” file. WARNING: 70 Mbyte file!. The R data is contained in four 180 row x 360 column by 58 layer arrays. They start at 89.5N and -179.5W, with the first month being January 2001. There is an array for the albedo, for the upwelling and downwelling solar, and for the upwelling longwave. In addition, there are four corresponding 180 row x 360 column by 57 layer arrays, which contain the first differences of the actual data.
Willis, have you considered using something like DropBox to back up your files? It functions transparently as a normal folder on your machine, but will immediately back everything up to the cloud upon saving. It would avoid you having to go through this pain again.
[RESPONSE: I use Time Machine on my Mac … but it, like DropBox, can only back up what I’ve remembered to actually save. As is often the case, the problem is pilot error … -w.]
“but a couple of things mitigate against it.”
Typo. Make that “militate.”
[Thanks, fixed. -w.]
The graph is interesting – but why is the “change in net sun” greater for the north and south poles and there is relatively little change in the net solar energy received by equator?
One would expect that the equator which receives more than 6 times the radiation of the poles to have a larger absolute change in net solar radiation.
Myrrh,
What’s ridiculous is the mindless unthinking regurgitation of AGWScience Fiction memes from their meme producing department. The Sun is not a laser, duh.
You claim that shortwave light from the sun cannot heat. A blue laser’s light is solely shortwave light, and it heats pretty well. No matter how many baseless and ill-informed insults per second you can write, you cannot change facts. Any electromagnetic wave, of any wavelength (including visible light), can only pass by, be reflected or be absorbed by any substance it encounters, and if absorbed, it becomes heat. Now keep trolling.
Joe Born says:
June 13, 2012 at 4:25 am
No way, that’s a fool’s errand. Give a citation to your claim or go home, I’m not going to chase something only to have you say Sorry, that’s not what I was talking about.
w.
cd_uk says:
June 13, 2012 at 3:27 am
Huh? I just tried it on real data. In this case I’m using the Excel function LINEST to compute the slope of the linear trend. If I calculate
then I find, as I said above, that
So I fear I’m not clear on your claim.
w.
Nick Stokes says:
June 13, 2012 at 3:17 am
Many thanks, Nick, your contribution is always valuable.
Regarding the spike near the equator shown in Figure 2, you say
Actually, since I’ve posted the data and all, before speculating you could take a look at something like this:
Note that Figure 2 relates the month-to-month change in the variables, not the absolute values.
All the best,
w.
Willis Eschenbach: “No way, that’s a fool’s errand. Give a citation to your claim or go home, I’m not going to chase something only to have you say Sorry, that’s not what I was talking about.”
Well, I’m no stats expert, and I confess to being even shakier in that discipline than in others that arise at this site, but I believe I’ve had exactly the same experience as P. Solar: I’ve bounded the slope by first using ordinary-least-squares analysis of one variable against the other to arrive at one slope and then taking the reciprocal of what I got by the same approach with the variables reversed–and the two slopes thus obtained were significantly different in the problem (whose specifics I confess to being unable to bring to mind just now) I applied this to. So what P. Solar said rang true to me. And cd_uk gave what appears to be the explanation.
In any event, the Wikipedia entry for “ordinary least squares”, to which I had thought my suggestion above would directly lead you, deals with an approach different from thus bounding the best slope estimate, but it gives what to this layman is a helpful explanation of the problem P. Solar identified. I particularly commend its second diagram to your attention.
My high opinion of your research skills had led me to believe that locating the Wikipedia entry for that phrase would not overtax them. I’m grieved that you felt imposed upon as a result.
Willis, re your computer crashes and code loss, might I make a suggestion.
Try using a public code hosting service, that way your code is safe, you can
store multiple versions, and it’s easy to point people to your code, and just as
easy for them to utilise it, and even give back to you by fixing errors, or just
making it less “user aggressive”.
There are many such services, as long as your code is open source. For a nominal amount you can have a private and public repository. (It’s cheating a bit, but you can even archive your data)
I can recommend bitbucket for ease of use and flexibility, but github is the big one.
Here is a link to useful info if you do consider it.
http://en.wikipedia.org/wiki/Comparison_of_open_source_software_hosting_facilities
I hope I’m not pointing out the bleeding obvious here 🙂
Joe Born says:
June 13, 2012 at 9:33 am
Joe, if you wanted to refer me to the Wikipedia least squares entry, you just had to say so. Your insults about my Google-fu are nasty and uncalled for. I’ve played that game before, someone says “Just Google X” with the sub-text of “you idiot”. Then when I do and I find something, they say “Sorry, that’s not at all what I was talking about”. So I’ve stopped playing that game, there’s no cheese at the end of the maze. If you have something you want me to look at, just tell me.
Moving on to your point, I looked at the Wikipedia article, I looked at the second diagram, and I found … well … nothing that clears up P. Solar’s point. Look, Joe, TRY P. SOLAR’S CLAIM OUT ON SOME ACTUAL DATA. Everyone always wants to get all theoretical. Give us an example with some real data of exactly what you (or P. Solar) are talking about. That’s what I did, and it confirmed what I said above. The slope of the linear regression of Y on X is simply the reciprocal of the slope of the linear regression of X on Y. It does not bound the error in the first linear regression. So I fear that behind your insults, I find nothing at all.
w.
Oscar Bajner says:
June 13, 2012 at 10:00 am
Oscar, my problem was simple. I got so engrossed in doing the work I forgot to save it … and there’s no code hosting service of any kind that can fix that. As we used to say, the problem is the nut that holds on the steering wheel …
w.
@ur momisugly Willis Eschenbach June 12, 3:01 pm
re: Stephen Rasey June 12 2:09 pm (adjust the X-axis, not the Y data)
I blundered. You were using (i think) the cos(latitude) to adjust the Y axis of Figure 2.
My suggestion was to apply the cos(latitude) to the X-axis, but conceptually, you want to make the size of the bin the function of the cos(latitude). To do that, you plot your unadjusted Y-axis slopes by the sin(latitude) from -1 to +1 for latitudes 90 S to 90 N. When you integrate to get the width of each bin, the sin() integrates to cos().
Dolphinhead says:
June 12, 2012 at 3:05 pm
Anthony these posts from Willis are excellent. Already comments from Robert Brown and Chris Colose. You should keep them as a sticky at the top of the home page for a few days
_____________________________
MAy I suggest a file of Willis “thermostat” posts as you have the “climategate” files or as a a secondary file under references?
P. Solar says:
June 12, 2012 at 3:57 pm
Excellent follow on from your earlier articles , Willis. This is building into something more solid.
One word of warning with fitting “linear trends”. Any and all methods of linear regression that you are going to be based totally on an assumption that there is minimal error in the independent variable…..
_________________________________________
I am no mathematician but if I recall correctly from my first year calculus class, you can use a straight line to approximate a curve if the distance between the two points on the curve is short.
At this point we know we have a 30 year half cycle of the ~ 60 year ocean cycles and that these cycles effect the weather. Is a five year time period short enough to be considered a “straight line” as an approximation?
Myrrh says:
June 13, 2012 at 3:05 am
When you’re standing in the Sun and feeling the heat it is longwave, thermal infrared, you’re feeling.
Myrrh, I recently watched an episode of “The Universe” and a lady astrophysicist explained just what you said to the viewing public. So you’ve got at least one person on your side about what it is we feel as warmth from the sun.
“””””…..Gail Combs says:
June 13, 2012 at 11:25 am
…………………
I am no mathematician but if I recall correctly from my first year calculus class, you can use a straight line to approximate a curve if the distance between the two points on the curve is short…..”””””
Should have stayed awake Gail, or not passed notes around to the other “Girls”; you can’t approximate a curve with a straight line. As pointed out to me (personally) by a very nice and also very smart Canadian Nobel Physics Prize winner (one of the real, non-ersatz ones), any portion of a circle, no matter how small, has the same finite curvature; and NO straight line however short, has any curvature exceeding |+/- zero|.
We were chatting over a beer, not 20 feet from where I am sitting typing this; but he was quite sober.
“””””…..Nylo says:
June 13, 2012 at 6:49 am
Myrrh,
What’s ridiculous is the mindless unthinking regurgitation of AGWScience Fiction memes from their meme producing department. The Sun is not a laser, duh.
You claim that shortwave light from the sun cannot heat. A blue laser’s light is solely shortwave light, and it heats pretty well. No matter how many baseless and ill-informed insults per second you can write, you cannot change facts. Any electromagnetic wave, of any wavelength (including visible light), can only pass by, be reflected or be absorbed by any substance it encounters, and if absorbed, it becomes heat. Now keep trolling……”””””
A big waste of effort Nylo, Myrrh is the last living specimen of that species that believes we get “heat” (and even “light”) from the sun.
We don’t get either one of course; we make ALL of the heat right here on earth by wasting the sun’s electromagnetic radiation energy mostly in the ocean, instead of collecting it with PV solar cells, and turning it into electricity.
And the “light” of course, we make right in our eyeballs and brain, out of a small sliver of the EM radiation spectrum energy; we even give it a completely new set of units; Lumens and the like, as distinct from Watts and Joules. I’m not even sure we ever bothered to name the “light” equivalent of Joules.
Willis Eschenbach: “The slope of the linear regression of Y on X is simply the reciprocal of the slope of the linear regression of X on Y. It does not bound the error in the first linear regression. So I fear that behind your insults, I find nothing at all.”
Gee, you guys at this site constantly throw around statistics jargon I find hard to follow, so maybe I’m seeing something that’s not there. If so, I’m sorry I wasted your time.
But here’s what I thought I found in the pairs that follow: y = 0.36x + 4.0 and y = 1.5x – 0.5.
1.19421945 4.84266446
1.169281461 5.03766177
2.294703511 5.534849024
1.115321035 4.484476829
2.667609819 3.594026393
1.532614844 3.12160353
2.77529536 3.006381135
2.557049838 4.825775731
2.785710156 6.564167154
2.817737262 3.447464588
2.080464611 3.765876239
4.111601729 3.527566694
4.057893282 5.655552923
2.698540812 6.494516826
3.101522315 4.98722445
3.831531789 4.679177019
3.421675179 5.200428647
3.610477106 6.589079457
4.243898375 7.308141653
4.708463447 5.707350745
5.978687507 5.422677067
5.963371443 7.151352875
5.928046423 7.329831452
4.850960707 5.624223684
5.354640614 7.919432681
5.104124164 6.793141389
7.137547588 4.651621071
7.225532347 5.826935514
7.369925687 4.974631855
7.195873601 7.716728484
I take it you instead get identical estimates independently of whether you regress the right against the left or the left against the right?
Myrrh says:
June 13, 2012 at 4:57 am
Nylo says:
June 13, 2012 at 2:27 am
Dear Myrrh,
Please put your hand in front of a working blue laser for a few minutes, and then tell me that shortwave cannot heat. That’s absolutely ridiculous.
==========
What’s ridiculous is the mindless unthinking regurgitation of AGWScience Fiction memes from their meme producing department. The Sun is not a laser, duh.
But maybe you can’t see it because you’ve been blinded by all that blue light in the sky?
Which, by the way, is visible light refracted/reflected by being bounced around by electrons of the molecules of nitrogen and oxygen – electronic transition level which is the electrons absorbing visible light.
What is truly ridiculous Myrrh is your continuing to regurgitate the same rubbish when the correct physics has been explained to you many times! By the way your ‘explanation’ of Rayleigh scattering is also wrong, there is no ‘electronic transition’ involved and no ‘electrons absorbing visible light’.
Willis:
Your graph show peak thermal emission from earth at 10 microns. In most graphs this is shown at ca 20 microns. Where did you get this graph?
Light beyond the visible to 2 microns is known as the near IR and there is a lot energy in these photons.
In a real dark room, turn on electric stove, put your hand above element and watch it. Note the amount of heat being emitted before you can just barely see very faint red glow.
Greenhouse gases don’t absorb in the near IR.
When you have a one-box model applied to a cyclical forcing, then the maximum lag you can have is tau. You can test this with the formula: lag = (period * atan(2pi*tau/period))/2pi. So with your averaged tau = 2.35 months, I thought I would test it against the lags for ENSO and the 11 year solar TSI cycle. With a two-box model, I’d expect the lags to be longer than predicted with a one-box model.
When I do a lag analysis of HadCrut3 vs NINO3.4 (both detrended), I get a statistical dead-heat between a 2 or 3 month lag. Given a reasonable error range, this is *not inconsistent* with your tau where I’d expect the lag to be a touch under 2.35 months.
As for the 11 year solar cycle, I actually get the temperature forcing TSI. This just tells me it’s hard to calculate the lag of a relatively low amplitude signal in a noisy environment, but again it is *not inconsistent* with your tau. Grant and Rahmstorf used a 1 month lag (again *not inconsistent*):
http://iopscience.iop.org/1748-9326/6/4/044022/fulltext/
Then we could look at the 100,000 glacial cycle. Any detectable lag would refute your one-box model. According to Roe in his “In defence of Milankovitch” paper, the best fit is with a zero lag (again *not inconsistent*):
http://earthweb.ess.washington.edu/roe/GerardWeb/Publications_files/Roe_Milankovitch_GRL06.pdf
If someone were a “my glass is half-full” type of guy, they’d probably say your one-box model *is consistent* with the observational cyclical evidence.
Willis: The slope of the linear regression of Y on X is simply the reciprocal of the slope of the linear regression of X on Y.
that’s seldom true (that is, the probability of it being true is 0.) If you perform the linear regression of y on x and call the slope estimate b(Y|X), and do the regression of x on y and call the slope estimate b(X|Y), then b(Y|X) =/= 1/b(X|Y).
You can look this up on Wikipedia under the topic “Deming Regression”. I have found Wikipedia entries on statistical topics to be quite good. You can also look it up on Mathematica.
The inverse function of the estimated linear regression does not equal the inverse of the regression estimate taken in the other order (X vs Y VS Y vs X) is the short form, because (to put it briefly) the inverse function is nonlinear.
P. Solar: Any and all methods of linear regression that you are going to be based totally on an assumption that there is minimal error in the independent variable…..
NOT ALL. There are “errors in variables” regression, Deming regression as I mentioned, canonical variables and principal components.
Willis, thank you for the code segments. My skill in R is poor, and I am building some libraries of code to learn from.
“””””…..Harold Pierce Jr says:
June 13, 2012 at 6:01 pm
Willis:
Your graph show peak thermal emission from earth at 10 microns. In most graphs this is shown at ca 20 microns. Where did you get this graph?
Light beyond the visible to 2 microns is known as the near IR and there is a lot energy in these photons.
In a real dark room, turn on electric stove, put your hand above element and watch it. Note the amount of heat being emitted before you can just barely see very faint red glow.
Greenhouse gases don’t absorb in the near IR……”””””
Check again. The spectral peak for Black Body radiation for a source at 288K, the purported average earth surface temperature (about 59 deg F) is around 10.1 microns, when the spectral radiant emittance is plotted against WAVELENGTH, in Watts per m^2 per micron.
Some people plot BB radiation graphs on a wave number (frequency) basis, so they plot Watts per m^2 per wave number against wave number. They do that deliberately, because it puts the spectral peak nearer the CO2 absorption frequency, corresponding to around 15 microns.
But you see that per wavenumber plots exagerate low frequency effects, while per wavelength plots exagerate high frequency effects. Frequency is not one of the SI units, whereas Metres is, so I don’t pay any attention to wave number graphs; frequency is all in our minds; it isn’t real like length. Yes I know E=h (nu).
And you are wrong on GHGs absorbing in the near IR. H2O absorbs prodigiously in the near IR from around 0.7 microns in fact. It has absorption bands around 0.7, 0.77, 0.85,0.9, 1.13,1.3-1.5, 1.75-1.95, all in your “near IR” range. CO2 also has absorption bands around 1.4 microns, and 1.9, then H2O is strongly absorptive, from 2.25 to 3.2 microns, and has its highest absorption at 3.0 microns.
So BOTH H2O and CO2 contribute to global cooling, by absorbing significant portions of the incoming solar radiant energy from the sun, so it never reaches the deep oceans to get stored.
More H2O or more CO2 equals less solar energy capture by planet earth, so a cooler earth.
Data on atmospheric absorption for over 100,000 spectral lines has been compiled by the AFCRL, and can be obtained from NOAA.