Guest post by Nick Stokes,
Every now and then, in climate blogging, one hears a refrain that the traditional min/max daily temperature can’t be used because it “violates Nyquist”. In particular, an engineer, William Ward, writes occasionally of this at WUWT; the latest is here, with an earlier version here. But there is more to it.
Naturally, the more samples you can get, the better. But there is a finite cost to sampling limitation; not a sudden failure because of “violation”. And when the data is being used to compile monthly averages, the notion promoted by William Ward that many samples per hour are needed, that cost is actually very small. Willis Eschenbach, in comments to that Nyquist post, showed that for several USCRN stations, there was little difference to even a daily average whether samples were every hour or every five minutes.
The underlying criticism is of the prevailing method of assessing temperature at locations by a combined average of Tmax and Tmin = (Tmax+Tmin)/2. I’ll call that the min/max method. That of course involves just two samples a day, but it actually isn’t a frequency sampling of the kind envisaged by Nyquist. The sampling isn’t periodic; in fact we don’t know exactly what times the readings correspond to. But more importantly, the samples are determined by value, which gives them a different kind of validity. Climate scientists didn’t invent the idea of summarising the day by the temperature range; it has been done for centuries, aided by the min/max thermometer. It has been the staple of newspaper and television reporting.
So in a way, fussing about regular sample rates of a few per day is theoretical only. The way it was done for centuries of records is not periodic sampling, and for modern technology, much greater sample rates are easily achieved. But there is some interesting theory.
In this post, I’d like to first talk about the notion of aliasing that underlies the Nyquist theory, and show how it could affect a monthly average. This is mainly an interaction of sub-daily periodicity with the diurnal cycle. Then I’ll follow Willis in seeing what the practical effect of limited sampling is for the Redding CA USCRN station. There isn’t much until you get down to just a few samples per day. But then I’d like to follow an idea for improvement, based on a study of that diurnal cycle. It involves the general idea of using anomalies (from the diurnal cycle) and is a good and verifiable demonstration of their utility. It also demonstrates that the “violation of Nyquist” is not irreparable.
Here is a linked table of contents:
- Aliasing and Nyquist
- USCRN Redding and monthly averaging
- Using anomalies to gain accuracy
- Conclusion
Aliasing and Nyquist
Various stroboscopic effects are familiar – this wiki article gives examples. The math comes from this. If you have a sinusoid frequency f Hz (sin(2π)) samples at s Hz, the samples are sin(2πfn/s), n=0,1,2… But this is indistinguishable from sin(2π(fn/s+m*n)) for any integerm (positive or negative), because you can add a multiple of 2π to the argument of sin without changing its value.
But sin(2π(fn/s+m*n)) = sin(2π(f+m*s)n/s) that is, the samples representing the sine also representing a sine to which any multiple of the sampling frequency s has been added, and you can’t distinguish between them. These are the aliases. But if s is small, the aliases all have higher frequency, so you can pick out the lowest frequency as the one you want.
This, though, fails if f>s/2, because then subtracting s from f gives a lower frequency, so you can’t use frequency to pick out the one you want. This is where the term aliasing is more commonly used, and s=2*f is referred to as the Nyquist limit.
I’d like to illuminate this math with a more intuitive example. Suppose you observe a running track, circle circumference 400 m, from a height, through a series of snapshots (samples) 10 sec apart. There is a runner who appears as a dot. He appears to advance 80 m in each frame. So you might assume that he is running at a steady 8 m/s.
But he could also be covering 480m, running a lap+80 between shots. Or 880m, or even covering 320 m the other way. Of course, you’d favour the initial interpretation, as the alternatives would be faster than anyone can run.
But what if you sample every 20 s. Then you’d see him cover 160 m. Or 240 m the other way, which is not quite so implausible. Or sample every 30 s. Then he would seem to progress 240m, but if running the other way, would only cover 160m. If you favour the slower speed, that is the interpretation you’d make. That is the aliasing problem.
The critical case is sampling every 25s. Then every frame seems to take him 200m, or halfway around. It’s 8 m/s, but could be either way. That is the Nyquist frequency (0.04 Hz), relative to the frequency 0.02Hz which goes with as speed of 8 m/s. Sampling at double the frequency.
But there is one other critical frequency – that 0.2 Hz, or sampling every 50s. Then the runner would appear not to move. The same is true for multiples of 50s.
Here is a diagram in which I show some paths consistent with the sampled data, over just one sample interval. The basic 8 m/s is shown in black, the next highest forward speed in green, and the slowest path the other way in red. Starting point is at the triangles, ending at the dots. I have spread the paths for clarity; there is really only one start and end point.
All this speculation about aliasing only matters when you want to make some quantitative statement that depends on what he was doing between samples. You might, for example, want to calculate his long term average location. Now all those sampling regimes will give you the correct answer, track centre, except the last where sampling was at lap frequency.
Now coming back to our temperature problem, the reference to exact periodic processes (sinusoids or lapping) relates to a Fourier decomposition of the temperature series. And the quantitative step is the inferring of a monthly average, which can be regarded as long term relative to the dominant Fourier modes, which are harmonics of diurnal. So that is how aliasing contributes error. It comes when one of those harmonics matches the sample rate.
USCRN Redding and monthly averaging
Willis linked to this NOAA site (still working) as a source of USCRN 5 minute AWS temperature data. Following him, I downloaded data for Redding, California. I took just the years 2010 to present, since the files are large (13Mb per station per year) and I thought the earlier years might have more missing data. Those years were mostly gap-free, except for the last half of 2018, which I generally discarded.
Here is a table for the months of May. The rows are for sampling frequencies of 288, 24, 12, 4, 2, and 1 per day. The first row shows the actual mean temperature averaged 288 times per day over the month. The other rows show the discrepancy for the lower rate of sampling, for each year.
| Per hour | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 |
| 1/12 | 13.611 | 14.143 | 18.099 | 18.59 | 19.195 | 18.076 | 17.734 | 19.18 | 18.676 |
| 1 | -0.012 | 0.007 | -0.02 | -0.002 | -0.021 | -0.014 | -0.007 | 0.002 | 0.005 |
| 2 | -0.004 | 0.013 | -0.05 | -0.024 | -0.032 | -0.013 | -0.037 | 0.011 | -0.035 |
| 6 | -0.111 | -0.03 | -0.195 | -0.225 | -0.161 | -0.279 | -0.141 | -0.183 | -0.146 |
| 12 | 0.762 | 0.794 | 0.749 | 0.772 | 0.842 | 0.758 | 0.811 | 1.022 | 0.983 |
| 24 | -2.637 | -2.704 | -4.39 | -3.652 | -4.588 | -4.376 | -3.982 | -4.296 | -3.718 |
As Willis noted, the discrepancy for sampling every hour is small, suggesting that very high sample rates aren’t needed, even though they are said to “violate Nyquist”. But they get up towards a degree for sampling twice a day, and once a day is quite bad. I’ll show a plot:
The interesting thing to note is that the discrepancies are reasonably constant, year to year. This is true for all months. In the next section I’ll show how to calculate that constant, which comes from the common diurnal pattern.
Using anomalies to gain accuracy
I talk a lot about anomalies in averaging temperature globally. But there is a general principle that it uses. If you have a variable T that you are trying to average, or integrate, you can split it:
T = E + A
where E is some kind of expected value, and A is the difference (or residual, or anomaly). Now if you do the same linear operation on E and A, there is nothing gained. But it may be possible to do something more accurate on E. And A should be smaller, already reducing the error, but more importantly, it should be more homogeneous. So if the operation involves sampling, as averaging does, then getting the sample right is far less critical.
With global temperature average, E is the set of averages over a base period, and the treatment is to simply omit it, and use the anomaly average instead. For this monthly average task, however, E can actually be averaged. The right choice is some estimate of the diurnal cycle. What helps is that it is just one day of numbers (for each month), rather than a month. So it isn’t too bad to get 288 values for that day – ie use high resolution, while using lower resolution for the anomalies A, which are new data for each day.
But it isn’t that important to get E extremely accurate. The idea of subtracting E from T is to remove the daily cycle component that reacts most strongly with the sampling frequency. If you remove only most of it, that is still a big gain. My preference here is to use the first few harmonics of the Fourier series approximation of the daily cycle, worked out at hourly frequency. The range 0-4 day-1 can do it.
The point is that we know exactly what the averages of the harmonics should be. They are zero, except for the constant. And we also know what the sampled value should be. Again, it is zero, except where the frequency is a multiple of the sampling frequency, when it is just the initial value. This is just the Fourier series coefficient of the cos term.
Here is are the corresponding discrepancies of the May averages for different sampling rates, to compare with the table above. The numbers for 2 hour sampling have not changed. The reason is that the error there would have been in the 8th harmonic, and I only resolved the diurnal frequency up to 4.
| Per hour | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 |
| 1/12 | -0.012 | 0.007 | -0.02 | -0.002 | -0.021 | -0.014 | -0.007 | 0.002 | 0.005 |
| 2 | -0.004 | 0.013 | -0.05 | -0.024 | -0.032 | -0.013 | -0.037 | 0.011 | -0.035 |
| 6 | 0.014 | 0.095 | -0.07 | -0.1 | -0.036 | -0.154 | -0.016 | -0.058 | -0.021 |
| 12 | -0.062 | -0.029 | -0.075 | -0.051 | 0.019 | -0.066 | -0.012 | 0.199 | 0.16 |
| 24 | 1.088 | 1.021 | -0.665 | 0.073 | -0.864 | -0.651 | -0.258 | -0.571 | 0.007 |
And here is the comparison graph. It shows the uncorrected discrepancies with triangles, and the diurnally corrected with circles. I haven’t shown the one sample/day, because the scale required makes the other numbers hard to see. But you can see from the table that with only one sample/day, it is still accurate within a degree or so with diurnal correction. I have only shown May results, but other months are similar.
Conclusion
Sparse sampling (eg 2/day) does create aliasing to zero frequency, which does affect accuracy of monthly averaging. You could attribute this to Nyquist, although some would see it as just a poorly resolved integral. But the situation can be repaired without resort to high frequency sampling. The reason is that most of the error arises from trying to sample the repeated diurnal pattern. In this analysis I estimated that just from Fourier series of hourly readings from a set of base years. If you subtract a few harmonics of the diurnal, you get much improved accuracy for sparse sampling of each extra year, at the cost of just hourly sampling of a reference set.
Note that this is true for sampling at prescribed times. Min/max sampling is something else.
Bit disingenuous. We may well have been using min/max for many decades, but we haven’t been using min/max to show that we must fundamentally change our economy for all that time. If all this were just an academic exercise I couldn’t’t care less (except about the debasement of scicience) but it is not.
Let’s assume that lunar tides have some subtle effect on surface temperature via the ‘nearly’ diurnal cycle of expansion of atmosphere, or simply on the ocean currents velocity. Since the lunar cycles are not in sync with the earth’s rotating or orbital periodicity the effect would be aliasing on monthly, annual, multi decadadal and centenary time scales, following changes in intensity of the lunar tides across the time scales mentioned. Just using two simple periodicity of 27.3 and 365.25 days produces some of the familiar spectral components found in the global temperature anomaly.
Don’t forget the 19 year Metonic cycle used by the Babylonians.
All this discussion on whether or not the sampling method is adequate or not can be resolved by running simulations. You can create a dataset that represents a true continual temperature trend over the course of 10 years . You can calculate the value for the “actual” average temperature over different periods of time, and compare that to the values you get from various sampling methods.
I suspect that anyone who does such an experiment will find that developing a metric by taking 365 sets of readings in a year, recording the Tmin and TMax (rounded to the nearest whole numbers), and dividing by two, is adequate to determine the actual average annual temperature and that it’s good enough to be able to discern trends in the underlying data.
I’m somewhat frustrated by the whole min/max thing anyway. Living in E. Texas I get to experience “exceptional” weather if not weekly, certainly monthly. In the Winter we have cold fronts which stall just to the south, over the Gulf, and having brought cooler weather, then reverse, become warm fronts, switch the winds again and warm us up. Not infrequently the fronts are fast moving and drop the temperatures 30+ F/13 C in less than an hour. Now we get the min/max temperature reading whenever, but almost assuredly not on some arbitrary data input time. The old min/max thermometer can tell you what, but not when, and two samples a day aren’t accurate either unless they just happened to fall at the right time. Since most two sample protocols have the times defined, one is almost guaranteed to have the wrong values recorded.
One might say that in some areas the temperature curve is an arbitrary, continuous shape. To map the shape half way accurately one needs more than 24 samples a day, but sampling every second is probably overkill.
In any case, whether in UHI, or in the boonies, you live the weather that is there, not the weather that’s recorded.
Why would that be “wrong”. What are you aiming to achieve and in what way would a protocol with specific times be “correct”?
There is no specific time of day for minimum temperature. If you specify a time the result could vary considerably. In what way would a fixed time reading be “right”?
Averaging min/max temps makes no sense. What if it is cloudy from 6am till 1pm(cooler), then cloudless till 2pm,(warmer), then cloudy again till 6am the next day . Your max would be from 1 hour of warmer and your min from 23 hours of cooler ?
On average, the result would be average.
On the vast majority of days, one never encounters an average high or average low temperature. Averages are a construct of man, and not nature, they mean nothing to Gaia. Averages are a fantasy.
In any case, temperature measuring can not prove and separate out the difference between natural warming and man made. Our politicians have simply assumed that any temperature increase has been due to AGW. They have been aided and abetted by the devious climate scientists that have refused to use the Null hypothesis testing in any of their reports. Thayer Watkins has said that 85 % of all LWIR back radiation is due to clouds. NASA says 50%. I have shown that NASA must be wrong on this point and I have demonstrated the maximum effect of CO2 on temperature increase, assuming that we have even had a temperature increase in last 68 years.
http://applet-magic.com/cloudblanket.htm
Clouds overwhelm the Downward Infrared Radiation (DWIR) produced by CO2. At night with and without clouds, the temperature difference can be as much as 11C. The amount of warming provided by DWIR from CO2 is negligible but is a real quantity. We give this as the average amount of DWIR due to CO2 and H2O or some other cause of the DWIR. Now we can convert it to a temperature increase and call this Tcdiox.The pyrgeometers assume emission coeff of 1 for CO2. CO2 is NOT a blackbody. Clouds contribute 85% of the DWIR. GHG’s contribute 15%. See the analysis in link. The IR that hits clouds does not get absorbed. Instead it gets reflected. When IR gets absorbed by GHG’s it gets reemitted either on its own or via collisions with N2 and O2. In both cases, the emitted IR is weaker than the absorbed IR. Don’t forget that the IR from reradiated CO2 is emitted in all directions. Therefore a little less than 50% of the absorbed IR by the CO2 gets reemitted downward to the earth surface. Since CO2 is not transitory like clouds or water vapour, it remains well mixed at all times. Therefore since the earth is always giving off IR (probably a maximum at 5 pm everyday), the so called greenhouse effect (not really but the term is always used) is always present and there will always be some backward downward IR from the atmosphere.
When there isn’t clouds, there is still DWIR which causes a slight warming. We have an indication of what this is because of the measured temperature increase of 0.65 from 1950 to 2018. This slight warming is for reasons other than just clouds, therefore it is happening all the time. Therefore in a particular night that has the maximum effect , you have 11 C + Tcdiox. We can put a number to Tcdiox. It may change over the years as CO2 increases in the atmosphere. At the present time with 409 ppm CO2, the global temperature is now 0.65 C higher than it was in 1950, the year when mankind started to put significant amounts of CO2 into the air. So at a maximum Tcdiox = 0.65C. We don’t know the exact cause of Tcdiox whether it is all H2O caused or both H2O and CO2 or the sun or something else but we do know the rate of warming. This analysis will assume that CO2 and H2O are the only possible causes. That assumption will pacify the alarmists because they say there is no other cause worth mentioning. They like to forget about water vapour but in any average local temperature calculation you can’t forget about water vapour unless it is a desert. A proper calculation of the mean physical temperature of a spherical body requires an explicit integration of the Stefan-Boltzmann equation over the entire planet surface. This means first taking the 4th root of the absorbed solar flux at every point on the planet and then doing the same thing for the outgoing flux at Top of atmosphere from each of these points that you measured from the solar side and subtract each point flux and then turn each point result into a temperature field by integrating over the whole earth and then average the resulting temperature field across the entire globe. This gets around the Holder inequality problem when calculating temperatures from fluxes on a global spherical body. However in this analysis we are simply taking averages applied to one local situation because we are not after the exact effect of CO2 but only its maximum effect. In any case Tcdiox represents the real temperature increase over last 68 years. You have to add Tcdiox to the overall temp difference of 11 to get the maximum temperature difference of clouds, H2O and CO2 . So the maximum effect of any temperature changes caused by clouds, water vapour, or CO2 on a cloudy night is 11.65C. We will ignore methane and any other GHG except water vapour.
So from the above URL link clouds represent 85% of the total temperature effect , so clouds have a maximum temperature effect of .85 * 11.65 C = 9.90 C. That leaves 1.75 C for the water vapour and CO2. This is split up with 60% for water vapour and 26% for CO2 with the remaining % for methane, ozone ….etc. See the study by Ahilleas Maurellis and Jonathan Tennyson May 2003 in Physics World. Amazingly this is the only study that quantifies the Global warming potential of H20 before any feedback effects. CO2 will have relatively more of an effect in deserts than it will in wet areas but still can never go beyond this 1.75 C . Since the desert areas are 33% of 30% (land vs oceans) = 10% of earth’s surface , then the CO2 has a maximum effect of 10% of 1.75 + 90% of Twet. We define Twet as the CO2 temperature effect of over all the world’s oceans and the non desert areas of land. There is an argument for less IR being radiated from the world’s oceans than from land but we will ignore that for the purpose of maximizing the effect of CO2 to keep the alarmists happy for now. So CO2 has a maximum effect of 0.175 C + (.9 * Twet). So all we have to do is calculate Twet.
Reflected IR from clouds is not weaker. Water vapour is in the air and in clouds. Even without clouds, water vapour is in the air. No one knows the ratio of the amount of water vapour that has now condensed to water/ice in the clouds compared to the total amount of water vapour/H2O in the atmosphere but the ratio can’t be very large. Even though clouds cover on average 60 % of the lower layers of the troposhere, since the troposphere is approximately 8.14 x 10^18 m^3 in volume, the total cloud volume in relation must be small. Certainly not more than 5%. H2O is a GHG. So of the original 15% contribution by GHG’s of the DWIR, we have .15 x .26 =0.039 or 3.9% to account for CO2. Now we have to apply an adjustment factor to account for the fact that some water vapour at any one time is condensed into the clouds. So add 5% onto the 0.039 and we get 0.041 or 4.1 % . CO2 therefore contributes 4.1 % of the DWIR in non deserts. We will neglect the fact that the IR emitted downward from the CO2 is a little weaker than the IR that is reflected by the clouds. Since, as in the above, a cloudy night can make the temperature 11C warmer than a clear sky night, CO2 or Twet contributes a maximum of 0.041 * 1.75 C = 0.07 C.
Therfore Since Twet = 0.07 C we have in the above equation CO2 max effect = 0.175 C + (.9 * 0.07 C ) = ~ 0.238 C. As I said before; this will increase as the level of CO2 increases, but we have had 68 years of heavy fossil fuel burning and this is the absolute maximum of the effect of CO2 on global temperature.
So how would any average global temperature increase by 7C or even 2C, if the maximum temperature warming effect of CO2 today from DWIR is only 0.238 C? This means that the effect of clouds = 85%, the effect of water vapour = 13 % and the effect of CO2 = 2 %. Sure, if we quadruple the CO2 in the air which at the present rate of increase would take 278 years, we would increase the effect of CO2 (if it is a linear effect) to 4 X 0.238 C = 0.952 C .
If the cloud effect was 0 for DWIR, the maximum that CO2 could be is 10%(desert) * 0.65 + (90% of Twet2) = 0.065 C + (90% *twet2)
twet2 = .26( See the study by Ahilleas Maurellis and Jonathan Tennyson May 2003 in Physics World.) * 0.585 C (difference between 0.65 and the amount of temperature effect for CO2 for desert) = 0.1521 C therefore Max CO2 = 0.065 C + (0.1521 * .9) = 0.2 C ((which is about 84% of above figure of 0.238 C. The 0.2 C was calculated by assuming as above that on average H20 is 60% of greenhouse effect and CO2 is 26% of GHG effect and that the whole change of 0.65 C from 1950 to 2018 is because of either CO2 or water vapour. We are disregarding methane and ozone. So in effect, the above analysis regarding clouds gave too much maximum effect to CO2. The reason is that you simply take the temperature change from 1950 to 2018 disregarding clouds, since the water vapour has 60% of the greenhouse effect and CO2 has 26%. If you integrate the absorption flux across the IR spectrum despite the fact that there are 25 times more molecules than CO2 by volume, you get 60% for H20 and 26% for CO2 as their GHG effects. See the study by Ahilleas Maurellis and Jonathan Tennyson May 2003 in Physics World. CO2 can never have as much effect as H20 until we get to 2.3x the amount of CO2 in the atmosphere than there is now.
NASA says clouds have only a 50% effect on DWIR. So let us do that analysis.
So according to NASA clouds have a maximum temperature effect of .5 * 11.65 C = 5.825 C. That leaves 5.825 C for the water vapour and CO2. This is split up with 60% for water vapour and 26% for CO2 with the remaining % for methane, ozone ….etc. As per the above. Again since the desert areas are 33% of 30% (land vs oceans) = 10% of earth’s surface , then the CO2 has a maximum effect of (10% of 5.825 C) + 90% of TwetNASA. We define TwetNASA as the CO2 temperature effect of over all the world’s oceans and the non desert areas of land. So CO2 has a maximum effect of 0.5825 C + (.9 * TwetNASA). So all we have to do is calculate TwetNASA.
Since as before we give the total cloud volume in relation to the whole atmosphere as not more than 5%. H2O is a GHG. So of the original 50% contribution by GHG’s of the DWIR, we have .5 x .26 =0.13 or 13 % to account for CO2. Now we have to apply an adjustment factor to account for the fact that some water vapour at any one time is condensed into the clouds. So add 5% onto the 0.13 and we get 0.1365 or 13.65 % . CO2 therefore contributes 13.65 % of the DWIR in non deserts. As before, we will neglect the fact that the IR emitted downward from the CO2 is a little weaker than the IR that is reflected by the clouds.
Since, as in the above, a cloudy night can make the temperature 11C warmer than a clear sky night, CO2 or TwetNASA contributes a maximum of 0.1365 * 5.825 C = ~0.795 C.
Therfore Since TwetNASA = 0.795 C we have in the above equation CO2 max effect = 0.5825 C + (.9 * 0.795 C ) = ~ 1.3 C. Now this is double the amount of actual global warming in the last 68 years, so since CO2 would not have more of an effect on a cloudy night versus a noncloudy night, the maximum effect could not be greater than the effect calculated, above, when not considering clouds. So clearly, NASA cannot be correct.
I fail to understand how climate scientists could get away with saying that water vapour doesnt matter because it is transitory. In fact the alarmist theory needs a positive forcing of water vapour to achieve CAGW heat effects. Since there is widespread disagreement on any increase in H2O in the atmosphere in the last 68 years, there hasn’t been any positive forcing so far. Therefore; the hypothesis is; that main stream climate science theory of net CO2 increases in the atmosphere has major or catastrophic consequences for heating the atmosphere and the null hypothesis says it doesn’t have major or catastrophic consequences for heating the atmosphere. Therefore we must conclude that we cannot reject the null hypothesis that main stream climate science theory of net CO2 increases in the atmosphere does not have major or catastrophic consequences for heating the atmosphere. In fact the evidence and the physics of the atmosphere shows that if we rejected the null hypothesis, we would be rejecting most of radiative atmospheric physics as we know it. So in the end, the IPCC conclusion of mankind increasing net CO2 into the atmosphere, causing major or catastrophic warming of the atmosphere; is junk science.
Please disregard the above post and substitute the following analysis of the maximum temperature effect of clouds and CO2.
http://applet-magic.com/cloudblanket.htm
Clouds overwhelm the Downward Infrared Radiation (DWIR) produced by CO2. At night with and without clouds, the temperature difference can be as much as 11C. The amount of warming provided by DWIR from CO2 is negligible but is a real quantity. We give this as the average amount of DWIR due to CO2 and H2O or some other cause of the DWIR. Now we can convert it to a temperature increase and call this Tcdiox.The pyrgeometers assume emission coeff of 1 for CO2. CO2 is NOT a blackbody. Clouds contribute 85% of the DWIR. GHG’s contribute 15%. See the analysis in link. The IR that hits clouds does not get absorbed. Instead it gets reflected. When IR gets absorbed by GHG’s it gets reemitted either on its own or via collisions with N2 and O2. In both cases, the emitted IR is weaker than the absorbed IR. Don’t forget that the IR from reradiated CO2 is emitted in all directions. Therefore a little less than 50% of the absorbed IR by the CO2 gets reemitted downward to the earth surface. Since CO2 is not transitory like clouds or water vapour, it remains well mixed at all times. Therefore since the earth is always giving off IR (probably a maximum at 5 pm everyday), the so called greenhouse effect (not really but the term is always used) is always present and there will always be some backward downward IR from the atmosphere.
When there isn’t clouds, there is still DWIR which causes a slight warming. We have an indication of what this is because of the measured temperature increase of 0.65 from 1950 to 2018. This slight warming is for reasons other than just clouds, therefore it is happening all the time. Therefore in a particular night that has the maximum effect , you have 11 C + Tcdiox. We can put a number to Tcdiox. It may change over the years as CO2 increases in the atmosphere. At the present time with 411 ppm CO2, the global temperature is now 0.65 C higher than it was in 1950, the year when mankind started to put significant amounts of CO2 into the air. So at a maximum Tcdiox = 0.65C. We don’t know the exact cause of Tcdiox whether it is all H2O caused or both H2O and CO2 or the sun or something else but we do know the rate of warming. This analysis will assume that CO2 and H2O are the only possible causes. That assumption will pacify the alarmists because they say there is no other cause worth mentioning. They like to forget about water vapour but in any average local temperature calculation you can’t forget about water vapour unless it is a desert. A proper calculation of the mean physical temperature of a spherical body requires an explicit integration of the Stefan-Boltzmann equation over the entire planet surface. This means first taking the 4th root of the absorbed solar flux at every point on the planet and then doing the same thing for the outgoing flux at Top of atmosphere from each of these points that you measured from the solar side and subtract each point flux and then turn each point result into a temperature field by integrating over the whole earth and then average the resulting temperature field across the entire globe. This gets around the Holder inequality problem when calculating temperatures from fluxes on a global spherical body. However in this analysis we are simply taking averages applied to one local situation because we are not after the exact effect of CO2 but only its maximum effect. In any case Tcdiox represents the real temperature increase over last 68 years. You have to add Tcdiox to the overall temp difference of 11 to get the maximum temperature difference of clouds, H2O and CO2 . So the maximum effect of any temperature changes caused by clouds, water vapour, or CO2 on a cloudy night is 11.65C. We will ignore methane and any other GHG except water vapour.
So from the above URL link clouds represent 85% of the total temperature effect , so clouds have a maximum temperature effect of .85 * 11.65 C = 9.90 C. That leaves 1.75 C for the water vapour and CO2. This is split up with 60% for water vapour and 26% for CO2 with the remaining % for methane, ozone ….etc. See the study by Ahilleas Maurellis and Jonathan Tennyson May 2003 in Physics World. Amazingly this is the only study that quantifies the Global warming potential of H20 before any feedback effects. CO2 will have relatively more of an effect in deserts than it will in wet areas but still can never go beyond this 1.75 C . Since the desert areas are 33% of 30% (land vs oceans) = 10% of earth’s surface , then the CO2 has a maximum effect of 10% of 1.75 + 90% of Twet. We define Twet as the CO2 temperature effect of over all the world’s oceans and the non desert areas of land. There is an argument for less IR being radiated from the world’s oceans than from land but we will ignore that for the purpose of maximizing the effect of CO2 to keep the alarmists happy for now. So CO2 has a maximum effect of 0.175 C + (.9 * Twet). So all we have to do is calculate Twet.
Reflected IR from clouds is not weaker. Water vapour is in the air and in clouds. Even without clouds, water vapour is in the air. No one knows the ratio of the amount of water vapour that has now condensed to water/ice in the clouds compared to the total amount of water vapour/H2O in the atmosphere but the ratio can’t be very large. Even though clouds cover on average 60 % of the lower layers of the troposhere, since the troposphere is approximately 8.14 x 10^18 m^3 in volume, the total cloud volume in relation must be small. Certainly not more than 5%. H2O is a GHG. So of the original 15% contribution by GHG’s of the DWIR, we have .15 x .26 =0.039 or 3.9% to account for CO2. Now we have to apply an adjustment factor to account for the fact that some water vapour at any one time is condensed into the clouds. So add 5% onto the 0.039 and we get 0.041 or 4.1 % . CO2 therefore contributes 4.1 % of the DWIR in non deserts. We will neglect the fact that the IR emitted downward from the CO2 is a little weaker than the IR that is reflected by the clouds. Since, as in the above, a cloudy night can make the temperature 11C warmer than a clear sky night, CO2 or Twet contributes a maximum of 0.041 * 1.75 C = 0.07 C.
Therfore Since Twet = 0.07 C we have in the above equation CO2 max effect = 0.175 C + (.9 * 0.07 C ) = ~ 0.238 C. As I said before; this will increase as the level of CO2 increases, but we have had 68 years of heavy fossil fuel burning and this is the absolute maximum of the effect of CO2 on global temperature.
So how would any average global temperature increase by 7C or even 2C, if the maximum temperature warming effect of CO2 today from DWIR is only 0.238 C? This means that the effect of clouds = 85%, the effect of water vapour = 13 % and the effect of CO2 = 2 %. Sure, if we quadruple the CO2 in the air which at the present rate of increase would take 278 years, we would increase the effect of CO2 (if it is a linear effect) to 4 X 0.238 C = 0.952 C .
If the cloud effect was 0 for DWIR, the maximum that CO2 could be is 10%(desert) * 0.65 + (90% of Twet2) = 0.065 C + (90% *twet2)
twet2 = .26( See the study by Ahilleas Maurellis and Jonathan Tennyson May 2003 in Physics World.) * 0.585 C (difference between 0.65 and the amount of temperature effect for CO2 for desert) = 0.1521 C therefore Max CO2 = 0.065 C + (0.1521 * .9) = 0.2 C ((which is about 84% of above figure of 0.238 C. The 0.2 C was calculated by assuming as above that on average H20 is 60% of greenhouse effect and CO2 is 26% of GHG effect and that the whole change of 0.65 C from 1950 to 2018 is because of either CO2 or water vapour. We are disregarding methane and ozone. So in effect, the above analysis regarding clouds gave too much maximum effect to CO2. The reason is that you simply take the temperature change from 1950 to 2018 disregarding clouds, since the water vapour has 60% of the greenhouse effect and CO2 has 26%. If you integrate the absorption flux across the IR spectrum despite the fact that there are 25 times more molecules than CO2 by volume, you get 60% for H20 and 26% for CO2 as their GHG effects. See the study by Ahilleas Maurellis and Jonathan Tennyson May 2003 in Physics World. CO2 can never have as much effect as H20 until we get to 2.3x the amount of CO2 in the atmosphere than there is now.
NASA says clouds have only a 50% effect on DWIR. So let us do that analysis.
So according to NASA clouds have a maximum temperature effect of .5 * 11.65 C = 5.825 C. That leaves 5.825 C for the water vapour and CO2. This is split up with 60% for water vapour and 26% for CO2 with the remaining % for methane, ozone ….etc. As per the above. Again since the desert areas are 33% of 30% (land vs oceans) = 10% of earth’s surface , then the CO2 has a maximum effect of (10% of 5.825 C) + 90% of TwetNASA. We define TwetNASA as the CO2 temperature effect of over all the world’s oceans and the non desert areas of land. So CO2 has a maximum effect of 0.5825 C + (.9 * TwetNASA). So all we have to do is calculate TwetNASA.
Since as before we give the total cloud volume in relation to the whole atmosphere as not more than 5%. H2O is a GHG. So of the original 50% contribution by GHG’s of the DWIR, we have .5 x .26 =0.13 or 13 % to account for CO2. Now we have to apply an adjustment factor to account for the fact that some water vapour at any one time is condensed into the clouds. So add 5% onto the 0.13 and we get 0.1365 or 13.65 % . CO2 therefore contributes 13.65 % of the DWIR in non deserts. As before, we will neglect the fact that the IR emitted downward from the CO2 is a little weaker than the IR that is reflected by the clouds.
Since, as in the above, a cloudy night can make the temperature 11C warmer than a clear sky night, CO2 or TwetNASA contributes a maximum of 0.1365 * 5.825 C = ~0.795 C.
Therfore Since TwetNASA = 0.795 C we have in the above equation CO2 max effect = 0.5825 C + (.9 * 0.795 C ) = ~ 1.3 C.
Now since the above analysis dealt with maximum effects let us divide the 11C maximum difference in temperature by 2 to get a rough estimate of the average effect of clouds. Therefore we have to do the calculations as per the above by substituting 5.5C wherever we see 11C.
So we have then 5.5C + 0.65C = 6.15C for the average effect of temperature difference with and without clouds.
So according to NASA clouds have a maximum temperature effect of .5 * 6.15 C = 3.075 C. That leaves 3.075 C for the water vapour and CO2. This is split up with 60% for water vapour and 26% for CO2 with the remaining % for methane, ozone ….etc. As per the above. Again since the desert areas are 33% of 30% (land vs oceans) = 10% of earth’s surface , then the CO2 has a maximum effect of (10% of 3.075 C) + 90% of TwetNASA. We define TwetNASA as the CO2 temperature effect of over all the world’s oceans and the non desert areas of land. So CO2 has a maximum effect of 0.3075 C + (.9 * TwetNASA). So all we have to do is calculate TwetNASA.
Since as before we give the total cloud volume in relation to the whole atmosphere as not more than 5%. H2O is a GHG. So of the original 50% contribution by GHG’s of the DWIR, we have .5 x .26 =0.13 or 13 % to account for CO2. Now we have to apply an adjustment factor to account for the fact that some water vapour at any one time is condensed into the clouds. So add 5% onto the 0.13 and we get 0.1365 or 13.65 % . CO2 therefore contributes 13.65 % of the DWIR in non deserts. As before, we will neglect the fact that the IR emitted downward from the CO2 is a little weaker than the IR that is reflected by the clouds.
Since, as in the above, a cloudy night can make the temperature 11C warmer than a clear sky night, CO2 or TwetNASA contributes a maximum of 0.1365 * 3.075 C = ~0.42 C.
Therfore Since TwetNASA = 0.42 C we have in the above equation CO2 max effect = 0.3075 C + (.9 * 0.42 C ) = ~ 0.6855C.
Since this temperature number is the complete temperature increase of the last 68 years, NASA is ignoring water vapour’s role which is 60% of the effect of GHGs. So clearly, NASA cannot be correct.
However let us redo the numbers with average temperature difference of 11/4 = 2.75 C between a cloudy and non cloudy day.
So we have then 2.75C + 0.65C = 3.4C for the average effect of temperature difference with and without clouds.
So according to NASA clouds have a maximum temperature effect of .5 * 3.4 C = 1.7 C. That leaves 1.7 C for the water vapour and CO2. This is split up with 60% for water vapour and 26% for CO2 with the remaining % for methane, ozone ….etc. As per the above. Again since the desert areas are 33% of 30% (land vs oceans) = 10% of earth’s surface , then the CO2 has a maximum effect of (10% of 1.7 C) + 90% of TwetNASA. We define TwetNASA as the CO2 temperature effect of over all the world’s oceans and the non desert areas of land. So CO2 has a maximum effect of 0.17 C + (.9 * TwetNASA). So all we have to do is calculate TwetNASA.
Since as before we give the total cloud volume in relation to the whole atmosphere as not more than 5%. H2O is a GHG. So of the original 50% contribution by GHG’s of the DWIR, we have .5 x .26 =0.13 or 13 % to account for CO2. Now we have to apply an adjustment factor to account for the fact that some water vapour at any one time is condensed into the clouds. So add 5% onto the 0.13 and we get 0.1365 or 13.65 % . CO2 therefore contributes 13.65 % of the DWIR in non deserts. As before, we will neglect the fact that the IR emitted downward from the CO2 is a little weaker than the IR that is reflected by the clouds.
Since, as in the above, a cloudy night can make the temperature 11C warmer than a clear sky night, CO2 or TwetNASA contributes a maximum of 0.1365 * 1.7 C = ~0.232 C.
Therfore Since TwetNASA = 0.232 C we have in the above equation CO2 max effect = 0.17 C + (.9 * 0.232 C ) = ~ 0.3788C.
As you can see the effect calculated for CO2 is still more than 50% of the actual temperature increase for the last 68 years. Clearly this is wrong since water vapour is 2.3 times the effect of CO2 as per the above study by Ahilleas Maurellis et al. So we must conclude that NASA is wrong and that the difference effect of temperature with and without clouds must be due mainly to clouds which makes intuitive sense. Thayer Watkins number must be closer to the truth than the number of NASA.
***************************************************************************************************
I fail to understand how climate scientists could get away with saying that water vapour doesnt matter because it is transitory. In fact the alarmist theory needs a positive forcing of water vapour to achieve CAGW heat effects. Since there is widespread disagreement on any increase in H2O in the atmosphere in the last 68 years, there hasn’t been any positive forcing so far. Therefore; the hypothesis is; that main stream climate science theory of net CO2 increases in the atmosphere has major or catastrophic consequences for heating the atmosphere and the null hypothesis says it doesn’t have major or catastrophic consequences for heating the atmosphere. Therefore we must conclude that we cannot reject the null hypothesis that main stream climate science theory of net CO2 increases in the atmosphere does not have major or catastrophic consequences for heating the atmosphere. In fact the evidence and the physics of the atmosphere shows that if we rejected the null hypothesis, we would be rejecting most of radiative atmospheric physics as we know it. So in the end, the IPCC conclusion of mankind increasing net CO2 into the atmosphere, causing major or catastrophic warming of the atmosphere; is junk science.
Alan
You said, “I fail to understand how climate scientists could get away with saying that water vapour doesnt matter because it is transitory.” Any given molecule may have a short residency; however, the WV is continually replenished in the source areas. In modern times, that means evaporation from reservoirs and irrigated fields, and water produced from the combustion of hydrocarbons. These are sources that didn’t exist before modern civilization.
“In modern times, that means evaporation from reservoirs and irrigated fields, and water produced from the combustion of hydrocarbons. These are sources that didn’t exist before modern civilization.”
And is negligible at the side of feedback from GHG warming …..
“Direct emission of water vapour by human activities makes a negligible contribution to radiative forcing.
However, as global mean temperatures increase, tropospheric water vapour concentrations increase and this represents a key feedback but not a forcing of climate change. Direct emission of water to the atmosphere by anthropogenic activities, mainly irrigation, is a possible forcing factor but corresponds to less than 1% of the natural sources of atmospheric water vapour. The direct injection of water vapour into the atmosphere from fossil fuel combustion is significantly lower than that from agricultural activity. {2.5}”
https://www.ipcc.ch/site/assets/uploads/2018/02/ar4-wg1-ts-1.pdf
Direct emission of water vapour by human activities makes a negligible contribution to radiative forcing.
…
Direct emission of water to the atmosphere by anthropogenic activities, mainly irrigation, is a possible forcing factor but corresponds to less than 1% of the natural sources of atmospheric water vapour.
Unfounded claim (1st). And misleading (2nd sentence) because irrelevant.
The question is not how large the extra WV is that is produced by man via such ways as irrigation, but whether the extra WV which is thereby put into the air is larger than the extra WV that is supposed to get into the air merely because of the minute warming caused by CO2.
That minute amount of extra WV due to CO2 direct warming is supposed to give the actual total greenhouse effect (x3 or x4 the original CO2 effect).
The extra WV put into the air via land use changes and irrigation is actually quite significant. Entire sea’s (Russia) and rivers (e.g. Colorado river) are used up for irrigation. The local climate in a large part of India has changed (notably) due to irrigation and also in Kansas (10% more WV).
Irrigation (and similar water use for agriculture) has increased by a very large amount since 1950 as that was needed to keep up with population growth,
Also irrigation has become significantly less wasteful than it was in the earlier part of the 20th century. No more sprinkling on top, but feeding low to the ground via tubes. The idea that this was better is fairly new in most regions, and only introduced in many places after the 1990’s… perhaps as late as this century. So while population and agriculture output kept growing, the use of water has not rising that much since about 2000…
Get it?
Anthony Banton
You apparently quote an IPCC source that claims “Direct emission of water vapour by human activities makes a negligible contribution to radiative forcing.” Yet, water vapor and CO2 are both produced by combustion. How is it that WV, which is supposedly more powerful than CO2 in impeding IR radiation is negligible when CO2 is important? After all, CO2 is produced primarily by combustion, while WV is produced by many other human activities as well!
Anecdotally, Phoenix (AZ) cooled off nicely at night in the 1950s and ‘swamp coolers’ were adequate for daytime cooling. Today, swamp coolers are inefficient. That is, since the city has built many golf courses, far more swimming pools than was the norm in the ’50s, installed cooling misters at bus stops, gas stations, and backyard patios, and increased the number of automobiles.
“Irrigation (and similar water use for agriculture) has increased by a very large amount since 1950 as that was needed to keep up with population growth,”
And the evaporation of which (not bottomless) is negligible compared to the 70% of the planet that has a water surface (bottomless).
Why is that fact not obvious??
“The extra WV put into the air via land use changes and irrigation is actually quite significant. Entire sea’s (Russia) and rivers (e.g. Colorado river) are used up for irrigation. The local climate in a large part of India has changed (notably) due to irrigation and also in Kansas (10% more WV).”
Merely your assertion – that the world’s experts do not agree with.
Some science please and not mere hand-waving.
How about working out the evaporation (continuous) from the 70% water surface.
a) from the hot tropical oceans.
b) from the north Atlantic/Pacific and the south Pacific – where evap is accelerated by strong winds.
c) consideration of the total water area presented to the atmosphere by man in comparison to the ~ 360 million Km^2 that the oceans present…. that is bottomless and stays hot through the 24 diurnal cycle to evap nearly as much as during the day.
Land surface soil moisture cools overnight.
d)landmasses are a LOT less windy that the oceans (lower evap).
A slight absence of common-sense re the relative proportions here.
Get it?
“Please disregard the above post ”
No probs, when I saw the length I did not even start reading.
This is quite a thought-provoking article.
Ultimately though, we’re talking about “integrating the time-series of measured temperature values, to produce a useful average for a day”. The rather obvious problem is that a T-max and T-min are extremes on what must nominally be a fairly noisy curve-of-the-day. Averaging them isn’t the best idea. The bigger problem is that for the longest time, it was standard equipment and quite-easy-to-manufacture, a “min, max recording thermometer”. With either alcohol or mercury, able to record the min and max extremes. Indefinitely. Without future calibration.
So, that’s the core of why min, max was used. And (min+max)/2.
Again though, coming back to the original premise, the real answer is integration, and the problem then becomes one of choosing representative values for each point-in-the-day that one wishes to record. With noisy data, it tends (over time) to average out. But there are some diurnal events (dawn, dusk) where “catching it on the wrong side” systematically over (or under) estimates the temperature for the band-of-time in question. Systematically, meaning, “affecting a whole run of days”. Which isn’t good.
The only real way to do this is to take raw measurements fairly frequently “internal to the instrument”, and average them over the longer reporting (recording) sample rate that is desired.
The instruments (dealing with similar issues, but for variables quite different from temperature, most of the time) I’ve built, which deal with long recording times (years) tend to measure things on a 1 second timeframe; if the experimenter wants “samples every 15 minutes”, well … that’s easy enough. Add up 900 of the 1 second measurements, and divide by 900. The “anti-aliasing method” is to pseudo-randomly choose 10% of the 900 samples, (90 of them), and average them out. This avoids the “aliasing” between raw-sample-rate (1 sec) and any so-called beat-frequencies of the phenomenon being measured over the 15 minute interval. For perfectly random-variation input data, this diminishes the accuracy of the average by a tiny bit, but again … overall, it leads to better results avoiding systematic sampling errors.
But here’s the big kicker: on a longer time scale the only competent way to also reduce the same kind of systematic sampling errors is to NOT sample “every 15 minutes”. Instead, either the instrument would be better set up to randomly sample “100 times a day”, at random intervals, or again pseudo-randomly, the every–15-minute samples would be better reduced to 5 minutes, and 10% of those accumulated “randomly” to constitute the daily average.
Statistics. Fun stuff, and not too hard.
Just saying,
GoatGuy
The central question in the previous post was whether averaging Tmin and Tmax was a good estimate of Tmean. It is not, and this was clearly demonstrated for a few test cases. I see the temperature cycle usually lasting about 24 hours, so sampling once an hour looks like following Nyquist sampling rules. Your analysis supports that since for one hour sampling, the difference in the means is less than the uncertainty in the thermometers. However, your analysis also implies that we can estimate the uncertainty in the older (Tmin+Tmax)/2 calculations. This may be of some worth. A nice test of this would be to find locations that used both the older style mercury in glass thermometers reading min and max, and had a modern platinum resistance thermometer, and compared the data.
Loren Wilson
You said, “…so sampling once an hour looks like following Nyquist sampling rules.” That may be adequate for an approximation of the shape of the temperature envelope, and provide a reasonable estimate of the true mean for all seasons. However, might there be more information that can be gleaned from the higher frequency data, such as instability or turbulence that is driven by more than just raw temperatures? I’m of the opinion that Doppler radar has provided meteorologists with much information about the behavior of wind than they ever learned from wind socks and anemometers. Relying on anachronistic technology almost assures that progress in our understanding of the atmosphere will progress slowly.
Clyde,
+1
Hmm, no mention of why you shouldn’t average intensive properties from different locations.
Jeff ==> If one admits the fact that averaging intensive properties from different locations is non-nonsensical, then there is simply nothing to discuss and most of Climate Science goes out the window. Thus, it is never mentioned.
The whole subject of Global Average Temperatures is, for the most part, non-scientific in nature and effect. Some of the worst aspects come into play when efforts are made to infill temperatures where no measurements were made.
In that regard, try to understand the science behind “kriging” to find AVERAGE temperature values for places and times where temperature was not measured. It is not that the mathematical process isn’t valid, it is that the mathematical process does not and can not apply to average temperature in any meaningful sense.
“The central question in the previous post was whether averaging Tmin and Tmax was a good estimate of Tmean.”
The relevant question is whether averaging Tmin and Tmax provides a useful metric. I can think of no reason why the difference between this metric and the “True Mean” will vary over time.
Steve O
You said, “I can think of no reason why the difference between this metric and the “True Mean” will vary over time.” The shape of the temperature envelope will change from an approximately symmetrical sinusoid at the equinoxes to ‘sinusoids’ with long tails at the solstices. Additionally, the shape can be distorted by storm systems moving across the recording stations during any season. The measures of central tendency (e.g. mean and mid-range value) are only coincident for symmetrical frequency distributions.
Compare the average shape of 3650 days over the last 10 years. How much different will it be to the average shape of 3650 days from 60 years ago? What reason will there be for any change from one decade to the next? Will there be any appreciable difference even from any one year to the next?
Thanks Nick for the work you put into this. It is an excellent demonstration of the productive value of good scientific critique. As a bonus, it twigged you onto an improvement in reducing the error of averaging. Without Ward’s Nyquist limit critique in temperature sampling, you probably would not have come upon your improvement. Bravo!
However, my critique is well defined by wsbriggs’s comment (whose excellent book on statistics “Breaking The Law of Averages” I continue to struggle with) .
https://wattsupwiththat.com/2019/01/25/nyquist-sampling-anomalies-and-all-that/#comment-2604272
Do you have a few examples of the actual shape and variety of diurnal temperatures. Do they approximate a sine wave satisfactorily? I’ve done a number of ground magnetometer mining exploration surveys and the diurnal correction curve is pretty much sinusoidal. On one survey, the stationary unit for recording diurnal failed and I had to run loops from the gridded survey, back to the same station periodically, approximately every hour (on snowshoes so more frequently was too much of a chore!). It seemed fully adequate.
Thanks, Gary,
” Do they approximate a sine wave satisfactorily?”
The main thing is that the harmonics taper reasonably fast. Here are the Fourier coefs for Redding, diurnal cycle, May, taken from averaging 2010-2018 hourly data, starting each cycle at midnight, in °C:
cos terms
-4.7789, 0.9484, 0.2305, -0.1249 …
sin terms
4.0887, -0.2826, -0.5696, 0.0805 …
There was a post on this at Climate etc.
https://judithcurry.com/2011/10/18/does-the-aliasing-beast-feed-the-uncertainty-monster/
The problem was that HADCRUT used monthly averages that does do remove aliasing. If they had low pass filtered the daily data and resampled the result the problem would have been greatly reduced.
Aliasing can generate low frequency components (ie: slowly moving trends) that are reflections about the sampling frequency (or negative frequency if one is going to be purist)
I remember reading that article at the time but had completely forgotten about it. Thanks for the reminder.
This is not a pedantic issue. I have been on about improper resampling for years. There is a naive idea that averaging always works, by people who do not realise that it assumes that you only have a balanced distribution of random errors. If there is any periodic components simply averaging over a longer period will not remove it , it will alias into something you very likely will not recognise.
Here is one of my favorite examples of aliasing from the ERBE data. Due to some silly assumptions about constancy of tropical cloud cover during the day, they had a strong diurnal signal in the data. This interacts with the 36 day repetition pattern of the flight path to produce some very odd results.
Initial processing using a monthy averages produced a similar shaped alias with a period of around 6 months. This was picked up Trendberth. Later data was presented only as 36 day averages which was a lot less use since everything else is reduced to monthly time series.
First time I read that link , I thought is said : does-aliasing-breast-feed-the-uncertainty-monster ? May have been a more catchy title.
The coefficient to estimate the so called “daily average temperature” from tmax and tmin is 0.5. I am not sure if there is any justification that 0.5 yieds more accurate values than say 0.3 or 0.6. The only potential reason would be that the temperature profile is symmetric around a mean.
A cursory look at hi-res temp data indicates that it is not the case. Daily temperature values are not sinusoidal at all but more triangular. Hence the the average value should be closer to Tmin. By using a coefficient of 0.5, a warm bias is introduced into the temperature trends.
Tangentially, how much does the diurnal temperature pattern change/vary with displacement from the typical measurement times?
I recall Willis discussing the idea that the daily formation of the clouds in equatorial regions might occur earlier, or later, with “climate change”. Half an hour’s sunshine at midday probably makes quite a difference to the energy budget.
If we are going to the great expense of automating temperature sampling, then the sample frequency should at least match the interval of the slowest element, the thermometers. Doing so creates a database that is useful for any temperature studies that anybody might need.
Waste not want not.
You are right, this claim is bogus. I am working on an essay that I’ll finish one of these days.
To calculate an uncertainty of the mean (increased accuracy of a reading) you must measure the SAME thing with the same instrument multiple times. The assumptions are that the errors will be normally distributed and independent. The sharper the peak of the frequency distribution the more likely you are to get the correct answer.
However, daily temperature reading are not the same thing! Why not? Because the reading tomorrow can’t be used to increase the accuracy of today’s readings. That would be akin to measuring an apple and a lime and recording the values to the nearest inch, then averaging the measurements and saying I just reduced the error of each by doing the average of the two. It makes no sense. You would be saying I reduced the error on the apple from 3 inches +- 0.5 inch to 3 inches +- 0.25 inches.
The frequency distribution of measurement errors is not a normal distribution when you only have one measurement. It is a straight line at “1” across the entire error range of a single measurement. In other words if the recorded temperature is 50 +- 0.5 degrees, the actual temperature can be anything between 49.5 and 50.5 with equal probability and no way to reduce the error.
Can you reduce temperature measurement error by averaging? NO! You are not measuring the same thing. It is akin to averaging the apple and lime. Using uncertainty of the mean calculations to determine a more accurate measure simply doesn’t apply with these kind of measurements.
So what is the upshot. The uncertainty of each measure must carry thru the averaging process. It means that each daily average has an error of +- 0.5, each monthly average has an error of +- 0.5, and so on. What does it do to determining a baseline? It means the baseline has an error of +- 0.5 What does it do to anomalies? It means anomalies have a built in error of +- 0.5 degrees. What does it do to trends? The trends have an error of +- 0.5 degrees.
What’s worse? Taking data and trends that have an accuracy of +- 0.5 and splicing on trends that have an accuracy of +- 0.1 degrees and trying to say the whole mess has an accuracy of +- 0.1 degrees. That’s Mann’s trick!
When I see projections that declare accuracy to +-0.02 degrees, I laugh. You simply can not do this with the data measurements as they are. These folks have no idea how to treat measurement error and even worse how to program models to take them into account.
Nyquist errors are great to discuss but they are probably subsumed in the measurement errors from the past.
If we want to understand weather, then carry on with these land surface temperatures and averaging and related counting angels on a pinhead.
If we want to understand CO2 and any effect its concentration has on climate, then we need something besides a surface land temperature record to assess that.
The Earth is 70% surface ocean and its depth is immense — averaging more than 2 miles. Our global climate is controlled by the near SST of the surrounding oceans. Anything happening regionally or locally (like droughts, floods, hot or cold spells, severe stroms) is simply weather. If we want to really understand any trend in the global (energy content) climate (is it warming or cooling and what rate) the ocean water temps across the globe at 10-100 meter depth are the only meaningful measure.
Land surface temp in the same spot can vary greatly simply based on vegetation cover changes, even if everything else remains the same. And air temps rise and fall so much daily and seasonally with so much inter-annual variation, taking averages and expecting to know something about long-term changing climate is simply foolish.
Even the Argo buoys are misleading with samples down to 2000 meters. This is well below the thermocline for most of the ocean producing meaningless information for near-term policy making.
Fixed ocean buoys though are producing meaningful temperature records at the depth where climate change can be assessed. For example, here is NOAA’s TAO buoy at Nominal Location: 2° 0′ 0″ S 155° 0′ 0″ W.
Down to 125 meters, the water temps stay above 27 C. At 200 meters they have fallen to below 14 C. (If you were a military submariner trying to avoid surface-based sonar detection, stay at or below 200 meters.)
It is the water temps from 100 meters and up to the near surface to 10 meters that are the only reliable metric for assessing any global effect of increasing CO2.
Willis did a great job a year ago introducing and assessing the TAO network in the Pacific
https://wattsupwiththat.com/2018/01/24/tao-sea-and-air-temperature-differences/
Maybe on this 1 year anniversary of his 2018 TAO post he can update us?
I meant to include this link:
https://tao.ndbc.noaa.gov/refreshed/site.php?site=38
in the above post.
It is the 2S 155W TAO buoy with current temperature data.
Joel, I agree that oceans are the calorimeter. Land temps are a consequence and adding land + sea is physically meaningless and seriously invalid.
I disagree with saying only the top 100m counts. That is what determines weather but if the point of interest is the long term impact of GHG forcing then we need to know whether the “missing heat” is “hiding in the oceans”.
If it is ending up in the deep oceans we can probably write off the problem for the probable duration of the current civilisation period.
If the missing heat is not hiding then we need to know that because the models are worse than we thought and “it’s a travesty”.
Here is a comparison of temp anom. of 0-100m and 100-700m , we can see the heat energy flowing from one to the other during Nino/Nina years. The deeper layers are important to what happens on the surface.
If 2000m is irrelevant , that will how in the data too. ( you are probably right, data will prove it ).
My point is, once you get below the thermocline, the exchange time scale (overturning) is long enough that we’ll see it (the AGW signal if it is happenng) in the upper layers first. Trenberth’s lament about the deep ocean ate his missing heat may be right. And the reason he worries is because if that really is the case then there is no case for alarmism. The deep oceans are burying the heat of a few extra watts.sm-2 in a high entropy, 4 degree C temperature state, forever lost. The second law of thermodynamics tells us that buried heat isn’t going to rise out the depths and be of any consequence to climate, even in 400 years..
The temperature changes in the deep ocean below the thermocline are so small, and cycle time in the AMOC is like 800 years, that it will take centuries or more observations to know whether changes seen are real, noise, or aliasing.
Once again, the idea of averaging testicles with ovaries is obscured with numerological bafflegab.
Plot them separately and you have something meaningful.
That is not hard to understand- the work is in pretending you don’t.
“The average person has one Fallopian tube.”
– Demetri Martin
The problem that has arisen in Australia (and elsewhere) is that the weather service here (Bureau of Meteorology) has transitioned all of the 500+ weather stations in the country to automatic recorders with (AWS) electronic thermometers taking 1 sec readings. Instead of averaging the 1 sec readings over some time base – as done elsewhere in the world (UK 1 minute, USA 5 minutes) – they take the instantaneous 1 sec reading as the maximum. A daily maximum can now come from any instantaneous 1 sec reading during the day.
Prior to this we had the mercury/alcohol thermometers read at set time intervals during the day and providing daily maximum and minimum temperatures averaged over a long time constant.
The BOM have claimed they have undertaken extensive comparative studies and they are certain the two methods are equivalent and the records for each site can be joined together. Trouble is they will not release any of the studies to the public, so no non-government scientist can review their claims. Is it a coincidence a lot of new temperature records have been set following introduction of the new AWS network?
This practice sort of makes the Nyquist discussion irrelevant for Australia.
Anything which can not be independently verified is not science. This persistent and obstructive opacity from BoM is just the clearest indication possible that they are playing politics , not performing objective science.
A thermometer will take 3-4 minutes to stabilise. The wind can change in 3-4 seconds. Anyone claiming that snapshot 1 second readings will have the same variability as data from a 4 minute response device is an ignoramus or a liar.
It is up to the reader to decide which is the case for BoM climatologists.
Great post, Nick. Well presented.
Interesting post, but I think Nick is making some unwarranted assumptions.
I was first introduced to Nyquist when studying control engineering. I understand the theory, but some important points can be made without resorting to the maths.
The importance of Nyquist is to retain enough information to be able to reconstruct the signal you are interested in. MarkW makes this point above, and is correct.
If you don’t sample above the Nyquist theoretical limit (in fact probably 5 – 10 times the theoretical limit for practical applications), you run the risk of making bad mistakes when interpreting your data.
Nyquist just says: if the signal can move at a certain rate of change, your sampling regime has to be quick enough to ensure you do not miss important data.
You need to know the characteristic of your signal to know whether your sampling regime is fit for purpose. Sometimes I wonder if the analysis of the signal’s characteristics is ever done when collecting climate data – it seems to be a strategy of “collect something and we’ll try to fix any issues later”.
This is where William Ward does a good job. He shows examples where bias is introduced because of an inadequate sampling regime. He shows that retaining min/max is inadequate to represent the shape of the signal. In essence, he shows the effect of losing too much information.
Imagine quite a warm a costal location, but is prone to onshore breeze which can suddenly drop the local temperature. If this causes a short period of cool foggy conditions in an otherwise fine day, there will be a large error in min/max to represent the whole day. Tmin gives the impression that half the day was at Tmax, and half at Tmin.
But maybe this will sort itself out over time? If the assumption is that there will be equal distribution of days with the opposite behaviour, this absolutely needs to be demonstrated (and supported to hold true). It might not be true, and the only way to know whether it is true it to retain the shape of the signal to check. Therefore back to complying with Nyquist.
In addition, a research group could come along and decide that the min/max data needs to be “homogenised” to remove their assumptions of sources of bias. So they correct it, without ever appreciating that the data is biased for completely different reasons (failure to observe Nyquist).
The problem with Nick Stokes’s position is the assumption of a wave shape which is regular enough to be represented by its min and max. William Ward showed why this is a poor assumption.
A finally thought.
Nyquist does not only apply in the “temporal domain” (time sample interval), but also in the “spatial domain” (how quickly temperature can vary from point to point on the surface of the planet). Even if it could be demonstrated that temporal sampling is adequate, I believe there is very likely to be significant distortion due to aliasing in the spatial domain.
Nyquist matters. If you think it doesn’t, you need to be very confident that you understand the signal characteristics well enough to justify why.
” Sometimes I wonder if the analysis of the signal’s characteristics is ever done when collecting climate data ”
Most “climate data” was never intended to be climate data, it was weather data. We are now trying our best to extract some climate information from that data.
thanks nick.
I am sure some here will still object.
I have a question. suppose I only sample once a year
like when the Thames freezes and they had frost fairs.
nyquist means there was no LIA. right?
asking for a friend.
@Mosher,
What is the sample rate of an air to air missile ?
Just generally, don’t give up any secrets.
“Sparse sampling (eg 2/day) does create aliasing to zero frequency, which does affect accuracy of monthly averaging.”
Wrong. Try running a DFT with your criteria and then recompose the signal from such a undervalued harmonics count based maybe peak values.
You won’t be even close to the average because, happens, temperature signal is not made out of nice Sin functions.
Violating Nyquist is a pretty nice random numbers generator indeed.
Further.
Temperature is closely related to energy content. Surprise !
Therefore any methods of aliasing such as average, that do not conserve “second momentum”, think “square root of sum of squares” is meaningless, turning the signal into a one dimension suite of numbers.
Temperature statistics in Celsius of Fahrenheit are meaningless. Yep. Like, a cubic meter of whatever at -3C would lead to a negative energy content.
However that same matter at 270.15 Kelvin has a positive energy as it should be.
Never forget that climate statistics use all it takes to achieve “fit for public presentation” numbers.
Quite some time ago Physicists admitted that it had a major problem…with trying to accurately measure things that were very very small…these tiny fields and particles seemed to be behaving in a manner that didn’t conform to their understanding of how matter should behave.
I wonder when people are finally going to accept that there is also a problem with measuring very big things…like a ‘global average temperature’…sure you can torture the numbers, but this
old rotating, tilting planet, crisscrossed by high speed winds and meandering ocean currents will ensure that whatever number you come up with is essentially meaningless.
The real problem with big thing is it needs big money to do the number of measurement needed and the big money all goes on their expensive computer models and grease monkey ( their description of us in the pre PC days) work of data collection is pared to the bone. The cabinets are not even well maintained from the few I have seen and that induces an error greater than the differences we are talking about.
” a problem with measuring very big things…like a ‘global average temperature’”
They do not measure that. It’s wrong to say they measure it.
They calculate it. It’s numerology, not scientific measurement.
Measurements measure… physical values. Actual, very well defined, physical values. There is no such thing as a physical temperature for a non-equilibrium system. Averaging intensive values do not give you a parameter of such physical system. It is provable for very simple systems that doing such idiotic averaging will supply you the wrong results if used to compute something physical out of it. You will get ‘warming’ systems that are physically cooling and ‘cooling’ systems that are physically warming.
That’s what they studiously ignore.
Averaging a turtle with a hare is not meaningful, period.
But this entire page is devoted to a discussion of how to do it right.
There is no member of the null set. But here we have endless debate about the properties.
As you say- it’s numerology – it is mysticism.
Stokes
You say, ” If you have a variable T that you are trying to average, or integrate, you can split it:
T = E + A
where E is some kind of expected value, and A is the difference (or residual, or anomaly). ”
The implication of your claim is that as the choice of E approaches T, A becomes vanishingly small and the error disappears. So, choosing a base close to the current temperature(s) reduces the size of the anomaly and the inherent error associated with the anomaly. Why isn’t that done?
Clyde,
“The implication of your claim is that as the choice of E approaches T, A becomes vanishingly small”
No, the intention is to still evaluate the average of A. The idea is that we choose E in a way that makes it possible to get a better estimate of its average. If it then makes up a substantial part of the total, then that improvement is reflected in the result, since the evaluation of A is no worse. There are two things that may help
1. A is indeed reduced in magnitude, and so is the corresponding error of averaging
2. A becomes more homogeneous – ie more like an iid random variable (because you have tried to make the expected value zero). That means there is more benefit from cancellation, even with inaccurate averaging.
I think factor 2 is more important.
Stokes
The total error in T is partitioned between E and A in proportion to their magnitude. The only thing that one can state with confidence is that the error in A is less than the error in T. However, it is really T that is important because it applies to the T in Fourier’s Law.
But, you really didn’t answer my question, which happens all too often when you respond. Why don’t the data analysts, such as yourself, use the most current 30 years as a baseline and report the anomalies for all 30 years, including the most recent year? Besides reducing the anomaly error, it would also better reflect the current climate that is acknowledged to be changing.
Clyde,
“But, you really didn’t answer my question, which happens all too often when you respond. Why don’t the data analysts, such as yourself, use the most current 30 years as a baseline and report the anomalies for all 30 years, including the most recent year? “
That wasn’t really the question you asked. But it’s an interesting one, which I looked at in some detail here. It depends on what you want to learn from the anomalies. If you want to look at spatial distribution for a particular time, yes, although you can do better than just the average for the last 30 years. But if you want to track a spatial average over time, then it makes sense to use a reasonably stable anomaly base, so you don’t have to keep updating for that new anomaly base.
The compromise recommended by WMO is to update every decade. Some suppliers are more attached to the stability aspect.
Stokes
I asked, “Why don’t the data analysts, such as yourself, use the most current 30 years as a baseline and report the anomalies for all 30 years, including the most recent year? “ You replied, “That wasn’t really the question you asked.” Well, I did use different words the first time I asked: “So, choosing a base close to the current temperature(s) reduces the size of the anomaly and the inherent error associated with the anomaly. Why isn’t that done?” It seems to me that the questions are essentially equivalent.
The point being, if the primary purpose of anomalies is to make corrections and interpolate, one should use the anomalies with the least error. Doing an annual update with a running average is fairly trivial with computers. However, if the point is to have a scare tactic to employ, then using a baseline from decades earlier gives larger changes (With larger errors!). So are you and the ‘suppliers’ really interested in minimizing error for corrections or not?