A Smoldering Gun From Nashville, TN
Guest post by Basil Copeland
The hits just keep on coming. About the same time that Willis Eschenbach revealed “The Smoking Gun at Darwin Zero,” The UK’s Met Office released a “subset” of the HadCRUT3 data set used to monitor global temperatures. I grabbed a copy of “the subset” and then began looking for a location near me (I live in central Arkansas) that had a long and generally complete station record that I could compare to a “homogenized” set of data for the same station from the GISTemp data set. I quickly, and more or less randomly, decided to take a closer look at the data for Nashville, TN. In the HadCRUT3 subset, this is “72730” in the folder “72.” A direct link to the homogenized GISTemp data used is here. After transforming the row data to column data (see the end of the post for a “bleg” about this), the first thing I did was plot the differences between the two series:
The GISTemp homogeneity adjustment looks a little hockey-stickish, and induces an upward trend by reducing older historical temperatures more than recent historical temperatures. This has the effect of turning what is a negative trend in the HadCRUT3 data into a positive trend in the GISTemp version:
So what would appear to be a general cooling trend over the past ~130 years at this location when using the unadjusted HadCRUT3 data, becomes a warming trend when the homogeneity adjustment is supplied.
“There is nothing to see here, move along.” I do not buy that. Whether or not the homogeneity adjustment is warranted, it has an effect that calls into question just how much the earth has in fact warmed over the past 120-150 years (the period covered, roughly, by GISTemp and HadCRUT3). There has to be a better, more “robust” way of measuring temperature trends, that is not so sensitive that it turns negative trends into positive trends (which we’ve seen it do twice how, first with Darwin Zero, and now here with Nashville). I believe there is.
Temperature Data: Pasteurized versus Homogenized
In a recent series of posts, here, here, and with Anthony here, I’ve been promoting a method of analyzing temperature data that reveals the full range of natural climate variability. Metaphorically, this strikes me as trying to make a case for “pasteurizing” the data, rather than “homogenizing” it. In homogenization, the object is to “mix things up” so that it is “the same throughout.” When milk is homogenized, this prevents the cream from rising to the top, thus preventing us from seeing the “natural variability” that is in milk. But with temperature data, I want very much to see the natural variability in the data. And I cannot see that with linear trends fitted through homogenized data. It may be a hokey analogy, but I want my data pasteurized – as clean as it can be – but not homogenized so that I cannot see the true and full range of natural climate variability.
I believe that the only way to truly do this is by analyzing, or studying, how differences in the temperature data vary over time. And they do not simply vary in a constant direction. As everybody knows, temperatures sometimes trend upwards, and at other times downward. The method of studying how differences in the temperature data allows us to see this far more clearly than simply fitting trend lines to undifferenced data. In fact, it can prevent us from reaching the wrong conclusion, as in fitting a positive trend when the real trend has been negative. To demonstrate this, here is a plot of monthly seasonal differences for the GISTemp version of the Nashville, TN data set:
Pay close attention as I describe what we’re seeing here. First, “sd” means “seasonal differences” (not “standard deviation”). That is, it is the year to year variation in each monthly observation, for example October 2009 compared to October 2008. Next, the “trend” is the result of smoothing with Hodrick-Prescott smoothing (lamnda = 14,400). The type of smoothing here is not as critical as is the decision to smooth the seasonal differences. If a reader prefers a different smoothing algorithm, have at at it. Just make sure you apply it to the seasonal differences, and that it not change the overall mean of the series. I.e., the mean of the seasonal differences, for GISTemp’s Nashville, TN data set, is -0.012647, whether smoothed or not. The smoothing simply helps us to see, a little more clearly, the regularity of warming and cooling trends over time. Now note clearly the sign of the mean seasonal difference: it is negative. Even in the GISTemp series, Nashville, TN has spent more time cooling (imagine here periods where the blue line in the chart above is below zero) than it has warming over the last ~130 years.
How can that be? Well, the method of analyzing differences is less sensitive – I.e. more “robust” — than fitting trend lines through the undifferenced data. “Step” type adjustments as we see with homogeneity adjustments only affect a single data point in the differenced series, but affect every data point (before or after it is applied) in the undifferenced series. We can see the effect of the GISTemp homogeneity adjustments here by comparing the previous figure with the following:
Here, in the HadCRUT3 series, the mean seasonal difference is more negative, -0.014863 versus -0.012647. The GISTemp adjustments increases the average seasonal difference by 0.002216, making it less negative, but not enough so that the result becomes positive. In both cases we still come to the conclusion that “on the average” monthly seasonal differences in temperatures in Nashville have been negative over the last ~130 years.
An Important Caveat
So have we actually shown that, at least for Nashville, TN, there has been no net warming over the past ~130 years? No, not necessarily. The average monthly seasonal difference has indeed been negative over the past 130 years. But it may have been becoming “less negative.” Since I have more confidence, at this point, in the integrity of the HadCRUT3 data, than the GISTemp data, I’ll discuss this solely in the context of the HadCRUT3 data. In both the “original data” and in the blue “trend” shown in the above figure, there is a slight upward trend over the past ~130 years:
Here, I’m only showing the fit relative to the smoothed (trend) data. (It is, however, exactly the same as the fit to the original, or unsmoothed, data.) Whereas the average seasonal difference for the HadCRUT3 data here was -0.014863, from the fit through the data it was only -0.007714 at the end of series (October 2009). Still cooling, but less so, and in that sense one could argue that there has been some “warming.” And overall – I.e. if a similar kind of analysis is applied to all of the stations in the HadCRUT3 data set (or “subset”) – I will not be surprised if there is not some evidence for warming. But that has never really be the issue. The issue has always been (a) how much warming, and (b) where has it come from?
I suggest that the above chart showing the fit through the smooth helps define the challenges we face in these issues. First, the light gray line depicts the range of natural climate variability on decadal time scales. This much – and it is very much of the data – is completely natural, and cannot be attributed to any kind of anthropogenic influence, whether UHI, land use/land cover changes, or, heaven forbid, greenhouse gases. If there is any anthropogenic impact here, it is in the blue line, what is in effect a trend in the trend. But even that is far from certain, for before we can conclude that, we have to rule out natural climate variability on centennial time scales. And we simply cannot do that with the instrumental temperature record, because it isn’t long enough. I hate to admit that, because it means either that we accept the depth of our ignorance here, or we look for answers in proxy data. And we’ve seen the mess that has been made of things in trying to rely on proxy data. I think we have to accept the depth of our ignorance, for now, and admit that we do not really have a clue about what might have caused the kind of upward drift we see in the blue trend line in the preceding figure. Of course, that means putting a hold on any radical socioeconomic transformations based on the notion that we know what in truth we do not know.
Anthony, there is something magical about using data from 1979 or so onward.
Here, try this simple test. Ask the NSIDC why they will ONLY use data from 1979 to 2000 as the reference average for Arctic sea ice. How can it possible be that this is the best and only reference for all things Arctic. They don’t have a rational answer. The obvious conclusion is this is done for dramatic effect.
The NSIDC then has the gall to proclaim that what has been seen in the 30 year record of Arctic sea ice is falling outside the natural variability – REALLY! If that truly were the case, then no more studying of the Arctic need be done. Actually, anything falling outside 2 standard deviations of the 20 year average is considered outside the natural variability. As Church Lady would say, “how convenient”.
JJ (23:17:00) :
I didn’t say HadCRUT3 was “raw.” I referred to it as “unadjusted.” Now what I meant by that is that it should be the same as GISS before GISS applies its “homogeneity” adjustment.
No monthly data can ever be described as “raw,” because the “raw” observations are daily. And any data set that has NO missing observations is unlikely to be truly “raw.” For all the talk about going back to the “raw” data, I don’t think that is where the problem begins. From my work with US data (I do some consulting work where I have occasion to look at the truly “raw” data occasionally), NOAA does some “quality” control right off the bat in reading from the hand written daily records. I doubt that any systematic “warming bias” is introduced at that point.
The next problem faced with using “raw” data is the handling of missing data. I think this may need to be looked at a little more closely. To read how this is done, it sounds like a reasonable approach, but only superficially. It is done by correlating differences (that’s important, and a good thing) from “nearby” stations. This is probably okay, for the occasional missing observation, especially if this is being done to fill in the daily data (which I’m not sure about, but should read up on at some point). But if a lot of data is missing, there should be a point at which the station is just excluded, rather than fill it in with the method currently used. (Maybe this is done. Do you know?)
For its purpose, I think what I’ve done is defensible. If you go to the GISTemp web site, you get the option of downloading its “pseudo-raw” version of the data, i.e. the data before it applies its “homogeneity” adjustment. I could have used that, instead of HadCRUT3, and I believe that the results would have been similar, if not the same. Both CRU and GISS are, in a case like this, getting their data from the same place. I doubt that they started with the daily data. They are taking the monthly data as given to them by the various met agencies. So it is relevant, and interesting, to explore how they can take the same data, and come up with such different results. Which is all I did.
An enlightening post, Basil Copeland. Seasonal differences seems to be more appropriate for knowledge about temperature/climate change. ClimateGate or CRUCrud is the worst and most costly scientific scandal of our generation. Your conclusion is ONE OF THREE ideas I would like to see emphasized every time there is a discussion of Climate Gate or Global Warming.
ONE “I think we have to accept the depth of our ignorance, for now, and admit that we do not really have a clue about what might have caused the kind of upward drift we see in the blue trend line in the preceding figure. Of course, that means putting a hold on any radical socioeconomic transformations based on the notion that we know what in truth we do not know.” (Copeland)
TWO (A) A real-scientist/skeptical-public site where validated (by whom?) RAW DATA is kept. This seems to be a necessity if we are going to pull together reasonable conclusions for a general public. See paulhan (22:53:05) “I have just downloaded the Daily data from GHCN (Please tell me that this data is truly raw, otherwise I will have downloaded 1.7GB of nothing).”
TWO (B) If the RAW DATA needs ADJUSTING, a validated (by whom?) method for doing so. Also paulhan (22:53:05) “”I’ve seen lots of complaints about the various ways the raw data is adjusted, but I think if we are going to be taken seriously we need to come up with our own ONE way of adjusting the data….”
THREE. A CENTRAL LOCATION for “distributed computing” using raw data. Mike Fox (23:35:01) suggests surfacestations.org surveyors might take on the task using “formulas like Willis and Basil did” and asking standardized questions that speak to Basil’s conclusion — why we must do nothing now but wait for scientific research based on transparency, accountability and verification. Paulhan suggests Eschenbach’s surfacetemps.org. E.M. Smith also has collected a tremendous amount of data.
There already are so many smoking or smoldering guns to go along with the pre-1999 (before the conspiracy began in earnest) science plus the skeptic-scientist “banned” research of the last 10 years, that concerted action to urge resonable non-action seems, well, urgent. (And I still want jail time for the fraudsters, fines equal to the grants at tax-payers expense they squandered, and banishment from academic life.)
MikeC (22:47:20) :
John Goetz and EM Smith has gone a step further by analyzing GISS source code. The buck stops there.
http://chiefio.wordpress.com/2009/07/21/gistemp_data_go_round/
http://wattsupwiththat.com/2009/07/22/giss-step-1-does-it-influence-the-trend/
http://wattsupwiththat.com/2008/04/08/rewriting-history-time-and-time-again/
http://climateaudit.org/2008/02/09/how-much-estimation-is-too-much-estimation/
One thing I’d like to see on future raw/cooked and blinking temperature
graphs is the difference. There ought to be several cases where the
adjustments are the same at several stations as UHI and street light
adjustment heuristics are twiddled.
Basil (05:26:33) :
What’s a bleg? A request for something in a blog?
Awk would be an easy choice, my kneejerk reaction for almost anything outside of kernel code is Python (follow the link!) and that might be on your Linux system already, some distros use it for a lot of system configuration stuff. Perl would be good to, but one reason I learned Python was so that I didn’t need to learn Perl.
I don’t really have time, but if there are no takers….
Mike Fox (23:35:01) :
….
How about if each of us who was a surfacestations surveyor would take the “raw” data from the stations we surveyed, crank it through formulas like Willis and Basil did in their posts, and then upload the results to supplement our surfacestations.org surveys?
See what I just wrote to JJ about “raw” data. I certainly am not interested in going back to the truly “raw” data, which is daily.
In deciding where to begin, we have to decide whether we begin with the truly raw data, or the data that was received by CRU (or GISS) from the met agencies. Much of the ruckus over FOI’s has been simply to get the latter. And I think we now have some of that with the “subset” of HadCRUT3 that has been released. I may be mistaken — somebody correct me if I am — but I think this is supposed to be the data “as received” from the met agencies. They describe the data they are releasing thusly:
“The data that we are providing is the database used to produce the global temperature series. Some of these data are the original underlying observations and some are observations adjusted to account for non climatic influences, for example changes in observations methods.”
It is a little unclear from this statement as to who is responsible for the adjustments, but I think they mean the original met agencies, not CRU. I.e., if the data they got from a national met agency was adjusted for time of observation bias, that’s what they used. If it wasn’t adjusted for time of observation bias, that is still what they used. I do not think they are saying that they (here Hadley or CRU) adjusted some of the data for non climatic influences, but didn’t adjust other data. They just went with what they were given.
Assuming this is so, for monthly data, this is probably about as “raw” as it gets, without going back to the original daily data. And it is a start. GISS is taking this same data, and is doing something more to it. So a good starting point is to understand the differences between GISS and HadCRUT.
This is probably a little off topic, unless you consider motivations for all this crap that’s going on- CRUtape Letters, AlBore, Copenhagen, etc.
Look at this: When the UN’s involved: Follow the Money.
From World Net Daily:
TonyB (00:01:31) :
I’ve asked about it a couple of times, but have never received a satisfactory answer about why we should be trying to UHI-adjust the data. UHI is a type of AGW. I understand the desire to quantify it, but not as an “adjustment” to the data. Leave out trying to “adjust” the individual stations for UHI. Then, once we have good data on temperature for selected stations, then try to quantify the effect of UHI.
Which brings up an interesting question. At what stage is the UHI adjustment done? I’m guessing (!) that it is done when taking the data received from the met agencies, and trying to grid it into a world model. But I am very suspicious about the adjustment, and doubt that it is nearly what it ought to be. I’d rather than adjustment be left out of the “global temperature” data set, and then let climate scientists argue out in the literature about how much UHI is in the “global temperature.” The way it is now, alarmists can dismiss UHI because it supposedly has already been accounted for.
I really do think UHI needs to be studied like Peter did in the video Anthony posted a couple of days ago. Do not make it part of the “global temperature” data. The “global temperature” is “what it is” regardless of the source. Adjustments may be justified for time of observation bias, or other “non-climatic” influences, but UHI is definitely not a “non-climatic” influence. It is very much an influence on the climate.
I downloaded the CRU data recently released by the Met Office and was surprised to find only partial data sets for the UK stations, some ending in 1990 or earlier. I then went to the met office website and downloaded two stations that had long data sets.
http://www.metoffice.gov.uk/climate/uk/stationdata/
I have plotted the graphs of temperature over time on my website.
For Durham and Oxford. These two towns are central England
http://www.akk.me.uk/Climate_Change.htm
They show a steady decline. I would appreciate any comments, this was my method and admittedly it is only two sites.
I have taken the difference between tmax and tmin for each month and averaged them over the year. Where there was estimated data I have used the estimated data. Where there was missing data I interpolated the missing data using the previous and following month.
astonerii (00:29:32) :
Is this a joke? You can show warming trend from data that goes from 15.9C to ~15.425C? I am sorry, but something is wrong with the calculations, if your saying maybe there is a warming trend at the very end, that is one thing, but to say there has been net warming, that is pretty much unbelievable. Either the starting temperature is higher or lower than the ending temperature, there is no other choice. You did not even show what the actual temperature curve looked like, just a trend line.
Here’s your “actual temperature curve(s)”:
http://img692.imageshack.us/img692/1286/actual.png
With that much variation, it isn’t hard to imagine subtle differences having major impacts on the trend lines.
“to accept the depth of our ignorance” – To do anything else would be dishonest, and opens the door to bias, wild speculation, and puts leaders and politicians in the drivers’ seat, to steer it in any direction they wish, which in this case, was right off the damn cliff. Not only does one have to be aware of the level of ignorance, but we must also be painfully aware of the sheer complexity involved in attempting to instrument the climate of our planet with the intention of predicting it’s ‘long-term’ future state.
This requires, if it is even feasible (or possible), a great deal more observational data (ALOT more, read: ‘launch more satellites!!’) over a greater period of time, with a great deal more supercomputers to process the data and to start basically guessing with the models until the predictions actually become somewhat accurate.
Anyways, and not to pile on, but I’ve just started looking into the infrared radiative transfer models… Too early to say, but my impression is that the potential for more of the same is most definitely there…and again, it’s a case of not accepting the depth of our ignorance, in addition to it being on more shaky science ground than mercury thermometers on an asphalt parking-lot covered in latex paint.
And the satellite calibration and data and it’s future.. well, this one is harder to investigate, but apparently the plan is to have a U.N. agency write the software and probably have control over the database and who gets access etc….
maybe we should consider building an open-source, massively distributed computing framework on the internet, as an additional means of oversight, (plus it could be a great deal of fun)… might even turn out be an asset for the big government agencies.
Doesn’t Pasteurization require applying heat?
Off-topic specifically but on-topic generally, I would like to recommend this post by Alan Sullivan of Fresh Bilge. It is a re-post of something he wrote in 2008, but it is even more relevant today. He is an excellent writer and a good science generalist. He neatly summarizes most of the true scientific “consensus” regarding the inter-relationships of forces and energies affecting the climate.
See http://www.seablogger.com/?p=18358
Basil,
“I didn’t say HadCRUT3 was “raw.” I referred to it as “unadjusted.””
The HadCrut3 data you used was not raw data. It is adjusted data. Specifically, it is homogenized data. The Met office tells you that on the page that you linked to above. This:
Some of these data are the original underlying observations and some are observations adjusted to account for non climatic influences, for example changes in observations methods.
means homogenized. Homogenized means ‘adjusted to account for non climatic influences’.
“Now what I meant by that is that it should be the same as GISS before GISS applies its “homogeneity” adjustment.”
That is not true. The HadCrut3 is not raw data. It is homogenized data. The Met tells you that on the page that you linked to above. It is not the same as GISS before before GISS applies its homogeneity adjustment.
You are not comparing GISS adjusted data with unadjusted data, as you claim. You are comparing two different adjusted datasets. What you say about that comparison is false. Please correct your error.
“For its purpose, I think what I’ve done is defensible.”
It isnt. You dont understand what you have done, and are describing both the datasets and the comparison you made with them incorrectly.
“If you go to the GISTemp web site, you get the option of downloading its “pseudo-raw” version of the data, i.e. the data before it applies its “homogeneity” adjustment. I could have used that, instead of HadCRUT3, and I believe that the results would have been similar, if not the same.”
[snip] You have the option to actually get the unhomogenized data from GISS, but instead you got homogenized HadCRUT3 data, and pretend that it is the same? Why would you do that? Honestly, why? That makes absolutely no sense. If you could have used unhomogenized GISS data where your intended analysis called for unhomogenized GISS data, why on earth didnt you?
“So it is relevant, and interesting, to explore how they can take the same data, and come up with such different results. Which is all I did.”
No it isnt. What you did was compare two homogenized datasets, and claim that one was homogenized and the other wasnt. And you did this, even though you admit to having access to what you knew to be the unhomogenized data you claim you were using.
Why on earth would you do this?
What caused that sudden “jerk” in the differences (Fig. 1) on about 1963? Was there a station move?
Trevor Cooper (02:13:35) :
An interesting approach to dealing with step changes. But I think your final section, labelled ‘An Important Caveat’, is misleading. To see if there has been warming recently, you should choose a period (say the last thirty years) and simply look at the average of the seasonal differences over that period. Your approach of fitting a line to the graph of seasonal differences is in fact measuring any acceleration in warming, a very different thing. There again, I may have misunderstood.
Your suggestion has merit. The problem you will encounter is that the standard deviation is so high that it is all but impossible to conclude anything from doing that. For example, for the GISTemp data set, I compared the mean of the last 30 years to the mean of all the years preceding:
Null hypothesis: Difference of means = 0
Sample 1:
n = 360, mean = 0.027222, s.d. = 2.5256
standard error of mean = 0.133111
95% confidence interval for mean: -0.234553 to 0.288997
Sample 2:
n = 1186, mean = -0.024872, s.d. = 2.699
standard error of mean = 0.0783719
95% confidence interval for mean: -0.178635 to 0.128891
Test statistic: t(1544) = (0.027222 – -0.024872)/0.160045 = 0.325496
Two-tailed p-value = 0.7448
(one-tailed = 0.3724)
For a graphic of the test, look here:
http://i46.tinypic.com/b4j38z.jpg
The difference between the means is “not significantly different than zero.”
Here’s the “problem:” natural climate is…variable. It is very difficult — in my view it is impossible, given the present state of knowledge — to say that the last 30 years are outside the range of natural climate variability.
We need to coin a new aphorism: “hide the variability.” Because besides “hiding the decline,” a lot of the alarmist claims are dependent on “hiding the variability.”
I am not a mathematician and I never studied statistics so please someone correct me if I’m wrong.
It seems as though any oscillating curve (even a sine wave) can be fitted with a trend line that points either up or down, depending on where you choose to start and end the curve that you are fitting.
Basil, more of a question. You are using the entire year’s data. What happens when you look just at the daily temps for the two ends of the climate, Jan and July? That is, what is the trend for all the Jan months for the series and same with the July months?
I have long speculated that what we are seeing is not a true increase in temps, but a narrowing of varation below the maximum temps, which will tend to increase the average daily temps, but no real physical increase is occuring. That is, what we are seeing are shorter warmer winters, with no change in summer temps, which gives an increase in average temps over time.
Also, I’m speculating that spring can come earlier and fall come later, which one can see with the temp changes in those transition months.
This could be the killer of AGW if this is the case because there is no real increase in temps, just less variation below max temps for each month. The alarmism of a catastrophic future dissapears then doesn’t it.
I’m going to test this speculation myself as I’m currently downloading, from Environment Canada’s website, all the daily temps for all 1600 Canadian stations from 1900 to the present. It will take about 10 days to complete that download. Then I’ll import it into Access or SQL Server to do the analysis.
acob (02:51:10) :
I very much disagree that “less cooling” means warming.
If the trend is still negative there’s no way to call it warming.
It’s the same as “less than expected job-losses” not meaning a recovery – it’s still getting worse.
cheers, acob
I understand your point. It is a question of semantics. I’m simply trying to acknowledge that something may be taking place that is raising the earth’s temperature, compared to what it would be if that thing were not taking place. Now I’ve purposely left ambiguous what that “thing” might be, for the reasons discussed in the post.
If it has not already been put up here and I missed it, this is interesting:
CRU: Artificial Corrections, Fudge Factor
Nick Stokes (05:05:39) :
Instead of just picking one station here and there, why not look at the effect of adjustments overall. This is what Giorgio Gilestro did for the GHCN set that Willis analysed for Darwin alone. . He shows the distribution of the effects of the adjustment on trend. It looks quite symmetric. Adjustments are just as likely to cool as to heat.
Then how do you explain this?
http://www.ncdc.noaa.gov/img/climate/research/ushcn/ts.ushcn_anom25_diffs_urb-raw_pg.gif
If I’m not mistaken — and I might well be! — the CRU dataset takes any adjustments like this, that have already been made by the met agencies, as they are. Gilestro only looks at what CRU has done to the data that was given to it. I’m a bit of “middle of the roader” here, but I’m not particularly leery of HadCRUT. I’m less sanguine about GISS.
In any event, I wasn’t comparing CRU “raw” to CRU “adjusted.” I was comparing CRU “whatever it is, as recently released by the UK met office” to GISS “homogenized.” I suspect that the kind of biases I show here will NOT all average out to zero, but reflect some intrinsic issues with respect to the GISS homogenization process.
Here is some rough excel code. It works.
NOTE that the excel sheet must be saved as a .XLSM macro enabled worksheet and macros will have to be enabled.
excel7
click developer tab
type a new macro name
e. g. YearPerRowToClmn
Assign to keybord if you like
click [create]
then between:
Sub YearPerRowToClmn()
End Sub
paste this (but not the “———“:
‘——————————
ActiveCell.Offset(1, 0).Range(“A1:A11”).Select
Application.CutCopyMode = False
Selection.EntireRow.Insert , CopyOrigin:=xlFormatFromLeftOrAbove
ActiveCell.Offset(-1, 2).Range(“A1:K1”).Select
Selection.Copy
ActiveCell.Offset(1, -1).Range(“A1”).Select
Selection.PasteSpecial Paste:=xlPasteAll, Operation:=xlNone, SkipBlanks:= _
False, Transpose:=True
ActiveCell.Offset(11, -1).Range(“A1”).Select
‘ must “hide the incline”
‘ beware the ides of march
repeattranspose:
If Len(ActiveCell.Text) < 2 Then GoTo stopp
ActiveCell.Offset(1, 0).Range("A1:x11").Select
Application.CutCopyMode = False
Selection.EntireRow.Insert , CopyOrigin:=xlFormatFromLeftOrAbove
ActiveCell.Offset(-1, 2).Range("A1:K1").Select
Selection.Copy
ActiveCell.Offset(1, -1).Range("A1").Select
Selection.PasteSpecial Paste:=xlPasteAll, Operation:=xlNone, SkipBlanks:= _
False, Transpose:=True
ActiveCell.Offset(11, -1).Range("A1").Select
GoTo repeattranspose
stopp:
'———————–
to use:
get data in text form with 1 year of 12 months + other stuff
Select data [ctrl]+a selects the lot
Copy data [ctrl+c
open blank sheet in the workbook containing the macro
paste the data at cell a1
You now have a single column of data one year per row. If your excel is set up differently it may convert the data to columns automatically, ifnot
select the first to last year:
click first – scroll to last and click the last year whilst holding shift
select [data] tab
click text to colums
click delimited if you KNOW that there is always a certain character (space, comma etc) between monthly data
or click fixed width
click next (select delimiter character if necessary)
check the colums are correctly selected – move, delete or add. If station number is in data this usually has the date attached without space. If so add a separator to separate the date from the station.
If station number is in first column click next and select station number to be text click finish
else click finish
You should now have the data separated into columns.
Click the cell containing the first date (or the first cell to the left of january’s temperature)
Save the workbook as the next step is not undo-able.
Run the macro above (use keyboard shortcut or go to the developer tab and double click the macro name.
The macro should stop on the last line of data (looks for empty cell). However if it does not press [ctrl]+[break] a number of times. select end or debug from the options according to taste.
No guarantee is given with this software!!!!!
I sail third officer deep sea now and the position, temperature, humidity, and wind as well as sea state are recorded at the end of every 4 hour watch as a matter of law (I speak of the US flagged). All of our logs are required to be retained by law. On one ship I had to launch a bathythermograph every watch for a NOAA guy that rode with us. These results were logged on a computer.
Does anyone know if there has been a large scale attempt to collect all ships logged weather data? They must cover a substantial part of the sea. Wx reporting to NOAA is done on some ships, but this is optional and subject to the Mates patience. Almost 100% of US flagged ships instruments (Barometer, dry/wet bulb thermometer) are calibrated by NOAA every so often. I would guess there is more care taken in the accuracy of these reading because we heavily rely on the resulting Wx maps (Provided by NOAA) to forecast and route.
Basil,
Turning to the balance of your analysis, you can save yourself considerable effort next time when calculating your ‘mean seasonal difference’ statistic. It is not necessary to create a difference series and tally all of its values to do that. Mathematically, your ‘mean seasonal difference’ statistic simplifies to the following equation:
(Te – Ts)/Y
Where:
Ts= Temperature at the start of the analysis period.
Te= Temperature at the end of the analysis period.
Y = number of years in the analysis period.
It should be much faster for you to calc that statistic next time. This does raise the following questions, however.
You have 130 years of data at the Nashville site, which amounts to 1,560 monthly average temperatures. First, you throw out 92% of that data by choosing only to look at October. Then you throw out 98% of that October data, by using a ‘mean seasonal difference’ statistic that is determined by the values of only two of the October monthly averages – the endpoints of the period of analysis.
Why is using only 0.13 % of the data to calculate a temperature trend superior to calculating a temperature trend from a larger portion of the data? Like, say, all of it?
Given that your ‘mean seasonal difference’ statistic only uses two datapoints (the endpoints of your seasonal series) it should be apparent that the choice of those two points is … pretty important. Just moving one of the endpoints of your analysis period forward or backward by one year could dramatically change the ‘mean seasonal difference’ trend that you calaculate.
In fact, on a dataset with essentially zero trend (such as the homogenized GISS dataset for Nashville that shows much less than 0.1C warming over a century) you could completely flop the trend from warming to cooling and back with only tiny moves of the endpoints. And that warming or cooling trend could pretty easily be an order of magnitude larger than the miniscule warming trend calculated from all of the data.
Is that what you mean when you say this method is ‘robust’?
Oldgifford you say:
“I have taken the difference between tmax and tmin for each month and averaged them over the year. ”
It is not surprising that there is a declining trend in the difference between maximum and minimum tempertatures since as far as I know generally in the UK there has been a trend of higher minimum temperatures and this has been higher than the rise in maximum temperatures.