This is an excerpt of a larger post that deserves the wider attention WUWT can bring to it. The image at left is from my article: A look at treemometers and tree ring growth – Anthony
Screening: Now with good+bad treenometers.
Written by: Lucia Liljegren
In my recent posts, I discussed how screening treenometers (or any proxy) based on correlation with temperature during the calibration period can screwup bias a proxy reconstruction in ways that deceive the analyst who is either unaware of or refuses to believe in the types of biases that screening introduces. With respect to blog spats about AGW, the short unnuanced interpretations of the two limits of what screening can do are:
- “If you screen, a batch of treenometers that do contain some signal, you can exaggerate a hockey stick.”
- “If you screen, treenometers that contain no signal at all, you can create a hockey stick from trendless data.” and
Both issues clearly make it difficult to decree that any hockey stick contained in a reconstruction where the specific proxies were selected by screening with correlation.
(Note: What I mean by screening here is “peeking” at the data to decided what to toss out. It’s ok to believe certain conditions result in high correlation, decree you will go out and collect trees from pre-designated sites, and then use all tree cores from all sites. What you must not do is collect the trees and afterwards toss out any tree cores or sites based on correlation with temperature during the calibration period.)
Naturally, based on my two extremes, people whose intuition says there must be some advantage to screening want to know two things:
- Suppose you pick sites, and you were somewhat successful, but some of the treenometer were temperature sensitive and others weren’t.
- Couldn’t you figure out how to “improve” results from a batch of “good” treenometer and bad ones?
Today, I’m going to what your results would look like in two cases. In one, by picking sites, you got batch of “quite good treenometers” mixed with a batch of “not treenometers at all” and in another, you got “adequate treenometers” mixed in with “not treenomters at all”
Read the post here in entirety, well worth your time. The results speak for themselves.
“not treenomters at all”
You can’t get temps from trees….the most you can get is the length of a growing season
Good to see the wider attention, but hope the discussion there doesn’t get drowned in the noise. It would be good to persuade some of the more numerate of the warmers to try and understand what this means – it is a fairly subtle point for anyone not used to dealing with real data!
It appears that there is a need for stats primer for climate researchers.
Is there one available?
WUWT is doing a great job in educating the public in how stats can be made to lie (deliberately or by accident).
It would be a good idea however to have available (by link maybe) to a primer on stats with special attention to time series data and how to avoid all of the key errors that people currently make.
Lucia, you are being a very unreasonable person. Gergis is no match for you. Don’t be a bully ;))
If A, then hockey sticks are exaggerated, if B, then hockey sticks are meaningless, therefore if either A or B, hockey sticks are uninstructive. Is that where we are?
Steve Richards said @ June 19, 2012 at 1:27 pm
William Briggs:
http://wmbriggs.com/blog/
I have been trying to get to the website “rankexploits” for a few days and get the Access denied message. I have never visited it before so I dont see how I could have been kicked off it.
Anyone have any idea why this might be happening?
Plain Jane says:
June 19, 2012 at 2:51 pm
I get the same. I live in Thailand and my IP adress is blocked by her website and I get a 403 error message. You should be able to follow a link to send her an e-mail and she will give you access.
I haven’t got round to e-mailing her yet but I will. Lucia is one very very smart cookie!
Jane I can’t access Lucia’s site anymore, at the moment I’m getting a scary screen 403 denied access page.
Lucia has tried to help me but so far it doesn’t work. I often looked in to read articles and read the comments as I do here, but a few weeks ago everything changed.
I think I’ll have to give it up and get reports of some of her articles from reports from other sites.
It seems to me that fundraising is required to try and find a cure for green dementia, the late 20th and early 21st century illness that warmists suffer from.
The problem with tree rings aside from the fact that every environmental factor changes how much growth occurs in a given season, and they ignore the winter almost entirely. Is that warmer temperature improves growth to a point and then it hinders it. So the very phenomena claimed to be observed looks exactly the same as it being too cold too dry or too wet.
■“If you screen, a batch of treenometers that do contain some signal, you can exaggerate a hockey stick.”
■“If you screen, treenometers that contain no signal at all, you can create a hockey stick from trendless data.” and
There should be no comma after “screen”.
Steve Richards asks: “It would be a good idea however to have available (by link maybe) to a primer on stats with special attention to time series data and how to avoid all of the key errors that people currently make.”
At a basic level, here’s a good book to have on your shelf:
http://www.amazon.co.uk/How-Lie-Statistics-Penguin-Business/dp/0140136290/ref=sr_1_1?s=books&ie=UTF8&qid=1340153107&sr=1-1
Info for those having trouble accessing Lucia’s website: she has had a lot of trouble with bots trawling her site and gobbling all the bandwidth. As a result, she had to implement some fairly draconian responses, including completely blocking servers used by the bots. If you email her, she will try to get your site access back.
“Today, I’m going to what your results would look like in two cases.”
What?
You’ve Slashdotted Lucia’s site Anthony. ( verb, to Slashdot)
In the light of the recent upsurge in Hockey Stick slaying — motivated by Gergis et al — it will be interesting to see how they (Gergis) play their hand with regard to their held paper. Recent posts at ClimateAudit and now Lucia’s have provided the gritty detail on why correlation screening will produce a flawed result, though that isn’t why Gergis was withdrawn. Will they risk a second exceptionally embarrassing forced withdrawal?
For those with problems accessing Lucia’s blog; that blog is protected by some (necessary) half-bakery to prevent site scraping – scraping is a large traffic problem and can cause sites to crash if they are unable to protect themselves.
(I’m being deliberately obtuse because I don’t want the real bad guys to implement the work-around)
Blackboard protection depends at least in part on the browser that is being used and historically on the IP address from which it has been used. There are certain user agent identity elements which Lucia identifies as being “significant” and therefore screens on them, based on the belief that they are characteristic of “scrapers”. It is most-likely that certain scrapers are “imposters” by employing such elements in their user agent identification when accessing the site. If you use a browser that legitimately identifies itself with elements preferred by the imposters, then you are assumed to be “hazardous”. And your IP address gets black-holed for a number of days.
All guys wearing black hats are bad guys.
If your activity has been classified as hazardous, change the hat. If your web browser doesn’t directly provide that function consider an add-on like “User Agent Switcher”. You’ll definitely need it if you’re using un renard feu out of the box, in a popular Linux distribution from Nuremberg.
Hosting providers MUST find ways of providing better control to webmasters to detect and inhibit scraping. (Similarly; crawling.) Web servers such as Apache are capable of detecting “suspicious” clients crawling/scraping. Unfortunately, the providers don’t automatically provide the hooks/tuneables for their paying webmasters to make good use of those capabilities.
DavidA
Dreamhost was down. It wasn’t just my blog. Anthony does send lots of traffic, but that’s not the cause of my site crashing. (At least the bots aren’t making it crash 12 hours on an off every day like it did back in Nov. March.!)
Neville– I’m posted asking people to suggest a legitimate reason your browser is asking for non-existant files and presenting cookies my site doesn’t set. Maybe someone will know– but that’s why you are being blocked.
Plane Jane, I have no idea why you are blocked. I would at a minimum need to know your IP and roughly when you were blocked.
Hi Lucia
I tried to get on your site sometime earlier in the last fortnight and got a similar message to today, as follows:
Access Denied
The owner of this website (rankexploits.com) has banned your IP address (59.154.42.139). (Ref. 1006)
Timestamp: Tue, 19 Jun 2012 22:15:05 -0700
Your IP address: 59.154.42.139
Requested URL: rankexploits.com/
Error reference number: 1006
Server ID: FL_23F5
Process ID: PID_1340169305.363-1-13912214
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2
My browser is safari and I also tried with firefox and got the same.
Thanks
I can access now. I used to visit before without any problem but today for the first time got a list of errors to chose from, one of which was server overload. There might be more than one problem.
My nan hangs her t-bags out on the line to make ’em last, she has commented that it is taking longer to dry them out, using special proxy powers I have extrapolated that the weather in the UK has been getting wetter over the past few months.
/Sarc.
If they intend to re-submit a revised paper then my sincere advice would be to send it to Steve McIntyre first. The reason is that Steve will certainly take a look at any re-submitted paper after publication.
If real (climate) science is to move ahead it needs genuine peer review. The peer review process is intended to try and keep out bad science / mistakes etc.Sadly, WUWT / Climate Audit etc. has shown that in many cases it’s Pal Review at work and this can’t be good for science.
@Shevva
If your nans teabags are too wet, I’d better buy a Prius !
Plain Jane says:
June 19, 2012 at 2:51 pm
I have been trying to get to the website “rankexploits” for a few days and get the Access denied message. I have never visited it before so I dont see how I could have been kicked off it.
Anyone have any idea why this might be happening?
___________________________
The first try I was blocked using this: http://rankexploits.com/
then I tried this: http://rankexploits.com/musings/
And the website came up.
Dreamhost was down last night. They’ve been shifting servers. That’s got nothing to do with my screening. But back before I wrote my “rules”, people got that intermittently for a total of about 6 hours a day for months. That would happen because ‘bots were hammering my site. (This was evident from my server logs. It’s not just my imagination.)
So… some of my rules are draconian. Some might even be wrong. But right now, I err on the side of blocking stuff that looks suspicious even though all sorts of people lecture me that I ought to do the opposite. But if I do as they say, my site crashes.
bill says:
June 19, 2012 at 2:17 pm
If A, then hockey sticks are exaggerated, if B, then hockey sticks are meaningless, therefore if either A or B, hockey sticks are uninstructive. Is that where we are?
It’s more like “When even random numbers give you a hockey stick, it’s time to revisit your code.”
Unless, of course, it was your intent to produce a hockey stick in the first place…
Plain Jane–
The block you saw was at cloudflare. That means someone with your IP broke ‘a rule’ that is set locally by my blocking software and that got escalated. (They all get escalated to spare my server.)
Your IP is now unblocked at cloudflare. But if something on that IP breaks a rule later one, it will get blocked again. Its easier for me to detect why the IP got blocked if someone communicates to me sooner rather than later.
I don’t mind your discussing blocking here. But Anthony might prefer his blog comments to be about climate change. So… if you really want to visit my blog it would be better to email me. My email address is easy to remember “lucia” at “the domain name for my blog”. If you know my blogs domain name (i.e. rankexploits.com) you know my email address!
To the tune of:
Raindrops Keep Falling On My Head
Hockey sticks are endin’ up in shreds
And just like the Mann whose ego’s too big for his head
Nothin’ seems to fit
Those hockey sticks are endin’ up in shreds, they keep shredin’
Hansen did him some talkin’ to the sun
And he said he didn’t like the way he’ got things done
Heatin’ up the globe
Those hockey sticks are endin’ up in shreds, they keep shredin’
But there’s one thing I know
CO2 he says will heat me won’t defeat me
It won’t be long till honest ones step up to greet me
Hockey sticks are endin’ up in shreds
And that does mean my world will soon be turnin’ red
Watermelon’s not for me
Cause he’s never gonna stop the sane by complainin’
Because I’m free
CO2’s not worrying me.
OOPS! Typo!
“And that does mean my world will soon be turnin’ red”
Should be “And that mean’s my world won’t soon be turnin’ red”