More on the now infamous “moon landing denier“” statistical paper.
Question – how did this title for a scientific paper:
Understanding Statistical Trends
Turn into this?
NASA faked the moon landing|Therefore (Climate) Science is a Hoax: An Anatomy of the Motivated Rejection of Science
Easy. You get ethics approval from your university for the first and use it to push the second. According to UWA rules, Dr. Stephan Lewandowsky was required to obtain approval for his survey from the UWA Ethics Committee. He got that, the process took a week.
At Australian Climate Madness, Simon Turnill obtained information on this process through an FOI, request, which he details here. The FOI released email exchanges are available here.
Steve McIntyre writes about it:
The information showed that Lewandowsky used bait-and-switch. Lewandowsky had obtained approval for a project entitled “Understanding Statistical Trends”. The original proposal had nothing to do with his bizarre online conspiracy theory. Lewandowsky switched the proposal in August 2010.
In addition to Simon’s points, note that Lewandowsky stated the following in his ethics proposal:
Because I am interested in soliciting opinions also from those folks, I would like to withhold my name from the survey as I fear it might contaminate responding”
Nonetheless, Lewandowsky’s name was prominently displayed at some of the anti-skeptic blogs. Lewandowsky’s fears that the survey would be contaminated seem to have been justified.
What is even more interesting, is that when Lewandowsky asked the Ethics Committee for a change to the procedure, switching from a written passed around survey at UWA to one done on the Internet, that approval took only about 18 hours.
Lewandowsky was so surprised at the speed he wrote:
My question now is whether those last minute changes violated some required review procedures. The question is whether or not the changes were at the sole discretion of ethics committee chair Kate Kirk or if they required a wider review. If the latter, I’d FOI the results of that review.
Based on the timeline for the change approval, my suspicion is that Ms. Kirk just waived it through without really looking at it or consulting anyone else. That may or may not be procedurally kosher according to UWA ethics rules.
She seemed flippant in this exchange:
As advertised publicly in this other UWA online survey project:
For any ethical concerns regarding this research project please contact:
Kate Kirk (Kate.Kirk@uwa.edu.au, Ph:08 6488 3703).
I strongly advise any readers against sending hate mail, but instead ask how she allowed herself to be victimized by this apparent bait and switch by professor Lewandowsky and if that 18 hours from request to approval was mostly waiting for Ms. Kirk to read the email in her inbox, or if she actually sent it out to others for review.
Simon Turnhill deserves props for following this through. I advise visiting ACM and reading his full essay, leaving some thanks in the tip jar.
Am in partial agreement with Skiphil’s (October 12, 2012 at 8:30 am ) arguments about the unethical nature of a study with the aim of smearing opponents. It is not just smearing opponents that was the purpose of the study, but “scientifically” finding another reason for contrary views not to be heard.
I believe that it is also unethical
– not to state the small proportion of skeptic responses.
– the small minority of all responses who supported the non-climate conspiracy theories.
– not to include a conspiracy theory or two that the pro-climate science types believe in, such as the role of big oil funding.
– But most of all the misleading claim that scientists are custodians of vastly superior knowledge that is beyond the comprehension of lesser mortals, whilst finding every excuse not to compare and contrast the statistical arguments.
As cognitive psychologists they should know that if you present deliberately misleading results, then when found out you will spread distrust and rejection of their own discipline and climate science.
Denis Kuzara
“Is it possible that a survey as stated in the application was actually done, but the results were not acceptable to the alarmists and was buried? In any case, what happened to the original survey?”
Yes, Lewandowsky did publish a paper based on the in initial survey for which he sought the Ethics Committee approval. The paper is called “Popular consensus: Climate change set to continue”, in which Lewandowsky finds that “presentation of climate data can counter claims that warming has stopped”.
Dennis Kuzara says…. at 11:40am.
……….”No graphs”……………..
I have a feeling that Davis Balestracci- the author of :
Data ‘Sanity’: Statistical Thinking Applied to Everyday Data
Abstract: This publication exposes eight common statistical “traps”. They are: 1. Treating all observed variation in a time series data sequence as special causes, 2. Fitting inappropriate “trend” lines to a time series data sequence, 3. Unnecessary obsession with and incorrect application of the Normal distribution, 4. Incorrect calculation of standard deviation and “sigma” limits, 5. Misreading special cause signals on a control chart, 6. Choosing arbitrary cutoffs for “above” average and “below” average, 7. Improving processes through arbitrary numerical goals and standards, 8. Using statistical techniques on “rolling” or “moving” averages.
Keywords: Process-oriented thinking – Time Series Data – Variance Reduction
http://asq.org/statistics/1998/06/data-sanity-statistical-thinking-applied-to-everyday-data.html?shl=088344#rate
would of been willing to provide a few examples (graphs) of some common traps when using statistics.
Never understood before what ethical commissions are good for. Now I know.
Sorry in advance for this – a tad off topic but it really made me laugh.
I came across this comment in the UK Guardian (I like a joke the same as the next bloke):-
” TurningTide
11 October 2012 5:16PM
Why do we play down the horror of climate change?
Play down?
We’ve had decades of media hype on the subject, tempered only by the fortuitous intervention of climategate.
You’ve cried wolf too often before and no one is listening now.
And before anybody points out there was a wolf in the story of the little boy who cried wolf – yes: there was: and it ate the irritating little sod who kept trying to frighten people without good reason.
…………………………………..
Priceless = totally priceless.
“The question is whether or not the changes were at the sole discretion of ethics committee chair Kate Kirk or if they required a wider review.”
Nope, Mrs Kate Kirk is no “chair”. She is an Administrative Officer. Manager is Mr. Peter Johnstone.
I understand that blink rates may also be proportional to embarrassment or nervousness. The absence or notable paucity of a discernible blink, aka. the reptilian look (minus the nictitating blink), helps convey a look of assurance and certitude. However, when sustained just a smidgin too long, something advised against by those who coach media savvy, it has the effect of engendering disquiet by conveying a sense of derangement, obsession, mania or as it appears, frank delusion. And in the case of those with the ‘vacant stare’, it’s a done deal.
(a) What are the possible benefits of this research?
…
(ii) Better understanding of how people process statistical information , in particular temperature records relating to climate change.
Since nature does not appear to be playing along with the alarmists, a huge effort now seems to be under way to improve ‘climate change communication’ aka propaganda.
Everything these social ‘scientists’ do seems to be aimed at improving the take-up of the propaganda amongst the disinterested, the gullible, the dimwits and the clueless.
What will they do when this doesn’t work either? Labor camps? Climate Gitmo?
I have, of course, assumed that skeptics are not prone to propaganda and therefore do not fall into any of the 4 categories I quoted above!
Radio Free Delingpole XXII: Fighting Windmills
We talk about FOI – an invention of the statist left which has backfired horribly because it’s now being used by the libertarian right to expose ugly truths the political class would rather keep hidden from us.
http://blogs.telegraph.co.uk/news/jamesdelingpole/100184632/radio-free-delingpole-xxii-fighting-windmills/
kakatoa says:
October 12, 2012 at 12:32 pm
Nice. However, I would hazard the ‘eight common statistical “traps”. ‘ are 10. If agreement can be reached that #6 and #8 are, in actuality not dual.
I have had a good laugh out of this thread and I needed it. WUWT is the roxor.
Dear Kate,
If the FOI emails displayed at WUWT are genuine, it would appear that at best you have been wrong footed by
an unscrupulous ‘activist’ into what can only be described as an unethical stance.
I suggest that in future you scrutinise any applications submitted my Mr Lewandowsky more carefully to avoid
appearing complicit in his ‘un-scientific’ and sometimes incendiary schemes.
The whole affair reflects poorly on UWA.
Yours Sincerely
Charles.
Either you take a survey or you don’t.
To try to avoid certain sectors it might reach, invalidates it.
Pointman, Duster and Manfred….what I noticed was his eye movement at the precise moment he declared his absolute belief in CAGW….as though on some very deep level he didn’t actually believe in it himself!
In a paper by Brian Martin, ‘Scientific Fraud and the Power Structure of Science’, written in 1992 he stated:
“Fraud is what scientists tell each other is fraud.
http://www.bmartin.cc/pubs/92prom.html#fn0
“My general answer is that the social definition of fraud is one which is convenient to most of the powerful groups associated with science.
“One of the most common misrepresentations in scientific work is the scientific paper itself. It presents a mythical reconstruction of what actually happened. All of what are in retrospect mistaken ideas, badly designed experiments and incorrect calculations are omitted.
“Another common misrepresentation of research work is exaggeration of its quality, progress and social importance….A modest and honest grant application stands little chance of success: the applicant, to obtain money, must puff up the quality and importance of previous work and give a highly unrealistic assessment of the likely results….
“Honesty in research grants, annual reports and media reports stands about as much chance as honesty in advertising, because this sort of misrepresentation of science is, indeed, a form of advertising.
“Misrepresentation is also common in the curriculum vitae, the formal record of a scholar’s career. ‘Creative curriculum vitae writing’ is a fine art: minor honours are inflated, administrative duties are exaggerated, major credit is claimed for collaborative research, and every possible publication is listed (perhaps including duplicate conference papers and ‘in press’ papers that have not yet been accepted or even submitted for publication). Most of all, failings are omitted from the vitae.
“ ‘Shoddy science’ includes things such as poor experimental design, bungled statistics, incomplete data sheets, improperly tested hypotheses, inaccurate reference to previous work, uncorrected minor mistakes in computer programmes, failure to test alternative hypotheses, and conclusions that do not reflect the body of work. Shoddy science is widespread. Lots of it gets into scientific journals, and much more is rejected. But an occasional rejection is about the only penalty for poor work. More common is the reward of promotion for producing so much of it.
“An eminent behavioural scientist, who must remain anonymous, wrote the following:
“I can however assure you that I have had more than one graduate student who has subsequently become eminent and who I know was fudging data….In addition I know of several people in biology and psychology whose results cannot be replicated, who refuse to give access to their raw data….but universities tend to protect them because employing fraudulent staff is not good for the image of the university. …”
“………science is ruled by an oligarchy of mediocrity: in the chaos of fashionable but pointless research done by less-than-competent researchers, cheating can escape unnoticed.
“Also tolerated are biased viewpoints, including those linked to powerful vested interests. Many scientists are employed by or receive research funds from companies or government bodies, and both expect and are expected to come up only with results useful to those bodies.
“It is almost always other scientists who are most aware of the cheating that goes on………but most don’t want to undergo the personal confrontation involved in making allegations of fraud. Administrators often are reluctant to raise the matter too widely since that would hurt the reputation of their institution.
“All this suggests that the priority is on limiting not fraud but damage to the reputations of the institutions concerned.
“In protesting scientific fraud, the whistleblower soon realizes that he or she will have few allies…..Its response is sufficiently savage to make whistleblowing professionally suicidal…..
“Structural changes that would affect the level of misrepresentation and bias in science include reducing the power of scientific elites, untying the link between quantity of publication and career advancement and reducing the impact of government and industry funding on science..”
The paper is well worth reading as it is entirely relevant to the CAGW activist branch of climate science and the parasites from other pseudo-subjects who feed at the trough provided by taxpayers in collusion with dishonest politicians.
http://www.bmartin.cc/pubs/92prom.html#fn0
On the subject of blinking when under stress, for whatever reason, look at Clinton before the Grand Jury:
@Charles Gerard Nelson.
It’s not their eyes you watch to see if they’re lying. After they’ve delivered the lie, leave a little pause and you’ll see the ghost of a smile. It’s actually a suppressed smirk.
Pointman
The reason one becomes an ethics professor is to learn how to do unethical things in an ethical way.
The description of the ethics application is as such variance with the content of the
Pape r that the paper has to be retracted. When it comes to this, there is no
Wiggle room. On submission to journal, authors w ould have been asked to declare
They had appropriate ethics clearance. Without that they cannot publish.
Furthermore, this is serious misconduct, you absolutely should be scrupulously factual
In this. The uwa chair of ethics should be asked to write to journal giving them direction
To withdraw paper. If the chair does not, then the uwa is effectively condoning
Violations of their own and national guidelines for human research.
Yes this work was using grant money – from the paper:
And from Lewandowsky’s CV:
Further – the ethics application sates there was no grant funding involved.
This was potentially correct at the time of the initial application, however as noted above, this clearly changed and was acknowledged in the paper.
Email sent to Ms Kirk, which included a parenthetical suggestion: “keep track of the number of legitimate questions you receive resulting from Watts’ blog and actual hate mail. It would be nice to see an irrefutable tally of each.”
Since she is the one who brought up the ‘hate mail’ thing on her own, I’m guessing in a nod to the big ‘Aussie death threats that never were’ controversy……. http://wattsupwiththat.com/2012/05/11/death-threats-against-climate-scientists-story-deader-still/
And more – the application shows the “Conspiracy” questions were using a 9 part Likert scale and the life satisfaction questions using a 7 part Likert scale. The Heath and Gifford free market questions are noted in a summary but are not shown – either he did not include them in the application or the FOI did not provide them – the Likert scale is not noted for them. .
It is quite clear Lewandowsky created and performed an entirely new project, albeit using some of the same questions. Despite a wholly different goal – to question “denizens” of climate blogs thru an online survey (as opposed to the general public thru a manually administered in person form) – he did not submit a new application but rather bootstrapped this wholly new project under the prior approval.
IMO this adds new insight into the comments at that time from John Cook at SKS about working with Lewandowsky on a survey. This appears to provide evidence that they potentially “Cook”ed up this follow-on work between them.
posted this at CA but probably worth repeating here:
The original application was for a simple 5 minute in person process involving the participant looking at a handful of graphs and them making predictions from them. Further, the application indicated it would target the public at large including UWA people, and would be completed in person.
This original project had the stated goal:
The completed work noted above may be similar, but according to Lewandowsky’s amendment request was not likely related to this application.
In the amendment request Lewandowsky submitted a completely different proposal.
They dropped the graphs altogether, and substituted new questions – on conspiracy theories, free market, and life satisfaction – identified in his request as “H&G&Kahan,” “BCTI” and “Happ&Sat” … which he notes are attached.
However, that is not what was “attached” – the conspiracy questions from Swami 2009 are not mentioned in his amendment letter, nor are by all appearances the Heath and Gould free market questions.
They also now intended to target “denizens” of climate science blogs, and used an online survey to collect their responses.
He characterized his request as an “amendment” to the earlier work yet this new proposed work bears no relation to the scope or purpose of the originally described work.
Further, although he includes apparently the conspiracy questions he makes no mention of them – ridiculously claiming the “survey will only be modified “slightly” – by removal of the core original “graphical extrapolation task” and replacing pretty much completely the survey questions.
Most importantly he makes no mention that the focus of the paper, as we now well know, had changed dramatically … from “Understanding Statistical Trends” to showing an association between conspiracy ideation and motivated rejection of science – associating climate skepticism with the Moon Land is a Hoax conspiracy theory..
He asserts … despite that:
(a.) the original premise,
(b.) the original questions, (c.) the original sample group
(d.) the original sample methods
… had all changed significantly … that “in all other respects, the approved procedure remains unchanged”
Sorry – I don’t see a single material part of the original work that was approved remains in the “amended” project. I think characterizing this “amendment” as fraudulent is entirely warranted by the evidence.
I would also note that those who defended Lewandowsky – claiming he was not advocating for the “cause” (as opposed to advocating his professional work) are proven wrong – by Lewandowsky’s own admission – stating he wanted permission to withhold his name from the survey details, giving the following rationale:
There is no better admission of guilt, or indictment, of his harsh and vitriolic rhetoric – his outright partisan cause advocacy – than his own admission and words, and his noting the results.
This is a perfect example why scientists when I grew up were expected to avoid advocacy on positions related to their research. You cannot be an advocate and the dispassionate researcher and relator of science both.
That the ethics office jokes about this shows how deeply flawed the system as it stands has become.
Any research without a proper ethics application cannot be published. The occasional
Ph.D. thesis sometimes have to strip work out (and be totally gutted and therefore no PhD)
by failure to get proper ethics clearance.
Ms Kirk should be advised that the UWA has totally failed in its compliance duties and
at a minimum she should immediately advise the chair of the UWA ethics committee
who should very quickly come back with a recommendation that the paper be withdrawn.
The diligence applied by Ms Kirk in this matter was less than satisfactory so she
may have some questions to answer.
There will probably be a signed declaration at the journal that the work had suitable ethics
clearance. This probably constitutes serious misconduct and/or academic fraud.
Signoffs regarding ethics are very serious documents.
If a funding body is named, then correspondence to the funding body that the
named grant be withdrawn would be appropriate due to violations of NHMRC
national guidelines.
In the scheme of things, the issue of whether agenda driven poor research was
done incomptently pails into insignificance in comparison with an ethics application.
that does not properly describe, the intent and methodology of a research project
involving human subjects.