@nytimes promotes an “eff you level” of irreproducible science

From the website: “Statistical Modeling, Causal Inference, and Social Science” by Andrew Gelman. h/t to Willie Soon

So the NYT yesterday has a story about this study I am directed to it and am immediately concerned about all the things that make this study somewhat dubious. Forking paths in the definition of the independent variable, sample selection in who wore the accelerometers, ignorance of the undoubtedly huge importance of interactions in the controls, etc, etc. blah blah blah.

But I am astonished at the bald statement at the start of the study: “The data, analytic methods, and study materials will not be made available to other researchers for purposes of reproducing the results or replicating the procedure.”

Why shouldn’t everyone, including the NYT, stop reading right there? How does a journal accept the article? The dataset itself is public and they didn’t create it! They’re just saying Fuck You.

I was, like, Really? So I followed the link. And, indeed, here it is: (screencap from the open access PDF)

The Journal of the American Heart Association published this? And the New York Times promoted it?


Complete, utter, failure on the part of the NYT the NIH, and AHA.

This is even worse than the famous line written by climate researcher Phil Jones:

“We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it. ”

This heart study should be published in The Journal of Irreproducible Results

0 0 votes
Article Rating
50 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
October 19, 2018 11:24 am

Some (lots and lots) of people at NIH should be assigned to studying the health of gangbangers in Chicago, if it is not possible to send them to Iraq.

john
Reply to  Tom Halla
October 19, 2018 2:54 pm

Better yet, send them to test their patience!

Latitude
October 19, 2018 11:24 am

….we’re in the new Ford/Kavanaugh era

Reply to  Latitude
October 19, 2018 4:40 pm

We believe tree ring proxy survivors!

Mr.
October 19, 2018 11:26 am

As a person of advanced years who has marveled at the capacity of mankind to advance knowledge through (mostly) honest scientific endeavours just in the course of my lifetime, I am appalled at where things are now.
My reaction can be summed up thus –

Joel Snider
October 19, 2018 12:09 pm

Why should their science reporting be any different than the rest of their content?

Cube
Reply to  Joel Snider
October 19, 2018 7:26 pm

Exactly. The NYT has published nothing but lies for fifty years. Why should this article be an exception?

Reply to  Cube
October 19, 2018 8:26 pm

The Times has had problems with reality since it’s founding apparently. Walter Duranty’s reporting on the Ukrainian “famine” was a classic NYT series.

October 19, 2018 12:10 pm

National Geographic:
It would be far worse than expected
https://youtu.be/EFmPZv7LIdo

rbabcock
Reply to  vukcevic
October 19, 2018 12:34 pm

Ha ! The consequences of no moon and runaway global warming seem eerily similar ..

TonyL
Reply to  vukcevic
October 19, 2018 2:46 pm

What a stunningly bad bunch of BS. So bad, you could mistake it for a Halloween story mad up to frighten little children.

@ rbabcock:
I noticed the same thing.

Robert of Ottawa
Reply to  vukcevic
October 19, 2018 3:20 pm

But what of the consequences of no SUN? Would the planet cease to be warm?

Nat Geo was goo when I was 11 years old, or at least it seemed to be to me at age 11. Rather like the Reader’s Digest.

Robert of Ottawa
Reply to  Robert of Ottawa
October 19, 2018 3:20 pm

Nat Geo was gooD when …

nw sage
Reply to  Robert of Ottawa
October 19, 2018 6:08 pm

I like goo better. Keep it!

Writing Observer
Reply to  nw sage
October 19, 2018 9:50 pm

Both. Was good, and is now goo.

Pop Piasa
Reply to  Robert of Ottawa
October 20, 2018 11:28 am

National Geographic got me in trouble in the third grade. My parents moved from an all white farmtown to an integrated larger city that year. My first day I was on the playground, curious about the black kids that were so new to me. I saw a very pretty second grader girl with pierced ears and large hoops in them. Trying to impress her with my knowledge gained from NG, I announced “I know what you are, you’re a cannibal! I then did my best native dance for her. She ran off crying, and after recess I got called into the hall by her teacher, also a black person. After her pointing out to me that I had hurt her feelings and catching me up to 1966 racial etiquette, I decided that I would rather read Popular Mechanix and MAD Magazine…

Mark
Reply to  Pop Piasa
October 21, 2018 3:36 am

With that legacy, you can never run for public office or be a Justice.

eyesonu
Reply to  vukcevic
October 19, 2018 8:50 pm

I lost the moon last week but part of it has returned so maybe it was just hiding in the deep ocean.

Rich Davis
Reply to  eyesonu
October 20, 2018 9:07 am

It has already recovered more than half its size, eyesonu. The recent growth has been unprecedented since we began observations more than 10 days ago. All responsible moon scientists (lunatics) agree that in just a few years the entire sky will be filled with moon.

Pop Piasa
Reply to  Rich Davis
October 20, 2018 11:39 am

😒🌛⚠💲❗
(blog mime)

ResourceGuy
October 19, 2018 12:35 pm

Technology and the speed of information has made the 80:20 Rule obsolete. It is fast evolving into the 90:10 Rule with only 10 percent doing the real and credible work and the rest operating in a fast and loose environment of lowered standards and lagging efforts to fact check if ever.

john
Reply to  ResourceGuy
October 19, 2018 2:58 pm

BS/FACTS = >9.0

Juan Slayton
Reply to  john
October 19, 2018 4:44 pm

Beware of dividing by zero.

Red94ViperRT10
Reply to  ResourceGuy
October 19, 2018 9:12 pm

It is fast evolving into the 90:10 Rule

More like 97:3. 😀

Admin
October 19, 2018 1:11 pm

Post normal science – publishing data or method might endanger your grant flow by revealing your incompetence. Keep it all secret at all costs.

Don
Reply to  Eric Worrall
October 19, 2018 1:36 pm

Job security through obscurity, eh?

Admin
Reply to  Don
October 19, 2018 2:57 pm

Ha 🙂

john
Reply to  Eric Worrall
October 19, 2018 3:00 pm

I recommend sending them their grant cheque without the bank transfer numbers.

Danley Wolfe
October 19, 2018 1:31 pm

The cascades of propaganda from the climate change mandate from the United Nations / IPCC is truly jaw dropping. The majority of the reporting in the press and especially the likes of the NYT is utterly worthless and just propaganda … spreading the word on the virtues of worshiping the gods of climate change. And a lot of it is for the cause of anti-establishmentarianism.

Pop Piasa
Reply to  Danley Wolfe
October 20, 2018 11:55 am

So, we skeptic sorts should be disestablishmentarian and seek to sever the ties between their religion and the government.

Rich Davis
Reply to  Pop Piasa
October 20, 2018 2:22 pm

Never thought I’d see the day that I have to give up my antidisestablishmentarianism
🙂

ken Mitchell
October 19, 2018 1:48 pm

Isn’t independent reproduction of results sort of the DEFINITION of “science”?

LdB
Reply to  ken Mitchell
October 19, 2018 7:47 pm

It is the study above is a PR or propaganda pamphlet.

M Courtney
October 19, 2018 1:48 pm

This is the second time today that the urge to say ,b>”I blame the journal” has been so strong that I have.

The journals are choosing content that they think they can sell on to more popular media in order to boost advertising revenue. They are not choosing content that has any scientific content.

It’s probably the fault of Facebook, indirectly.

john
Reply to  M Courtney
October 19, 2018 3:03 pm

That is something I hadn’t thought of. An interesting and rational probability. Also, perhaps an access point to get media outlets to swear off of this as certainly an unsupportable practice. Who calls out the media for their BS? Don’t they have any self governing body?

william Johnston
Reply to  john
October 19, 2018 6:52 pm

I believe it is called freedom of the press. Evidently there are no responsibilities that now go with same.

Dave N
Reply to  john
October 19, 2018 6:54 pm

“Peer review” = “clicking like”?

October 19, 2018 3:15 pm

It feels like the “not” was a typo. I emailed the lead author and hope that he will either confirm or correct.

MikeN
Reply to  Curious George
October 19, 2018 5:24 pm

My reaction as well.

Reply to  Curious George
October 19, 2018 6:56 pm

There is some explanation here. It isn’t a typo, but results from the Journal adopting a requirement that authors specifically answer whether they will supply data. The awkward wording is attributable to that rigidity. The authors apparently argue that the data is bulky, not theirs, and publicly available, so people should get it from source.

LdB
Reply to  Nick Stokes
October 19, 2018 7:58 pm

The old it’s too hard defense, it is a complete rubbish they had to collate the results to do the study. This is why there are so many fraud papers in the medical and pharma areas because they don’t have to prove the link and aren’t held to full disclosure. Another journal helping push a field rapidly into pseudoscience.

dodgy geezer
Reply to  Nick Stokes
October 19, 2018 9:49 pm

Thanks for a useful comment!

hunter
October 19, 2018 11:31 pm

The real oligarchs are in the West, not Russia.
The US oligarchy has trashed our media, and is working hard on the rest.
The Amazon Post, better kniwn as The Bezos Sockpuppet, leads the way lately.
But the NYT, toadying up to corrupt lefties for nearly a century, has set a bar that is hard to beat.

Steven Mosher
October 20, 2018 3:13 am

Nick’

“There is some explanation here. It isn’t a typo, but results from the Journal adopting a requirement that authors specifically answer whether they will supply data. The awkward wording is attributable to that rigidity. The authors apparently argue that the data is bulky, not theirs, and publicly available, so people should get it from source.”

I get that the authors believe that pointing to the source is good enough.

However, on more than one occasion I have witnessed folks who bungled a download job.

So author A downloads data from source B. and points to source B as the data provider.

I go to get the data.

1. The data may have changed.
2. Author A may have bungled the data
3. I may bungle data.

That’s why I consistently ask for the data AS USED and code AS RUN

Alternatively I suppose they could supply a hash of the data, but then we would need

A) hash of the data as archived
B) hash of the data as used

And then I compare my hash after download.

Hard drives are cheap, cloud storage is cheap.

I dont see why journals dont provide Archives so authors can store their data.. yes yes there will be some monster DBs.. rather make an exception for super large files and establish a good rule

LdB
Reply to  Steven Mosher
October 20, 2018 9:37 am

The problem with pointing at the source means they might not have done the study at all AKA it’s fraud. You can’t even do a cursory check without repeating all the work so why did they bother to write the paper in the first place, it’s junk from there and you certainly can’t trust it?

The whole burden of proof has been reversed in some perverse way and you can make stupid claims without ever having to show any thread of any real work. Imagine you ran the test and got a different result you have given blatant fraud an out they just say they made an honest mistake.

There is considerable motives in many fields for fraud the same as there is with sports and athletics and you have seen what happens the moment you have poor oversight. Climate Science may enjoy the loose and fast rules but I would encourage any other field to go the zero tolerance, nuke and burn path.

Retired_Engineer_Jim
Reply to  Steven Mosher
October 20, 2018 4:40 pm

Thanks, Steve – very rational.

Clyde Spencer
Reply to  Steven Mosher
October 24, 2018 11:00 am

Mosher,
+1

Sheri
October 20, 2018 10:05 am

I stopped trusting medical journals a long ways back. The medical profession is as political and greed driven as any other “science” of today. Just check out the “lawsuits” (aka make the frigging lawyer rich suits) for all kinds of crap that science backs. Look at the advertising for “miracle drugs” that aren’t. It’s useless to trust any research without reading it all the way through. Sorry, but we live in the Age of Lies and trust should be very, very limited.

Pop Piasa
Reply to  Sheri
October 20, 2018 12:16 pm

IMHO, skillful deception has become a necessary tool in all politically infiltrated aspects of society.

Frank
October 21, 2018 12:00 am

Andy: If you read on about the study population, it says:

“NHANES collects data on various health and behavior indicators, including physical activity and self-reported diagnosis of prevalent health conditions such as diabetes mellitus, coro- nary artery disease, stroke, and cancer. These data were merged with mortality records from the National Death Index, available through December 2011. The National Center for Health Statistics links mortality records available through the National Death Index with individuals in the respective NHANES database using various individual information such as social security number, first name, month of birth, sex, race, and others.9 The matching process/linkage provides vital status for each individual that has been shown to accurately identify 87% to 97% of the true death records and is fairly accurate across race/ethnicities.10–12”

https://www.ahajournals.org/doi/pdf/10.1161/JAHA.117.007678

In medical studies, it is totally unethical to publish information about named patients without their consent. In the process of their work, these researchers were able to combine data from several different sources and identify individual patients even though those sources had not provided them with patient names. Those researchers, of course, are not going to provide data that identifies individual patients without their information nor clearly explain how others could do the same thing.

The same problem exists in other field: economists, criminologists and others may be given access to government databases (say the IRS) if they promise to release raw data that might identify individuals.

In both cases, someone wishing to reproduce the author’s work can request access to the same confidential data and attempt to repeat the entire analysis without the benefit of working with exactly the same raw data set as the original authors.

%d bloggers like this: