From Lockdown Sceptics
Stay sane. Protect the economy. Save livelihoods.
An experienced senior software engineer, Sue Denim, has written a devastating review of Dr. Neil Ferguson’s Imperial college epidemiological model that set the world on a our current lock down course of action.
She appears quite qualified.
My background. I wrote software for 30 years. I worked at Google between 2006 and 2014, where I was a senior software engineer working on Maps, Gmail and account security. I spent the last five years at a US/UK firm where I designed the company’s database product, amongst other jobs and projects.
She explains how the code she reviewed isn’t actually Ferguson’s but instead a modified version from a team trying to clean it up in a face saving measure.
The code. It isn’t the code Ferguson ran to produce his famous Report 9. What’s been released on GitHub is a heavily modified derivative of it, after having been upgraded for over a month by a team from Microsoft and others. This revised codebase is split into multiple files for legibility and written in C++, whereas the original program was “a single 15,000 line file that had been worked on for a decade” (this is considered extremely poor practice).
She then discusses a fascinating aspect of this model. You never know what you’ll get!
Non-deterministic outputs. Due to bugs, the code can produce very different results given identical inputs. They routinely act as if this is unimportant.
This problem makes the code unusable for scientific purposes, given that a key part of the scientific method is the ability to replicate results. Without replication, the findings might not be real at all – as the field of psychology has been finding out to its cost. Even if their original code was released, it’s apparent that the same numbers as in Report 9 might not come out of it.
Ms. Denim elaborates on this “feature” quite a bit. It’s quite hilarious when you read the complete article.
Imperial are trying to have their cake and eat it. Reports of random results are dismissed with responses like “that’s not a problem, just run it a lot of times and take the average”, but at the same time, they’re fixing such bugs when they find them. They know their code can’t withstand scrutiny, so they hid it until professionals had a chance to fix it, but the damage from over a decade of amateur hobby programming is so extensive that even Microsoft were unable to make it run right.
Readers may be familiar with the averaging of outputs of climate model outputs in Climate Science, where it’s known as the ensemble mean. Or those cases where it’s assumed that errors all average out, as in certain temperature records.
Denim goes on to describe a lack of regression testing, or any testing, undocumented equations, and the ongoing addition of new features in bug infested code.
Denim’s final conclusions are devastating.
Conclusions. All papers based on this code should be retracted immediately. Imperial’s modelling efforts should be reset with a new team that isn’t under Professor Ferguson, and which has a commitment to replicable results with published code from day one.
On a personal level, I’d go further and suggest that all academic epidemiology be defunded. This sort of work is best done by the insurance sector. Insurers employ modellers and data scientists, but also employ managers whose job is to decide whether a model is accurate enough for real world usage and professional software engineers to ensure model software is properly tested, understandable and so on. Academic efforts don’t have these people, and the results speak for themselves.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
7 May: Politico: What COVID-19 scientists can learn from their climate change colleagues
The epidemiologists and virologists helping governments are now thrust into the public eye — which also makes them target
by RICHARD BLACK
(Richard Black is director of the Energy and Climate Intelligence Unit. He was formerly BBCscience and environment correspondent for 12 years)
The late climatologist Stephen Schneider titled his memoirs “Science as a Contact Sport” — and for him and his colleagues, either side of the explosive 2009 U.N. climate summit in Copenhagen, life was exactly that…
And hence the defenestration of Imperial College London’s Neil Ferguson by newspapers whose comment pages speak to an abhorrence for lockdown policies. Newspapers happy to call a scientist whose advice probably saved many thousands of lives “the bonking boffin” and “Professor Lockdown.”…
Climate scientists have been at this a lot longer than their coronavirus peers. So what can the latter usefully glean from the formers’ experience?…
The detractors of climate science no longer have currency anywhere it matters (outside the White House) because their claims, whether “climate change is all natural” or “reducing emissions is economic suicide,” have been clearly shown to be wrong.
COVID-19 science, as Ferguson has just found out, may currently be a contact sport. But the experience of climate science suggests it is a sport that good scientists will eventually win.
https://www.politico.eu/article/coronavirus-what-covid-19-scientists-can-learn-from-their-climate-change-colleagues/
The Covid-19 panic-demic/scam-demic is as fake as climate change.
NY, NJ, MA, IL and CA together have more Covid-19 cases than the ENTIRE rest of the US.
NY, NJ, MI and MA together have more Covid-19 deaths than the ENTIRE rest of the US.
The top ten states have 84% of the cases and 90% of the deaths.
New York state is in sixth place among global deaths. Good job!!
Why?
BLUE states and BLUE cities with BLUE administrations.
Coincidence is cause, is that not correct?
Covid-19 is NOT a US national problem.
The US, Italy, Spain and UK together have more confirmed Covid-19 cases and more Covid-19 deaths than the ENTIRE rest of the world combined.
Covid-19 is NOT a “global” pandemic.
Shutting down the entire US and global economies was driven by the speculative and erroneous theories of “experts” that Covid-19 spread exponentially when the DATA!!!! was clearly second order.
Much like IPCC’s RCP 8.5, a computer model stacked on assumptions with a 0.1% connection to reality.
Then the public and politicians were stampeded to the precautionary principle by a fake news MSM propaganda machine intent on getting Trump.
Remember that come November, unless before then the lying, rabble-rousing, fact free, shit stirring fake news MSM left-wing propaganda machine goes “all in” and deposes Trump by fomenting a coup.
they just released some info
MORE – like 60+% of the NY cases were people who HAD stayed inside at home!
6 May: US Spectator: Neil Ferguson’s remarkable fall from grace
It’s time our tribal chief found a more sensible adviser. This witch doctor, like so many before him, has turned out to have feet of clay
by Toby Young
I originally had Neil Ferguson down as a kind of Henry Kissinger figure. The professor of mathematical biology at Imperial College London seemed to have bewitched successive prime ministers, blinding them with his brilliance. Whenever a health emergency broke out, whether it was mad cow disease or avian flu, there he was, PowerPoint in hand, telling the leaders of the United Kingdom what to do. And they invariably fell into line…
But it turns out to be less a case of Dr Strangelove than Carry On Doctor. On Tuesday night, we discovered that the furrowed-browed scientist, who has been at the Prime Minister’s side throughout this crisis, is in fact Austin Powers in a lab coat…
Truth be told, though, I’m not that bothered about the double-standards. My hope is it will be an emperor’s-new-clothes moment, breaking the spell this Rasputin figure has cast over Boris and the cabinet. God knows, there’s enough evidence that his computer model, which predicted 250,000 would die if the government didn’t place the country under house arrest, is about as reliable as Paul the Octopus.
Exhibit A is the team of scientists at Uppsala University who plugged Ferguson’s numbers into their computer in early April, at a time when many in Stockholm were hoping to frighten their government into imposing a lockdown. Imperial had provided the ammo: if the Swedish authorities continued to pursue its ‘reckless’ mitigation strategy, it was argued, the healthcare system would be overwhelmed 40-fold and approximately 96,000 would die of COVID-19 by the end of the year. By May 1, the model predicted, the death toll would be 40,000. In fact, at the time of writing, Sweden’s death toll from coronavirus is 2,854 and its hospitals are nowhere near the projected collapse.
It’s time our tribal chief found a more sensible adviser. This witch doctor, like so many before him, has turned out to have feet of clay.
https://spectator.us/neil-ferguson-remarkable-fall-grace/
Hey, Paul the Octopus had a much better record!!!!
A Henry Kissinger figure, as in ‘I wonder who’s kiss inger now.”
a new model fit for mass hysteria:
3 Apr: Nature: Special report: The simulations driving the world’s response to COVID-19
How epidemiologists rushed to model the coronavirus pandemic.
by David Adam
An earlier version of the Imperial (College London/Neil Ferguson) model, for instance, estimated that SARS-CoV-2 would be about as severe as influenza in necessitating the hospitalization of those infected. That turned out to be incorrect…
The true performance of simulations in this pandemic might become clear only months or years from now…
“Forecasts made during an outbreak are rarely investigated during or after the event for their accuracy, and only recently have forecasters begun to make results, code, models and data available for retrospective analysis,” (John Edmunds, who is a modeller at the LSHTM) and his team noted last year in a paper6 that assessed the performance of forecasts made in a 2014–15 Ebola outbreak in Sierra Leone…
Media reports have suggested that an update to the Imperial team’s model in early March was a critical factor in jolting the UK government into changing its policy on the pandemic. The researchers initially estimated that 15% of hospital cases would need to be treated in an intensive-care unit (ICU), but then updated that to 30%, a figure used in the first public release of their work on 16 March…
Ferguson says the significance of the model update might have been exaggerated…
https://www.nature.com/articles/d41586-020-01003-6
Ferguson’s old group not lockdown enough for David King & The Guardian?
4 May: Guardian: Public’s trust in science at risk, warns former No 10 adviser
Ex-chief scientific adviser sets up rival panel of experts over Covid-19 ‘lack of transparency’
by Hannah Devlin
Prompted by growing concern about the lack of transparency around the government’s Scientific Advisory Group for Emergencies (Sage), ***Prof Sir David King has convened a panel of experts that he says will act as an independent alternative.
The group, which will broadcast live on YouTube and take evidence from global experts, will hold its first meeting on Monday…
The group, which includes a range of leading scientists working across public health, computer modelling, behavioural science and intensive care medicine, aims to present the government with “robust, unbiased advice”.
King argues that the official Sage is compromised by the fact that 16 of the 23 known members of the committee, including the prime minister’s strategist Dominic Cummings, are employed by government.
The Independent Sage meeting will cover seven areas, including the criteria for lifting lockdown, testing and tracing and quarantine and shielding policies for vulnerable groups. It will formally submit its recommendations to the health and social care select committee, placing pressure on the government to explain the advice behind its lockdown exit strategy, parts of which are expected to be unveiled in the coming week.
King said that the biggest potential pitfall in weeks ahead would be to relax lockdown measures too soon and that he believes the government’s so-called five tests for whether it is safe to ease restrictions are inadequate.
“My own feeling is that the extent to which the virus is still in the population means we are not yet close … to coming out of lockdown,” he said. “Undoubtedly the biggest potential pitfall is removing lockdown too early and too quickly.”
A second peak, he said, could not only increase casualties, but could also lengthen the overall period of time before the country is able to fully exit lockdown. “If you go into a second peak, it just becomes more and more difficult to end the pandemic,” he added.
King previously held the chief scientific role, now occupied by Sir Patrick Vallance, between 2000 and 2007 and served as the UK’s climate envoy from 2013 to 2017.
The independent advisory group will include some vocal critics of the government’s Covid-19 policies, such as the global public health expert Prof Anthony Costello, as well as ***former and existing Sage experts.
https://www.theguardian.com/world/2020/may/03/publics-trust-in-science-at-risk-warns-former-no-10-adviser
***will Ferguson be joining?
This left wing nonsense SAGE group, is being chaired by the same Sir David King (adviser to Tony Blair) who predicted we would all be living in Antarctica due to run away man made global warming.
He was barmy then and clearly things have not changed.
Here is the full story from 2004.
https://www.independent.co.uk/environment/why-antarctica-will-soon-be-the-only-place-to-live-literally-58574.html
David King is the kind of “scientist” who gave morons a bad name.
Sir D King :
– are we supposed to believe such a clown ?
https://notalotofpeopleknowthat.wordpress.com/2017/04/19/does-sir-david-king-still-believe-the-drivel-he-spouted-in-2004/
Well Petit Barde the climate alarmists will believe anything their religion tells them to, so there are those people who will “believe” him. The safe option is to remain part of the realists group or normal people who study science and allow the warmists/alarmists to worship just about anybody daft enough to agree with their flawed understanding of what is actually going on in real life.
“poor praxis” are all Microsoft products with no exemption.
Gmail, chrome, and Google maps are not much better.
Neil is a scientist, not a programmer.
Two fatal errors (of several — and, of course, one is enough…):
1. Propagation of Error
(Dr. Pat Frank has explained this vis a vis the IPCC’s climate computer simulations here: https://www.youtube.com/watch?v=THg6vGGRpvA&t=7s (“Propagation of Error and the Reliability of Global Air Temperature Projections” discussed here: https://wattsupwiththat.com/2019/09/07/propagation-of-error-and-the-reliability-of-global-air-temperature-projections-mark-ii/)
2. Groupthink Due to Management’s Order to Repair Instead of Re-create
Sue Denim (https://lockdownsceptics.org/code-review-of-fergusons-model/ )
***********************
And one more thing… an all-too-human lack of moral courage by the techs:
“Baker: Did you ever have any qualms about what you were doing? … did you ever think of saying, ‘I do not think this is quite right.’ …
Porter: Yes, I did.
Baker: What did you do about it?
Porter: I did not do anything.
Baker: Why didn’t you?
Porter: … because of the fear of the group pressure that would ensue, of not being a team player.
,
(The Mind of Watergate, Leo Rangell, M.D., http://leorangell.semel.ucla.edu/published/books/mind-watergate-1980 )
well..on one side you have a wrongly observed mortality rate, on the other hand a guess about possible spread of epidemic..
it gives a raw number of deaths..
and you can play with models..with simulate social interactions , population and dynamic of epidemic..
it helps a bit to show how effective a measure to fight epidemic can be..
what amazes me is people thinking models are the actual world..
a well coded program would not really help..because actual uncertainty are huge..
and guessing is important..
if seems obvious that public transportation helps spreading epidemic..may be school …thing like that..
let make a model to confirm or quantify that…
It is worth noting a couple of things.
Firstly the author does not appear to have read or analysed the code.
Their analysis appears to be based solely on other people’s comments on github rather than an actual analysis. No examples are given of actual errors in the code, i.e. stating which lines in which files are wrong.
Secondly the main issue appears to be the non-determinisic nature of the code. Which the authors argue is a feature not a bug since the code trys to model what people will do and that is always uncertain so the best thing anyone can do is to provide an estimate plus the associate error. Plus the non-deterministic nature appears to be due to the code-writers using un-seeded random number generators (again judging from the comments in the essay). Whether or not this is a bug is open to debate but it is certainly not best practice.
Finally the estimates given using the code are in line with estimates from other modellers and also from
simple back of the envelope calculations. In the USA there are currently about 1 million cases and 70 thousand deaths. The population of the USA is about 330 million so most people have still not been infected or exposed and if we assume that eventually 10% get infected (less than on various cruise ships where the infection rate was 20%) then there will be 30 times more cases 2.1 million deaths. This is to be compared with the estimate of less than 1.5 million deaths produced by Ferguson’s model. And who is to say that a bad model will produce an over estimate of the number of deaths.
A must See Fauci should be arrested/fired immediately
https://kfyi.iheart.com/content/2020-05-06-coronavirus-watch-dr-anthony-faucis-ex-employee-tell-all/
already blocked from view
The main problem may be one that the piece doesn’t consider, that is, the excessive quest for detail under the impression that this is related to accuracy.
You see this in business models all the time. You are trying to forecast takeup of some new product type. The department starts out with a back of the envelope one liner of total sales and price.
As the investment required rises, in the effort to satisfy senior management that they are thinking rigorously, the department breaks every significant parameter down further. So they end up forecasting by product type, by region, by single, married, city, rural…. etc.
By the time this process get through, and this happened in the notorious wireless spectrum auctions during the dotcom bubble, you have a model in Excel covering hundreds of pages with extensive use of macros. Now the code will probably be unstructured and uncommented. But that is not the real problem with it.
The real problem is that it has become impossible for decision makers to have a sensible argument about the key parameters. They no longer have anything on which they can bring their knowledge and experience to bear.
Whereas at the start they could sit around a table and argue whether those sales estimates were reasonable, and draw on examples from experience, they could happen in this or that mix or this or that blend of different prices, now they sit staring at the output from a model they have not seen and cannot understand while some bright spark from Finance or Marketing explains to them that this is what the model shows. And offers of proof of its legitimacy that everything has been carefully modelled down to the last detail.
As in Ferguson’s model – its modelling, apparently, hotels differently from other vectors. When you read that, you know immediately that we are in the realm of arbitrary assumptions at an excessive level of detail. But you can’t see how arbitrary all the assumptions are, because they are buried somewhere in pages of code, and you can’t question the result without having to question all the detail, which you can’t get at. Certainly not in a meeting or reasonable length.
Any model which is going to be used as the basis for public policy should fit on one A4 and have no macros or VB in it. Then managers can actually bring their experience and intuition to bear on the key drivers. The way its usually done, and the effect of the Ferguson model, is to turn experienced and qualified people into Yes/No switches.
Its tough for generalists to get their heads around this stuff. A friend of mine tried to overcome the difficulty by using Crystal Ball. It did not help. The problem was it required management to have intuitions about the shape of the probability curves of various parameters. They were very competent people, quite used to arguing through the case for various outcomes. But they didn’t have intuitions expressed in this form. He dropped it and went back to a simple spreadsheet model, expressed in terms they were used to.
If by the way anyone wants to do what CB does, but without the huge outlay, you can do something similar in Gnumeric. A lot more technical, but it can be done.
A very good insight into the problem, michel, thanks.
Dr. Neil Ferguson and his gang have been way off in near all of their work, look how accurate they were with Foot and Mouth (killing large numbers of uninfected and unexposed animals), Bird Flu (going by their figures most of us should be dead already), Swine Flu (same as Bird Flu) CJD (we should all be infected to some degree) and several others how and why the UK Government still uses Imperial College makes one wonder what friends in high places or what information they hold over people?
Going by their track record (similar to global warmists) they’ve missed the mark consistently. In business they’d be broke within days.
James Bull
I have been a programmer for 30 years as well, and I’m struggling to understand how a program can give different outputs for identical inputs. One of the things I had to do was debug code, and the one thing we could rely on was the fact that if you took a copy of the original input, and made sure the system date was set to the date of the original run, the output would be the same. The code is the same, no matter how buggy, and programs are deterministic, so the same input values must create the same output values each time. In fact, had such a thing happened, where everything was identical except for the results, I would have to conclude that the laws of physics are not invariant over space and time. Crazy stuff.
As an aside, I know that climate models often give hugely divergent results from small differences in starting values. But that reinforces my point: the starting values had to differ.
How about a random number generator somewhere using the cpu clock time as a seed?
In that case, how would the random number generator output not be considered an input? In other words, not all the inputs would be the same from one run to another.
One varying value can produce widely differing results if there are enough later steps making progressive calculations based on earlier steps’ results.
If they are using a random number generator against an assumed probability distribution, this wouldn’t be so. There may be no one starting value. If one of the inputs variables is defined as a random pick from a normal distribution with mean and SD of a given value, every time you run it, it will have different inputs. Now, run it long enough, and then successive runs should converge on the same output values. These will be probabilistic of course. But every run, especially with lower numbers of trials, may be different.
The potentially valuable thing about this approach is that it will allow you to model how the probability distributions of the variables interact in a total scenario. But its very hard to get generalists, politicians or managers, to have reliable intuitions about probability distributions. And if all they do is accept your suggestions, your modelling has actually subtracted value.
Vincent,
Uninitialized variables and/or buffers. This is quite easy to do in C, and since local variables are on the stack, they are not set to zero when the program starts. The stack contents are typically not initialized either, so depending on what processes were/are running on the machine, the stack area can literally contain anything and can be different run from run. I’ve seen this exact failure more many times in my career.
In one part of my career I wrote test software used to test various digital electronics. As with all human endeavour from time to time we had to fix. My boss never liked my response to his usual question of “Is it working now?” of “It seems to be”. I could have added until we find the next bug but that would really have upset him.
One fault in a compenent from one manufacturer took about 18 months to identify, thanks to a chance remark that it “only does it when”. But what was interesting was how difficult it was to persuade the manufacturer that his component was faulty. Americans going into meltdown and banging tables a feature of one high level meeting. Demonstrating to engineers was a lot easier.
The end result was a lot of expensive scrap and a different set of people to deal with. Hopefully, at least one person less inclined to shoot the messenger
Stop using Imperial garbage !
Adopt the International System of Units !
🙂
I just read the original thread on this by Sue Denim.
On that thread (and here) are people with misconceptions about how stochastic models run in computer code.
The stochastic model element runs via a random number generator. However, a random number generator is not truly random. It is an algorithm that generates a sequence of numbers that appear to be random when subjected to statistical tests. The output sequence is usually referred to as “pseudo-random”.
However, and this is the key point for software testing and reproducibility, the random number generator algorithm is initiated be a seed number (typically a large, odd, integer eg 385671). When given the same start seed, the random number generator produces the same sequence of pseudo-random numbers.
So for testing code which contains random number generators, it will produce the same results if the same seed is used. This is critical for checking the code. Ferguson’s code appears to be producing different results even when the same seed is used for initialisation. This is very bad.
If the code is running correctly, then multiple runs of the code with either different start seeds (not ideal, but commonly used) or looping many times from the same initial seed/generator combination (much better) produces multiples stochastic simulations, an “ensemble” which can then be analysed statistically. As an aside, generating random seeds is a bad idea as it prevents both reproducibility and tracking down bugs. It also makes the simulation (technically, usually trivially) less random than using a single seed and a long sequence generator.
My background is in stochastic seismic inversion simulation – I have designed and built several very large scale and mathematically complex 3D spatially coupled stochastic simulation models using geostatistical techniques. Similar techniques are used in Oil Industry stochastic (static) reservoir model schemes for simulating geology, again an area in which I have a high level of expertise.
Ironically, I also taught geostatistics as a Visiting Lecturer at Imperial College for 14 years. So when I see Ferguson’s code from Imperial cannot reproduce the same output from the same input seed red flags are out immediately.
The similarities with the mess of climate models and code is also apparent.
“The similarities with the mess of climate models and code is also apparent.”
somebody has not looked at GCM code..
Go get MITGCM.
you wont
Best-written GCM codes still can’t predict air temperature.
Admit that.
You won’t.
That reminds me, Steve and the other trolls quite frequently declare that only those they recognized as having expertise in an area have a right to comment.
Based on that logic, what business does anyone who doesn’t have a degree in computer science have trying to write code?
In the area of modelling , garbage in , garbage out is a well known idea .
But there is another one , good date in treated like garbage results in garbage out.
So even when you feed a model good data , if the manner in which you use it is poor , you lose rather than gain value from the results.
And these ideas are true of ANY MODELLING approach no matter what is used for , even the born perfect ‘science ‘ of climate doom and we known this partly how often they been proved WRONG.
I have great problems with the Imperial team’s modelling. However I have not seen a single criticism of the modelling in Sue Denim’s paper.
You have got to distinguish between the model (an attempt at a mathematical idealisation of reality, probably including some randomness) and the code, which is an attempt to implement that model.
It is one of the major fallacies of the 21st century to confuse these two.
Indeed, I would argue that “ensemble modelling”, as perpetrated by climate scientists, is, in part, an example of that fallacy in practise.
From the Lockdownsceptics page is an interesting comment by commenter Anne which rather obviates the concerns about Ferguson’s code. Previous modelling predictions from Ferguson’s code are quoted as:
Bird Flu = prediction = 200m globally. Actual = 282
Swine Flu prediction = 65,000 UK. Actual = 457
Mad Cow prediction = 50-50,000 UK. Actual = 177
On Mad Cow, the late Christopher Booker in “Scared to Death” pointed out that more farmers died by committing suicide over government policy to tackle Mad Cow than actually died from the disease itself.
The real problem for Ferguson and the government (and now the population at large) is that the ability to model these diseases appears to be lacking and unproven and yet huge decisions are based on “stochastic ensemble” output of the computer simulations. Climate models anyone? Academics are very good at dressing up any old crap as pseudo-science and government Ministers and Civil Servants are stupid enough to believe it.
Just had a quick look, its 99% C code that is compiled as C++ and uses multithreading and multidimensional fixed arrays- collectively a guaranteed way to shoot both feet off with one bullet every time.
There are these things called scaling frameworks that take care of the fiddly stuff in ensuring that things run in parallel without depending on the phase of the moon to complete reliably & repeatedly. The original coder had only to go down the corridor to the Comp Sci Dept and ask… Cobbling together your own thread handling framework and making data exchange work reliably is something the best of us have trouble getting right.
The code should be put out of its misery and deleted. The original coder should take up knitting.
Like the National Academy of Sciences recommendation that the writing of climate science papers should include collaboration with a decent statistician in order to eliminate some of the many statistical faux pas often found therein?
That must have been before Mann was elected to the NAS?
even more exaggerated than the Imperial College model was the one Harvard’s Marc Lipsitch spread around via a media call, as follows. 2% fatalities from 70% of US population infected would have been approx. 4.6m deaths, tho no media ever bothered to calculate this.
22 Feb: WaPo: Coronavirus outbreak edges closer to pandemic status
By Carolyn Y. Johnson, Lena H. Sun, William Wan and Joel Achenbach; Min Joo Kim in Seoul, Amanda Coletta in Washington and Chico Harlan and Stefano Pitrelli in Rome contributed to this report
Harvard epidemiologist Marc Lipsitch estimates that 40 to 70 percent of the human population could potentially be infected by the virus if it becomes pandemic. Not all of those people would get sick, he noted. The estimated death rate attributed to covid-19 — roughly 2 in 100 confirmed infections — may also drop over time as researchers get a better understanding of how widely the virus has spread…
https://www.washingtonpost.com/health/coronavirus-outbreak-edges-closer-to-pandemic/2020/02/21/03afafc0-5429-11ea-9e47-59804be1dcfb_story.html
shortly after the above was published, Lipsitch was asked by Harvard’s Feldman, a Democrat, (who made an appearance at the Trump impeachment inquiry) if WaPo had quoted him correctly as to 40 to 70% of the global population getting infected; Lipsitch said it was roughly correct, but he should have said of the adult population and he was making amends, but didn’t say where or how:
AUDIO: 25m49s: 28 Feb: Stitcher: Deep Background with Noah Feldman (Harvard)
The Coronavirus Isn’t Going Away
Marc Lipsitch, an epidemiologist at Harvard University, predicts that between 40 to 70 percent of adults in the world will become infected with the coronavirus.
https://www.stitcher.com/podcast/pushkin-industries/deep-background-with-noah-feldman/e/67663436
nonetheless, Lipsitch was still pushing the “world’s population” meme days later:
2 Mar: CBS: Coronavirus may infect up to 70% of world’s population, expert warns
by Jim Axedrod
CBS News spoke to one of the country’s top experts on viruses, Marc Lipsitch from Harvard University, who cautions that 40-70% of the world’s population will become infected — and from that number, 1% of people who get symptoms from COVID-19, the disease caused by the coronavirus, could die…
If it really does spread as widely as that projection says, and that’s what I think is likely to happen, then there are gonna be millions of people dying. And I don’t think there’s any way to get around that…
https://www.cbsnews.com/news/coronavirus-infection-outbreak-worldwide-virus-expert-warning-today-2020-03-02/
same date he was tweeting:
Tweet: Marc Lipsitch, Harvard T.H. Chan School of Public Health
Because I am now less certain of where the R0 will end up (and how it may vary geographically) I am going to revise downward the range of outcomes I consider plausible to 20%-60% of adults infected. This involves subjectivity about what range of R0 may turn out to be true.
3 Mar 2020
To preempt the critique that the earlier figures were alarmist: I update my beliefs when the available data change, as any rational person would do. The available data are pointing to a different (and better) outcome than before. So I’m updating…
3 Mar 2020
Tweet: Summary: Should have said 40-70% of adults in a situation without effective controls.
25 Feb 2020
Postscript: My original quote was in the @wsj which I thought had huge circulation. Around the same time I said the same to the @TheAtlantic. The WSJ article made some ripples, but the Atlantic one went completely viral. Not what I expected.
25 Feb 2020
https://twitter.com/mlipsitch/status/1234879949946814464?lang=en
The Atlantic article that went viral:
24 Feb: The Atlantic: You’re Likely to Get the Coronavirus
Most cases are not life-threatening, which is also what makes the virus a historic challenge to contain.
by James Hamblin
The Harvard epidemiology professor Marc Lipsitch is exacting in his diction, even for an epidemiologist. Twice in our conversation he started to say something, then paused and said, “Actually, let me start again.” So it’s striking when one of the points he wanted to get exactly right was this: “I think the likely outcome is that it will ultimately not be containable.”…
Lipsitch predicts that within the coming year, some 40 to 70 percent of people around the world will be infected with the virus that causes COVID-19…
https://www.theatlantic.com/health/archive/2020/02/covid-vaccine/607000/
27 Mar: UK Independent: Coronavirus: Dr Deborah Birx making ‘fundamental scientific errors’ in rush to reopen US, warns expert behind White House data
Marc Lipsitch condemned Deborah Birx for presenting a best-case scenario as likely
by Andrew Naughtie
A leading US epidemiologist has accused one of the doctors on the White House’s coronavirus task force of “false reassurance” after she said a model he helped develop to predict the spread of the virus overstated the number of people likely to develop Covid-19 – when in fact it referred to something more like a best-case scenario.
Marc Lipsitch, a professor of epidemiology at Harvard University, has previously criticised the US government for a “feckless” response that has failed to slow the epidemic’s progress, and called for intense social distancing policies coupled with a “massive expansion” in testing capacity…
In response, Professor Lipsitch wrote that “Our modeling (done by @StephenKissler based on work with @ctedijanto and @yhgrad and me) is one of the models she is talking about” – and that on that basis, he found Dr Birx’s explanation misleading…
Johns Hopkins public health academic Tom Inglesby: “Anyone advising the end of social distancing now, needs to fully understand what the country will look like if we do that. COVID would spread widely, rapidly, terribly, could kill potentially millions in the year ahead with huge social and economic impact across the country.”…
https://www.independent.co.uk/news/world/americas/coronavirus-harvard-scientist-deborah-birx-scientific-error-a9429516.html
don’t know if this involves the above modelling or not:
14 Apr: Science: Report: Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period
Stephen M. Kissler1,*, Christine Tedijanto2,*, Edward Goldstein2, Yonatan H. Grad1,†,‡, Marc Lipsitch2
1Department of Immunology and Infectious Diseases, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
2Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
https://science.sciencemag.org/content/early/2020/04/24/science.abb5793
30 Mar: Wired: The Mathematics of Predicting the Course of the Coronavirus
Epidemiologists are using complex models to help policymakers get ahead of the Covid-19 pandemic. But the leap from equations to decisions is a long one.
by Adam Rogers, Megan Molteni
That’s where Chris Murray and his computer simulations come in.
Murray is the director of the Institute for Health Metrics and Evaluation at the University of Washington. With about 500 statisticians, computer scientists, and epidemiologists on staff, IHME is a data-crunching powerhouse…
You can see how fiddling with the numbers could generate some very complicated math very quickly. (A good modeler will also conduct sensitivity analyses, making some numbers a lot bigger and a lot smaller to see how the final result changes.)
Those problems can tend to catastrophize, to present a worst-case scenario. Now, that’s actually good, because apocalyptic prophecies can galvanize people into action. Unfortunately, if that action works, it makes the model look as if it was wrong from the start. The only way these mathematical oracles can be truly valuable is to goose people into doing the work to ensure the predictions don’t come true—at which point it’s awfully difficult to take any credit…
Speaking at a White House briefing on Thursday, Deborah Birx, response coordinator for the Coronavirus Task Force, admonished the press against taking those models too seriously, even as New York governor Andrew Cuomo begged for federal help with acquiring ventilators and protective equipment for health care workers. “The predictions of the models don’t match the reality on the ground,” Birx said.
Responding to Birx in a thread on Twitter, Harvard infectious disease epidemiologist Marc Lipsitch said Birx had been talking about work from his lab, which the federal government had asked for two days prior. In a preprint (so not peer-reviewed), his team had used an SEIR model with numbers tweaked to simulate the tightening or loosening of social distancing measures, as well as a potential flu-like seasonal variation in Covid-19 infections. He was varying R0, essentially. In the model, putting a stop to strict social distancing (without something like a vaccine or a cure coming along) allowed infections to climb right back up to their peak of about two critical cases per 1,000 people—which could be 660,000 Americans getting seriously ill or dying. And even with the strictest lockdown-type measures lasting from April through July, his team’s model finds that the disease surges back in autumn…
If Lipsitch’s team is right, the characteristics of Covid-19 might require a cyclical flux between strict social distancing and viral resurgence, on and on, perhaps until 2022…
https://www.wired.com/story/the-mathematics-of-predicting-the-course-of-the-coronavirus/
Lipsitch criticises IHME’s less exaggerated model in mid-April, because Trump!
17 Apr: StatNews: Influential Covid-19 model uses flawed methods and shouldn’t guide U.S. policies, critics say
By Sharon Begley; Helen Branswell contributed reporting
A widely followed model for projecting Covid-19 deaths in the U.S. is producing results that have been bouncing up and down like an unpredictable fever, and now epidemiologists are criticizing it as flawed and misleading for both the public and policy makers. In particular, they warn against relying on it as the basis for government decision-making, including on “re-opening America.”
“It’s not a model that most of us in the infectious disease epidemiology field think is well suited” to projecting Covid-19 deaths, epidemiologist Marc Lipsitch of the Harvard T.H. Chan School of Public Health told reporters this week, referring to projections by the Institute for Health Metrics and Evaluation at the University of Washington…
The IHME projections were used by the Trump administration in developing national guidelines to mitigate the outbreak. Now, they are reportedly influencing White House thinking on how and when to “re-open” the country, as President Trump announced a blueprint for on Thursday.
The chief reason the IHME projections worry some experts, Etzioni said, is that “the fact that they overshot will be used to suggest that the government response prevented an even greater catastrophe, when in fact the predictions were shaky in the first place.” IHME initially projected 38,000 to 162,000 U.S. deaths. The White House combined those estimates with others to warn of 100,000 to 240,000 potential deaths…
Believing, for instance, that measures well short of what China imposed in and around Wuhan prevented a four-fold higher death toll could be disastrous…
There are two tried-and-true ways to model an epidemic. The most established, dating back a century, calculates how many people are susceptible to a virus (in the case of the new coronavirus, everyone), how many become exposed, how many of those become infected, and how many recover and therefore have immunity (at least for a while). Such “SEIR” models then use what researchers know about a virus’s behavior, such as how easily it spreads and how long it takes for symptoms of infection to appear, to calculate how long it takes for people to move from susceptible to infected to recovered (or dead).
“The fundamental concept of infectious disease epidemiology is that infections spread when there are two things: infected people and susceptible people,” Lipsitch said…
IHME uses neither a SEIR nor an agent-based approach. It doesn’t even try to model the transmission of disease, or the incubation period, or other features of Covid-19, as SEIR and agent-based models at Imperial College London and others do…
https://www.statnews.com/2020/04/17/influential-covid-19-model-uses-flawed-methods-shouldnt-guide-policies-critics-say/
when Stanford University epidemiologist John Ioannidis published this:
17 Mar: StatNews: A fiasco in the making? As the coronavirus pandemic takes hold, we are making decisions without reliable data
By John P.A. Ioannidis
At a time when everyone needs better information, from disease modelers and governments to people quarantined or just social distancing, we lack reliable evidence on how many people have been infected with SARS-CoV-2 or who continue to become infected. Better information is needed to guide decisions and actions of monumental significance and to monitor their impact.
Draconian countermeasures have been adopted in many countries. If the pandemic dissipates — either on its own or because of these measures — short-term extreme social distancing and lockdowns may be bearable. How long, though, should measures like these be continued if the pandemic churns across the globe unabated? How can policymakers tell if they are doing more good than harm?…
The data collected so far on how many people are infected and how the epidemic is evolving are utterly unreliable…
https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-as-the-coronavirus-pandemic-takes-hold-we-are-making-decisions-without-reliable-data/
Lipsitch was ready to pounce the very next day:
18 Mar: StatNews: We know enough now to act decisively against Covid-19. Social distancing is a good place to start
By Marc Lipsitch
In a recent and controversial First Opinion, epidemiologist and statistician John Ioannidis argues that we lack good data on many aspects of the Covid-19 epidemic, and seems to suggest that we should not take drastic actions to curtail the spread of the virus until the data are more certain.
He is absolutely right on the first point…
We spoke by phone on Tuesday, not long after his article appeared, and found that we had more in common than it appeared when I first read it…
https://www.statnews.com/2020/03/18/we-know-enough-now-to-act-decisively-against-covid-19/
the following day CBC weighed in:
19 Mar: CBC: Prominent scientist dares to ask: Has the COVID-19 response gone too far?
Leading epidemiologists publish duelling commentaries, igniting debate on social media
by Kelly Crowe
It’s a clash of titans — an epic battle between two famous scientists over the world’s response to the COVID-19 pandemic.
In one corner, influential Stanford University epidemiologist John Ioannidis, who wrote a commentary asking whether taking such drastic action to combat the pandemic without evidence it will work is a “fiasco in the making.”
Across the mat, prominent Harvard epidemiologist Marc Lipsitch punched back with a defiant response titled: “We know enough now to act decisively against COVID-19.”
Watching from the sidelines? Everybody else…
https://www.cbc.ca/news/health/coronavirus-covid-pandemic-response-scientists-1.5502423
CBC definitely weights the article against Ioannidis.
this may be of interest
https://www.sciencedirect.com/science/article/pii/S1755436518300306
BBC pandemic experiment
Would be intersting for insurance companies to offer products related to ‘climate change’, ‘coronavirus’ etc.
In that way we could be sure of professional actuarial standards of modelling, and there would be a gold standard to which journos could cross-check these essentially self-serving academic models.
Another way would be for bookies to offer odds on academic prognostications ….. or at least for journos and/or the general public to ask academic authors to take wagers on their claims.