Reposted from the Cliff Mass Weather Blog
Wednesday, April 29, 2020
Things are not going well these days regarding predicting the future of coronavirus in the U.S., with the epidemiological community, including critical government agencies, not succeeding in these important areas:
- They do not know the percentage of the U.S. population with active or past COVID-19 infections.
- They do not have the ability to quality control and combine virus testing information into a coherent picture of the current situation. This is a big-data problem.
- The epidemiological simulation models used by U.S government agencies or American universities have a poor track record in their predictions, with their quantification of uncertainty unreliable.
But there is a group in the U.S. with deep experience and a highly successful track record in predicting complex environmental threats. A group that is masterful in taking observations, combining them to create a good description of reality, building and testing predictive models, providing uncertainty information, and communicating the information to decision makers for critical life-threatening situations.
You know them these people: meteorologists involved in the large U.S. numerical weather prediction community. And perhaps meteorologists can help epidemiologists and the U.S. government to get a handle on the coronavirus situation.
Now don’t take this blog as one uppity weather guy trying to give advice “outside his lane.” A published paper in the Journal of Infectious Diseases (2016), said much of the same, with the authors noting the huge similarities in the work meteorologists and epidemiologists do and suggesting that the epidemiological community is roughly 40 years behind the numerical weather prediction enterprise. They observed that both epidemiological and numerical weather prediction models are attempting to simulate complex systems with exponential error growth, and thus have great sensitivity to initial conditions.
So perhaps the experience of meteorologists, who spend much of their time thinking about how to improve weather forecasting, may be relevant to the current crisis.
The First Step in Prediction: Describing the Initial State of the System
To predict the future you need to know what is happening now. The better you can describe the initial starting point of forecasts, the better the forecast.
Meteorologists have spent 3/4 of a century on such work, first with surface observations and balloon-launched radiosondes, and later with radars and satellite observations. Billions have been invested in the weather observing system, which gives us a three-dimensional observational description of atmospheric structure. Big data. And we have learned how to quality control and combine the data with complex data assimilation techniques, with the resulting description of the atmosphere immensely improving our predictions. This work is completed operationally by large, permanent groups such as NOAA and NASA, with large interactions with the research community.
Contrast this to the unfortunate state of epidemiologists predicting the future of the coronavirus.
They have very little data on what is happening now. They don’t know who in the population is currently infected or has been infected. They don’t even know the percentage of the current population that is infected. Without such information, there is no way epidemiologists can realistically simulate the future of the pandemic. They are trying, of course, but the results have been disappointing.
What they do have is death information and limited testing of those that are sick, but that information is insufficient to determine the state of current and past infection in the community, or essential parameters such as transmission rate and mortality rates.
Obviously, the U.S. needs massive testing of the population to determine how the virus has invaded our communities and who is now immune. The lack of such testing is terrible failure of multiple levels of government.
But just as big a failure is the lack of random sampling of the population to determine the percentages of infection and how that varies around the nation.
We do have enough testing capability to do this (remember national political polls only use thousands of samples, not millions). Why is the epidemiological community and our political leaders not calling for such intelligent sampling of the population? With random sampling we would KNOW what is going on and not act out of ignorance (as we currently are muddling by). Why is the media not baying about this?
Quality control is another major problem faced by the epidemiological community, who deals multiple types of tests of various quality that need to be brought together to produce an integrated picture of reality. Death information is unreliable, because of non-reports or problems with determining the primary cause of death. Quality control is a difficult task, faced by the meteorological community as well, one that we have dealt with in our data assimilation systems (e.g., observations weighted by their past quality and sophisticated consistency checks).
Simulation Models
Starting with an initial description of the system one is predicting (the 3-D atmospheric structure for meteorologists, the initial disease state of the population for epidemiologists), simulation models are used to predict the future.
Meteorologists use complex, full-physics models comprised of equations that predict the future evolution of the atmosphere. Then we apply statistical corrections to make the forecasts even better.
Epidemiologists use three types of forecast models:
- SEIR/SIR models is the most “traditional” approach, one in which the population is divided into different groups (susceptible, exposed, infected, recovered), using relatively simple equations to describe how folks move from one group to another, all of which have assumptions about how the disease is transmitted, the effects of social interactions and more. The UK Imperial Model is an example of this approach.
- Statistical models that don’t really simulate what is going on, but are really curve-fitting exercises, in which theoretical curves (often gaussians) are used to predict the future, adjusting the curves based on the evolution of disease in the past or at other locations. There are many assumptions in this approach and they cannot properly consider the unique characteristics of the region in question. The UW IHME model is a well-known user of this approach.
- Agent-based modeling actually try to simulate the community at an individual level and it is the most complex and computer intensive approach. Although dependent on several assumptions (such as the transmission rates between individuals) this approach is the closest to the numerical weather prediction used by meteorologists. The GLEAM model from Northeastern University (and others) is an example of this.
The trouble is that none of these epidemiological models have proven particularly skillful and produce vastly different results, something noted in some of the media, social media, and several new research papers. The UW IHME model, often quoted by local and national political leaders, has been particularly problematic (this paper describes some of the issues), including the fact that its probability forecasts are highly uncalibrated. The UK Imperial Model in mid-March predicted 1.1-1.2 million deaths in the U.S., even with mitigation (so far the U.S. death toll has been about 60,000). Many of the coronavirus prediction efforts have evinced unstable forecasts, with great shifts as more data becomes available or the models are enhanced.
The poor performance of these models in predicting the coronavirus is not surprising: the lack of testing and particularly the lack of rational random sampling of the population results in no viable description of what is happening now. The favored IHME model is only based on death rates, not on the infection state of the community. Can you imagine if meteorologists tried to predict weather only using data around active storms? Very quickly, the forecasts–even of storms–would become worthless. The same happens with coronavirus.
You cannot skillfully predict the future if you don’t have a realistic starting point. Furthermore, some of the models are highly simplistic and not based on the fundamental dynamics of disease spread (like the curve-fitting IHME approach).
The U.S. has a permanent, large, well-funded governmental prediction enterprise for weather prediction, one that has improved dramatically over the past decades. No such parallel effort exists in the government for epidemiological modeling. Instead, University groups, such as UW IHME, have revved up ad-hoc efforts using research models.
The Bottom Line:
Our government and political leadership have been making extraordinary decisions to close down major sectors of the economy, promulgating stay-at-home orders, moving education online, and spending trillions of dollars.
And they have done so with inadequate information. Decision makers don’t know how many people are infected or were infected. They don’t know how many people are already immune or the percentage of infected that are asymptomatic. They are using untested models that have not been shown to be reliable. This is not science-based decision making, no matter how often this term has been used, and responsibility for this sorry state of affairs is found on both the Federal and state levels.
The meteorological community has a long and successful track record in an analogous enterprise, showing the importance of massive data collection to describe the environment you wish to predict, the value of sophisticated and well-tested models to make the prediction, and the necessity to maintain a dedicated governmental group that is responsible for state-of-science prediction.
Perhaps this approach should be considered by the infectious disease community. and the experience of the numerical weather prediction community might be useful.
Big data, but bad theories, at least among climatologists (long term meteorologists)
http://phzoe.com/2020/04/29/the-irrelevance-of-geothermal-heat-flux/
meteorologists can often provide a good guess for the weather 2 to 3 days ahead, beyond that it goes to shit.
I think I could guess COVID-19 stats 2 to 3 days ahead with accuracy as good as a weather forecast.
Not sure meteorology has a lot offer here.
In the age of big data you would think it would be easy to find automated comparisons of predictions to results, so we can see how well meteorologists do.
Anyone found such a resource?
Nope. Not three days ahead of time, no way.
They say three days of sun but sadly, the next day is full cloudy, or the day after.
Live your life one day at a time and don’t trust anybody that can predict beyond two days.
Any use is more than the junk Zoe is pedaling .. Anthony should ask for payment for advertising .
Anyone that believes the #19 models were any different from AGW models with the aspect of being political and with worse case scenarios only being hyped is naive.
That “outside his lane” mem is just another PC club to keep people in line and shut down discussion.
It is well known that cross discipline knowledge helps with insight and invention.
It’s also well known that cross discipline knowledge among climatologists is largely absent leading to the obtuse artlessness of climate science.
Well said, c2.
Yeah, I can dig that.
Seriously.
Regarding testing for this virus, I have not seen anything which gives me confidence that the testing being done anywhere is specific to this virus. The testing started early so I suspect that it is not specific but for a range of such viruses. Can anyone help me here?
There are tests that are specific, currently only lab-based. isothermal nucleic acid amplification & reverse transcription polymerase chain reaction .
“That “outside his lane” mem is just another PC club to keep people in line and shut down discussion.”
During my career, essentially all professional accomplishments (including publications & patents) were “outside my lane.” My lane was Polymer Science with Masters thesis on epoxy-metal-ion polymers (Rice U).
Cases in point are excerpts from some of the patents where I am way outside my lane:
1. “An apparatus for decontaminating ground areas where toxic chemicals are buried . . .”
2. “. . . the invention employs high voltage arc discharge between the electrodes for heating a ground region to relatively high temperatures at relatively low power levels. . .”
3. “The annulus between the molybdenum rod electrode and the graphite collar is suitably filled with a conductive ceramic powder that sinters upon the molybdenum rod . . .”
4. “A signal processor is coupled to the sensors to receive an electrical signal generated by a sensor, and generate a signal which is encoded with information which identifies the sensor . . . .”
So “cross discipline knowledge helps with insight and invention” is an understatement imo.
I am STILL looking for information on whether this virus (or any OTHER virus) is capable of traveling on the prevailing winds. Meteorologist’s, with their 40 year head start on the epidemiologists’ SHOULD be able to shed some light on this, I would think. It seems, to me, at least, that almost all , if not all, of these viruses originate in the Asian continent, and we seem to get winds primarily from either Asia or the North Pole regions, here in the Northern hemisphere. So it seems logical to think that the winds passing OVER Asia could easily ‘pick up’ these viruses floating around, (since they do seem to flourish in very cold conditions) and carry them down wind to the USA and later to Europe, etc. and eventually to the rest of the world. This would explain how people living in remote, far flung regions, where there are NO air ports or other means of contact with the ‘civilised’ world become infected.
There is also a question about whether birds might be involved. We just don’t know, yet, HOW this disease travels. I would think that someone, a LOT smarter than myself, would be thinking about this.
You mean, a lot more knowledgeable.
That I can answer for you definitively. No, for two reasons, both related to it being an enveloped virus.
1. Inactvated by sunlight in minutes (new DHS lab finding) when it takes many days to cross an ocean.
2. Maximum survival ex vivo is about three days on benign hard surfaces indoors, when it takes many days to cross an ocean.
Bitds are likely not involved in Wuhan; it is strictly a mammalian virus.
Unlike influenza, where most strains start as avian, then jump to pigs, and then to humans. H1N1 1918 pandemic was classic. Fall 1917 Missippi duck flyway to Kansas hog farms. Probably a hog ate a dead duck, literally. Incubation in hogs for a year. Jump to Kansas hog farmers fall 1918. Jump to WW1 troops training in Kansas immediately thereafter. Barry’s The Great Influenza is a good read.
2009 swine flu incubated in pigs in Mexico before jumping to humans.
They travel by by bus or plane,sometimes first class
If you can smell a guy smoking then you are in range. If you can smell the guy Fart in the car ahead of you, you are in range.
Isn’t the initial state, unlike for climate predictions, well known? Nobody has symptoms. Nobody is infectious. Nobody is infected. Nobody has died. A certain percentage, C₀, will have natural immunity, but that is probably a small number and parallel studies can be done assuming 0%, 5%, 10%. Assuming 0% resistant will give conservative output, for most purposes. Actual percentage can be back-estimated, if need be, by studying later statistics and/or testing.
All true. But for epidemiological modeling you need the initial R0 and a CFR. Looks now like R0 is at least 2.5 absent social distancing and other mitigation measures, and we still dunno CFR. Based on South Korea, maybe about 2%. All commented on previously.
We do now know much other useful epidemiological stuff about Wuhan that was unknown just three months ago. Mean time from infection to symptoms a bit over 5 days. 97.5% of symptomatics display within 11.5 days. Main symptoms fever >100.5F, dry cough, malaise. 20% of positives NEVER develop symptoms implying my naivete hypothsis. Positives can shed infectious virus for something between 1 and 3 days before symptoms, implying VERY hard to control without super testing and 14 day quarantines. Symptomatics progress for about 9-10 days to a bifurcation: recovery, or rapid worsening to serious critical viral pneumonia, then cardiac and renal failure and thrombosis when the virus enters the blood stream via damaged alveoli capillaries.
GIGO … all of it … there are no credible viral models … none of them can predict a flu season and they have had decades of model fitting vs real world to try …
none of it matters if we protect the at risk elderly … there is almost no risk for the healthly below 65 …
its weaker than the flu for the below 65 …
we are focused on the wrong end of the stick …
Can any of these models use the current data and back into how many people were really infected in NY in January.
Every epidemic essentially starts each new infection location with one new index patient expanding exponentially outwards. What would be gained by reversing the process, it’s hard enough tracking the contacts moving forward
Thanks Rud – useful summary.
Could the manbearpig have been a virus?
Co is actually a significantly large number.
The virus arrived in UK at least 6 weeks before lockdown. The London underground ran for that 6 weeks crammed full of people 5 million journeys a day most of them daily return journeys. Even after lockdown the tube continued still crammed thanks to running less trains. The numbers infected are extremely low for a supposed very contagious virus.
This is for several reasons the number of ACE receptors per endothelial cell in Europeans are lower than those in Asian genotypes so the virus finds infection of asian genotypes easier. Also the Chinese diet is low in Selenium and Zinc whereas in Europe most people are up to the recommended daily intake. Selenium blocks the ‘docking’ of the virus on the endothelial cells intracellular zinc prevents corona viruses hijacking the cell’s RNA transcription and multiplying.
This is why only 5% or so is shown by antibody testing to have been infected and created antibodies. The remaining 95% may well have inhaled or swallowed some of the virus – but they are effectively immune to corona viruses.
The models assume that every person is identical as the epidemiologists assume R0 based on all people being identical. This is not the case the hypothesis expressed in the model has been falsified not by lockdown but because the majority of people cannot catch COVID-19 due to genotype and/or diet.
“But there is a group in the U.S. with deep experience and a highly successful track record in predicting complex environmental threats. A group that is masterful in taking observations, combining them to create a good description of reality, building and testing predictive models, providing uncertainty information, and communicating the information to decision makers for critical life-threatening situations.”
Climate modellers? They do exactly the same thing don’t they. “masterful” I like that.
Maybe climate modelers dream that they do the exact same thing.
He’s not talking about climate modellers, he’s talking about weather modellers, meteorologists, “The U.S. has a permanent, large, well-funded governmental prediction enterprise for weather prediction, one that has improved dramatically over the past decades.”
“one that has improved dramatically over the past decades.”
Something we can’t say in regards to climate models. The range for possible ECS has been unchanged for decades.
Funny.
Cliff wrote “meteorologists” and Loydo saw “climate modellers”
One day you might catch up, lad.
Unlikely. Loydo either does not have the mental capability to learn, or is a paid troll. Don’t know why anybody even responds to him/her.
Meteorologists incorporate realtime feedback; climatologists are not grounded in reality.
I wonder if Cliff could enlighten us as to whether there are any actual substantial differences or how much overlap there is rather than resorting to cliches.
OK dispute this:
Meteorologists incorporate realtime feedback into their models; climatologists do not incorporate realtime feedback into their models.
I dispute that. GCMs are used by meterologists and climatologists. Both input realtime data. Meteorologists are going to input fresh data more frequently because they are constantly updating short-term predictions. But other wise not much difference. Both “masterful”.
I get it, you’re itching to make a distinction because otherwise the “model bad” narrative is shown to be a lie and we can’t have that.
“Loydo April 30, 2020 at 4:22 am
GCMs are used by meterologists and climatologists.”
GCMs *ARE* the problem. Just like the models used for COVID-19. All bollox!
LOLoydo, frequent feedback is what makes meteorologists model projections somewhat accurate. Without that realtime connection to reality their models would be garbage just like climate models are.
Meteorologists do not use climate models.
No climate model has ever produced a result that could match up with what is happening in the real world.
“…I dispute that. GCMs are used by meterologists and climatologists…”
Of course, by “GCM” do you mean “general circulation model,” which can be at the heart of meteorologist modeling, or “global climate model,” which is exclusive to climate modeling? There are people who either don’t understand the difference or intentionally use GCM in the hopes of tricking people into thinking the models are the same.
“…Meteorologists are going to input fresh data more frequently because they are constantly updating short-term predictions. But other wise not much difference…”
Well that is a huge difference, and not the only huge one.
For example, meteorologists know things “blow up” into garbage if they try to look more than a few days or weeks ahead (depending on what they are looking at), trying to model the non-linear chaotic systems with a fine resolution. Climate modelers portray their models are accurate decades out. And to keep things from getting “chaotic” (literally and figuratively), they use constraints to artificially dampen results or use approximations that don’t represent reality well.
One of the more amusing differences: when a meteorologist says it is going to be 3 degrees warmer than normal in Raleigh, NC, and 5 degrees colder than normal in Juneau, AK, he needs to be right on both accounts. He needs the +3 in Raleigh and -5 in Juneau to be accurate. When a climate modeler gets it backwards and says it should be 3 degrees warmer than normal in Juneau and 5 degrees colder than normal in Raleigh, he is “right” because the anomalies balance-out. It doesn’t matter which one is +3 and which one is -5. Only in climate science can you be wrong all over the globe with temperature, precipitation, cloud cover, etc., but as long as your global temperature anomaly adds-up close to being right, your model is “accurate.”
Here you go, Loydo
https://wattsupwiththat.com/2020/04/30/model-madness-parallels-between-failed-climate-models-and-failed-coronavirus-models/
Like most of Loydo’s thoughts, this one is also completely wrong.
As the climate alarmists are prone to saying, weather is not climate. And weather models have nothing in common with climate models.
Weather models take current conditions and try to run the camera forward to see how those conditions change over time, they also only deal with local to regional forecasts.
Climate models take a set of presumed conditions, and then tries to determine what average climate for the entire earth is going to be like under those conditions.
Well many climate models do try to get down to regional forecasts and sometimes somewhat close to “local” depending on how you define that (I think 100 km x 100 km grids are in the coarse category today, and that can be encompassed in “local radar” on your “local news”). They are just embarrassingly bad about it. But if you add up all of those wrongs, you get to a global value that may seem halfway right, and that is supposed to somehow be valid.
Indeed, indeed, like chalk and cheese.
How did we get here? It was fast. It happened before we recognized what was going on. It was like falling into a tiger trap. Just a simple pit on a forest trail covered with leaves. At first you are told we just want to flatten the curve. Now they want to stop all chance of death. Climate Alarmist were all set up, trained and ready to deploy. They just changed the labels on their cheesy polyester lab coats to Corona Alarmist. Same phony arguments about adherence to a non-existent science with rubbery models as their only evidence. Same one sided argument and complete disregard for economic consequences.
Yes, I noticed how the message changed from “flattening the curve” to “slowing the spread” to now “stopping it”. I wish people would realize that we already achieved the first one, and that the other two are impossible. Even with a vaccine (a year away), some people will still get it and some will die. Just like every other similar virus (i.e. SARS-1, H1N1, etc.)
Flattening the curve and slowing the spread mean pretty much the same thing.
Texas is one of the better run states, but their report is very bureaucratic. 66 pages, inadequate read, but didn’t find fresh, open air. In the century old case that almost killed my grandfather in France, they realized the importance of open air. Here in Aransas County, one of their rural, minor places, parks, boat ramps and similar fresh airs have been closed. The report approaches the length of warranty, etc., documents. I am in the most susceptible group, survived the worse polio epidemic and my major mentor was a parasitologist. Maybe we should be quarantined, but would rather see immunity develop.
https://gov.texas.gov/uploads/files/organization/opentexas/OpenTexas-Report.pdf
Page 44 “Rural counties may, on an individualized basis, increase capacity for restaurants, retail, shopping malls, museums, libraries, and/or movie theaters if the county judge certifies and affirms to DSHS that the following standards have been investigated and confirmed to be met: …” It was advised by a huge committee. We can increase in-place restaurant availability from 25 to 50%, but only if….“The attestation form, including the supplemental county information, to be completed by the county judge, can be found on the Department of State Health Services Coronavirus Disease 2019 (COVID-19) website at https://dshs.texas.gov/coronavirus/.” Our county judge closed the boat ramps, doubt that the virus liked salt water. It was not well received.
Cliff Mass musings are usually very well grounded. This time, not so much.
Epidemiologists have been trying to understand disease propagation since Dr. Snow took the pump handle off the Broad Street well to stop London’s 1854 cholera epidemic.
They often get it medically wrong (HRT, egg cholesterol being examples in my book The Arts of Truth), but more often ‘right’ as a disease is better understood.
The novelty of this virus, its R0, its comorbidities were all unknown educated guesses from analogies just two months ago. Heck, Fauci still thinks Wuhan is highly seasonal like flu, when main route of transmission is known common cold like and summer colds are, well, common.
Rud
But isn’t that exactly why we should have been doing randomized population testing from the beginning, so that we could have learned about these unknowns earlier in this event? My point about the modeling is that society is making huge gambles and major sacrifices based on unreliable models.. Are you saying that is was a wise approach?….cliff
PS: I am glad that my “musings” are usually “well grounded.”
So which ones did they get right, Rud?
Swine flu? no.
SARS? No.
MERS? No.
Bird flu? No.
Zika? No.
CJD? No.
COVID-19? No.
All of these viruses were over hyped with prophecies of mass worldwide deaths.
Were any lessons learned? No.
One name stands out in all these doomsday prophesies and that is Niel Ferguson. He was also responsible for the unnecessary mass slaughter of animals during the UK foot and mouth outbreak.
Ferguson says his computer code is 13 years old and contains thousands of lines of undocumented code. He won’t release it. Where have I heard that before.
It took me 1 hour to write a SEIR model in Python and most of that time was writing code for drawing graphs. 40 lines of code and I didn’t receive a single penny from Bill Gates.
Do you seriously expect me to give up all my political rights based on a few differential equations?
“All of these viruses were over hyped with prophecies of mass worldwide deaths.”
With good reason because they were unknown viruses at the time. It was prudent to assume they were dangerous until proven otherwise. It will be the same way in the future when another unknown virus appears.
I was reading an article last night that claimed that when they had time to review death certificates and preserved samples from autopsies after the outbreak had been controlled, that deaths from SARS, was between 7 and 15 times greater than what was known at the time.
Can Meteorologists Help Epidemiologists with #Coronavirus? Forecasting in both fields is incredibly shaky, it’s kind of like the mute providing diction lessons for the deaf.
BabylonBee: Scientists Who Didn’t Predict A Single Thing Accurately For Last Two Months Confident They Know What The Weather Is Going To Be Like In 100 Years
You have to put on hip-waders to get through this. They are basically saying that because the medical modelling community is “40 years” behind the weather/climate modelers you can still trust the climate models. This has been the big issue up to now; how bad the IHME and others were at modelling the number of deaths. How can we trust climate models then?
There is no mention of climate modelers. Weather modelers and climate modelers are not the same thing. Weather models and climate models are not the same thing.
They’re closer than you think. Some differences are that climate models run at lower resolutions, don’t account for as many higher order factors and use longer time steps. Both suffer from the divergence problem limiting their usefulness for long term predictions.
The big difference as mentioned before is the feedback. Weather models can be improved because you have a constantly increasing test set. If the weather forecast models have a big bust the research community figures out what happened and tries to correct it. The new model is implemented and revised again when it runs into a problem it cannot handle. You can verify a weather forecast model pretty well but you will never be able to verify a climate model’s GHG sensitivity. Of course even when a model can be tested it will eventually run into the divergence problem and other issues so there is a point when the weather model long-term forecasts cannot be expected to provide useful forecasts.
There is no comparison between the two.
Weather models start with current conditions and try to project those forward.
Climate models assume certain conditions and try to create a static projection for what the average climate will be like at the time of those conditions.
Weather models try to predict how fast a weather front will move over the next few days.
Climate models don’t even factor in weather fronts under the assumption that weather just averages out.
Yes they both deal with the atmosphere, but how they deal with the atmosphere is completely different.
“…Obviously, the U.S. needs massive testing of the population to determine how the virus has invaded our communities and who is now immune…”
(1) it seems that testing is “massively” unreliable with plenty of false negatives
(2) there are doubts that simply having it once makes one immune (or, to put it another way, reports that there is no evidence recovered patients are immune)
From the start the CDC elected to go with no data when decent data could have been acquired.
Way back in February, Singapore was using Serum Antibody Tests to do surveillance. The tests used were not accurate enough for testing individuals, but initially estimated at ~80% accurate were good enough for gathering population statistical data and for tracking.
So here we are 4 months into the epidemic with only a vague notion of the infection rate…the most important pieces of data when trying to model an epidemic…the “how many” and the “where”.
The only excuse I’ve heard for this lack of early Antibody Testing is that the tests were not accurate enough. I don’t accept that answer…if that is their answer….not sure SINCE NOBODY’S ASKING THE QUESTION. The accuracy and specificity of the early tests were known. The accuracy was good enough to provide very good population data with (large enough sampling sizes). The accuracy and specificity of these tests improved steadily. But the CDC elected to fly blind rather than even attempt to get the data required to make good predictions. Errors at the level of 3 orders of magnitude isn’t good enough for making existential policy decisions with…but that’s what we did.
Even with inaccurate antibody tests by taking 2 (or 3) rather than just one you increase the specificity of the tests. With automation this should not cause a problem and you could probably create a version of the test with three blood wells on the sample collector.
It does seem that medical metrology has no consistent governance (that is metrology as in measurment/recording). So iterative models based on the poor metrology will rapidly deviate from reality as Lorentz showed some time ago. Then the gullible innumerate politicians panic – and here we are probably with a cure that was worse than the disease.
Keep immune systems functioning with Vitamin D3, Selenium and Zinc and the models will be even further off.
One huge problem is the loose handling of lingua franca of the trade.
When I hear the word testing for COVID-19, I wonder are they intending to mean a PCR based test that detects the viral RNA in an actively infected person? Or are they meaning a serological test for IgM/IgG antibodies specfic to the virus as “correlate of immunity” and evidence of past infect ion resolved.
When an ER or clinical medical staff discusses testing, they generally mean a PCR test to determine if someone is actively infected and thus affect how they are treated for symptoms they may be presented. These testing results of course get reported, and are most of what has been discussed. But PCR tests for virus RNA do not identify past infected (and assumed now immune) individual.
For testing large representative samples of asymptomatic individuals, the serologic test for specific antibodies is done. The serologic test for antibodies, most notably IgG and usually also shorter duration IgM/D, is called a “correlate of immunity”. Serologic tests for pathogen specific antibodies is only 1/2 the immunoliogcal question. Antibody testing tells us little about the very important T cell memory response and how durable that is that must also be formed to provide long-lasting immunity. But if IgG is present, then that typically correlates” with longer term T-cell immunity (years likely gives years instead of months for IgM immunity, because the T cells didn’t get involved.)
Remember correlation is not causation. IgG antibodies are merely correlates of assumed immunity that includes the much harder to assess T cell responses. This is because to get to Gamma globulin (IgG) by B-cells (and their differentiated daughters cells called plasma cells), a complex dance must occur in the lymph nodes between antigen-specific B cells and antigen-specific T cells for the T-cell dependent Ig class switch from IgM to IgG output to occur. T-cell dependent B-cell/antibody response also drives (immunologists call it “licensing”) a very important process given the name “affinity maturation.” In affinity maturation many moderate to high affinity B-cells compete with each for the virus antigen for stimulation whilst they each make adjustment at the variable regions of their antibody binding sites, the Tcells are also doing the same thing, and those B cells that present the best processed antigen to the T cells, the T cells reward with survival signals. Those B cells that don’t stimulate the T-cells then don’t get the survival signals provided by T cells (CD40L generally but others too) and go off and die. During affinity maturation, actual molecular changes to the DNA coding bases (the triplet codons actually change, DNA is purposefully altered) for the antibody production that will then make mRNA that goes to ribosomes to make Ig. The amino acid sequences on the B cells’ antibody binding site are actually changed and thus their antigen specificity for the one’s that make a successful modification gets ever higher binding affinity (sub-nanomolar in many cases) to viral epitopes in order to effectively neutralize a virus. Positive selection thus occurs to drive only the highest binding affinity antibody presenting B cells to survive and ultimately form the pool of differentiated plasma cells which secrete large amounts of soluble IgG from their niches in the bone marrow. Lower affinity B cell antibody makers that go straight to the bone marrow and skip the affinity maturation class switching from M to G globulin.
So when we talk about “testing” for Corona Virus, we need to be sure we are all talking about the same thing. These very important distinctions (between tests) are easily lost on the public when they hear something about Covid-19 testing.
Thanks!
During affinity maturation, actual molecular changes to the DNA coding bases (the triplet codons actually change, DNA is purposefully altered)
In our university immunology class s this system was called “generator of diversity” or G.O.D. Is that term still used?
This:
was followed almost immediately by this:
We do have enough testing capability to do this (remember national political polls only use thousands of samples, not millions).
So which is it?
Or door #3: Just another pitch for government experts.
In my area they did some testing to see if people had had the disease. It was supposed to be random but word got out on social media and people who had been sick late last year or early this year ran over to get tested. Dr. Mass is talking about testing with a plan to ensure randomness. Statisticians can calculate how big a sample size is needed and how it should be done so that you get a representative sample of society. Representativeness is a big deal for meteorologists.
You can argue that we don’t have enough tests, or you can argue that we’re not sampling correctly. You can’t argue both simultaneously.
Epidemiologists can’t even get this illness’ etiologies right, so they’ll never get the models right. Pathogenesis, iatrogenesis, co-factors, co-morbidities, environmental toxins, etc.. Epidemiology is so much more than R(n) and graphs.
I just read that Antarctica is losing ice at the rate of .0007% pa.
That’s a lot of James Bond cocktail cubes. And probably just as fictional.
Call me a stirrer, but is this the sort of expertise we need to shake up the Covid19 response?
In Georgia, new policy is door-to-door blood collection for Covid-19.
Between you and me, this is a futile exercise that won’t solve the virus problem as many will contract it after bloodletting.
The best would be open up the country, allow people’s immune system to regain its strength after months of immune inactivity via social distancing and masks.
And forget weather predictions.
https://www.washingtontimes.com/news/2020/apr/28/georgia-kicks-chilling-door-door-covid-19-blood-co/
Again, the type of testing is critical to specify.
No matter what HIPAA comes into play. The states can’t violate HIPAA and the privacy behind serologic testing.
Door to door pcr based infection testing what most of the community based testing is going for. Finding sources of active virus shedding falls under community health and safety. Health officials can issue a you a valid quarantine order if you are found to be actively infected, whether its TB or COVID-19 or any of a number of other contagious pathogens.
But serologic testing for any past infection (corona virus or otherwise) is another matter. Privacy here is yours and HIPAA protections apply. And a negative serologic result today says nothing about what a serologic test taken in two weeks will read-out with a highly infectious virus but frequently asympotomatic infection like the WuFlu/Chicomm-19/KungFlu/COVID-19.
Dr Birx mentioned in one press briefing that they had considered actually quarantining people separately from their household, but decided it may not be possible in America. It seemed they thought it would significantly slow down the spread.
One of the UN WHO officials a month ago suggested it might be needed to forcibly separate same house-hold family members when one tests PCR positive for an infection. The Libertarians all said on social media, “You better bring a big Army and a lot of body bags with you Mr blue-helmeted UN guy if you try that here in the US.”
That is correct. No one is taking my children from me over my live body.
The thing the meteorologists do well is data assimilation. That’s what the epidemiologists need to learn.
Meteorologic data doesn’t have privacy considerations.
1. They use data assimilation.
2. they had to wait for dead bodies.
FFS.
Always a pleasure to read Cliff Mass. Thank you. Some important breakthroughs in science have come from cross disciplinary transfers in research methodology.
In sum, if it’s not real, then it’s useless.
How can we accept a test, whose result might NOT indicate the presence of the virus that it is testing for? … a test that is prone to false positives? … false negatives? … to define a “case” that merely requires the positive that could be false for the test that might NOT indicate that the virus being tested for is present?
Great googlie mooglie! — it’s a nightmare of uncertainty that we are using as the basis for killing ourselves economically!
Wake up!
Everyone has a lifetime of experience with the weather … but only a few months experience with a partial COVID-19 cycle still in progress.
So, which subject is most likely to have a useful model?
Yeah because COVID-19 is the first novel disease we’ve encountered.
Okay, Mr. Smarty Pants Jankowski, tell us how the COVID19 story ends.
-What’s that smell, a wire burning?
-No, just ozone.
-Explain ozone ?
-Nature’s way of telling us that the weather guy who sent us here is not with us to enjoy the upcoming show.
-Ok, I got it, thunderstorm cells forming all around.
-Exactly, let’s see how you handle a weather diversion call before you fill your diaper!
Guess anyone doing predictions of what nature will do next should compulsory spend time with a seasoned grumpy steam-gauges aircraft captain.
where do you think the Chinese communists got their playbook?
Regarding testing for this virus, I have not seen anything which gives me confidence that the testing being done anywhere is specific to this virus. The testing started early so I suspect that it is not specific but for a range of such viruses. Can anyone help me here?
Well you put “How does a coronavirus test work” in an internet search engine and then you start reading.
The short answer you won’t understand is they rely on either
1.) immunoassay (detection of proteins associated with the virus ) .. marginal error rate
2.) They detect nucleic acid (virus’s genetic code) .. low error rate
However each test kit when accepted by a country will publish the expected error rate.
Now go read.
Imperial model was made 13years ago
On Monday, April 20 (I missed it at the time) there was an opinion by Andy Kessler in the Wall Street Journal. I made it mid-way in the first of 4 columns and started to laugh.
The title is “Upgrade Our 8-Track Government.”
Do your best to find and read this.
– – – –
We live in Washington State about 100 miles east of Seattle. The county has a large area and a small scattered population – students are gone from the local university, and so on.
As of today the county has 14 confirmed cases with maybe 650 tested. Number of deaths = Zero.
This is like living in a rain shadow in a state noted for rain and mist. Meteorologists might be able to help with such issues.
You’re getting F’d with a one-size fits all approach because Inslee is desperate and deranged to see Trump gone.
Proposing we exchange one set of epidemiologists and their weak models for another set of meteorologists pushing bad weather/climate models, seeking to ‘trespass’ well beyond their skill set into epidemiology, is the height of arrogance unsupported by evidence of reproducible achievement.
” Without such information, there is no way epidemiologists can realistically simulate the future of the pandemic. They are trying, of course, but the results have been disappointing.”
maybe cliff should look at spaghetti plots for Hurricanes?
With a huge data base of past hurricanes, with real time information,
weather forecasters still can only output spaghetti
Useful spaghetti, but spaghetti
Steve…don’t understand your point. Spaghetti plots are a way, one way, of displaying ensemble forecasts. This is useful information to see the variability in the solutions. What is the issue?..cliff
Compare those spaghetti plots with plots from 20 to 30 years ago. They have gotten much better.
Unlike climate models.
“We do have enough testing capability to do this (remember national political polls only use thousands of samples, not millions). Why is the epidemiological community and our political leaders not calling for such intelligent sampling of the population? With random sampling we would KNOW what is going on and not act out of ignorance (as we currently are muddling by). Why is the media not baying about this?
Huh?
There are several different purposes for testing, Figuring out the percentage infected is LOW
on the priorities.
First for PCR testing ( testing whether you CURRENTLY HAVE COVID) you do NOT WANT
a RANDOM sample.
Since you cant test everyone you want to prioritize THOSE WITH SYMPTOMS.
Why? so you can isolate and track the positives and get them the medical help they need.
So for DIAGNOSTIC TESTING ( PCR testing DO YOU HAVE IT NOW!!!) you do not want a
random sample, and you would not even KNOW HOW TO CONTSTRUCT a random sample.
would you sample mean and women equally? all ages equally? smokers non smokers?
various BMI? city dwellers? commuters? police? all these factors can effect the positive
rate you would find if you did it “randomly” because defining random cant be done in a vacuum.
So nobody with a brain is calling for RANDOM diagnostic testing, because diagnostic testing has
a PURPOSE: Identify why this person has a fever or is short of breath or has lost their sense
of smell.
That’s why in Korea they test those with symptoms and their contacts. not random, because it has
a purpose. Find the sick, find the higher probability asymptomatic and treat and isolate.
So maybe you are talking about serological testing?
This is blood test to tell if you HAD covid.
There are three purposes here.
1. Employee screening. You’d like to know how health workers, teachers, transit employees,
police, fire, people with HIGH CONTACT jobs are doing. Can you them some measure
of comfort. within these groups new york is testing randomly. It has no policy
implications.
2. Plasma donor screening. Since you have a pool of people you know had the disease, you probably
don’t need t do random testing to find more donors. but you could if tests were free and widely
available. No policy implications.
3. Disease prevalence testing. there is an academic question how many people had the disease
This allows you t better estimate the death rate. But this is not that important. We already know
the disease is deadly enough. Count the bodies.
So there is One reason you want random testing: disease prevalence and adjusting death rates.
The problem here again is how you construct a random sample. Lets take New Yorks recent
attempt to randomly sample: they went to grocery stories and got people. Was that random?
Well it all depends. Did they walk to the store? take a car? or mass transit? Do they live in
a house, large apartment building? alone? with others? work in a large office or small
Another approach is to use a marketing firm like they did in LA. Random with respect to age, sex and race
But was random with respect to Social distancing? did they select people who were heavily skewed
to those staying home 100% of the time pr not?
You see random isn’t all that easy.. because the sample has to be REPRESENTATIVE with respect to
factors that may drive infection.
Last, we won’t have widespread serology for a while. It takes time
Steve,
When we only had the ability for a few tests, then testing only those with symptoms should have priority. But even then, a small percentage of the tests should have been used for random sampling. Today, we have enough testing capability to do both and many tests are being used for cases that are unlikely to be COVID.. We are closing down vast parts of our economy without knowing the real story….random testing is the only way to do it. I am sure we can figure out reasonable ways to do the sampling…some might even use blood samples that have already been collected for other reasons…cliff
“Agent-based modeling actually try to simulate the community at an individual level and it is the most complex and computer intensive approach. Although dependent on several assumptions (such as the transmission rates between individuals) this approach is the closest to the numerical weather prediction used by meteorologists. The GLEAM model from Northeastern University (and others) is an example of this.”
effective agent based modelling is difficult. It’s like modelling the molecules that make up the wind.
And yes they are dependent on assumptions. WHY? because there is no physics of disease spread.
Modelers have to decide.. Do we model schools? businesses? mass transit? homes?
Do we model how many times young people meet old people? ( some do). Do we have any
data on how and where people meet? How many hermits are there? Hw many social butterflies?
Do we model gyms? churches? weddings? funerals? concerts? sports venues?
Structurally they are “like” weather models or models we use to predict war.
Biggest difference?
in a weather model YOU KNOW THE DATA YOU WANT TO COLLECT and that data happens every day
With a disease what data do you want to collect? Comorbidities turns out to be important!
Well that’s private data, ask yourself… do we have a table that segments the US population
by
1. Age
2. gender
3. Comorbidities
4. Occupation
5. Social interaction
6. Smoking
7. Household size
8. method of transportation
Thats just a start.
Imagine you were building a weather model and the only data you could get was
Did it rain?
Given that, predict the rest of the stuff of interest.
It is odd that governments of advanced countries do not involve their data collection and analysis agencies. In the UK with have the ONS, the office of national statistics. The produce good quality data that the gov can rely on.
They produce lists of random members of the population and then call or visit then to gather data.
Such a list of data subjects could easily include the words “you have been randomly selected to be checked for testing for covid19. Please book a visit to your nearest drive in test centre”
We have the testing infrastructure in place. All we need is a list of randomised subjects.
I wonder why they do not do it?
That would not give you a random sample.
“We have the testing infrastructure in place. All we need is a list of randomised subjects.
I wonder why they do not do it?”
1. because the testing infrastructure is being used to Diagnose those with symptoms .
you don’t learn anything by randomly sampling in your method, because it’s not
random. it might, for example, bias toward people who have cars and dont ride mass transit.
it might bias toward people who are stay at home mom’s with time on their hands. heck
in Korea I get texts 2-3 times a day saying ‘A person with Covid was within 5 minutes
of you, please go to testing” Do I? of course not. I wash my hands and don’t touch my face.
I wear a mask. I have no symptoms. Not gonna drive to a test. I don’t own a car.
2. because random sampling is HARD.
let me explain why random sampling is hard.
Lets start with a simple example where you know some things. (numbers made up to make it
easy to understand)
1. you know that men are about 50% of the population and women the other 50.
2. you want to understand the average weight of americans
3. you randomly sample .
4. you compute the average. 1t’s 160 lbs.
5. you check your sample.. it 60% men. OPPS
6. You adjust your average based on the population.
OR
7. you sample again taking care to select 1000 men (50%) and 1000 women (50%)
Now you can report the average and the average by gender.
Why? because weight varies with gender. So you have to control your random sampling
with respect to variables that MATTER, so you dont have a bias.
Now you are testing for covid.
1. you dont know what proportion of the population are BAD HAND WASHERS.
You dont know what portion are constant face touchers.
In reality, if you knew, imagine that 25% are bad hand washers and constant face touchers
2. You “randomly” choose 1000 names from some marketing database.
3. you dont know what portion are bad hand washers and constant face touchers.
4. your “random” test shows 50% of your sample is infected.
5. you never ask the question “do you wash your hands properly with soap for at least 20 seconds”
what can you conclude from the “random” test?
Was your sample representative of FACTORS THAT MATTER?
Do you even know what factors matter? you know gender matters to weight, what matters t becoming
covid infected? hand washing? train riding? you dont know with a lot of certainty.
So it could be that your random sample had 50% bad hand washers as opposed to the population
average of 25%. you dont know unless you
1. know the factors that contribute to being infected.
2. select your random sample in line with these factors.
random sampling is good, the issue is the sample also has to be representative of unknown factors that contribute to getting infected
Steve, I’ve gotten on you in the past for cryptic, drive-by flamethrowing. Recent posts , while still somewhat terse, are helpful in explaining your thoughts and have been a huge improvement. Thank you.
Steve
Good points. But we are still left with a population sample which is very useful
In Scotland, 50k people have been tested, 10k are positive. Probably from people showing symptoms. A good sample size.
First define what the sample is and how it was obtained. Everyone is clear on what you have measured and what questions you can reasonably ask. i.e. define the limits of the study.
Then you can apply randomness to your sample. For the 50k population, Take say, 200 random samples. There are proven methods of how to do this. For instance, the more random samples from your population sample you have, the closer the mean of means approaches the true sample mean.
The normal stats analyses can then be applied.
This of course ONLY applies to your original sample population. Not to the whole population.
People are rightfully suspicious of statistics. But done properly they are integral to understanding problems.
We don’t have the testing infrastructure in place.
There are testing stations, but few & far between.
The major problem in the UK & I’m guessing other countries, is that our local Environmental Health services, have been largely dismantled, as their need had been removed, with infectious diseases controlled by vaccination & drug treatments.
Thus the people are no longer there, to go & visit those who are selected for testing.
What will you respond to better, somebody from your local area, ringing your door bell, telling you who they are and asking you if they can do a test & explaining why, or a letter from some faceless bureaucrat, telling you to do a round trip of say 25 miles & report to your test centre?
Testing’s all well & good, practical measures are needed to control the infection.
Like working out where people are most likely to contract the infection?
The answer’s probably in hospital.
https://www.medrxiv.org/content/10.1101/2020.04.14.20065730v1
44% of the Chinese patients in this study, contracted Covid-19 in hospital.
Conclusion?
Dedicated facilities for handling suspected Covid-19 patients.
Cliff,
The problem with this epidemiology etc starts with two lacks.
1. There is a lack of private enterprise incentivisation at all stages of the work. Bureaucracy again makes the mistake of assuming it can be better, contrary to history in many fields.
2. There is no clear money path promoting the profit motive, thus for allowing reward for excellence and a kick up the backside for underperformance.
We have seen these 2 factors help to make climate research so poor.
Here is an example. Try, as an interested citizen scientist, to download daily atmospheric, global, digital data for CO2 for the year 2020 to date. It is an unholy mess of poor data, missing data, contradictory data and especially data that the primary agencies will not release (with the exception of Scripps and NOAA at the one site of Mauna Loa, in my efforts to date). Geoff S
They do not know the percentage of the U.S. population with active or past COVID-19 infections.
Test the sewage. Results are much quicker than waiting for someone to get sick or other testing methods.
From the article: “The UK Imperial Model in mid-March predicted 1.1-1.2 million deaths in the U.S., even with mitigation (so far the U.S. death toll has been about 60,000).”
Isn’t this old news and no longer relevant?
If I recall correctly, a mathematical error was found in this model and was subsequently corrected.
From the article: “The lack of such testing is terrible failure of multiple levels of government.”
It’s a failure on past governments to foresee such needs.
Current governments had this problem dumped in their laps unannounced and are now playing catch-up.
They had to practically start from scratch but they have done an outstanding job of ramping things up. This is not a failure. Nobody is calling a failure on ventilators now because Trump managed to ramp up production from nothing to a surplus in a matter of weeks.
The same thing is happening with the testing which is getting better and more available every day and will soon be in the category of the ventilators: More than we need.
Both weather and pandemics models are subject to sensitivity to initial conditions problems — as known from Chaos Theory — thus short term predictions are as good as our knowledge of the system and and skill of our models — but mid- and long-term predictions break down and produce wildly differing results, even from initial conditions data that are very similar.
No Weatherman would attempt to predict whether it would rain on any particular day in a particular city a year from now. Weather is far too complex and chaotic (in the Chaos Theory sense) — but because Weatherpersons have a hundred years of recorded weather history to look back on, they can tell you when the rainy season is in Cincinnati, and maybe even a probability figure for rain in the first week of June.
Pandemics are far easier than weather — less complex, less chaotic. Influenza pandemics also have a known history and we have past experience with them. They are not all the same, but they are very similar.
Let the Weatherpersons inform the Epidemiologists — teach them from their weather experience.
Note that epidemiologists not involved in the fame-producing public-posturing Panic Game have far different views than those that created the world-wide economic crash.
John P.A. Ioannidis has produced a couple of papers pointing out the misguided lockdowns — Knut Wittkowski has been so outspoken that YouTube pulled one of his video interviews as “dangerous” as he basically says that the school closings and lockdowns have produced negative effects and voices lack of support for social distancing – except for those at real risk: Old Folks and those with serious co-morbidities.
Why do we need testing? Either you’re sick or you’re not. All of this data gathering is just job security for academics. If an epidemic is starting: >65 with comorbidities, quarantine, the rest of you stay home if you’re sick (cough or fever).
For the same reasons models don’t work, can’t be trained, and depend on factors nobody can measure, the “stop Covid” bluetooth contact app can’t be trained, can’t be parameterized correctly, and can’t be useful.
I would be looking at long range weather forecasts to predict spikes in the daily death totals, maybe when the Arctic Oscillation shifts negative.