Can You Trust the Polls?

Guest Essay by Kip Hansen – 17 July 2022

Just the other day I wrote about how the NY Times failed to mention that the opinion poll they were reporting about had found that the general public had little confidence that the mass media was being honest or fair in their journalism.   

The reality is that not only does the mainstream media not report honestly or fairly about the results of opinion polls, but the opinion polls themselves are often biased.  Polls can be and are biased by their introductions – the preambles — to their questions, by the exact wording of the questions themselves, the choice of answers offered the persons answering the questions and the selection of the cohort of respondents (who is being polled).

We see a lot of poll results concerning climate, climate change, and climate change policy — but can we trust those polls to reflect the real opinions and views of the general public? Here I look at a poll on another subject as an object lesson.

Since writing the piece on confidence in the mainstream media, I have stumbled on a great example in yet another NY Times/Siena College poll.  This poll was performed before the recent Supreme Court decision on NYSRPA v. Bruen regarding a New York State law on the issuance of licenses allowing persons to carry a concealed handgun.

The poll in question was performed on 7-9 June 2022 and results reported in a press release on 16 June 1022.  On this particular issue, Siena reports:

 “Likewise, more than three-quarters of voters want the Supreme Court to uphold New York’s decades-old law requiring a license to carry a concealed handgun, including 72% of gun owners and 79% of Republicans (even more than the 77% of independents),” Greenberg said.”

Well, fair enough so far …. but what exactly did the Siena Poll ask and what answer-choices were offered?

Digging into the details of the poll (see pg. 6) we find the exact patter and question about this issue:

“Q31. New York State has had a law for decades that only allows licensed adults over 21 the right to carry a concealed handgun. Would you like to see the Supreme Court uphold that law and let New York continue to require a license for a concealed handgun,


would you like to see the law overturned, eliminating the need to have a license to carry a concealed handgun? “

                                               Total      Dem      Rep      Ind/Other

Uphold law                            79%       82%     79%      77%

Law overturned                    15%       13%     16%      18%

Don’t know/ No opinion       6%         4%        5%        5%

The first sentence is the “patter”  — like that of a magician before and as he (or she) performs a stage magic trick.  I call it patter because it has the same function as the magician’s patter – to frame the scene, distract the audience, in this case the person being polled, and predispose the audience to a certain state of mind or view of what comes next.

That patter — the set-up sentence or question preamble — is exactly true.  There is no trick there.  NY State, along with 43 of the 50 U.S. states, has a law that requires persons to have a license to carry a concealed handgun.

The trick is in the full question:  “Would you like to see the Supreme Court uphold that law and let New York continue to require a license for a concealed handgun… OR  would you like to see the law overturned, eliminating the need to have a license to carry a concealed handgun? “

In June, newspapers and TV news in NY State had been talking about the Supreme Court case on NY’s gun law for months leading up to this poll, especially so in June 2022 due to a recent mass shooting incident at a Texas elementary school.  

Here’s the rub:

The Supreme Court case in question, NYSRPA v. Bruen, was not a question of whether or not New York could have a law requiring a license for a concealed handgun and overturning the existing law would not eliminate the need for a license to do so.  In other words, the question misrepresents the Supreme Court case almost entirely.

The case has now been decided, and the Supreme Court did overturn the NY State law on concealed handgun carry licenses.  But not the requirement to have a license, only how they are administered.

The Wiki has a pretty good summary:

NYSRPA v. Bruen

“The case concerned the constitutionality of the 1911 Sullivan Act, a New York State law requiring applicants for an unrestricted license to carry a concealed pistol on their person to show “proper cause”, or a special need distinguishable from the general public, in their application.”

“It made the possession of a handgun a crime without a permit, and instituted issuance of concealed carry permits at the discretion of local law enforcement. The law states that to obtain a permit, the applicant must “demonstrate a special need for self-protection distinguishable from that of the general community or of persons engaged in the same profession”.

The Supreme Court, in its ruling on the matter, found:

“In a 6–3 decision, the majority ruled that New York’s law was unconstitutional, and effectively ruled that the possession of pistols in public was a constitutional right under the Second Amendment. State licensing of firearms was not declared an infringement on that right as long as states stay within the much more common “shall-issue” systems, which may only condition licenses upon satisfying objective criteria, such as passing a background check rather than “may-issue” systems, which may be based on “arbitrary” evaluations of need made by local authorities.”  [ Wiki ]

The decision of the Court is contained in the Syllabus (pdf).  The short-form decision is:

Held: New York’s proper-cause requirement violates the Fourteenth Amendment by preventing law-abiding citizens with ordinary self-de­fense needs from exercising their Second Amendment right to keep and bear arms in public for self-defense.”


“The constitutional right to bear arms in public for self-defense is not “a second-class right, subject to an entirely different body of rules than the other Bill of Rights guarantees.” McDonald, 561 U. S., at 780 (plurality opinion). The exercise of other constitutional rights does not require individuals to demonstrate to government officers some special need. The Second Amendment right to carry arms in public for self-defense is no different. New York’s proper-cause requirement violates the Fourteenth Amendment by preventing law-abiding citizens with ordinary self-defense needs from exercising their right to keep and bear arms in public. “

The Supreme Court ruling in NYSRPA v. Bruen does not affect the ability of the various states to require proper background checks and other reasonable restrictions on the issuance of concealed-carry licenses and in no way “eliminat[es] the need to have a license to carry a concealed handgun”. 

With the U.S. mid-term elections coming up in November, we will see a flood of polls intended to influence U.S. voters about candidates and issues over the next four or five months. Readers beware!


1.  The wording of the Siena College poll was created either by someone incredibly ignorant of the case (which seems unlikely as the case has been a hot topic in NY State for years and Siena College is located in Loudonville, a suburb of Albany, NY, the state’s Capital) or someone intentionally biasing the question and answers for use as a political tool in the weeks just prior to the Court’s expected decision. 

2.   No number returned by, or report about the finding of, any public opinion poll should ever be accepted as factual (one could almost stop right there…) until you have personally dug in and found the full details of the poll:  the preambles, the exact questions, the allowed answers, and the details of the selection of the respondents.  It is far easier to lie with a POLL than it is to lie with statistics.

# # # # #

Author’s Comment:

I hope that comments can be restricted to a discussion of the issue at hand:  biased polls, polls twisted to be used for political or propaganda purposes and exactly how this is done. 

All of this follows from the concepts put forth in an essay published here at WUWT seven years ago: “What Are They Really Counting”.  Numbers are just numbers – all the important facts about those numbers lie in the details of exactly what has been counted and how it has been counted.  

I have personal opinions on the issues involved in NYSRPA v. Bruen and, more generally, the subject of gun control in the United States — BUT — I will not be discussing them in the comments.

Of interest, today’s news has a story in which an armed bystander, The holder of a concealed carry license, shot and killed a man in an Indiana mall, cutting short a mass shooting, saving an unknown number of lives.

Remember to ask:

What exactly are they really counting?

Is the thing they counted really a measure of the thing being reported?

# # # # #

5 17 votes
Article Rating
Newest Most Voted
Inline Feedbacks
View all comments
Tom Halla
July 19, 2022 6:15 pm

The wording of the question asked mostly missed the matter of dispute, whether New York could have arbitrary choice on who gets a permit, without clear standards.
This poll clearly fits Rush Limbaugh’s assertion that polls were mostly to hang a news narrative on, not determine actual public opinion

jeff corbin
Reply to  Tom Halla
July 21, 2022 11:34 am

Most of the conservatives I know in both parties do not respond to polls for the reasons you have pointed out. I think, this is the reason the television news media was blind sided when Trump was elected. This was very surprising to me since the fib plot progression of the poll compost (60-80 polls, mostly local papers) pointed clearly to a Trump win 8 days out from the election. As far as the media was concerned, a beast had risen silently from the deep. Since then it has been Katy bar-the door.

July 19, 2022 6:20 pm

Don’t about half the states now have constitutional carry where no license is required for concealed carry? Don’t know where the 43 comes from.

Old Man Winter
Reply to  Kip Hansen
July 19, 2022 7:28 pm
Reply to  Kip Hansen
July 20, 2022 10:16 am

Kip I see where you got that number though 25 have constitutional carry and most of those do have the option of getting a permit I think that is where the disconnect happened, most of them are shall issue still, Vermont being the exception ironically since they don’t have a permitting process and allow constitutional carry.

Paul Penrose
Reply to  Kip Hansen
July 20, 2022 10:43 am

In the states that have constitutional carry AND licensing procedures, the license is optional and only to allow the citizen to carry in states that require a license (and have CCL reciprocity).

July 19, 2022 6:22 pm

Kip, do you plan to do an article on the way pollsters pick their samples in order to influence the results?

Last edited 28 days ago by MarkW
Reply to  Kip Hansen
July 19, 2022 7:55 pm

Of course a “nationally representative sample” leaves open the question of determining what a “nationally representative sample” is and then how do you convince yourself that your actual sample matches your target sample.
But we are getting off topic so I will shut up now.

Reply to  MarkW
July 19, 2022 11:46 pm

That’s easy. Represent age groups, income, place of residence , gender , race , education levels. It’s no big secret.
What happens is they get older and white people easily, it’s harder for younger , lower education , non white ethnicity and more rural areas.

As above it’s the questions that are more problematic

Reply to  Duker
July 20, 2022 9:10 am

So it’s OK if all your respondents are Democrats, so long as the other boxes get ticked?

July 19, 2022 6:32 pm

The Doran and Zimmerman 2008 paper is a case in point. The question they asked in their survey was “Do you think human activity is a significant contributing factor in changing mean global temperatures?”. The results were reported as “97 per cent of scientists agree that climate change is caused by humans”. (eg.

Reply to  Mike Jonas
July 19, 2022 7:57 pm

The biggest problem with that question is that the word significant is never defined.
To a statistician, significant is anything that is non-trivial. To many lay people, it means a large fraction, or perhaps even a majority.

Reply to  MarkW
July 19, 2022 8:47 pm

Agree. In most contexts, 10% or even 5% would be regarded as significant, as in eg. 10% of annual deaths in Oman are from road accidents
Another possible measure could be statistical significance, but not really in general discussion.

Reply to  Kip Hansen
July 20, 2022 10:08 pm

In medicine there lots of statistically significant results in studies that are clinically insignificant. The larger the study population the easier it is to get a P-value of <0.05 but still it may not be of any practical importance. Google “p-hacking”.

Reply to  Mike Jonas
July 20, 2022 6:56 am

Polling the public on global warming/climate change is relentless since the practice is effective to raise public awareness and concern to support emissions policies. I did a review of questions and answer options based on studies from Stanford researchers.

“Thus it is that survey results are influenced greatly by the design of the questioning process. Helpfully, the Stanford program provides this history of the questions put to participants over the years. Below are the result categories, some showing the evolving form of questioning, and others just the most recent form for brevity.”

July 19, 2022 6:38 pm

Another polling issue almost no one talks about is what to do about the people who refuse to answer poll questions. These can easily be as many or more people than the number who are willing to answer. So we assume those who refuse would have the same proportion of responses as those who do answer? Why? Because that’s all you can do. But that fundamental problem makes polling very error-prone. I don’t bother believing any polls provide useful information. Not much better than astrology.

Last edited 28 days ago by Hoser
Reply to  Hoser
July 19, 2022 7:12 pm

“Not much better than astrology.’

That describes CAGW “science” as well as polls.

Paul Penrose
Reply to  Kip Hansen
July 20, 2022 10:46 am

Especially when the polls are long, as was this case in the one you have pointed out here.

James F. Evans
July 19, 2022 7:06 pm

No you can’t.

July 19, 2022 7:27 pm

I have a conspiracy non-ideation, a conspiracy fact, that at least one series of political polls were made up in their entirety in France, those for the primary of the right and center when François Fillon was opposed to Alain Juppé.

It’s obvious that the whole saga doesn’t add up and:

  • Alain Juppé, the right wing candidate favored by the left, was propped the whole time
  • at the end, they back adjusted the bias to zero
  • they did so in a urgent way

The story is that Alain Juppé was nicknamed Ali Juppé and that a fake news re: “la grande Mosquée de Bordeaux”, distributed by the “fachosphère” (the French alt right?), a building that never existed wrecked his candidacy.
But I never heard about in in the “fachosphère”. The Islamophile tendencies of Juppé were well established. The promotion of the fake “fake news” about the “grande mosquée”-that-never-was was done by the anti conservative MSM: we were warned for days and days to NOT pay attention to the story of Juppé agreeing to built a mosque.

That was like a preemptive Patriot bouquet sent against non flying missiles! Imagining sending a bunch of missiles and ending up carpet bombing residential areas with a bunch of lost antimissiles.

The pro Juppé carpet bombing of Patriots made non sense.

I checked at the time on Google that the interest for the story only grew AFTER the MSM carpet defended Juppé! Clearly the MSM crashed Juppé, not the fachosphère! It’s factual, based on Google stats. But then, it doesn’t make sense. Why would they?

Clearly Juppé was their favorite candidate. Then they trash him?

I was following social media a lot and people on the right didn’t like Juppé at all. He was only supported by the left as a “less evil” right wing candidate. He clearly had very little support. I was surprised he made up to the second turn of the primary, with support from the left (open primary as we aren’t allowed to declare political affiliation in France).

There was no way Juppé could win. But they couldn’t accept a discontinuity between Juppé polling high and an humiliating defeat in the second turn of the primary.

So they adjusted the bias to zero. Like President Clinton setting civilian GPS to bias to zero. Except GPS bias is a well documented thing and allowed to exist, and political polls bias would not only be secret, illegal but also a clear interference in elections.

But the data can only be explained like I just did:

Juppé primary candidacy polls crashed well before the spike in interest in “grande mosquée” on Google.

It’s cheating beyond any reasonable doubt.


  • Juppé was promoted by the left
  • he was always hated by the true patriots
  • MSM ostensibly defended Juppé against defamation will harming him as much as they could
  • MSM was manipulated into doing that by the deep state pollsters to cover for bias backtracking
  • Juppé would never have won anyway

The rest of my message has less to do with polls and everything to do with the deep state.

These are obvious facts. I have a theory that the deep state also absolutely wanted Fillion to win the primary because they knew of an October surprise coming up in January 2017.

Of course Fillion being a very weak and silly pro politician, he reacted badly and the surprise crushed him. We literally had 14 days of the MSM only talking about his minor “scandal” and then commentators complaining about the fact we didn’t focus enough on … that “scandal”.

So the deep state made him win. I know it sounds contradictory with the fact that he could only crush Juppé, but that’s because the deep state must be a weakly coordinated non entity, a bunch of shadowy actors doing bad things with no one in charge.

I insist that these are extremely parsimonious explanations. I tried a lot to explain away those facts without shadowy agents. I doesn’t work.

Old Man Winter
July 19, 2022 7:39 pm

I don’t do polls because I’ve seen how badly the results were distorted in the two-question
glow-bull warming poll. Since the MSM is mostly BeeEss, I assume the opinions will reflect
that. It’s just another way for them to repackage their garbage & is a total waste of my time!

Last edited 28 days ago by Old Man Winter
Old Man Winter
Reply to  Old Man Winter
July 19, 2022 8:28 pm

The repackaging is essentially the propagandist creating a harmless third entity- like a ventriloquist
doll or a sock puppet- through whom they can spew their propaganda without having the
appearance of it being from them, thus removing any resistance the viewers/readers/listeners may
have toward them. Sleight of hand, so to speak!

Polls used to carry a certain authority as we had been taught to give people the benefit
of the doubt & assumed they asked fair questions & tabulated the results to give us an
honest answer. Given only 11% of the people trust TV news & a few more the
newspapers, they lost our trust & the authority polls used to have. That’s their fault.

Last edited 28 days ago by Old Man Winter
Old Man Winter
Reply to  Kip Hansen
July 20, 2022 9:39 am

Those K-tel & Ronco ads in the ’70s taught us to be skeptical! 😮

Nick Graves
Reply to  Old Man Winter
July 19, 2022 11:44 pm

I do. I usually give the answers they don’t want.

If there is a ‘comments’ box at the end, I invariably leave a comment about ‘loaded questions’.

It’s probably futile, but at least it makes me feel better that they know I’ve rumbled them.

Gunga Din
Reply to  Nick Graves
July 20, 2022 7:47 am

How do you know your answer won’t be spun to support the answer they do want?

Q: Are you concerned about what is being done to combat climate change? Yes/No

I could answer “yes” to that because what they are doing is a waste of time and money and is harming people.
(AOC could answer “yes” because she thinks not enough is being done.)
I could answer “no” for the same reasons. Climate Change is not a problem so we don’t need to anything.

Either answer could be spun to conclude I agree with AOC, not enough is being done, or spun to say we should stay the course and continue to eliminate fossil fuels.

Frank S.
July 19, 2022 7:45 pm

“It is better to take refuge in the Lord, than to trust in man.” Ps. 118:8 God’s guidance has never let me down.

The Emperor's New Mask
July 19, 2022 7:56 pm

Lies, damned lies, and polling.

Roger Bournival
July 19, 2022 8:26 pm

As far as conducting a poll is concerned, always remember what I call the ‘deli guy thumb’ on the scale in favor of Democrats whenever polls are conducted. Polls of all stripes for the past 20 years (at least) almost always oversample Democrats by a few or a lot of percentage points and that fact isn’t necessarily disclosed, or it takes effort to dig up.

If there is an issue that can be wagered on, like Brexit and Trump getting elected as just two obvious examples of this divergence polls and betting sites, you’re much better to follow sites like Ladbrokes to see where people are putting money on it instead of polling outfits trying to blow smoke up my ass.

July 19, 2022 9:10 pm

The real problem is that there are so many bad polls or bad reporting on polls that I don’t believe anything concerning polls. That is a shame.

Shoki Kaneda
July 19, 2022 9:10 pm

I trust polls. I trust them to be misleading, deceptive and incorrect.

July 19, 2022 9:23 pm

Psephologist – sociologist who studies election trends.

These people focus on electorates, polling booths, electoral rolls and the past elections voting trends (and more information) to calculate and predict future election results.

The same can be applied to conducting polls by phoning people in areas surrounding polling booths where the vote favoured the side the pollster wants to promote.

Andy White
July 19, 2022 10:31 pm

This clip from a BBC series “Yes Minister” addresses the ability to manipulate poll results well.
Yes Prime Minister – Sir Humphrey shows poll rigging – YouTube

July 19, 2022 11:47 pm

This is a reply to: Andy White July 19, 2022 10:31 pm (awaiting approval)

Bloody brilliant!

My favorite poll is the Gallup: Nation’s Most Important Problem

They ask :

What do you think is the most important problem facing the country today?

Then they sort the answers into categories.

The poll is done once a month and some of the categories are sensitive to what’s in the news. For instance ‘Guns/Gun control’ were 8% in June, up from less than 1% in March.

‘Environment/Pollution/Climate change’ is 2% and hasn’t changed in a few months.

‘Energy/Lack of energy sources’ is also 2% and does change.

My guess is that there’s a committed 1% who will always answer ‘Climate change’. I also guess that the majority of people see energy as a bigger problem than global warming.

Given the heat wave in Europe right now, and the coming shortage of (Russian) natural gas, I wonder what kind of results the Most Important Problem poll would get in Europe. 🙂

Richard Page
Reply to  commieBob
July 20, 2022 3:13 pm

West Europe where it’s rather warm or East Europe where it isn’t?

Andy H
July 20, 2022 12:36 am

Here is how it works. The pollsters have a long list of people they can poll. They have previously polled them and know their views on a wide range of views, from energy to choice of cheese. For example, they know that 80% of people who like blue cheese and 40% of people who don’t like blue cheese will be in favour of issue X. (If they aren’t sure, they can do a small poll). So if they make 90% of the people polled blue cheese lovers then they will get a result of 0.8*0.9+0.1*0.4= 76% of people they polled are in favour of issue X. This probably isn’t representative of the population, but it gets them the result they want. Don’t believe polls.

Reply to  Andy H
July 20, 2022 6:04 am

The MSM conducts and uses polls to shape opinion, not reflect opinion.

That is the reason behind the ‘loaded’ or leading questions that Andy May pointed out in the article.

Old Man Winter in a comment upthread gave the ‘why’ for taking polls. The MSM is using them as a ventriloquist dummy, a *ahem* neutral 3rd source to back their propaganda.

The MSM points to the polls to say in effect, “See? What we’ve been pushing on you in our medium is what people are thinking, too.”

Or they can do the poll first and use the predetermined, desired result to say, “See? This is what most people are thinking. You should be thinking that too.” And then they put out supporting propaganda.

The polls you see from the MSM are used to shape opinion, not to reflect opinion. When you see a poll in the news you should ask yourself, “What narrative are they trying to push with this poll?”

Reply to  Kip Hansen
July 20, 2022 9:32 am

You write, “many polls” and my point is that all of them conducted by the MSM are not to be trusted.

Our (U.S.) MSM’s job is to shape public opinion and cover for the Dems when things go pear shaped. Polls are just one tool in the toolbox.

You will hear of ‘internal polls’ from time to time, particularly around elections. Those polls are conducted by the candidates or the parties to gauge their candidate’s standing. Even those may be untrustworthy on the rare occasion they are released, and they are only released when they support the narrative.

Those polls are used to help determine where to spend campaign money and how much to spend.

July 20, 2022 12:37 am

since I suspect there will be a mysterious absence of articles on yesterday’s UK record temperatures, I’ll repost this here where its current:

The UK temperature record set only 3 years ago was broken in 29 different places across England yesterday… by 1.5 degrees, with multiple instances of the first 40C temp records for the UK. New record of 34 for Scotland (21 is hot in Scotland in the summer!).

given that before yesterday 7 of the top ten UK temps had been this century and that all the 10 hottest years have been since 2002, are Watts readers still going to insist this is ‘normal’ for the UK and somehow an isolated incident?

It is plainly climate change…

and if you look across Europe where even the Netherlands is seeing temps approach 40C… heatwave in Spain, Portugal, France, Italy…

Well I’ve leave you now to prepare your feeble excuses for the heat dome in the US and approaching new Texas record temp…

Reply to  griff
July 20, 2022 4:30 am

No, my dear griff. God is unhappy with the fact that we have killed her and is giving us a taste of the afterlife we’ll earn if we don’t shape up.

Pick your favorite issue and use a perfectly natural heat wave as a way of pushing it.

We’ve only had thermometers since the Little Ice Age. What was going on during the Holocene optimum? All you can say is that it hasn’t been this hot in a couple of hundred years.

Reply to  griff
July 20, 2022 7:59 am

“… I suspect there will be a mysterious absence of articles on yesterday’s UK record temperatures …

and as usual, you would be wrong:

Hump Day Hilarity – EU ‘Smack in the Face’ Heatwave Edition

It was totally predictable that the media would start screeching the usual screechiness they employ whenever there is a heat wave anywhere in the world. Some headlines were a bit over the top

Gunga Din
Reply to  Kip Hansen
July 20, 2022 8:32 am

Should we take a poll about moving griff’s comment to one or both of the articles he says isn’t there? 😎

Gunga Din
Reply to  Kip Hansen
July 20, 2022 10:37 am

But sometimes it’s fun!

Brent Wilson
Reply to  griff
July 20, 2022 8:23 am

Weather, Griff.

Richard Page
Reply to  Brent Wilson
July 20, 2022 3:16 pm

Actually it’s probably all UHI and jet exhaust! Alarmist hysteria and scare-mongering.

July 20, 2022 1:10 am

Let’s do our own polls. Zogby and others will do the poll if you pay them, less than $10k, depending on the sample size. Start a gofundme or whatever to raise the money or do it here

Reply to  jimmywalter
July 20, 2022 6:06 am

And let’s be sure to let them know the results we expect, eh?

James F. Evans
July 20, 2022 1:12 am

It’s out of order, the World Economic Forum & the United Nations(in coordination) issued a statement that 2030 goals must be achieved by 2030.

Polls… who manipulates the Polls?

This is the time,.. people are facing how expensive & unrealistic the Green Cult is.

People are more open to the science than, perhaps, since all this madness started.

It’s a jump ball.

July 20, 2022 1:17 am

I hope this is an issue in the voting:

Over 100m Americans under heat warnings as wildfires rage in 12 statesBlazes have scorched over 3m acres as temperatures expected to reach 100F in much of the Plains and Mississippi Valley

Ed Zuiderwijk
July 20, 2022 3:45 am

The words ‘poll’ and ‘pollution’ derive from the same root?

July 20, 2022 4:19 am

Kip, there’s an incredible expose on polling at the “Expert Source Bureau. The article can be found at . The article was reviewed by one of the best pollsters in the world for clarity and integrity. It’s a bit old, but still “right on.”

Reply to  McGrats
July 20, 2022 6:23 am

👍 👍 I had not run across that article before, McGrats. But I have reached similar conclusions, though just based on observations and things pointed out to me by others.

Old Man Winter hit on much the same as this article discusses earlier in this comment thread.

That article is a really good read. Thanks for the link.

Smart Rock
July 20, 2022 4:57 am

As far as I know, opinion polls are usually (always?) done by telephone. In recent years, I’ve found that many people don’t answer their phones unless they recognise the caller, and sometimes even if they do recognise the caller. This social trend is going to skew poll results in unpredictable ways.

It would be an interesting study to look at the demographics of who would answer the phone when a pollster calls, and who wouldn’t.

Here’s an example that I think demonstrates my point: Last year, during the pandemic panic, there was a poll in the UK that said (IIRC) 40 percent of respondents wanted mask wearing to be mandatory for everyone, for ever and ever. I though that this was highly unlikely, and I did a simple thought experiment asking myself who would be most likely to answer the phone at that time and in that place. And my answer was, folk who had swallowed the most dire warnings from our so-called leaders, and who stayed at home, wearing masks, sheltering in terror from the swarms of angry viruses battering against their doors and windows – they would probably be the dominant phone-answering group because they would all be bored stiff and looking for even the most minor distraction – and a human contact without fear of infection. And of course there would be a very high proportion of lockdown enthusiasts in that group.

Tim Gorman
July 20, 2022 6:09 am

Polls can be and are biased by their introductions – the preambles — to their questions, by the exact wording of the questions themselves, the choice of answers offered the persons answering the questions and the selection of the cohort of respondents (who is being polled).”

Never forget to add that the questioner can bias the answers using their spoken emphasis and intonation while reading the question and possible answers.

Walter Sobchak
July 20, 2022 8:10 am

Once upon a time a very long time ago, I was a graduate student at a major public university in a social science. One of my methodology courses was a an advanced course in statistics. As part of that course we had full access to an important body of raw survey data.

In the course of my work I came the conclusion that polling data is worse than useless. Everything I have seen since then has reinforced that idea.

The problems I saw then were rooted in the baked in bias of the way questions were worded. The necessity of being concise enough to ask a question that does not leave the average respondent saying huh, means that key details like those discussed above must be elided.

Further the questions are written by pollsters who are political junkies and who speak a language of politics that is not understood by ordinary people. Thus a question like are you a conservative on a 1 to 10 scale and are you a liberal 1 to 10 that should, in the pollsters minds, produce a 100% correlation, instead produced a 0% correlation.

Then there is the issue of social subtext. Most polling is conducted by having one person ask questions of the respondent. The mere act requires the negotiation of a social relationship between the pollster and the respondent. Since most people are polite and conflict averse they will try to answer questions in a way that they feel the pollster would want them to. Add vast differences in class, race, and language, and there is almost no way that the answers to polling questions reflect the true opinions of the respondents.

A final issue is the difficulty of finding people who will answer polling questions was large then and a million times worse now. At the beginning of polling, pollster went door to door to find respondents. By the sevnties most people wold no longer admit strangers into their houses for fear of crime . Then they went to telephone polling, and that has gotten to be almost impossible as well because many people will no longer answer the phone because of spaming.

Bottom line. Polls are worthless except as objects of derision.

Walter Sobchak
Reply to  Kip Hansen
July 20, 2022 11:48 am

I wouldn’t bother to spend the time to figure out which ones were any good.

Walter Sobchak
Reply to  Kip Hansen
July 20, 2022 5:43 pm

Good luck.

Christopher Paino
July 20, 2022 8:50 am

A percentage without the whole number the percentage is based on is meaningless.

Tim Gorman
Reply to  Christopher Paino
July 20, 2022 10:00 am

A measurement without an uncertainty interval is useless as well. What’s the uncertainty on all of these temp measuring stations?

July 20, 2022 9:36 am

The simple answer to “Can You Trust the Polls?” is no.
I haven’t participated in a poll in about 40 years. I have read the questions on mailings from both sides of an issue, since I belong to different institutions that are diametrically opposed to each other. The ones that are opposed to my own beliefs are required for my employment so it’s not possible to drop them for a few years (hopefully) but I am still inundated with the propaganda. Even with the ones that I agree with show an obvious bias.
Both sides are not so much interested in my opinion but my financial backing. Most of the questionnaires are nothing more than appeals for a check or, God forbid, a credit card.

July 20, 2022 12:51 pm

All polls are suspect.
An average of polls is a little better.
The trend of an average of polls is best.
Which is why, when you look at the Real Clear Politics average of polls regarding job approval for President Biden, you know that he’s toast – from +20 approval in Jan 21 to -20 disapproval now.

Richard Page
Reply to  Kip Hansen
July 20, 2022 3:24 pm

Indeed he appears to have been having a ‘senior moment’ for over 2 years straight. Could have been worse though; here in the UK our outgoing PM married his ‘difficult time.’

Call me a skeptic
Reply to  Richard Page
July 20, 2022 6:45 pm

The problem is you can’t trust democrat operatives. They will attempt to steal as many elections as they can. A reckoning is coming and it won’t be pretty.

%d bloggers like this: