Can You Trust the Polls?

Guest Essay by Kip Hansen – 17 July 2022

Just the other day I wrote about how the NY Times failed to mention that the opinion poll they were reporting about had found that the general public had little confidence that the mass media was being honest or fair in their journalism.   

The reality is that not only does the mainstream media not report honestly or fairly about the results of opinion polls, but the opinion polls themselves are often biased.  Polls can be and are biased by their introductions – the preambles — to their questions, by the exact wording of the questions themselves, the choice of answers offered the persons answering the questions and the selection of the cohort of respondents (who is being polled).

We see a lot of poll results concerning climate, climate change, and climate change policy — but can we trust those polls to reflect the real opinions and views of the general public? Here I look at a poll on another subject as an object lesson.

Since writing the piece on confidence in the mainstream media, I have stumbled on a great example in yet another NY Times/Siena College poll.  This poll was performed before the recent Supreme Court decision on NYSRPA v. Bruen regarding a New York State law on the issuance of licenses allowing persons to carry a concealed handgun.

The poll in question was performed on 7-9 June 2022 and results reported in a press release on 16 June 1022.  On this particular issue, Siena reports:

 “Likewise, more than three-quarters of voters want the Supreme Court to uphold New York’s decades-old law requiring a license to carry a concealed handgun, including 72% of gun owners and 79% of Republicans (even more than the 77% of independents),” Greenberg said.”

Well, fair enough so far …. but what exactly did the Siena Poll ask and what answer-choices were offered?

Digging into the details of the poll (see pg. 6) we find the exact patter and question about this issue:

“Q31. New York State has had a law for decades that only allows licensed adults over 21 the right to carry a concealed handgun. Would you like to see the Supreme Court uphold that law and let New York continue to require a license for a concealed handgun,

OR

would you like to see the law overturned, eliminating the need to have a license to carry a concealed handgun? “

                                               Total      Dem      Rep      Ind/Other

Uphold law                            79%       82%     79%      77%

Law overturned                    15%       13%     16%      18%

Don’t know/ No opinion       6%         4%        5%        5%

The first sentence is the “patter”  — like that of a magician before and as he (or she) performs a stage magic trick.  I call it patter because it has the same function as the magician’s patter – to frame the scene, distract the audience, in this case the person being polled, and predispose the audience to a certain state of mind or view of what comes next.

That patter — the set-up sentence or question preamble — is exactly true.  There is no trick there.  NY State, along with 43 of the 50 U.S. states, has a law that requires persons to have a license to carry a concealed handgun.

The trick is in the full question:  “Would you like to see the Supreme Court uphold that law and let New York continue to require a license for a concealed handgun… OR  would you like to see the law overturned, eliminating the need to have a license to carry a concealed handgun? “

In June, newspapers and TV news in NY State had been talking about the Supreme Court case on NY’s gun law for months leading up to this poll, especially so in June 2022 due to a recent mass shooting incident at a Texas elementary school.  

Here’s the rub:

The Supreme Court case in question, NYSRPA v. Bruen, was not a question of whether or not New York could have a law requiring a license for a concealed handgun and overturning the existing law would not eliminate the need for a license to do so.  In other words, the question misrepresents the Supreme Court case almost entirely.

The case has now been decided, and the Supreme Court did overturn the NY State law on concealed handgun carry licenses.  But not the requirement to have a license, only how they are administered.

The Wiki has a pretty good summary:

NYSRPA v. Bruen

“The case concerned the constitutionality of the 1911 Sullivan Act, a New York State law requiring applicants for an unrestricted license to carry a concealed pistol on their person to show “proper cause”, or a special need distinguishable from the general public, in their application.”

“It made the possession of a handgun a crime without a permit, and instituted issuance of concealed carry permits at the discretion of local law enforcement. The law states that to obtain a permit, the applicant must “demonstrate a special need for self-protection distinguishable from that of the general community or of persons engaged in the same profession”.

The Supreme Court, in its ruling on the matter, found:

“In a 6–3 decision, the majority ruled that New York’s law was unconstitutional, and effectively ruled that the possession of pistols in public was a constitutional right under the Second Amendment. State licensing of firearms was not declared an infringement on that right as long as states stay within the much more common “shall-issue” systems, which may only condition licenses upon satisfying objective criteria, such as passing a background check rather than “may-issue” systems, which may be based on “arbitrary” evaluations of need made by local authorities.”  [ Wiki ]

The decision of the Court is contained in the Syllabus (pdf).  The short-form decision is:

“Held: New York’s proper-cause requirement violates the Fourteenth Amendment by preventing law-abiding citizens with ordinary self-de­fense needs from exercising their Second Amendment right to keep and bear arms in public for self-defense.”

and

“The constitutional right to bear arms in public for self-defense is not “a second-class right, subject to an entirely different body of rules than the other Bill of Rights guarantees.” McDonald, 561 U. S., at 780 (plurality opinion). The exercise of other constitutional rights does not require individuals to demonstrate to government officers some special need. The Second Amendment right to carry arms in public for self-defense is no different. New York’s proper-cause requirement violates the Fourteenth Amendment by preventing law-abiding citizens with ordinary self-defense needs from exercising their right to keep and bear arms in public. “

The Supreme Court ruling in NYSRPA v. Bruen does not affect the ability of the various states to require proper background checks and other reasonable restrictions on the issuance of concealed-carry licenses and in no way “eliminat[es] the need to have a license to carry a concealed handgun”. 

With the U.S. mid-term elections coming up in November, we will see a flood of polls intended to influence U.S. voters about candidates and issues over the next four or five months. Readers beware!

BOTTOM LINE:

1.  The wording of the Siena College poll was created either by someone incredibly ignorant of the case (which seems unlikely as the case has been a hot topic in NY State for years and Siena College is located in Loudonville, a suburb of Albany, NY, the state’s Capital) or someone intentionally biasing the question and answers for use as a political tool in the weeks just prior to the Court’s expected decision. 

2.   No number returned by, or report about the finding of, any public opinion poll should ever be accepted as factual (one could almost stop right there…) until you have personally dug in and found the full details of the poll:  the preambles, the exact questions, the allowed answers, and the details of the selection of the respondents.  It is far easier to lie with a POLL than it is to lie with statistics.

# # # # #

Author’s Comment:

I hope that comments can be restricted to a discussion of the issue at hand:  biased polls, polls twisted to be used for political or propaganda purposes and exactly how this is done. 

All of this follows from the concepts put forth in an essay published here at WUWT seven years ago: “What Are They Really Counting”.  Numbers are just numbers – all the important facts about those numbers lie in the details of exactly what has been counted and how it has been counted.  

I have personal opinions on the issues involved in NYSRPA v. Bruen and, more generally, the subject of gun control in the United States — BUT — I will not be discussing them in the comments.

Of interest, today’s news has a story in which an armed bystander, The holder of a concealed carry license, shot and killed a man in an Indiana mall, cutting short a mass shooting, saving an unknown number of lives.

Remember to ask:

What exactly are they really counting?

Is the thing they counted really a measure of the thing being reported?

# # # # #