Why does political polling suck so much?

Back in the 2016 election cycle, I texted back and forth with a lobbyist friend in Washington DC nearly every day sharing counterpoints to political polling. Nearly everyone I knew was voting for Trump, and the folks who were voting for Clinton had lost it mentally in ways I could not describe. They were becoming so obsessed with politics – a topic they had shown zero interest in previously – that they were calling their now-former best friends Nazis and fascists on social media, and then writing long confessional posts about how they were being socially blackballed. These used to be normal human beings, but they had been radicalized online like freaking terrorists. It was utterly bizarre. And many people I knew had had similar interactions with friends and family members who had started to speak like political jihadis. It was like the “toasters” being activated in Battlestar Galactica. Like someone whistled a tune and they became something totally other, and everyone around them was shocked they had been carrying around that potential all along. Many folks thought it would end with the election, but it clearly didn’t.

I had lost count of the number of lifelong Democrats I knew who were switching parties, partly because they genuinely loved Trump’s tough guy antics and partly because they felt alienated by the “new” Democratic Party. These are people who had voted for Obama twice, defended him through his presidency, and then became adamant Republicans. Why? During the Obama era, Planned Parenthood replaced labor unions (company labor unions, not public labor unions) as the largest source of campaign funding for the Democratic Party. With widespread approval of right-to-work laws, that gravy train ended, and Planned Parenthood stepped in to fill the money vacuum. With new financial masters, the Democratic Party became solely focused on identity politics – because that is what drives Planned Parenthood’s business model – which many, many, many people absolutely hated. Politics ceased to be a team sport for them. They did not want to belong to the party of drag queens and partial birth abortion. They cared about the economy and taxes and wars. That was the stuff Trump talked about. It remarkably wasn’t even a tough decision. The new Democratic Party repulsed them.

My lobbyist friend and I had more accurate electoral projections than any political pollster and certainly any talking head (we were off by two states, but only because both flipped in the other direction, creating an essentially correct call). It seemed fascinating to me back then how awful polls were. I could not tell if people in politics were innumerate or if there was a deliberate reason polls were the way the way they were. How do you have soooooo much money spent on a product and have the results be so terrible? That’s something that would never happen in business. If someone dumps millions of dollars on a service, they expect it to work. The only explanation is the polls are “working” for someone, it’s just their desired function is now something other than calling elections accurately.

And now it seems like we are watching a literal repeat of 2016, but on crack. I have yet to meet a single person under the age of 65 who is excited about voting for Biden. And really, most of the people over the age of 65 loathe the guy and think he will be a puppet for the scary socialist left. But polls are showing that Biden is not only leading, but has a truly monstrous double-digit lead over Trump, even in historically conservative states. It’s hysterical. Massive numbers of Republicans are switching parties! Look at all the people in the “Republican” security apparatus who have endorsed Biden! Colin Powell, who hasn’t supported a Republican since the George W. Bush era, supports Biden! Something about Republicans wanting to get out of never-ending wars in the Middle East seems to have offended the security state, so weird – but I bet it will be wildly unpopular with ordinary Americans too!

If anything, Biden is looking at losing a large portion of what was traditionally the Democratic base. I have never encountered so many Black conservatives in my life, and Black conservative commentators have amassed millions of followers in the Trump era. I don’t think BLM riots and Google’s “Black Trans Lives Matter” doodles are playing well in culturally conservative Black churches. I don’t think the scamdemic closing down Black churches indefinitely is playing well either, which is probably a bigger deal. Whether that means they will vote for Trump or simply sit out the election remains to be seen. But neither is good news for the senile curmudgeon mask lord.

Beyond that, I think most of America is existentially exhausted with career politicians going around breaking shit over decades-long petty grudges and ego trips. The scamdemic was the last straw, seeing what this dysfunction can achieve in terms of wrecking people’s lives par excellence. And Trump may say crazy stuff, but he’s always on the side of normalizing things, not seeing how far the batshit crazy can go. Only the most rabid partisan leftists don’t realize that at this point. Trump is Mr. Let’s Go Back to Normal Days and Make Some Money. That’s his entire platform. That doesn’t sound like Hitler to ordinary people. Especially when the people he is up against are aggressively tearing up social institutions.

There is no denying that political polls nowadays are bunk from a mathematical standpoint. Scott Hounsell has a great explanation today on the bizarre mechanics of polling now:

On September 1, 2020, The Economist/YouGov released a National Presidential “Poll” [Quotes Added] in which they declare Joe Biden has a monstrous 11 point lead on Donald Trump.  As a polling nerd, I decided to take a deeper look at it.

In order to analyze the data, we must first identify what we are looking at here. First, is what polling firms call “topline” or the data that is presented to you as the consumer. The topline data is the direct answers to the questions for the poll.   We ask 100 people, for “who are they voting?” 52% say Bob, and 48% say, Steve.   That is topline data.

Further down in most polls, you find cross-tab and methodology information. It shows how people, with certain demographic and other identifying information, respond to certain questions within that poll. If the questions for 100 people is, “for whom are they voting?”,  topline data will say that 52 – 48 with Bob in the lead, but cross-tabs, or applying a filter of how people who answered a question one way answered another question, will tell us that 88% of Democrats are voting for Steve, and 95% of Republicans are voting for Bob, as applied to the topline number.

Primarily, when I look at polls, topline means very little to me in the long-run.  I want to see the methodology (how the people who participated were chosen) and the cross tabs (how those people voted and if possible, why).

When reviewing polls in 2016, I came across bad poll after bad poll as a result of oversampling problems as well as push-poll lines of questions or questions that are framed in a way to influence the way someone answers.  For instance, if a polling firm asked a question “Do you agree with Joe Biden’s call for all Americans to unite?”, it doesn’t leave any room for someone saying they disagree with the premise of the question, that is, Joe Biden even having that ability in the first place.  Another example of a push poll question is “Do you support Donald Trump’s divisive rhetoric?”  Maybe the respondent doesn’t believe Trump’s rhetoric is divisive. Maybe the respondent thinks the Democrats’ rhetoric is divisive.  Either way, a respondent is less likely to respond in the affirmative to supporting divisive rhetoric and therefore would lead to a skewing of the results.

Polls should also not be opt-in.  I will only say this once: ANY OPT-IN POLL WILL DELIVER AN FLAWED RESULT.  Opt-in polling allows for people to seek out a poll or even be paid to participate in the poll.  The moment a poll removes the random and unbiased nature of the results, those results are invalid, full stop.

Polling should ALWAYS be only of Registered Voters.  Asking a non-registered voter or an ineligible voter their opinion on a voting issue is the equivalent of asking the Queen of England who is she is voting for in the US Presidential election.  In other words, including “US Adults” in a poll should never (and I repeat NEVER) be considered an accurate poll.  Questions asking respondents whether or not they are registered are worthless as they are self-reported and could be completely wrong, yet polling will reflect their answer as if they are registered to vote.

Allowing for voters to self-report voter frequency is another poll-faux pas that polling firms commit left and right.  As stated above, most voters over-state their voting behavior when questioned.  If they are only a causal voter, most respondents will not admit to being so as that would suggest they are not as well-versed as other, more frequent voters.  The voters contacted by polling firms should ALWAYS be chosen for contact based upon their propensity.

Another trait of a good poll is one that has correct sampling.  If a subsection of a population we are attempting to poll is 40% Republican, 40% Democrat, and 20% Independent/DTS/NPP, then you will either want to make sure your poll reflects that sampling or is weighted for that sample. If my sample for my poll ends up being 50% Republican, 25% Democrat, and 25% Independent/DTS/NPP, then I have incorrectly polled the population and my results will almost definitely incorrectly favor Republicans.

As voter data is public, the ability of polling firms to correctly acquire a sample for a poll should be pretty easy.  Without a doubt, polling firms could determine someone’s registration status AND propensity BEFORE they are even contacted.  In 30 states where party preference is available as well, polling firms could contact those they need to to achieve an accurate sample.  In the 20 states that you cannot obtain voter party preference from the public data, allowing for respondent-defined voter preference is acceptable.

The problem with the vast majority of polls is that they make some pretty horrible mistakes when it comes to the above-mentioned traits of a good poll.

Let’s examine the Economist / YouGov poll published on September 1, 2020. (<— follow the link if you’d like to follow along)

First, as a matter of attempting to compliment the “poll,” the organization they present here is fantastic.  My ability to navigate the poll from the first five pages is excellent.  I wish more polling firms would release their data like this, linking questions to the results and allowing people to navigate the poll with a few clicks.  While that doesn’t have anything to do with how the results were reached, bravo to them for doing this part.

Unfortunately, that’s where my compliments end.  The vast majority of the rest of this poll is a disaster.  To start, there was a minimum of 130 questions in this poll (minimum because there were identifying questions asked for cross-tabbing).  Talk about respondent fatigue.  Respondent fatigue occurs when people get tired of the length of a poll and start firing off answers in an effort to wrap up the poll rather than think about each question.  It is evident in the responses, as it appears respondents stop responding to some of the questions as the total responses drop off as you move through the poll.

If you scroll to the last page you will find the methodology section of the poll.  First, you will recognize that the “Sampling Method” section states that the results were compiled from “YouGov’s opt-in Internet Panel using sample matching.”  Remember, OPT-IN POLLS ARE GARBAGE.  Twitter polls? Facebook polls? All garbage.  This is one step above that.

Under the Number of Respondents, you will see 1500.  Why on this earth would they allow a non-registered voter to participate in the first place?  I’m all shoulders on that one. While it appears they remove the non-registered from some of the questions, but then include them in others.  No rhyme or reason as to why.

Next, they make another huge mistake:  Allowing respondents to self-identify whether or not they are registered voters.  No one wants to be an irresponsible person or disengaged citizen.  Asking someone if they are a registered voter is almost the same as asking someone if they hate people who kick puppies.  Very few people are going to answer in the negative.

On the bottom of the methodology section, you’ll find that they say “30 questions not reported.”  What were those questions?  Why were they not reported?  There’s no way of knowing what those questions were (presumably they were demographic questions?) and leaving unknown aspects of any data immediately leads to questions about the accuracy of the poll.

Now, as we look at the sample in the crosstabs section, on Page 20 of the poll, you will find the results of the question, “How enthusiastic are you about voting for President in the upcoming presidential election in November?”  Again, if you ask a question in a way that could lead to potential shame (enthusiasm for voting at all vs. enthusiasm in voting for a particular candidate) you’re going to get skewed results.  On page 21, let’s examine the 2020 vote breakdown and the Party ID breakdown.

In the 2020 Vote section it shows that Trump has a 86 to 82 advantage over Biden in any positive enthusiasm (Extremely, Very, Somewhat) section.  Remember, this poll gave Biden an 11 point lead on the topline.  Under Party ID, you’ll find similar results as 87% of Republicans have a positive enthusiasm (Extremely, Very, Somewhat) in voting in the Presidential Election, compared to just 70% for Dems.  Further down on Page 36 and 37 you will find the question, ” Who do you think will win the 2020 Presidential Election?”  Again, only 69 percent of Biden voters believe he is going to win, while 86% of Trump voters believe he is going to win.

These two questions show that Biden voters are neither as enthusiastic nor confident in their nominee as are Trump voters in theirs’.

Then how would this poll show Biden with an 11 point lead?  Simple:  Sampling is WAY off.  Depending on a poll, you may see a poll give one party a 3-5% advantage over the other. Much above that and your sample will taint the results of the poll.  In this poll, beyond the already-identified-above flaws with it, you’ll find they oversampled Democrats by nearly 10 points.

In Question #10, found on pages 24 and 25 of the results, respondents were posed, “If an election for president were going to be held now and the Democratic nominee was Joe Biden and the Republican nominee was Donald Trump, would you vote for…?”

This question was posed to just registered voters and gave Joe Biden a 51% to 40% lead in the poll.  Crazy, right?  Except on page 25, we can see how respondents answered and how many from each party answered. Of the 1,207 respondents to the question, 494 self-identified as Democrats and only 314 self-identified as Republicans (the remaining 399 were independent/third party). Simply put, of the respondents polled, nearly 41% self-identify as Democrats, while only 26% self-identify as Republicans.  A 15 point advantage for Dems built into the data.   That number isn’t included in the methodology… I wonder why??

Welp, there’s your problem.  Even if they gave Democrats a 5 point advantage, that’s still an oversampling of Dems by 10 points!!! If we were to reapply these same results, but adjust them with Dems still having a 5 point advantage AND eliminating effects from enthusiasm, you’d still have a Trump lead.

Does anyone think the people administering the polls are unaware of these problems? I don’t.

A friend hypothesized to me that polling now exists to support Washington DC’s “campaign industrial complex” in their fundraising efforts. I think that is the best explanation for this phenomenon that I have seen yet.

After eight years of gigs in the sprawling bureaucratic machine, most Obama and Clinton alums ended up as political consultants. They have endless Russian doll arrangements of LLCs and “nonprofits” that exist only to enrich themselves personally. (Truly if you ever want to transform Washington DC for the better, enact radical changes to the definition of what count as nonprofits in the federal tax code. You could make 90% of the corruption evaporate instantly.) Every subsequent Democratic campaign inherits these legions of lifers that they have to feed and support through non-stop fundraising. Trump, the perennial political outsider, does not have this problem.

The key to fundraising is always making it seem like you are a winner, even when most people hate your guts. It’s like a Ponzi scheme in finance. Look at those returns! Just keep the money coming in, and we will never stop “winning”! No Hollywood celebrity or tech entrepreneur wants to throw millions on what they perceive to be a lame horse. Hillary Clinton was a flaming bitch with baggage out the wazoo and a pedo husband who made his barely legal intern sit on his cigar, but she’s going to win in a landslide! Joe Biden is almost 80 years old – so old he will be a lame duck on day one in the White House – and is so detached from reality after nearly 50 years (!) in government that he needs to be asked scripted questions everywhere he goes, and he even fucks that up. But he’s going to win in a landslide! See a pattern?

In fact, that is one thing that both Clinton and Biden have in common: they are so awful around ordinary people that their campaigns actively tried to keep them from having any real world interactions whatsoever and scripted even the most minor details to make them seem human. Clinton was given debate questions in advance so she could memorize scripted responses. Biden has scripted responses ready for the rare occasions he surfaces outside his house. It’s bonkers, but it is becoming “normal” for the Democratic campaign industrial complex to treat candidates this way. Candidates that a significant portion of their own party did not want and do not like, but who were the only people available in that political moment that the machine approved of, precisely because the monied interests funding the whole shebang are driving the party in obscene new directions.

An alternative explanation is that polls try to demoralize Trump voters. I do not think this is true. I think they fully appreciate there is no half-life to Trump’s enthusiasm advantage. A boat parade in South Florida this past weekend attracted 11,000 yachts. It was a world record for boats in one place, apparently. Traffic came to a stop on the causeways as people got out of their cars to cheer on the boats. I don’t think the campaign industrial complex is under any impression Trumpers are ripe for conversion. But by pretending they do not exist, they can get people to push money their way and keep the millions in consulting fees going for another cycle. And they are playing to what remains of their own base, which at this point also thinks kindergartners are at risk of dying from a cold virus. By definition, they will believe anything that seems statisticky or an “expert” opinion, even if it’s obviously bunk and their “experts” are obviously career grifters. So from the machine’s perspective, bad polls are good polls.

A more interesting question is how long we are going to be stuck with this behavior. How long can the DC population be a running joke with ~80% of the country?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s