View: Why polls today are more about psychology and political economy than statistics

By Surjit S Bhalla & Abhinav Motheram
India 2014. Britain 2016. The US 2016. India 2019. And now the US 2020. What do the above five national elections have in common? The fact that seasoned professional and previously accurate pollsters got significant elements of the elections wrong is a pointer. Identification of the cause(s) is important.
Not for the mere desire for more accurate polls, but also because opinion polls today both reflect and form public opinion. The latter, enhanced by social media, to a much greater degree than ever before. Opinion polls are the bread and butter for the cottage industry of poll aggregators (e.g., Fivethirtyeight). Their models seek to ‘inform’ the public with polling odds; getting it right should be an important part of business. Is there a pattern to so many election forecasts going wrong in recent years? We believe there is. The last time pollsters got it majorly wrong was in 1948, when the Chicago Tribune headlined a story, ‘Dewey Defeats Truman’. That is how far in memory space you must go. Seventy-five years later, there are five consecutive surprises in a row (3 in 4 in the US, including the midterms in 2014 and 2018, which looks even worse when you include down-ballot races such as Senate and the House).

Was 2020 close to warrant soul-searching? Don’t look at the aggregate vote, or the electoral college. Both are indicative, but no cigar. Look at the closeness of the races statewise. Look at the percentage difference in the vote share — 2016 was very similar to 2000, e.g., less than 2% separated the two parties in six seats, seven seats if the bar is 2.5%. The 2020 results are likely to parallel 2016. The last time the same closeness was observed in two consecutive elections was 1976 and 1980. The world was different then. There are several elements to the pattern, some methodological and others, perhaps, ideological. 2016 was a certain election for Hillary Clinton (one of us [Bhalla], had predicted a landslide for Clinton!). Trump won but there was still optimism among the experts certifying that ‘Polls are all right’.

The more academic American Association for Public Opinion Research’s (AAPOR) postmortem report on the 2016 election polls recognised that state polls ‘clearly underestimated Trump’s support in the Upper Midwest’. The polls did the same in 2020. Not all pollsters are wrong to the same degree. Ann Selzer, who leads the Des Moines Register Poll in Iowa, a respected authority, uses a random digit dialling (RDD) method of polling — i.e., randomly selected respondents, rather than derived from traditional voter registration lists. Her method (Des Moines Register) correctly predicted Trump’s margin in 2016, and 2020. (She had Trump winning by 7% in Iowa — he won by 8%).

Several of the reasons are statistical and, therefore, easily correctable. More important errors have to do with a changing world order. A new world where there is an ongoing Battle of the Elites. The old elite, the Establishment; the new elite, the Upstarts. As usual, cricket provides a useful parallel — Gentleman vs Players. This ongoing social struggle is an important cause of polling errors. Remember Mitt Romney’s 47%, and Hillary’s ‘deplorables’? Trump received a higher fractionof African American and minority support than any Republican. Because of his tax cuts, the bottom twothirds of the population witnessed income growth after at least 35 years of stagnation (since 1980).

There may be an analogy here with ‘perspectives’ on lockdowns. The old elite welcomed them because they could work from the comfort of their homes; the rest detested them because it meant a significant loss in incomes. Even viruses are political. To account for this changing social-political demographic, we need to correct the polling data for the realistic possibility of lying. In this lying, there is an anger against the system. A correction was tried and tested on elections in India the late 1980s-early 1990s. The method looks for inconsistencies in answers (a polite representation of a lie). It does not assume who is lying (or being inconsistent). It attempts to estimate net inconsistencies. In the study, ‘Are Election 2020 Poll Respondents Honest About Their Vote?’, 11.7% of Republicans said they would not report their true opinions — roughly double the number of Democrats (5.4%).

The fractions who are lying need not be very large to cause major errors. Assume one candidate has a 10-point lead — A is at 55 and B is at 45. If only 5 percentage points (PPT) of Camp A lie, a 10-point lead is turned into a dead heat. The consistently good polling company Trafalgar Group seems to have a method that works, and one which in principle is not that far away from our 1989 method. To mitigate the social desirability bias, they ask the respondents not only how they themselves will vote, but also how they think their neighbours plan to vote. It is very plausible that this correction broadly adjusts for lying, or misrepresentation, in traditional polling methods. What this means is that modern day polling does not have to worry that much about statistics, but much more about psychology and political economy. Think of it this way — the traditional pollster has today been replaced by an app.

(Bhalla is executive director, International Monetary Fund (IMF) representing India, Sri Lanka, Bangladesh and Bhutan, and Motheram is research analyst at National Council of Applied Economic Research-National Data Innovation Centre (NCAER-NDIC). Views are personal)