Wrong!!!
The ongoing challenge of public polling is to reconcile popular expectations about what polls “mean” at election time with our own desire to provide the public with information about mass opinion on politics and policy. We begin with the realization that polling results provide an account of public attitudes only at the time the data are collected. However, publicly released polls tend to be taken as a prediction of what will happen on Election Day. As much as we would like this to be the case, and as pleased as we are when the polling results comport with the eventual reality, we don’t, in the end, view the results in this way.
A situation with (a) a lot of unformed or non-existent opinions of candidates and (b) active campaigning in multicandidate races with no distinguishing party labels in a notoriously low-turnout election was, and is, likely to create volatility in results and uncertainty about the composition of the electorate. This volatility, particularly in the weeks leading up to an election, as voters slowly begin to pay attention, is why campaigns invest in daily tracking polls if they can afford them. As several candidates found out Tuesday, the past, even the relatively recent past, is always an imperfect guide to the present.
In our own polling, to assess the state of the primary elections, we screened “likely voters” from the larger sample of registered voter respondents — people who told us that they intended to vote in a particular party’s primary and, in addition, said that they were “very” or “somewhat” interested in politics and had voted in “every” or “almost every” one of the past few elections. Even among this group, many expressed no candidate preference in a number of races. With the election just around the corner, we forced them to make a decision — asking which candidate would get their vote in each race if “don’t know” was not among the options. In sum, we reported the results for people who seemed to be “likely” primary voters at some distance from the actual primary election. This screen, like any screen, is arbitrary, but has, in the past, been particularly robust and, maybe even more important to us, is purposefully agnostic about the eventual composition of the electorate.
As someone who has criticized that poll and called on Henson and Shaw to do an after action review on it, I commend them for doing so. I’m sure this has not been a fun week for them.
Now that they have undertaken this job, let me make a couple of suggestions to them. I don’t see why the screening process for primary voters needs to be complicated. We have a very good idea of who the likely voters in a primary election are – the people who have voted in the primaries before. Look at the turnout levels for the last three primaries – they’re in a pretty tight band for both parties. It’s the same thing for Houston’s odd-year elections. Pre-screen for those who have voted in two of the last three such elections – which is to say, do what the campaigns themselves do – and be done with it. Sure, the electorate gets expanded sometimes – 2008 for the primaries, 2010 for Republicans in the general; Democrats are working to make 2014 be like that for themselves – but you’ll be right more often than not, and in the exceptional years you’ll very likely have some external data telling you that this time it’s different. If that’s not easily done within the confines of their YouGov panel model, well, maybe that should tell them something.
The other thing I’d suggest is that it’s OK for “I don’t know” to be the majority answer. A poll result that said Kesha Rogers led David Alameel by nine percent to seven percent, with 76% undecided, is admittedly unsexy and unlikely to get picked up with Politico and the Washington Post, but it’s also unlikely to result in you writing a mea culpa after being roundly mocked for your crap-ass predictions. Seems like the better choice to me.
They reinforce that point later:
Additionally, what these tables don’t show is how uninformed and underdeveloped the attitudes of the electorate were in the final weeks of the campaign — an element that was sure to create volatility (that is, broad but potentially uneven changes in preferences that affect the totals for the candidates). Additional data elaborate the point: About a fifth of GOP voters for each of the lieutenant governor candidates did not register either a positive or negative opinion toward their preferred candidate. In addition, roughly half of the potential GOP primary voters surveyed in the attorney general and comptroller races originally stated that they hadn’t thought enough about the race to form an opinion. This is almost certainly why Debra Medina polled so high among people forced to choose: They recognized her name.
The Democratic side of the ledger was even more disheartening for anyone who wants to assume the existence of a large, engaged and informed electorate. U.S. Senate candidate Kesha Rogers’ strong initial polling — driven in large part by African American respondents who, in the end, didn’t vote — was also buoyed by the roughly three-quarters of our respondents who initially said that they had no opinion in that primary. (As with the Republicans, those who initially chose no one were then asked which way they leaned.)
There was a ton of self-loathing on the Democratic side at the new of Rogers leading that poll. I guess we can take a small measure of comfort at the news that despite the coverage and the millions spent, a bunch of Republicans had no idea whom to support in these races. Of course, none of their choices would be as offensive to them as Rogers is to us, so it’s not quite the same. Be that as it may, this is what I’m talking about above. By Henson and Shaw’s own admission, these voters are highly likely to be swayed by late campaign activity. If so, why would you want to push them for an answer when you know it’s very much subject to change? If this experience doesn’t let that lesson sink in, I don’t know what would. I’m glad they’re reviewing their approach, but I think they ought to keep thinking about it.