TV debate polls: not all that they seem

Welcome to the 111th edition of The Week in Polls (TWIP), which takes a look at the polling of TV debates and why there’s more to the figures than meets the eye.

Then it’s a look at the latest voting intention polls followed by, for paid-for subscribers, 10 insights from the last week’s polling and analysis. (If you’re a free subscriber, sign up for a free trial here to see what you’re missing.)

First though, this week’s eyebrow raising is courtesy of the front page of The Independent which headlines in a lurid red box “Labour poll blow” its report of a poll… putting the Conservatives 24 points behind and lower than they ever were with that pollster under Liz Truss.

---

Been forwarded this email by someone else? Sign up to get your own copy here.

Want to know more about political polling? Get my book Polling UnPacked: the history, uses and abuses of political opinion polling.

---

Polling TV debates: what should you measure?

Stephan Shakespeare, co-founder of YouGov, put it well at a previous election:

There has been great interest not only in the election debates, but in the polling of who “won” or “lost”. At first sight, it may seem obvious how pollsters should measure that, but on closer inspection, it isn’t obvious at all.¹

Whose opinions are such polls trying to measure? Regular voting intention polls have a standard answer. They are trying to measure the views of those who will vote. Not everyone watches a TV debate, however. Should you be polling just those who saw the debate? What about those who watched the debate but aren’t going to vote? Or what about likely voters who did not watch the debate but may be influenced by how it went? Or perhaps all those groups are too hard to define tightly, so is it better to revert simply to adults overall?

These groups are not all the same in their views, so which you choose as your target varies what results you get.

There is no simple right or wrong answer as to who such polls should try to measure. To make matters worse, knowing for sure the make-up of those who watched a debate is very difficult, making it nearly impossible for pollsters to weight any samples to make them representative of those who watched, which might be the most obvious solution to go for otherwise.

As a result, practice varies greatly as to whom pollsters are trying to measure in such polls. To give a practical example from this week, Conservatives were more likely to watch the debate. Weighting to debate watchers therefore is a rather different matter from weighting to the public overall.

So no surprise that while the headlines were grabbed by YouGov’s super-quick poll after the end of the Sunak-Starmer debate, its findings were not matched by those of other pollsters whose results came out a little later.

First though those YouGov findings:

A headline win, just, for Sunak - which given his overall popularity is much lower than Starmer’s sounds pretty decent, indeed.

But, but, but… YouGov also found that slightly more thought Keir Starmer performed well (60%) than Rishi Sunak (55%). He also came out ahead of Sunak on being trustworthy, likeable and in touch with ordinary people, only losing just on being Prime Ministerial.

Moreover, Jeremy Corbyn similarly outperformed his popularity in 2019, only losing 49%-51% to Johnson in the equivalent YouGov poll back then. That over-performance, however, was followed by crushing defeat.

Turning to JL Partners, their poll earned a positive headline for Suank in The Sun but it was based on what 2019 Conservatives. The headline figure with this poll was a comprehensive win for Starmer by 53%-33%. The poll also had Starmer winning on being persuasive, competent and less boring.

Savanta also had Starmer winning, 44%-39%. Again too the other questions had Starmer repeatedly outperforming Sunak: “Starmer beats Sunak on every major issue and personality-based question in overnight poll” as the pollster put it.

Redfield & Wilton was another pollster finding a Starmer win, by 38%-32%.

Finally, More in Common had a… yes, you’ve guessed it, Starmer win. With them it was by 37%-34% though with a big difference between those who watched the debate live and those who saw clips of it afterwards:

Overall then, the pollster that got the most immediate attention had it as a Sunak win, but then four other pollsters had it as a Starmer win. A reminder as ever not to pay attention to just the one poll.

The news headlines could have been:

Four out of five pollsters say Sunak loses
Prime Minister, already well behind in the polls, fails to get his game changer

Though perhaps none of this matters very much. Because as this week’s evidence-touting party pooping political scientist, Nick Anstead, points out:

Busy media day today with lots of interest in TV debates. Big spoiler from the academic research: they tend not to change anything in terms of how people vote… but that doesn’t make them any less interesting as political events.

For polling on the seven-way TV debate, see More in Common’s polling here.

---

National voting intention polls

The table below includes a new name, Verian. They used to be Kantar who in turn used to go under various other names. Through the names changes, they’ve had quite a low profile as a pollster but also one of the best records. What makes this poll particularly interesting is that it is the closest we’ve had to a true random sample poll. The methodology also excludes don’t knows rather than trying to remodel them (placing them more to the left of last time’s diagram).

For more details and updates through the week, see my daily updated table here and for all the historic figures, including Parliamentary by-election polls, see PollBase.

---

Last week’s edition

Why pollsters don't simply believe what people tell them.

---

My privacy policy and related legal information is available here. Links to purchase books online are usually affiliate links which pay a commission for each sale. Please note that if you are subscribed to other email lists of mine, unsubscribing from this list will not automatically remove you from the other lists. If you wish to be removed from all lists, simply hit reply and let me know.

---

Prospects for Conservatives winning over Reform supporters, and other polling news

The following 10 findings from the most recent polls and analysis are for paying subscribers only, but you can sign up for a free trial to read them straight away.