Some polling you should ignore: Telegraph/YouGov, round 2

Welcome to the 92nd edition of The Week in Polls, which sees a return to my self-defeating editorial proclivity of writing a newsletter about polling which tells readers to ignore some polling.

Then it’s the usual look at the latest voting intention polls followed by, for paid-for subscribers, 10 insights from the last week’s polling and analysis. (If you’re a free subscriber, sign up for a free trial here to see what you’re missing.)

As ever, if you have any feedback or questions prompted by what follows, or spotted some other recent polling you’d like to see covered, just hit reply.

---

Been forwarded this email by someone else? Sign up to get your own copy here.

Already a reader and know others who might enjoy this newsletter? Refer a friend and you can get up to 6 months of free subscription to the paid-for version.

Want to know more about political polling? Get my book Polling UnPacked: the history, uses and abuses of political opinion polling.

---

It’s time to ignore some polling

The short version of last time’s piece about that YouGov MRP polling published by the Daily Telegraph was ‘ignore the paper’s write-up but pay attention to the polling results’.¹

More YouGov polling has since appeared in the Daily Telegraph, again fronted by Lord Frost and paid for by the anonymous donors of the Conservative Britain Alliance. This time the short version is ‘ignore the paper’s write-up and also ignore the polling results’.

That’s because the polling question was, let’s say, sub-optimal.²

Here’s the question asked:

Please imagine that we are well into 2024, and the next UK general election is being held. Suppose by this time, the Conservatives had replaced Rishi Sunak and elected a new leader and prime minister, and they had made the following changes:

• Taking a much tougher approach on people coming to the UK illegally on small boats from France

• Introducing tough new measures on crime and anti-social behaviour

• New measures which significantly reduced legal migration

• Bringing in new tax cuts for working people

• Successfully getting waiting times for operations with the NHS falling

And in this hypothetical election, which of the following would you prefer to be prime minister?

The new Conservative leader
Keir Starmer
Not sure

The Daily Telegraph generously wrote this up as:

A YouGov poll of 13,000 voters suggests that a new Tory leader, championing core Conservative values, could secure a convincing victory over Labour.

When people were asked who they would prefer as prime minister – Sir Keir or a new, tax-cutting Tory leader with a tougher approach to legal and illegal migration – voters in 322 constituencies in England and Wales preferred a new Tory leader, while Sir Keir came out on top in only 164 seats.

And once again, just days after it last had to publish a list of corrections to the Telegraph’s reporting of its poll, YouGov had to take to its website to point out that its data didn’t really say what the newspaper was reporting:

The experiment does not ask (nor suggest) anything about vote intention or what would likely happen at an election given this entirely hypothetical scenario. It only asks which of two options respondents think would make the best prime minister…

Hypothetical polling of this nature is complicated and must be reported and interpreted with many caveats.

Those caveats went missing from the newspaper’s coverage.

But beyond the questions over how the Telegraph reported the polls - and all those of us in professions who get scrutinised by the media over how we use data can only look on enviously at how generously the rest of the media has treated the Telegraph’s repeated fumbling of the numbers - the YouGov question wasn’t only hypothetical and not about voting intention, it also was about comparing a real, fallible human (Keir Starmer in this case) with an abstract, idealised fictional creation who, in this fantasy world, pulls off within a few months a series of impressive policy achievements.³

It’s not quite comparing like with like, is it?

Which is why it’s a poll finding best ignored.

A digress about the regulation of polling

There is a side-note of interest, which is whether it should be legitimate for a polling firm to be paid to ask such questions.

In one respect, the self-regulation of the British polling industry - the British Polling Council (BPC) - is very effective. It has brought about levels of transparency over individual polling questions that leaves countries like the US way behind (and don’t get me started on how some Australian pollsters refused to provide basic data to the last post-election polling inquiry there…).

The new FiveThirtyEight pollster transparency ratings in the US, for example, read oddly to UK eyes as the criteria used are nearly all the well established norm over here. They are what have been standard for decades, even from whoever the most controversial pollster of the day has been.⁴

Other potential areas for (self-)regulation beyond such transparency are, however, mostly left alone. Fo example, as I recounted previously:

A few years back I worked through all of the options available with a pollster and the relevant regulator over a polling question that presented as if it were true a false statement about a Liberal Democrat. I was politely rebuffed all the way down the line. Putting a false claim to someone in a poll, without even telling them after that it was false, was considered reasonable, acceptable and ethical.

In that case, the regulator was the Market Research Society (who some, but not all, pollsters are members of). Its standards in fact go considerably further than the BPC’s. But even it wasn’t moved to say such a question was a problem.

Some of the fuss around the Conservative Britain Alliance’s polling I think is fairly left by the polling industry to others. In particular, there has been a bit of coverage this week about the mystery of the funding behind the Conservative Britain Alliance who paid for the YouGov poll. Yet concerns about anonymous money in politics shouldn’t be limited just to polling. If that’s your concern (and I think it should be), then it’s a matter for the Electoral Commission and for Parliament as that money can be, and is, also spent on all sorts of things other than polling.

It’s also true that YouGov deserve credit both for their polling transparency and willingness to push back on the misreporting of their poll.

However, it is a fair question to ask whether having industry self-regulation that is really only about transparency over individual polling questions is the best choice. As the British Polling Council’s own statement this week on the matter states:

It should be made clear that the BPC is neither an arbiter nor a guarantee of quality of polling.

In many other professions, the (self-) regulator bodies do take on one or both of those tasks.

The current polling self-regulation model certainly does the one thing it is designed to do related to transparency well.

But is it enough to do just that?

A side-note to the side-note: I’ve referred to ‘transparency over individual polling questions’ as a piece of shorthand because although the BPC’s rules do go a wider than that, they also omit some important aspects relevant to understanding and evaluating a poll. They don’t provide for much transparency over how internet panels are put together or over how MRP models work. With media headlines increasingly grabbed by MRPs run using online panels and with non-online political polling now largely a thing of the past, there’s a separate discussion about whether the current transparency model is getting too dated even on its own terms.

Perhaps there’s one significant thing about this latest polling

As Stephen Bush points out, perhaps the one substantive thing we can draw from this polling is that the Daily Telegraph is happy to put the boot into a Conservative Prime Minister only months (or perhaps even weeks) out from a general election.

Being able to rely on heavy cheerleading from a large phalanx of right-leaning newspapers in the long run-up to polling day is a standard component of winning Conservative general election campaigns.

That’s not what we’re seeing at the moment.

---

National voting intention polls

The run continues of polls putting the Conservatives on 30% or less. They were last over 30% in June 2023.

Here are the latest figures from each currently active pollster:

For more details and updates through the week, see my daily updated table here and for all the historic figures, including Parliamentary by-election polls, see PollBase.

---

Last week’s edition

What's the reality behind the big Daily Telegraph/MRP story of the week?

---

My privacy policy and related legal information is available here. Links to purchase books online are usually affiliate links which pay a commission for each sale. Please note that if you are subscribed to other email lists of mine, unsubscribing from this list will not automatically remove you from the other lists. If you wish to be removed from all lists, simply hit reply and let me know.

---

Surprising polling on the Royal Mail, and other polling news

The following 10 findings from the most recent polls and analysis are for paying subscribers only, but you can sign up for a free trial to read them straight away.