Sorry YouGov, I don't quite believe your new poll

Welcome to this week’s edition which takes a sceptical look at a new poll finding from YouGov.

But first, this week’s polite one person standing ovation for good use of polling goes to Richard Vaughan, who over in The i reported not only on a poll commissioned by the newspaper but then put its results in context by quoting from three other pollsters, including results both similar to and different from The i’s own poll. (One of the abiding frustrations of much polling coverage is media outlets commissioning a poll and then only reporting on their own poll, ignoring other polls that would add to a proper understanding of what’s going on.)

As ever, if you have any feedback or questions prompted by what follows, or spotted some other recent polling you’d like to see covered, just hit reply. I personally read every response.

---

Been forwarded this email by someone else? Sign up to get your own copy here.

If you’d like more news about the Lib Dems specifically, sign up for my monthly Liberal Democrat Newswire.

---

My privacy policy and related legal information is available here. Links to purchase books online are usually affiliate links which pay a commission for each sale. Please note that if you are subscribed to other email lists of mine, unsubscribing from this list will not automatically remove you from the other lists. If you wish to be removed from all lists, simply hit reply and let me know.

---

Not all is quite as it seems with new YouGov poll

Every day there’s a steady trickle of new traffic to my post from the start of last year, YouGov: is it biased to the Conservatives? Short answer, no. Longer answer, with data, here.

Despite YouGov’s reputable track record, as with any other pollster its findings shouldn’t simply be taken at face value as there any many ways that reasonably worded questions from high quality pollsters can fail to tell us the full story.

So it is with this week’s release of a poll of MPs carried out by YouGov. It found that a quarter of Conservative MPs (24%) expect the general election to result in a hung Parliament in which their party successfully forms a government.

Also notable is that under half (47%) of Conservative MPs say that Brexit has been more of a success than a failure:

Back though to the next election: that 24% is a surprising finding as quite who do those Conservative MPs think their party would be able to deal with in a hung Parliament, enabling them to continue in government? Even the DUP’s Arlene Foster now regrets not voting Theresa May out of office.

But there are two levels of doubt about the poll’s findings. The first is that although MPs are asked to take part in such online surveys, in practice their staff often complete them.¹ That isn’t necessarily a show-stopper, as on many topics it’s reasonable to expect a member of staff to know their boss’s views. After all, many of those staffers spend much of their time writing letters and drafting speeches for their boss to put their name to, so they very much have to know what their boss thinks to do their job well. I’d be pretty confident of the accuracy of results for questions about, say, whether tea is better with or without milk, and pretty dubious about the accuracy of results for questions about, say, personal lives.

My hunch for expectations of election results is that MPs’ staff have a pretty decent idea of what their boss thinks. There’s likely though to be a bit of a positive slant to what MPs tells their staff, as after all saying ‘we’re doomed and you’re going to lose your job at the next election’ is not the most motivating of management styles.

However, there is a second problem, which is one that applies more widely to polling. It’s that people know that the results of polls get published, and so sometimes give answers that reflect what they’d like the media coverage to be like rather than answers that reflect what they think is the state of reality. As I wrote in Polling UnPacked about this problem more generally:

What is the most effective way to obtain the best write-up of the poll? It is to give your side the most positive answers possible, even if they exaggerate your own views. Tat is a perfectly rational, if calculating, way of answering a poll.

Even the public indulge in such expressive polling, so all the more reason to expect it from a more politically calculating audience.

Therefore, when surveying MPs what result to expect for their party, we should expect the results to be more optimistic than their actual views, both because they might impart an extra optimism to their staff who sometimes answer such questions and also because either they or their staff may want to help make the coverage of the poll more positive for their party.

In that context, a quarter of Conservative MPs apparently expecting not only a hung Parliament but also their party to be able to secure the support of others in order to stay in government, is rather less surprising. It’s very likely that their real views are more negative than that but that they or their staff nudged their views a little in the positive direction for the purpose of answering. Only a bit more positive, so as not to seem implausibly out of touch with public opinion and only a bit more positive to square with their conscience, but still a bit more positive than their real views.

Which makes for two conclusions. One is that if the views skew positive, the fact that (only) 53% of Conservative MPs picked their party remaining in power shows how low their own morale is.

The other is that the most useful figures are ones which come with trends because seeing how that 53% has fallen or risen (and the corresponding figure for Labour MPs too) would be instructive.

Even data that has to be caveated becomes more useful when it can be plotted over time.

---

Know other people interested in political polling?

Refer friends to sign-up to The Week in Polls too and you can get up to 6 months of free subscription to the paid-for version of this newsletter.

---

National voting intention polls

It’s a while since I’ve compared Rishi Sunak to Michael Foot so let’s do another check-in: two-thirds of pollsters have Sunak’s Conservatives doing as badly, or worse, than Labour did under Michael Foot in 1983.

Here are the latest figures from each currently active pollster:

For more details and updates through the week, see my daily updated table here and for all the historic figures, see PollBase.

Last week’s edition

Follow-ups and footnotes to previous stories.

Double dose of evidence that ‘war on woke’ unlikely to rescue the government, and other polling news

These weekly round-ups are for paying subscribers only, but you can sign up for a free trial to read them straight away.