Three polls in need of a second glance

Welcome to the 75th edition of The Week in Polls, written after I have returned from Liberal Democrat conference and as I am basking the glory of a series of polls each showing the party up by one point, all due no doubt to my conference speech.¹

Three polls have come out in the last few days all with headline grabbing figures but also worth a second glance.

Then it’s a look at the latest voting intention polls followed by, for paid-for subscribers, 10 insights from the last week’s polling and analysis. (If you’re a free subscriber, sign up for a free trial here to see what you’re missing.)

As ever, if you have any feedback or questions prompted by what follows, or spotted some other recent polling you’d like to see covered, just hit reply. I personally read every response.

---

Been forwarded this email by someone else? Sign up to get your own copy here.

Want to know more about political polling? Get my book here.

Or for more news about the Lib Dems specifically, get the free monthly Liberal Democrat Newswire.

---

A second glance at three polling headlines

Is The Sun right?

“Sadiq Khan risks being booted out as London Mayor by Tory maverick, exclusive poll shows” was The Sun’s headline this week on a poll showing him only 3 points ahead of Conservative candidate Susan Hall.

The poll was a legit poll from a respectable pollster, JL Partners, who releases all the proper details of its polls. But should we believe it, given both how heavily London voted Labour in the 2019 general election and the 2022 council elections, not to mention the apparent weaknesses of the Conservative candidate?

First up, the headline 3% figure comes from - unusually - including don’t knows. If you exclude the don’t knows, as is the common UK polling practice, the headline lead becomes 4%.

But potentially more significantly, it looks like the poll didn’t use weights/quotas to ensure it had the right mix of London’s ethnic minority population in its sample.² Given their distinctive voting patterns, that could lead to noticeably off results.

That said, the previous Mayor of London poll, from Redfield & Wilton, had Susan Hall only one point behind Sadiq Khan, and we have to go back to Survation in June for a poll putting Khan 12 points ahead - though that one didn’t name candidates.

But before you get too firm in your conclusions, the R+W poll had some surprising details in its cross-breaks which would otherwise give a pause for thought, and there’s a big difference in the results for the Lib Dems between the two pollsters.

All the bits of evidence therefore have reasons to be treated with caution beyond the usual ‘it’s only one poll’ caveat.

So the best conclusion to draw? Please do another poll, naming the candidates this time, Survation.

Stonehaven’s MRP and the reason to always look up the North Shropshire result

We have a new MRP poll, this time from strategy consultancy Stonehaven with a trio of eye-catching details beyond the headline.

The headline is of a 13% Labour lead and a Labour majority of just under 100. Handily, all the advice in my guide to MRP from August apply to understanding this poll too, so I’d suggest pop on the hold music and go read that first.

Hold music by Oleksii Kaplunskyi from Pixabay.

Now you’re back, let’s take a look at three surprising details.

One is the sample size:

The current projection is based on a Stonehaven nationally representative poll of 2,000 voters from England, Wales, and Scotland, surveyed online between 29th Aug 2023 and 31st Aug 2023. However, this projection leverages a sophisticated model refined over the past two years with cumulative data from approximately 100,000 UK respondents.

Doing an MPR with a sample of 2,000 counts as the brave school of polling, producing some scepticism but not out right rejection from other experts. I hope to have a chance to dig into their methodology more for a future piece.

The second thing that catches the eye is a test I left out of that MRP guide. Let’s call it The Wem Test in honour of the town of Wem in North Shropshire. You may think Lib Dem Helen Morgan, dramatic by-election winner there, has a good chance of holding her seat. You might not. But how plausible is it for an MRP to predict that Labour will win the seat (by-election vote share: 9.7%) and that Helen Morgan will finish third? Electoral Calculus previously had a similar eyebrow raising set of figures for the constituency, before adjusting its MRP model to cater for seats which have had by-elections. I wouldn’t be at all surprised if we see a similar adjustment come to future versions of the Stonehaven MRP too.

The third thing to catch the eye we can call the The Lib Dem Lumpiness Test. As with North Shropshire, you might be expecting the Lib Dems to have a good seat tally at the next election. Or you might be expecting otherwise. But what would be odd is to predict both that the Lib Dems will do well in seat numbers and decently in vote share yet also be failing to do well at squeezing the Labour votes in top Lib Dem / Conservative contests.

Being successful while failing at one of the most obvious and important routes to success would be a somewhat baffling outcome.³ Much more likely that the Lib Dems will either succeed at the squeeze and so succeed overall, or fail in the squeeze and so fail overall. Predicting the good-yet-bad outcome is, again, a brave one and one that Stonehaven’s individual seat results look to do.

Why though call this The Lib Dem Lumpiness Test? Well, it’s because there’s a common feature of some MRP models, Electoral Calculus’s also included, which is to show a decent Lib Dem overall vote share but to have it distributed rather evenly across the country, rather than it being very lumpy with peaks in target seat and troughs elsewhere. MRPs that score lower on the Lumpiness Test therefore also show a surprisingly large number of close-ish three-way results; i.e. they suggest that far from a wave of successful anti-Conservative tactical voting at the next election we’ll see it flop. If the Conservative vote share is low enough in the MRP that flop doesn’t save Conservative seats, but even so suggesting such a rise in three-way contests seems is another brave prediction.

There’s more on that in the Wimbledon section of my MRP guide and this new MRP is a sign that we’re likely to see MRP polarise into two camps with their results: the lumpy and the smooth.

Labour lead only 10%?

Finally, the simplest second glance of the lot: Opinium’s latest poll for The Observer. Opinium’s methodology consistently gives a lower Labour lead than most other pollsters and than it did under its old methodology. It’s a bit of an outlier with its results, but sometimes the pollster that breaks from the pack is the one that’s right.

What has caught lots of eyes now, though, is its latest poll which, while individually showing Labour (-2) and Conservative (+3) vote changes that are within the margin of error, add up to a Labour lead of ‘only’ 10%. That’s around five points lower than the typical Opinium Labour lead and means that of the polls with fieldwork in the last half of September, 7 of the 11 had a fall in Labour’s lead (2 rises and 2 no changes completed the set).

Adam Drummond, head of political and social research at Opinium got the way to view this spot on with this comment in the press release that went with the poll:

This is the biggest change we’ve seen to Labour’s lead in a little while so we’ll need more polls to see if it’s the start of a trend or just an outlier. If we look back to the end of August when Labour were 14 points ahead, there isn’t much change in the Tory vote or how many of their 2019 voters they are holding on to (in both cases about 65%) and the prime minister’s approval rating is still deep underwater.

If the Labour lead were to fall into single figures consistently, even with just one pollster, that would be I suspect quite a psychological change even though the different between a 9% poll lead and an 10% point one means very little given margins of error. (And before I get too sanctimonious about over-interpreting such small changes, I will pause to remind myself of how much difference it makes to the mood of Lib Dems if a poll puts the party on 10% rather than 9%.)

---

Know other people interested in political polling?

Refer friends to sign-up to The Week in Polls too and you can get up to 6 months of free subscription to the paid-for version of this newsletter.

---

National voting intention polls

Once again, it a week without a poll putting the Conservatives on more than 30%, extending the run stretching back to late June (when a Savanta poll gave them 31%).

Here are the latest figures from each currently active pollster:

For more details and updates through the week, see my daily updated table here and for all the historic figures, including Parliamentary by-election polls, see PollBase.

Last week’s edition

Four insights from new British Social Attitudes survey.

---

My privacy policy and related legal information is available here. Links to purchase books online are usually affiliate links which pay a commission for each sale. Please note that if you are subscribed to other email lists of mine, unsubscribing from this list will not automatically remove you from the other lists. If you wish to be removed from all lists, simply hit reply and let me know.

---

What the public thinks of HS2, and other polling news

The following 10 findings from the most recent polls and analysis are for paying subscribers only, but you can sign up for a free trial to read them straight away.