How did the polls do in the Parliamentary by-elections?

Welcome to this week’s edition which takes a look at how pollsters did in this week’s batch of Parliamentary by-elections.

But first, my attempt last week to praise a journalist for their coverage of a poll was undermined by me getting their name completely wrong. I have a complicated series of excuses for such brazen incompetence, but as I’m not Boris Johnson I’ll just say sorry to The i’s Richard Vaughan, who deserved the plaudits.

Meanwhile, this week’s polite cough of disapproval goes to The Times for their piece ahead of the Parliamentary by-elections which, as Patrick Flynn pointed out used vox pops in Selby and Ainsty of people aged 56, 60, 72, 79 and 88. We do not have votes at 55.¹

As ever, if you have any feedback or questions prompted by what follows, or spotted some other recent polling you’d like to see covered, just hit reply. I personally read every response.

---

Been forwarded this email by someone else? Sign up to get your own copy here.

If you’d like more news about the Lib Dems specifically, sign up for my monthly Liberal Democrat Newswire.

---

My privacy policy and related legal information is available here. Links to purchase books online are usually affiliate links which pay a commission for each sale. Please note that if you are subscribed to other email lists of mine, unsubscribing from this list will not automatically remove you from the other lists. If you wish to be removed from all lists, simply hit reply and let me know.

---

How did the Parliamentary by-election polls do?

Three by-elections, two published, public opinion polls. How did their results match up against the election results?

In Selby & Ainsty JL Partners, with fieldwork ending two weeks before polling day, had it as Labour 41% and Conservative 29%. The result was 46%-34% in Labour’s favour, so the same lead as in the poll but with both of those party five points higher.

Given how early in the campaign the polling finished, that’s a plausible outcome as you’d often expect support for other parties to fall away under a tactical voting squeeze.

It was less good news though in Uxbridge & South Ruislip where similarly early polling by JL Partners had Labour ahead 41%-33%. Labour famously lost though to the Conservatives by 44%-45%.

Being three points out on the Labour score isn’t bad at all. However, having Labour comfortably ahead when they lost shows all was not right with the rest of the findings.

A big chunk of the miss was the poll may have been due to the poll being 6 points too high on votes for the very fringe candidates, including having an anti-ULEZ candidate who scored 1% in reality on 4% in the poll.

Indeed, James Johnson from JL Partners did point out in advance of polling day that:

With undecideds [23% on the initial voting intention question]² leaning Conservative and 4% currently going to an anti-ULEZ independent, the Tories could narrow the gap if voters see the election as a referendum on ULEZ.

However, tempting though it is to draw a similar conclusion about the poll to the one for Selby, that voters were squeezed between the poll and polling day, this doesn’t work so well as an explanation here.

While it’s plausible that anti-ULEZ candidates were squeezed by the Conservatives, Lib Dems and Greens dropped from 10 points between them in the poll to 5 points in the result, with the squeezed 5 points - if that was what it was, rather than polling error - more likely to have gone to Labour than the Conservatives.

It’s possible that the Labour candidate’s U-turn on ULEZ is a prime explanation for the difference between poll and reality as it came during the campaign just as the poll’s fieldwork was ending. But again it doesn’t quite fit as an explanation because why would that U-turn make someone switch from Labour to Conservative? Perhaps seeing a Labour candidate U-turn was enough to provoke Conservatives-turned-don’t-knows to turn back to the Conservatives? Perhaps.

That said, there’s a faulty premise in the previous paragraphs, that is the search for the one explanation. The reality of polling misses which have then had detailed post-mortems is that they find usually there is a mixture of different factors that added up to the overall error. As I wrote in that book:

Being a good pollster is like being a tightrope walker. Preparation, practice, expertise, fine judgement and good instincts are all required…

As you head out along the rope, repeated small adjustments are needed to keep upright. For those with skill, most of the time the little gusts of wind or twinges of muscles are coped with. But every now and again, there will be an unexpectedly strong gust of wind, just as a foot is slightly misplaced and a desperate urge to sneeze works its way through the nostrils. When all those factors combine against you at the same moment, disaster results.

Polling is the same. Polling is not a search for the perfect methodology that, once found, will solve everything. Rather, it is a constant battle against circumstances.

In west London, the JL Partners tightrope walking didn’t come off, possibly because reality moved between the poll and the result or possibly because the poll was off.

My hunch - hopefully well-informed by experience, including of doing internal constituency polls, but still only a hunch - is that the error came more from the poll being off than movements in support after the poll came out.

But, we’ve not seen enough of their polling followed by actual election results to be able to make a good call on which of those two scenarios it was. (They haven’t yet polled a general election and have only publicly done three by-elections, the earlier third of which also got the result right.)³

One other hunch applies to other pollsters too. I’ve covered before the big variation there is in the levels of support different pollsters find for the likes of Reform. Both the JL Partners polls had an over-estimate for a right-wing populist candidate: Reform by 8%-5% (Selby) and Reclaim by 5%-2% (Uxbridge). Again, only two data points and only over-estimates at the edges of the usual margin of error.

But added to the actual low performances for those candidates in the results, and the consistent absence of progress (or even candidates much of the time) in council by-elections or the big round of May elections, it’s suggestive of the pollsters with higher ratings for Reform being more likely to be the ones that are wrong than the pollsters with lower ratings for them.

By-elections bar charts bonus

Millimetre perfect bar charts, of course.

---

Know other people interested in political polling?

Refer friends to sign-up to The Week in Polls too and you can get up to 6 months of free subscription to the paid-for version of this newsletter.

---

National voting intention polls

It’s still grim for the Conservatives, with the majority of pollsters still putting Rishi Sunak’s party on a lower share than that secured by Labour in its near-party destroying landslide defeat under Michael Foot in 1983 (28%). The best poll for the Conservatives in the table below only puts them two points above that.

Here are the latest figures from each currently active pollster:

For more details and updates through the week, see my daily updated table here and for all the historic figures, including Parliamentary by-election polls, see PollBase.

Last week’s edition

Sorry YouGov, I don't quite believe your new poll.

Boost for Conservatives from pollster’s methodology change, and other polling news

These weekly round-ups are for paying subscribers only, but you can sign up for a free trial to read them straight away.