How accurate are the ConservativeHome leadership election surveys?

Welcome to the 122nd edition of The Week in Polls (TWIP) which diverts from ‘proper’ opinion polling to take a look at the Conservative leadership surveys run by ConservativeHome.

Then - with apologies to a couple of pollsters - it’s a summary of the latest national voting intention polls along with a new round-up of party leader ratings, followed by, for paid-for subscribers, 10 insights from the last week’s polling and analysis. (If you’re a free subscriber, sign up for a free trial here to see what you’re missing.)

---

Been forwarded this email by someone else? Sign up to get your own copy here.

Want to know more about political polling? Get my book Polling UnPacked: the history, uses and abuses of political opinion polling.

---

Should we believe the ConHome leadership election surveys?

Proper opinion polls of party members are hard to do, expensive, rare and - given the sampling and weighting difficulties¹ - risky. As a result, other forms of surveys have grown up during leadership elections, particularly from websites associated with specific parties.

I used to do such surveys first with the Lib Dem Voice team and then under my own steam for Lib Dem elections, for example. From a methodological perspective, it would be easy to be dismissive about features of those surveys such as the extent to which they were self-selecting. They were in some respects close to voodoo polling. But… they also had a good track record when compared with the reality exposed when the votes were counted.

Indeed, my finally foray into such surveying was just about the perfect combination: I produced figures that showed a much closer race than a proper poll had, that showed a closer race than other pieces of data, my figures were mocked by some in one campaign for clearly being off… and then when the results were declared, they were just about bang on perfect. By luck, I then got out of doing such surveys as I suspect that features of the next leadership contest would have resulted in my survey methodology performing badly.² I got out before reality caught up with me.

But still going, with higher stakes and a higher profile, are the leadership election surveys of Conservative party members done by ConservativeHome. They use the same panel as their monthly surveys, and as the site points out:

The ConservativeHome Party Members’ Survey is a self-selecting panel, not a demographically or geographically weighted poll – but the results have a strong track record and offer a unique insight into the views and thinking of the Conservative grassroots.

The ConHome record

How strong is that record, and is it stronger or weaker than that of opinion polls? Let’s take a look at those Conservative leadership elections which have gone to a ballot of party members and had at least one ConHome survey.

In what follows, I have recalculated numbers to exclude don’t knows or won’t vote where they were included in published headline figures.

• 2022 Truss vs Sunak

  • Result: 57%-43% (14% lead for Truss)

  • Average of final polls:³ 64%-36% (28% lead for Truss)

  • Final ConHome survey: 68%-32% (36% lead for Truss)

• 2019 Johnson vs Hunt

  • Result: 66%-34% (32% lead for Johnson)

  • Final poll:⁴ 74%-26% (48% lead for Johnson)

  • Final ConHome survey: 73%-27% (46% lead for Johnson)

• 2005 Cameron vs Davis

  • Result: 68%-32% (36% lead for Cameron)

  • Final poll:⁵ 67%-33% (34% lead for Cameron)

  • Final ConHome survey: 69%-31% (38% lead for Cameron)

The verdict

Being 22 points out on Truss’s lead was a sizable miss - and it wouldn’t have to be that close a contest for that size of miss to make for a wrong call as to who is winning.

But, to be fair, the polls were not that much better either, and overall? Three contests, three times got the winner right. That’s a good record for those ConHome surveys and on leads the surveys have been only a bit less accurate than proper polls.

Their final surveys were, though, too high on support for both Johnson (by 7 points) and Truss (by 11 points). So if there is another ballot of party members comprised of a ‘populist’ versus ‘technocratic’ contest,⁶ and especially if it is looking closer than those above, that is an important caveat to bear in mind.

---

Voting intentions and leadership ratings

A thin table so far as I have reset it to include only post-general election voting intention polls:

New from this week is also a summary of the the leadership ratings, which vary a lot depending on the wording but overall patterns are consistent:

* indicates slightly different fieldwork dates from the rest of the row.

For more details, such as over the different leadership question wording, and updates through the week, see my regularly updated table here.

For the historic figures, including Parliamentary by-election polls, see PollBase.

---

Last week’s edition

The riots: what the polls tell us.

---

My privacy policy and related legal information is available here. Links to purchase books online are usually affiliate links which pay a commission for each sale. Please note that if you are subscribed to other email lists of mine, unsubscribing from this list will not automatically remove you from the other lists. If you wish to be removed from all lists, simply hit reply and let me know.

---

Changing polling on immigration, and other polling news

The following 10 findings from the most recent polls and analysis are for paying subscribers only, but you can sign up for a free trial to read them straight away.