Campaigns need to hire real pollsters from here on out

We are making a minor adjustment to how FiftyPlusOne screens pollsters.

In the last month, we’ve seen an odd thing happen a couple of times: Candidates for federal office have been conducting surveys of their races independently, without the involvement of a professional pollster. The first such surveys we saw came from Peter Crosby, the Democratic candidate in Utah’s 2nd Congressional District. At the time, I thought it was unusual, so I reached out to the Crosby campaign for more information.

Crosby replied to me himself. This is unusual, as when we reach out to campaigns for polling methods information here at FiftyPlusOne, we usually hear back from the campaign’s communications team or campaign manager. Crosby explained that he has worked in survey research professionally for years, and provided me with some details about his methodology. His response made clear to me that he was an experienced pollster, and FiftyPlusOne aggregated the poll as we would any other campaign internal.

But then this Monday, October 5, we discovered a similar thing from another Utah Democrat: Jonny Larsen, candidate for Utah’s 4th Congressional District, also released a poll he conducted himself that raised some red flags for me. First of all, the initial ballot in the survey showed Larsen ahead of incumbent Republican Mike Kennedy by over 20 percentage points, wildly different from the only independent poll of the race (conducted on behalf of the Utah Debate Commission) which showed Larsen trailing by 9 points. The result from Larsen was highly improbable in a district Trump won by over 30 points in 2024.

Now at FiftyPlusOne, we generally don’t dismiss polls just because the results don’t align with our priors — but large differences often indicate to us that something is awry with a survey. When I looked deeper, a few things stuck out as strange. For example, in the methodology statement, rather than providing sample sizes for various demographic groups, the campaign included information about how much each group was weighted, which tells us very little about the nature of the sample (knowing that Democrats were weighted at 0.81 does not tell me how many Democrats were in your survey). The release said to treat the poll like “a focus group,” which is confusing, since these are two very different kinds of instruments for measuring public opinion.

All this sparked some internal discussion about how we should treat these kinds of surveys. On one hand, we have at least one competent pollster who also happens to be a candidate for office. On the other hand, due to easier access to technology for conducting polls, pretty much anybody who has $10,000 could get their hands on a voter file and start sending out text messages, yielding vast quantities of iffy data.

Ultimately, though, the more important issue is about accountability. When a professional pollster puts out surveys that are extremely inaccurate, it has consequences. They may lose clients or take reputational damage that could impact their careers. Even when conducting campaign internals, pollsters know that their reputation as a reliable and accurate source of data is on the line. And while internal polls have been shown to systematically overstate their candidate’s chances by around 4 or 5 points, that is a predictable bias that we can (and do) account for in our polling averages and models.

For candidates, no such incentives for accuracy exist. They can literally just say anything, and if they lose their race, they can go back to their regular job (that probably doesn’t include public opinion research) with no consequences. There is no reason for a candidate to produce accurate data, and inaccurate data might actually help them in some ways — perhaps by getting their name in the news, or improving fundraising, for example.

And this, poll lovers, is the crux of the problem for us. Because there are no systematic incentives for candidates to produce accurate results, FiftyPlusOne will no longer aggregate internal campaign polls that were conducted without the involvement of a professional pollster. Because this rule was not in place at the time of the survey releases, we will keep Crosby’s previous polls in our database and up on our site, but we will not publish any additional research conducted by the campaign, and we will not aggregate the survey from Jonny Larsen’s campaign. This change can now be found in our Methodology Statement and will apply to all future poll releases from candidates for office.