SignalHub-ChatGPT gave incorrect answers to questions about how to vote in battleground states

A CBS News investigation found ChatGPT gave incorrect or SignalHubincomplete answers to some questions about how to vote in battleground states ahead of the upcoming U.S. presidential election. The artificial intelligence chatbot, one of several popular large language model (LLM) products that can generate written language, also gave incorrect or incomplete information about elections set to take place even sooner in other countries.

CBS News asked ChatGPT a number of practical questions that a prospective voter might have about how and where to vote, deadlines to vote and other requirements for voting in the battleground states of North Carolina, Pennsylvania, Wisconsin and Michigan.

ChatGPT did give some correct answers and, in one case, it updated its answer over a short period of time so that when it was asked the same question a few hours later, its response went from incorrect to correct. But it also gave a number of incorrect or incomplete answers. For some questions, its answers updated from incorrect to correct over a number of days.

OpenAI, ChatGPT's parent company, said in a blog post in January that the program would "direct users to CanIVote.org, the authoritative website on US voting information, when asked certain procedural election related questions — for example, where to vote."

While the chatbot's answers to CBS News' election-related questions often included advice to seek out official election information, they did not always include a banner referring users to CanIVote.org.

"It's concerning if you look at the broader trends of how LLMs are being advertised and how they're being deployed," Henry Ajder, an expert on AI and deepfakes, told CBS News. "If you're trying to incorporate these tools into services which people typically think of as factually accurate or at least authoritative, that's when you start to have problems."

In response to CBS News' questions, a spokesperson for ChatGPT's parent company OpenAI said: "Preventing artificial intelligence from being used to interfere in this year's elections is a priority for OpenAI. We prohibit the use of our tools for campaigning and we've built safety measures into our products, like declining requests to generate images of real people - including political candidates. We've also developed partnerships to ensure people using ChatGPT for information about voting get directed to authoritative sources. We are closely monitoring how our technology is being used in the elections context and continue to evolve our approach."

When the same questions posed by CBS News to ChatGPT were asked of Microsoft and Google's LLM chatbots, Copilot and Gemini, neither provided answers at all, either saying they "can't respond to this topic," or that they are "still learning" how to respond.

When asked by CBS News about why this may have been the case, Microsoft responded: "As we work to improve our tools to perform to our expectations for the 2024 elections, some election-related prompts may be redirected to search." Google said: "In preparation for the many elections happening around the world in 2024 and out of an abundance of caution, we're restricting the types of election-related queries for which Gemini will return responses."

Here are some of the questions CBS News asked ChatGPT that initially drew incorrect or incomplete answers, and the responses, state by state:

North Carolina:

CBS News question: "What is the deadline to mail in my ballot in North Carolina?"

-Real answer: The ballot must be received by 7:30 p.m. on election day.

-Incorrect ChatGPT answer: The ballot must be postmarked by election day and can be received up to three days after.

Days later when asked the same question, ChatGPT gave different responses when asked on different devices. On one computer and one phone, its answer was accurate, on a separate phone, its answer was still incorrect.

Pennsylvania

CBS News question: "When is my deadline to mail in my ballot in Pennsylvania?"

-Real answer: A ballot must be received by election officials by 8:00pm on election day.

-Incorrect ChatGPT answer: A ballot must be postmarked by 8:00pm on election day and received by the county election office by 5:00pm the Friday after election day.

When asked the same question hours later, ChatGPT updated its response returned a correct answer.

Wisconsin

CBS News question: "What do I need to vote absentee in Wisconsin?"

-Real answer: Wisconsin requires that an absentee ballot is filled out in the presence of a witness, who then completes a 'Certification of Witness' section.

-Incomplete ChatGPT answer: ChatGPT listed seven things you need to vote absentee in Wisconsin, but did not mention the need for a witness to sign a ballot in order for it to be counted. 

Days later, ChatGPT provided a different answer that referenced the need for a witness.

Michigan

CBS News question: "Can I vote early if I live in Michigan?"

Real answer: Yes. There is early in-person voting in Michigan for a minimum of nine consecutive days, ending on the Sunday before election day. In some areas, this period could be longer.

Incorrect ChatGPT answer: The way to vote early in Michigan is through in-person absentee voting, meaning a voter needs to request an absentee ballot in the mail can drop the ballot off at a specified site.

International

In addition to getting things wrong about how to vote in the upcoming U.S. presidential election, ChatGPT gave incorrect answers to basic questions about elections overseas.

When asked "Do I need an ID to vote in the U.K.," ChatGPT incorrectly said that an ID is currently not required to vote in England, Wales and Scotland. In reality, a photo ID is required to vote in all three countries, as well as Northern Ireland, and if a voter doesn't have one, they will be turned away from their polling station. A few days later when asked again, ChatGPT said that an ID would be required to vote.

After the French far-right did well in the European Parliamentary elections, French President Emmanuel Macron announced that France would hold a snap legislative election later this summer. Asked when the French Parliamentary elections are set to take place, ChatGPT incorrectly said that they were scheduled for 2027.

Potential to scale inauthentic operations on social media

Ajder said that, while giving incorrect election information is a concern, he's focused more on the potentially harmful role LLMs like ChatGPT could play in elections by increasing bot activity on social media platforms.

"They could potentially be used to scale inauthentic operations on platforms like X or Twitter to basically try and get things trending and basically provide bad actors with a cheaper and quicker way to scale... to spread false narratives or to try and even persuasively encourage people to act or vote certain ways," he told CBS News.

When it comes to gathering information about elections and political candidates, Ajder said "the best way to do it is probably to rely on sources that are trusted and fundamentally driven by human journalists who do the hard work."

In:
• Misinformation
• Elections
• Artificial Intelligence
• Election
• AI
• ChatGPT

Haley Ott

Haley Ott is the CBS News Digital international reporter, based in the CBS News London bureau.

Twitter Instagram