Humans are reading ChatGPT users’ prompts to improve OpenAl’s models, and those chats can include sensitive, personal information, according to leaked internal documents and real prompts seen by 404 Media.

The news presents a major privacy risk for ChatGPT’s users, with people often using ChatGPT as a therapist, professional assistant, or digital friend, and providing it with all sorts of intimate details about their lives. The contractors don’t see ChatGPT usernames, and OpenAl says it tries to remove personal information before prompts reach the reviewers, but the company acknowledged sensitive details can still get through.

The news also dispels the misconception that these models are improving only because of OpenAl’s mass scraping of the internet, the talent of its well-paid engineering and Al teams, or the power of its newer models. An important and overlooked part are the outside contractors paid to read and review ChatGPT responses to real prompts over and over again. Anthropic confirmed to 404 Media it is also using human review to improve its models.

“No,” someone who works with the prompts said when asked if they think ChatGPT users know that humans are reading their chats. “I don’t think they would imagine some contractor somewhere […] is analyzing the conversations.”

Unpaywalled link here - https://archive.ph/98Wr5