As much as I love using ChatGPT, a few troubling developments from the company recently have made me question my trust in OpenAI. Ilya Sutskever’s departure is one of them and probably the most important.
The OpenAI co-founder was seen as the main person keeping AI in check at OpenAI. But it’s not just Sutskever who left the superalignment team, making us wonder who is overseeing AI safety at the company. Plenty of top engineers departed OpenAI. Then, CEO Sam Altman reorganized the team into a safety and security committee that answers to him.
Add the public fight with Scarlett Johansson over GPT-4o’s voice that sounded eerily similar to one of the actress’s AI-related roles, and you realize that OpenAI might be breaking too many eggs for its own good.
Fast-forward to this week, and we have an open letter out in the wild from a group of AI engineers that includes current and former OpenAI employees. They’re asking for better oversight of AI companies. This would include a system that encourages whistleblowers to come forward when they feel something isn’t right without risking backlash from the company they work for.
Considering the fast pace in the AI industry, whistleblowers might be the best thing that happens to ChatGPT and other AI products, especially if companies like OpenAI don’t fully understand how their AIs work. It turns out that’s also what’s happening with current AI chatbots.
The open letter is available on a Right to Warn website for anyone to read. It’s signed by former OpenAI, Google DeepMind, and Anthropic employees who were unafraid to put their names under the latter. But it also has anonymous signatures from former and current OpenAI employees who fear retaliation from the company.
Indeed, reports a few days ago said that OpenAI’s non-disclosure agreements might prevent former employees from publicly criticizing the company.
What the whistleblowers want
The signatories say they “believe in the potential of AI technology to deliver unprecedented benefits to humanity.” But they also “understand the serious risks posed by these technologies.”
“These risks range from the further entrenchment of existing inequalities, to manipulation and misinformation, to the loss of control of autonomous AI systems potentially resulting in human extinction. AI companies themselves have acknowledged these risks, as have governments across the world and other AI experts.”
The scientists say that guidance from the community, policymakers, and the public can mitigate the risks. But there’s not enough oversight in the nascent AI industry. Companies have too much control over what they know (and don’t know) about AI. There are no laws in place that would protect AI whistleblowers.
“Ordinary whistleblower protections are insufficient because they focus on illegal activity, whereas many of the risks we are concerned about are not yet regulated. Some of us reasonably fear various forms of retaliation, given the history of such cases across the industry. We are not the first to encounter or speak about these issues,” the letter says.
The AI engineers ask for AI companies like OpenAI to commit to a list of principles that would allow safe whistleblowing.
They want a mechanism under which whistleblowers can safely and anonymously voice their concerns about AI, like ChatGPT. That companies will consider the open criticism and not retaliate against those who dare speak their minds on potential risks. All of that should happen within a framework that also protects the release of confidential information unnecessarily.
The letter dropped hot on the heels of ChatGPT going down for several hours. It was the second such incident in a matter of days. Weirdly enough, Gemini, Claude, and Perplexity had their own share of issues on Tuesday.
How does ChatGPT work anyway?
A few days ago, Sam Altman said at an AI conference in Geneva, Switzerland, that OpenAI doesn’t fully understand how ChatGPT works, which is not unusual in the industry. Other companies are also looking to understand their AIs better.
Altman confirmed that OpenAI had not solved interpretability, per The Observer. Pressed on whether OpenAI should refrain from releasing new, more powerful models until they figure out how the GPTs work, Altman ultimately minimized the risks, saying that “these systems [are] generally considered safe and robust.”
“We don’t understand what’s happening in your brain at a neuron-by-neuron level, and yet we know you can follow some rules and can ask you to explain why you think something,” Altman said. The comparison is valid up to a point. No single brain might develop superhuman intelligence that could then pose a risk to the entire species.
Then again, you’d expect the guy who survived a major coup attempt and then ended up replacing the outgoing safety team with a team he oversees to say just that. It’s further proof that the AI space needs whistleblowers, just like any other industry. Maybe even more so than other niches, considering the massive theoretical risks associated with AI.
