The Stakes Are Incredibly High:Insights from Ex-OpenAI Staffers on Urgent Whistleblower Safeguards

The Stakes Are Incredibly High:Insights from Ex-OpenAI Staffers on Urgent Whistleblower Safeguards

In a potentially costly move for William Saunders, the former safety researcher who resigned from OpenAI in February, the decision to speak out carries significant risks. Like many who departed the company, Saunders signed a non-disparagement agreement to retain the option to sell his equity in the company. Despite assurances from OpenAI that they do not intend to enforce the agreement, and similar public commitments, Saunders acknowledges the gamble involved in his decision to speak publicly. "By speaking out, I might forfeit access to vested equity worth millions of dollars," he reveals to TIME. "But I believe it's crucial to foster a public conversation about the inner workings of these AGI companies.

Saunders is not alone in his sentiments. On Tuesday, a coalition of 13 current and former employees from OpenAI and Google DeepMind joined forces to advocate for stronger whistleblower protections within companies developing advanced AI. Their open letter urges these labs to grant employees the "right to warn" regulators, board members, and the public about safety concerns regarding AI technology. This call to action comes amidst a wave of notable departures from OpenAI, including chief scientist Ilya Sutskever, who was sidelined after voting to remove Sam Altman in November of the previous year. Sutskever has remained silent on the matter, leaving the reasons for his departure shrouded in mystery.

Another significant departure was that of senior safety researcher Jan Leike in May, who lamented OpenAI's prioritization of product releases over safety protocols. This trend of resignations continues with Daniel Kokotajlo, one of the voices behind the recent open letter, leaving in April. Kokotajlo openly expressed his loss of confidence in OpenAI's ability to act responsibly in the event of creating artificial general intelligence (AGI), a technology all major AI labs are striving to develop. Refusing to sign the non-disparagement agreement upon his departure, Kokotajlo believed he was sacrificing millions in equity. However, OpenAI revised its stance on the agreement following public scrutiny, assuring departing employees that criticizing the company would not result in equity forfeiture.

OpenAI emphasized the importance of public discourse surrounding advanced AI in a recent statement. "We're committed to our track record of providing the most capable and safest AI systems and believe in our scientific approach to managing risks," stated OpenAI spokeswoman Lindsey Held to the New York Times. "Recognizing the significance of this technology, we acknowledge that robust debate is essential, and we remain dedicated to engaging with governments, civil society, and global communities.

However, OpenAI declined to offer further commentary to TIME regarding the assertions made in the article. Similarly, Google DeepMind has not publicly addressed the open letter and did not respond to TIME's request for comment. Yet, two former OpenAI employees, Daniel Kokotajlo, a member of the company's governance team, and William Saunders, a researcher on the superalignment team, shed light on the challenges employees face beyond non-disparagement agreements. They noted that broadly defined confidentiality agreements prevalent in leading AI labs pose risks for employees speaking out about their concerns.

Both Kokotajlo and Saunders foresee significant advancements in AI capabilities in the coming years, with profound implications for society. "The stakes are incredibly high," Kokotajlo emphasized. In contrast to regulated industries like finance, where whistleblowers are protected by U.S. government provisions and may even receive a portion of fines resulting from reported violations, the lack of specific laws governing advanced AI development leaves whistleblowers in the AI sector vulnerable to legal repercussions for breaching non-disclosure or non-disparagement agreements.

The absence of preexisting whistleblower protections in this unregulated industry leaves employees exposed," Kokotajlo remarked. "AGI labs operate without real accountability," Saunders added. "True accountability necessitates the ability to share information about wrongdoing, a capability that is currently lacking.

Upon joining OpenAI in 2021, Saunders had hoped to confront challenging questions about the societal implications of AI advancement. "If a machine can perform the economically valuable tasks humans do, what does that mean for our societal power dynamics? Can we maintain democracy in a world where human labor is replaced?" he pondered. However, after the emergence of ChatGPT in November 2022, Saunders witnessed a dramatic shift within OpenAI. The company's value skyrocketed to tens of billions of dollars, with executives embroiled in a fierce competition to outpace rivals. The introspective questions Saunders anticipated grappling with at OpenAI seemed sidelined, overshadowed by the pursuit of releasing the next groundbreaking product.

In the recent open letter penned by Saunders, Kokotajlo, and fellow current and former employees, they advocate for significant reforms within AI labs. Among their demands are the cessation of non-disparagement agreements, the establishment of channels for employees to voice concerns to board members, regulators, and watchdog groups, and the cultivation of a culture conducive to open criticism. The letter also stipulates that if whistleblowers resort to public disclosure after unsuccessful attempts to address concerns through official channels, AI companies should commit to refraining from retaliation.

However, not all within OpenAI are in agreement with the open letter's stance. Joshua Achiam, a research scientist at OpenAI, criticized the letter on social media, contending that publicizing safety fears could hinder the labs' ability to address sensitive issues effectively. Achiam stressed the importance of maintaining trust and confidentiality within the organization to foster a functional safety environment.

Saunders, however, remains skeptical of OpenAI's leadership, particularly CEO Sam Altman. He perceives Altman as resistant to accountability, noting Altman's tendency to assert personal influence over oversight mechanisms, including the board and safety and security committees. "Sam Altman seems averse to any form of oversight," Saunders remarked. "His insistence on personal involvement in every oversight group raises serious concerns about the company's governance.

In conclusion, the discourse surrounding accountability and oversight within AI companies like OpenAI is at a critical juncture. As former employees like Saunders advocate for systemic reforms to ensure alignment with public interests and ethical principles, questions persist about the efficacy of current governance structures and the role of individual leadership. The reinstatement of Sam Altman and the confidence expressed by OpenAI's new board members underscore the complexity of these issues and the need for transparent and collaborative approaches moving forward. Ultimately, the pursuit of safe and beneficial AI necessitates a concerted effort to establish robust mechanisms of oversight and accountability that transcend individual personalities and prioritize the collective well-being of society.