OpenAI Adds AI Safety Researcher Paul Christiano to Its Foundation Board

 

Paul Christiano, a prominent AI researcher known for his work on AI alignment and keeping advanced AI systems under human control, is joining the OpenAI Foundation’s board of directors.

 

Christiano has warned that rapid advances in AI could create a serious risk of humans losing control over increasingly capable systems. In a social media post announcing his appointment, he said he believes there is a “meaningful risk” of catastrophic and irreversible loss of control in the near term.

 

He also argued that the AI industry, including OpenAI, is not yet doing enough to reduce that risk. He said he is joining the board because he believes OpenAI has an opportunity to make significant progress on AI safety.

 

One of his concerns is that AI systems could eventually be used to train more capable AI systems, potentially triggering a rapid acceleration in capabilities that humans may struggle to control.

 

Christiano will serve on OpenAI’s Safety and Security Committee, led by Carnegie Mellon University professor Zico Kolter. The committee plays a key role in decisions about whether OpenAI should release new models.

 

Christiano has a long history with OpenAI. He helped pioneer reinforcement learning from human feedback (RLHF), a technique that became an important part of training large language models. He left OpenAI in 2021 and later founded the Alignment Research Center, focusing on how to ensure advanced AI systems remain aligned with human interests.

 

He has also warned that reinforcement learning could potentially create incentives for AI agents to seek power, undermine human control, or conceal behavior if those actions help them achieve their objectives. Recent incidents involving AI agents have, in his view, made these concerns more than a theoretical possibility.

 

Christiano is also affiliated with the U.S. government’s AI safety efforts, where he has contributed to evaluations of advanced AI models. According to OpenAI, he will continue advising the government but will recuse himself from matters involving OpenAI and its model evaluations.

 

His appointment comes as OpenAI faces renewed scrutiny over AI safety and security, making the addition of one of the field’s most prominent AI-alignment researchers to its board particularly significant.