Artificial Intelligence: OpenAI Launches “Preparedness” Team To Guard Against Catastrophe By AI

The new OpenAI team would oversee and evaluate the development of “frontier AI models” to watch out for risks including nuclear war, cyberwarfare, and pandemics.

OpenAI is taking significant steps to mitigate the potential “catastrophic risks” associated with cutting-edge artificial intelligence models. This initiative, known as Preparedness, is tasked with overseeing and evaluating the development of highly capable AI models that could possess dangerous capabilities, particularly in areas like cybersecurity and nuclear threats. The team, integrated into OpenAI, will ensure that these models adhere to strict safety protocols and guidelines.

Some specific concerns include the ability of AI models to manipulate human users through language and perform autonomous tasks. Additionally, OpenAI is actively addressing “extinction-level threats” such as pandemics and nuclear warfare. (Forbes)

The leadership of Preparedness falls to Aleksander Madry, the director of MIT’s Center for Deployable Machine Learning, who has temporarily joined OpenAI.

Another key objective is the formulation and maintenance of a “Risk-Informed Development Policy.” This policy will outline OpenAI’s approach to managing risks posed by advancing AI models, especially as they approach “artificial general intelligence.”

OpenAI is also encouraging external input through a challenge, seeking ideas on how AI might be misused to cause real-world harm. This proactive approach reflects OpenAI’s commitment to responsible AI development.

The backdrop to these efforts includes a growing awareness within the AI community about the potential risks of advanced AI models. Prominent figures, including OpenAI’s CEO Sam Altman, have emphasized the importance of addressing these risks while continuing to advance AI technology.

Related Story:  In Five Years’ Time, AI Could Escape Human Control – Geoffrey Hinton, the Godfather Of AI

Photo by benjamin lehman on Unsplash

Free Industry News

Subscribe to our free newsletter for updates and news about alternatives investments.

  • This field is for validation purposes and should be left unchanged.


Latest Alternative Investment News
Venture Capital: HUGO BOSS Invests In Sustainability-Focused Fashion Venture Fund
December 6, 2023     ESG and Sustainability, News, Venture Capital

HUGO BOSS has reinforced its commitment to sustainability by becoming the inaugural investor in Collateral Good Ventures Fashion I, a climate-centric venture capital fund aimed at expediting sustainability initiatives in…
Artificial Intelligence: xAI, The Musk-Owned AI Startup, To Raise $1B

Elon Musk’s artificial intelligence startup, xAI, aims to secure $1 billion in equity financing to compete with industry leaders such as OpenAI, Microsoft, and Google. The company, co-founded by Musk,…
Digital Assets: Bitcoin Mining Company Phoenix Group Makes Impressive Listing Debut On Abu Dhabi Securities Exchange
December 5, 2023     Digital Assets, News

Cryptocurrency mining firm Phoenix Group (ADX: PHX) has achieved a significant milestone by debuting trading on the Abu Dhabi Securities Exchange, marking one of the Middle East’s initial publicly listed…
Venture Capital: Alterra, A $30B UAE Climate Fund, To Partner With BlackRock, Brookfield, TPG
December 5, 2023     ESG and Sustainability, News, Venture Capital

Lunate Capital, based in the UAE, is partnering with global asset managers to create a $30 billion climate fund named Alterra, as announced at the COP28 summit in Dubai. Managed…