Anthropic is an AI safety and research company.

Anthropic, a public benefit corporation and AI startup launched in 2020, raised a $124 million Series A in 2021 and has just announced a massive $580 million Series B round led by Sam Bankman-Fried, CEO of FTX. The round also included participation from Caroline Ellison, Jim McClave, Nishad Singh, Jaan Tallinn, and the Center for Emerging Risk Research (CERR). (Anthropic)

Anthropic, which is based in San Francisco, was founded by the brother-and-sister team of Dario Amodei and Daniela Amodei, who were vice presidents at AI research lab OpenAI LLC. Now much larger, its 40-person team focuses on the development of large-scale AI models that are safe.

“Large, general systems of today can have significant benefits, but can also be unpredictable, unreliable, and opaque: our goal is to make progress on these issues,” says the Anthropic website.

The company is researching multiple areas including natural language, human feedback, scaling laws, reinforcement learning, code generation, and interpretability. It also intends to develop the technical components necessary to build large-scale models which have better implicit safeguards and require fewer after-training interventions.

“With this fundraiser, we’re going to explore the predictable scaling properties of machine learning systems, while closely examining the unpredictable ways in which capabilities and safety issues can emerge at scale,” said Dario Amodei, CEO. “We’ve made strong initial progress on understanding and steering the behavior of AI systems, and are gradually assembling the pieces needed to make usable, integrated AI systems that benefit society.”

Related Story: Chinese AI Startup Parametrix Raises $100M, Turns Unicorn