The AI Pilot uses both vision and natural language to communicate its intent with other aircraft, whether manned or not.

Researchers at Carnegie Mellon University have developed an AI-powered system that pilots an autonomous aircraft through crowded airspace. This first-ever “AI pilot” guides the aircraft well enough to avoid collisions, anticipates the likely flight path of other planes in the vicinity and takes actions accordingly, and maintains radio communications with pilots and air traffic controllers (ATCs). (Carnegie Mellon University)

The researchers trained the AI on data collected at the Allegheny County Airport and the Pittsburgh-Butler Regional Airport. This daya included air traffic patterns, images of aircraft and radio transmissions.

This training enabled the AI to use both vision and natural language for communicating with other manual, or autonomous, aircraft. Vision is enabled through the use of six cameras and a computer vision system. Its automatic speech function is powered by natural language processing so that the AI pilot can process incoming radio messages as well as speak with other pilots and the ATCs.

As a result, the AI pilot is capable of safe and socially compliant navigation.

The AI pilot was tested on flight simulators (not real aircraft), and found to be competent. In the test, the researchers ran a flight test within the same airspace on two simulators, one controlled by the AI, the other by an actual human. The AI pilot could safely navigate itself, it was found.

“This is the first AI pilot that works in the current airspace,” said Sebastian Scherer, an associate research professor in the CMU’s Robotics Institute and a member of the team. “I don’t see that airspace changing for UAVs. The UAVs will have to change for the airspace.”

The AI pilot could extend the capabilities of autonomous aircraft that move people or goods, inspect infrastructure, protect crops, and for controlling poaching or deforestation.

Related Story: Underwater, Autonomous Glider Tracks CO2 Levels In Alaska’s Resurrection Bay