Raytheon BBN will develop the device for the Defense Advanced Research Projects Agency (DARPA).

DARPA has picked Raytheon BBN to develop goggles that are equipped with Augmented Reality (AR) and Artificial Intelligence (AI) capabilities to serve as a virtual medical assistant to combat medics on the field. The device would provide real-time, hands-free guidance for 50-different medical procedures. (Raytheon)

Medical Assistance, Guidance, Instruction and Correction (MAGIC)

Visual and audio sensors embedded inside the device dubbed Medical Assistance, Guidance, Instruction and Correction (MAGIC) will track tasks and guide field medics in performing procedures correctly and efficiently. The DARPA Perceptually-enabled Task Guidance program’s consumer off-the-shelf augmented reality goggles will be used for the project.

“The combat medical environment is challenging and chaotic,” explained Brian VanVoorst, Raytheon BBN scientist. “Our goal for the Raytheon BBN MAGIC AI tool is to help support personnel to provide guidance as needed without disrupting their concentration.”

The cameras and sensors continuously monitor the situation, feeding information to the AI. The AI would learn from more than 2,500 videos recorded in stereo for 3D effects, and over 50 million images for training it on the correct sequence of actions in various medical procedures.

The AI can provide on-demand assistance to the medic, delivering audio suggestions and even a video overlay inside the device to display the ‘how-to’ for a procedure.

The AI can also advise the medic on the next steps, follow-on actions, and critical measurements such as time elapsed and dosing. These are invaluable to the front-line medics, who usually do not have the experience of a surgeon or doctor to administer a specialized procedure.

Yet, carrying this action out on a timely basis could mean the all-important difference between life and death for a soldier and DARPA.

MAGIC is scheduled for its first demo in about 18 months.

Related Story: The Pentagon Harnesses AI For “Days-Ahead” Outlook On World Events

Image Credit: Raytheon