The Joint Artificial Intelligence Center has floated a “Request for Information” for cutting-edge testing and evaluation capabilities.

The Joint Artificial Intelligence Centre, or JAIC, is looking for tools to test and evaluate future artificial intelligence products. The tools must also certify the Pentagon’s products as safe and effective. (Nextgov) It’s a case of “watchers watching the watchers.”

The JAIC has been tasked by the Pentagon to oversee the adoption and implementation of artificial intelligence. The JAIC’s Test and Evaluation Office is responsible for developing standards for AI. It also must test AI algorithms, systems, and operations.

In February, the Defense Innovation Board recommended the Pentagon’s 5-point AI Principles after a 15-month study.

It’s a Request for Information, not a Request for Proposal

“The JAIC is requesting testing tools and expertise in planning, data management, and analysis of inputs and outputs associated with those tools. The introduction of AI-enabled systems is bringing changes to the process, metrics, data, and skills necessary to produce the level of testing the military needs, and that is why the JAIC is requesting information,” Dr. Jane Pinelis, Chief, Test, Evaluation and Assessment at the JAIC, said in a statement.

“Testing and Evaluation provides knowledge of system capabilities and limitations to the acquisition community and to the warfighter. The JAIC’s T&E team will make rigorous and objective assessments of systems under operational conditions and against realistic threats so that our warfighters ultimately trust the systems they are operating and that the risks associated with operating these systems are well-known to military acquisition decision-makers.”

What the Pentagon/JAIC is seeking

In the RFI the Pentagon/JAIC is seeking information on T&E software for AI, creation of AI T&E processes and best practices, and the ability of vendors to provide AI test and evaluation services.

The scope encompasses test technology and tools, test and evaluation services, and other technologies.

The first-named item covers emerging AI technologies including Machine Learning (ML), Deep Learning (DL), and Neural Networks / Neural Nets (NN) with focus on the following:

1. Conversational interface applications using voice to text.
2. Speech-enabled products and services for DoD applications and systems.
3. Image analysis, testing deep learning-based visual search and image classifier.
4. Natural Language Processing (NLP) enabled products and services.
5. Humans augmented by machines, to include human-machine interfaces and improved methods to measure warfighter cognitive and physical workloads, to include augmented reality (AR) and virtual reality (VR) test services.
6. Autonomous systems.

Prospective applicants must submit a capability statement and a white paper by [10:00] AM CST on 12 May 2020.

Related Story:     Artificial Intelligence: The Pentagon Adopts Five AI Principles