Pittsburgh Review

The Software Engineering Institute (SEI) and the Office of the Under Secretary of Defense for Research and Engineering (OUSD(R&E)) have joined forces to address the challenges of ensuring trustworthiness in AI systems. The newly established Center for Calibrated Trust Measurement and Evaluation (CaTE) aims to pave the way for safe and reliable AI systems in critical situations.

Kimberly Sablon, principal director for trusted AI and autonomy within OUSD(R&E), emphasized the importance of understanding the capabilities and limitations of AI systems for responsible use. She stated, "The human has to understand the capabilities and limitations of the AI system to use it responsibly."

Farnam Jahanian, President of Carnegie Mellon University, expressed the university's commitment to supporting the DOD in leveraging AI for national security. He said, "Developing and implementing AI technologies to keep our armed forces safe is both a tremendous responsibility and a tremendous privilege."

Paul Nielsen, SEI director and CEO, highlighted the significance of assuring the safety and reliability of AI systems for military personnel. He stressed, "When military personnel are deployed in harm’s way, it’s of the utmost importance to give them not only the greatest capability but also the assurance that the AI and autonomous systems they depend on are safe and reliable."

The collaborative efforts of SEI, OUSD(R&E), and partners in industry and academia aim to establish standards and practices for safe human-machine teaming in the context of DOD mission strategy. The initiative seeks to address the complexities and engineering challenges associated with AI systems to ensure calibrated levels of trust.

The Center for Calibrated Trust Measurement and Evaluation (CaTE) is the first of its kind, focusing on human-machine teaming and measurable trust in AI systems. Through the combined expertise of SEI, CMU, and other partners, the initiative aims to advance the use of AI to strengthen national security and defense.

For more information about the SEI’s AI engineering research, visit sei.cmu.edu/our-work/artificial-intelligence-engineering/index.cfm.