Kitware Secures $11.5M, Multi-Year DARPA Contract to Teach AI How to Make Difficult Decisions Aligned with Humans

Kitware is leading a team for DARPA’s In The Moment program to deliver trusted AI solutions that can perform human-level reasoning

Clifton Park, N.Y. (August 10, 2023) – Kitware, a leading provider of advanced computer vision research, has been awarded a $11.5 million, multi-year contract by the Defense Advanced Research Projects Agency (DARPA) for the In The Moment (ITM) program. The goal of the program is to develop AI solutions humans can trust to help make difficult decisions. ITM is initially focused on medical triage decision-making, but, if successful, this framework would be used to develop future decision-making technology in other areas.

Kitware has composed a team of experts to deliver AI and deep learning technology that can perform human-level reasoning incorporating decision-making attributes and moral considerations. To this end, Kitware will be responsible for developing the Aligned Moral Language Models with Interpretable and Graph-based Knowledge (ALIGN) system. ALIGN reasons over all possible choices by flexibly weighing different factors such as the morality of a decision, its relation to medical triage knowledge, and alignment with human decision-making attributes. For interpretability, ALIGN will also generate natural language explanations of its decisions. ALIGN will also be capable of dynamically aligning with different decision-makers and attribute spaces, allowing it to adapt to a variety of users, settings, and scenarios.

“ITM is a critical fundamental research program advancing the science of ethical AI and improving trust in the decision-making capabilities of AI algorithms,” said Arslan Basharat, Ph.D., assistant director of computer vision at Kitware and the overall Principal Investigator on Kitware’s ITM effort. “Our team of world-renowned academic researchers, including the 2023 MacArthur Fellow, Yejin Choi, will be supported by a well-rounded team of domain experts in medical simulation and triage as well as cognitive science to address the challenges of this exciting program.”

An important decision-maker attribute for trusted AI systems may be their ability to provide explanations for their decisions using explainable AI (XAI) techniques. Tools like ChatGPT are normalizing the use of AI for decision-making, but the results are not always reliable, and often the AI cannot provide the reasons underlying its decisions. Explainable AI reveals the rationale, characterizes strengths and weaknesses, and conveys an understanding of how the technology will behave in the future. For making critical decisions, including healthcare, criminal justice, and autonomous driving, it is essential that users, developers, and researchers understand how and when to trust complex AI models.

There are five subcontractors on Kitware’s team, including the University of Washington, which will apply its groundbreaking research in large language models, common sense reasoning, ethical decision-making, and human-AI alignment; the University of California, Berkeley, which will contribute its expertise in machine learning, linguistics, AI agent alignment, and decision-making; the University of Illinois at Urbana-Champaign, which will contribute its expertise in aligned decision-making, knowledge graph representations, and reasoning; MacroCognition LLC, which will apply cognitive science to support scenario development and advise on the human decision-making attributes; and the Geneva Foundation who will provide medical expertise to design scenarios for development and recruit trusted human decision-makers, including military medics and trauma surgeons, to support algorithm development.

Throughout this project, Kitware will develop open source software and contribute to the research community in areas such as decision-making, ethical AI, explainable AI, natural language processing, human physiology simulation, and others. For more information about Kitware’s expertise in these domains, contact Kitware.

This project is sponsored by the Air Force Research Laboratory and Defense Advanced Research Project Agency (AFRL and DARPA).

About Kitware
Kitware delivers innovative software solutions built on open source technology. Kitware’s world-class, interdisciplinary team of technical experts develops custom software for customers in the areas of medical computing, computer vision, and scientific visualization. Since its founding in 1998, Kitware has cultivated a reputation for putting people first. The company understands and prioritizes customer needs, offering dependability and transparency. Kitware also cares for its employees and is proud to be 100% employee-owned. For additional information, visit kitware.com or follow on Twitter, LinkedIn, and Facebook.

Featured Photo: U.S. Air Force photo by Johnny Saldivar courtesy of DVIDS

###