Making Machine Learning Safe for the World
Executive Summary
The productive impacts of state-of-the-art machine learning (ML) have been, and will continue to be, sharply limited given the difficulty or inability to integrate it with safety-critical applications – those in which failures of a system during operation would cause harm to individuals, the public, or the environment. Safety-critical domains are the nexus at which problems that plague state-of-the-art ML in lower-stakes domains meet, revolving principally around matters of reliability, human interpretability, and the ability to intervene in the system’s internal mechanisms during operation. That general-purpose ML models today cannot meet the bar for their adoption in safety-critical domains contributes to a sense that they are ever-present with only modest social and economic impacts.
ML, having compelled the world to accommodate it in its most versatile forms, must change if it is to succeed in the most sensitive domains of application. Such changes should follow in the lineage of the most impactful engineering marvels of recent history by providing guarantees on performance in safety-critical domains, minimizing the probability of harmful outputs and reducing their severity.
Policy Recommendations
- The National Artificial Intelligence Initiative Office (NAIIO), together with the Networking and Information Technology Research and Development (NITRD) Subcommittee, should instruct the National Artificial Intelligence Research & Development Interagency Working Group to coordinate investments in safety-critical neuro-symbolic AI research and development totaling $1.5 billion over a five-year period.
- The National Institute of Standards and Technology’s (NIST) Center for AI Standards and Innovation (CAISI) should inform – but not exclusively guide – the NAIIO’s investment coordination by convening a working group that establishes a new evaluation metrics research agenda specifically for safety-critical neuro-symbolic systems.
- A separate working group convened by NIST’s CAISI should determine whether Autonomy Readiness Levels (ARLs) for non-defense, safety-critical applications are warranted at the current stage of development. These are evaluation metrics specifically for models expected to perform autonomously over a potentially shifting range of circumstances in the execution of a human-given input. AI “agents” may be considered a part of this target group.
- The National Science Foundation (NSF) should parallel the NAIIO’s efforts by establishing a National AI Research Institute dedicated to a specific application area of neuro-symbolic AI. An initial investment worth $80 million to $100 million over five years should be made.
The views expressed in this article are those of the author and not an official policy or position of New Lines Institute.
