60NANB24D231

Title: CMU/NIST AI Measurement Science & Engineering Cooperative Research Center

Purpose: Establish the CMU/NIST AI Measurement Science & Engineering Cooperative Research Center (AIMSEC) to facilitate collaborative research and experimentation focused on advancing our national capability for test and evaluation of modern AI capabilities and tools.

There are three principal goals:
1. Advance modern AI risk management.
2. Validate evaluation approaches through stakeholder partnerships.
3. Translate assessment capabilities and methodologies to practice.

Research will focus on advancing measurement science for modern AI systems, including machine learning (ML) and generative AI (GENAI) systems, such as large language models (LLMs).

Activities to be performed: Create a practical taxonomy of threats against large language models (LLMs).

The proposed work includes developing techniques to reduce memorization and protect confidential information provably, even if it results in less exact outputs.

Develop new measures of robustness for ML models that better reflect a model’s ability to withstand attacks in practical scenarios.

Develop richer definitions of alignment, correctness, and intent to determine the suitability of generative AI systems for their intended uses.

Create a state-of-the-art suite of attacks against LLMs for testing new models.

These activities will be organized in three streams of effort aligned with NIST AI priorities:
- Validity, reliability, safety, privacy, and security
- Accountability, transparency, fairness, and explainability
- Generative AI evaluation

Expected outcomes: Development of AI system-level tooling, metrics, evaluation procedures, development processes, and best practices to ensure AI system builders consistently engineer safe AI systems.

Goals are to advance modern AI risk management, validate evaluation approaches through stakeholder partnerships, and translate assessment capabilities and methodologies to practice.

Research will focus on advancing measurement science for modern AI systems, including machine learning (ML) and generative AI (GENAI) systems, such as large language models (LLMs).

These efforts are expected to yield additional approaches to mitigation of weaknesses and vulnerabilities, privacy protection, a practical taxonomy of threats against LLMs, approaches to meet AI governance challenges, and new socio-technical definitions and metrics.

Intended beneficiaries: AI stakeholders, including government, industry, academia, and the general public, will benefit from this work of advancing capability for testing and evaluating modern AI capabilities and tools.

Subrecipient activities: There are no planned subawards.

Carnegie Mellon University

TO SUPPORT ACTIVITIES THAT DEVELOP, EXPAND, STRENGTHEN, OR SUSTAIN NIST PARTNERSHIP PROGRAMS AND/OR SUPPORT THE CONDUCT OF RESEARCH OR A RECIPIENT'S PORTION OF COLLABORATIVE RESEARCH IN A VARIETY OF AREAS INCLUDING, BUT NOT LIMITED TO: METROLOGY; STANDARDS; NANOTECHNOLOGY; ARTIFICIAL INTELLIGENCE; ADVANCED COMMUNICATIONS; ADVANCED MANUFACTURING; PROMOTION OF U.S. INNOVATION AND INDUSTRIAL COMPETITIVENESS; MEASUREMENTS IN SCIENCES; NEUTRON RESEARCH; GREENHOUSE GAS MEASUREMENTS; AND ENHANCING COORDINATION OF THE U.S. STANDARDS SYSTEM WITH GOVERNMENT AND PRIVATE SECTOR ORGANIZATIONS.

11.609 - Measurement and Engineering Research and Standards

National Institute of Standards and Technology (NIST) [DOC]

Pittsburgh, Pennsylvania 15213-3815 United States

Single Zip Code

Measurement Science and Engineering (MSE) Research Grant Programs (2023-NIST-MSE-01)