ISO/IEC 42105:2025 (E) - Information technology � Artificial intelligence � Guidance for human oversight of AI systems

0.1 General

This document provides guidance on human control and monitoring of AI systems, referred to as human oversight. It supports the safe, trustworthy, and responsible operation of AI systems by ensuring meaningful human involvement throughout the AI lifecycle.

It is applicable to all types of organizations and AI systems, regardless of size or sector, and extends ISO/IEC TS 8200 on AI system impact assessment and related trustworthiness concepts.

Example: In a high-risk AI system for medical diagnosis, human oversight ensures clinicians can review, override, or intervene in AI recommendations to protect patient safety.

0.2 Relationship with Other Standards

This standard complements ISO/IEC 42001 (AI Management System), ISO/IEC 23894 (AI Risk Management), ISO/IEC 42005 (AI System Impact Assessment), and the EU AI Act (particularly provisions on human oversight for high-risk systems).

Example: Integrate human oversight requirements from this standard with risk controls from ISO/IEC 23894 and transparency elements from ISO/IEC 12792.

This document provides guidance on human oversight of AI systems, including principles, roles, mechanisms, and processes for effective human control and monitoring. It covers the entire AI system lifecycle from design to decommissioning.

It does not prescribe specific technical implementations but offers flexible, risk-based approaches suitable for different AI applications and contexts.

Example: Guidance for deployers of autonomous AI agents in customer service to maintain human-in-the-loop controls for sensitive interactions.

The following documents are referred to in the text:

- ISO/IEC 22989, Artificial intelligence � Artificial intelligence concepts and terminology

- ISO/IEC TS 8200, Artificial intelligence � Impact assessment

- ISO/IEC 42001, Artificial intelligence � Management system

- ISO/IEC 23894, Information technology � Artificial intelligence � Guidance on risk management

Example: Use ISO/IEC 22989 for consistent terminology when defining oversight roles and responsibilities.

Key terms include:

- Human oversight: Processes enabling humans to understand, monitor, and intervene in the operation of AI systems.

- Human-in-the-loop: Human involvement in decision-making before or during AI system operation.

- Human-on-the-loop: Human supervision and ability to override or intervene after AI operation.

- Human-out-of-the-loop: Fully autonomous operation with minimal or no real-time human involvement.

Example: "Meaningful human control" refers to oversight that is timely, informed, and effective in influencing outcomes.

4.1 General

Human oversight should be proportionate to the risks and impacts of the AI system.

4.2 Core Principles

Principles include transparency, accountability, competence of overseers, timeliness of intervention, and adaptability to changing conditions.

Example: For AI in hiring, oversight principles ensure bias detection and correction by qualified HR professionals.

4.3 Context Considerations

Oversight design must consider the specific use case, risk level, regulatory requirements, and stakeholder needs.

5.1 Identification of Roles

Define roles such as AI system supervisors, reviewers, intervention operators, and governance overseers.

5.2 Competence and Training

Ensure individuals performing oversight have the necessary knowledge, skills, and understanding of the AI system.

5.3 Authority and Accountability

Assign clear authority to intervene and hold humans accountable for oversight decisions.

6.1 Monitoring Mechanisms

Tools and interfaces for real-time or periodic monitoring of AI behavior, performance, and outputs.

6.2 Intervention Capabilities

Mechanisms allowing humans to pause, override, or modify AI operations.

6.3 Explainability Support

Integration with transparency and explainability features to support informed oversight.

Example: Dashboard with confidence scores and counterfactual explanations for human reviewers in credit decision AI.

7.1 Integration in Design

Embed human oversight considerations from the earliest stages of AI development.

7.2 Risk-Based Approach

Scale oversight intensity according to the potential impact and risk level of the AI system.

7.3 Documentation

Maintain records of oversight design, implementation, and effectiveness.

8.1 Operational Controls

Procedures for day-to-day oversight during deployment and use.

8.2 Technical Enablers

Features such as alerts, human-AI interfaces, and fallback mechanisms.

8.3 Procedural Safeguards

Protocols for escalation, logging of interventions, and post-event review.

9.1 Oversight Effectiveness

Evaluate how well human oversight achieves intended safety, fairness, and reliability outcomes.

9.2 Metrics and Indicators

Use measurable indicators such as intervention frequency, override success rate, and user feedback.

9.3 Audits and Reviews

Conduct periodic audits of the human oversight framework.

10.1 Continual Improvement

Update oversight practices based on performance evaluations, incidents, and new insights.

10.2 Adaptation to Changes

Adjust human oversight when AI systems, environments, or regulations change.

Example: Retrain overseers and update interfaces after a major model upgrade.