prEN 18282 — Practical Cybersecurity Requirements for AI Systems

0.1 General — What You Actually Need to Do

In practice, you must implement a cybersecurity management process that protects AI systems against intentional attacks and unintentional failures throughout their entire lifecycle. This is not just traditional IT security — AI systems introduce new attack surfaces (training data, models, prompts, inference APIs) that require specific controls.

0.2 AI-Specific Cybersecurity Risks — The New Threats You Must Address

Traditional cybersecurity is not enough. You must explicitly address AI-specific threats such as:

• Data poisoning and model poisoning
• Adversarial examples (especially in computer vision and autonomous systems)
• Prompt injection and jailbreaking in generative AI
• Model extraction and inversion attacks
• Backdoor insertion during training or supply chain

In practice you must:

• Apply these cybersecurity requirements to every high-risk AI system you place on the EU market or put into service.
• Cover the full lifecycle: data collection, model development, training, testing, deployment, continuous monitoring, updates, and decommissioning.
• Address both intentional attacks and unintentional failures that could lead to safety or fundamental rights violations.

In practice you need immediate access to:

• EU AI Act (Regulation (EU) 2024/1689) — Articles 8–15 and Annex III
• prEN 18282 Cybersecurity — both Part 1 (normative) and Part 2 (guidance & examples)
• prEN 18286 AI Quality Management System — for integration
• ISO/IEC 42001 AI Management System — for management capability
• Relevant cybersecurity standards (ISO 27001, IEC 62443 for industrial AI, NIST AI RMF)

Key terms you will use daily:

• Threat = potential cause of an unwanted incident that may harm the AI system, users, or fundamental rights.
• Vulnerability = weakness in the AI system that can be exploited by a threat.
• Adversarial Example = input specifically crafted to cause the AI model to make a mistake.
• Prompt Injection = technique where an attacker manipulates the input prompt to override the model’s intended behaviour.
• Residual Cybersecurity Risk = risk remaining after all reasonable controls have been applied.

4.1 Establishing the Context — What You Do

At the start of every AI project and at least annually, conduct a structured threat modelling workshop with security, AI, legal and business teams. Document the AI system’s assets, data flows, deployment environment, and potential attackers.

4.2 Interested Parties — Who You Must Engage

Identify and consult internal teams (development, security, compliance), external parties (customers, regulators, security researchers), and affected users (especially vulnerable groups).

4.3 Cybersecurity Objectives — Setting Clear Goals

Define measurable cybersecurity objectives (e.g., “Reduce successful prompt injection attempts to less than 0.1% of queries”, “Achieve 99.5% robustness against known adversarial attacks on our dataset”).

5.1 Leadership and Commitment — Visible Actions

Top management must personally approve the AI cybersecurity policy, allocate budget for security tooling and red teaming, and review the cybersecurity risk dashboard at least quarterly. They must also ensure cybersecurity is embedded in the AI development lifecycle (not bolted on afterwards).

5.2 Cybersecurity Policy — Writing One That Works

Create a clear 1–2 page policy stating your commitment to protecting AI systems against attacks that could cause harm to health, safety or fundamental rights, and your requirement that residual cybersecurity risks are reduced to ALARP.

5.3 Roles, Responsibilities and Authorities

Appoint a Cybersecurity Owner for each high-risk AI system (can be the same person as the Risk Owner). Give them authority to block deployment if critical vulnerabilities remain unaddressed. Maintain a RACI matrix and up-to-date competence records for all security roles.

6.1 Threat Identification — The Process You Follow

Use structured methods (STRIDE, PASTA, or the threat catalogue in prEN 18282 Annex B) to identify all relevant threats. Pay special attention to AI-specific threats: data poisoning, model poisoning, adversarial examples, prompt injection, model extraction, and supply-chain attacks.

6.2 Vulnerability Assessment

Regularly scan for vulnerabilities in the full AI stack (data pipelines, training infrastructure, model serving, APIs, user interfaces). Include both traditional software vulnerabilities and AI-specific ones (e.g., model inversion risks).

6.3 Risk Estimation and Evaluation

Estimate likelihood and impact (including fundamental rights impact) for each threat-vulnerability pair. Evaluate whether residual risk after controls is acceptable. Use the risk matrix and acceptability criteria defined in your policy.

7.1 Resources — What You Must Budget For

Allocate dedicated budget and personnel for AI-specific security tools (adversarial robustness libraries, red teaming platforms, model scanning tools), external security audits, and continuous monitoring infrastructure.

7.2 Competence — Who Must Be Trained

All AI developers, MLOps engineers, security team members, and product owners must receive documented training on AI-specific threats, secure development practices for AI, and incident response procedures. Keep records for at least 5 years.

7.3–7.4 Awareness & Communication

Run regular awareness sessions (at least annually) covering real AI attack case studies. Establish clear channels for anyone to report suspected security issues without fear of retaliation. Publish security advisories when required.

8.1 Operational Planning and Control

Embed cybersecurity checkpoints into every stage of your AI development process (data collection, training, testing, deployment, monitoring). Require a signed cybersecurity risk assessment update before moving to the next stage.

8.2 Cybersecurity Controls Implementation — The Hierarchy You Must Follow

Apply controls in this order (most to least effective):

1. Inherently secure design (e.g., remove unnecessary data exposure, use differential privacy)
2. Technical protective measures (input sanitisation, adversarial training, robust model architectures, runtime monitoring)
3. Detection and response (anomaly detection, logging, incident response playbooks)
4. Information and instructions for users and operators

9.1 Monitoring, Measurement, Analysis and Evaluation

Define clear KPIs (e.g., number of blocked prompt injection attempts per day, model robustness score against adversarial examples, time to detect and respond to incidents, percentage of systems with up-to-date threat models).

9.2 Testing and Validation — What You Must Actually Test

Conduct regular testing that includes:

• Adversarial robustness testing on representative datasets
• Red teaming / penetration testing focused on AI-specific attack vectors
• Real-world condition testing (not just clean lab data)
• Supply chain security assessment (especially for pre-trained models and datasets)

9.3 Management Review

Top management must review the AI cybersecurity posture at least quarterly. Inputs include threat intelligence updates, test results, incidents, and resource needs. All decisions and actions must be documented.

10.1 Nonconformity and Corrective Action

When a vulnerability is discovered or an incident occurs, you must investigate root cause within defined timeframes (critical issues within 48 hours), update the threat model and risk assessment, implement corrective actions, and verify effectiveness before closing the issue.

10.2 Continual Improvement

After every significant incident, every major model release, and at least annually, review the entire cybersecurity process and ask: “What can we improve?” Update threat models, controls, testing methods, and training accordingly.

10.3 Incident Response — You Must Be Ready

Maintain documented incident response playbooks specific to AI systems (e.g., how to respond to a successful model extraction attack or a prompt injection that led to data leakage). Test these playbooks at least annually through tabletop exercises or simulated incidents.