AI Regulation: EU AI Act, NIST RMF & Governance Frameworks

#Why Regulation Matters Now

AI regulation is not a bureaucratic imposition -- it is a response to documented, large-scale harm. Hundreds of millions of people have been affected by AI decisions in hiring, credit, criminal justice, and healthcare. The regulatory frameworks emerging in 2024-2026 are the first systematic attempt to hold AI systems to the same accountability standards we apply to bridges, pharmaceuticals, and financial products.

Three forces converging at once are making this urgent:

#The EU AI Act: The World's First Comprehensive AI Law

The EU AI Act entered into force on 1 August 2024. It is the first legally binding, comprehensive regulation of AI anywhere in the world. Understanding it is now a baseline competency for any engineer building AI systems that affect EU residents -- which includes most commercial AI products given the EU's market size.

#Risk Tiers: The Four-Level Classification

The Act classifies AI systems into four risk tiers. Your obligations depend entirely on which tier your system falls into.

These systems are prohibited outright with no exceptions. Effective August 2024:

High-risk AI systems require conformity assessments, technical documentation, human oversight mechanisms, accuracy and robustness standards, and registration in the EU AI database before deployment. This tier covers:

• Biometric identification and categorization of natural persons
• Management of critical infrastructure (water, gas, electricity, transport)
• Education and vocational training (determining access, assessing performance, detecting cheating)
• Employment, worker management, and access to self-employment (CV sorting, interview assessment, task allocation, performance monitoring)
• Access to essential private and public services (credit scoring, insurance pricing, public benefits)
• Law enforcement (individual risk assessment, polygraph-equivalent testing, evidence evaluation)
• Migration, asylum, and border control
• Administration of justice and democratic processes

Systems that interact with users or generate content must disclose what they are:

• Chatbots must tell users they are interacting with AI
• Deepfakes must be labeled as AI-generated
• Emotion recognition and biometric categorization systems must notify subjects

Spam filters, AI in video games, simple recommendation systems, AI-powered search suggestions. The vast majority of AI applications fall here.

#High-Risk System Requirements

If your system is high-risk, here is what you must implement before deploying in the EU:

#GPAI: General Purpose AI Provisions

The EU AI Act includes a dedicated section for General Purpose AI (GPAI) models -- foundation models like GPT-4, Claude, Gemini, and Llama. If you are building or deploying these models, you have additional obligations.

All GPAI providers must

• Maintain technical documentation
• Publish a sufficiently detailed summary of training data (copyrighted content training data requires documentation and compliance with EU copyright law opt-out mechanisms)
• Make models available with a policy allowing downstream providers to comply with their own obligations
• Have in place a policy for copyright compliance

• Perform model evaluation and adversarial testing (red-teaming)
• Assess and mitigate systemic risks
• Report serious incidents to the European AI Office
• Implement cybersecurity measures and ensure adequate physical security

#Penalties

These apply to SMEs as well as large enterprises. The percentages are of global (not EU) revenue, which matters significantly for multinational companies.

#Implementation Timeline

The answer is C. Employment AI is explicitly listed in Annex III of the EU AI Act as a high-risk category. The company must implement conformity assessments, technical documentation, logging, human oversight mechanisms, and register the system in the EU AI database before deployment. The system is not banned, but the compliance burden is substantial.

#NIST AI RMF: The US Standard

The National Institute of Standards and Technology AI Risk Management Framework (NIST AI RMF), released January 2023, is the leading US voluntary framework for managing AI risk. It is increasingly becoming mandatory in practice through federal procurement requirements and enterprise buyer due diligence.

#The Four Core Functions

The NIST AI RMF is organized around four functions that mirror how mature risk management works in other engineering disciplines:

GOVERN is about building the infrastructure for everything else. Without it, MAP, MEASURE, and MANAGE are one-time exercises rather than continuous processes.

What GOVERN requires:

• Defined policies for AI risk management (what is your risk tolerance? who is accountable?)
• Roles and responsibilities assigned explicitly (who owns AI risk? who can block a deployment?)
• AI risk embedded in the software development lifecycle (not bolted on at the end)
• Processes for ongoing monitoring and updating of risk posture
• Workforce education and culture of responsible AI

GOVERN in practice: appoint an AI Risk Officer or embed AI governance in your existing GRC function. Build an AI risk review gate into your sprint process for features that affect real people. Maintain an AI system inventory.

MAP is where you apply GOVERN's policies to a specific AI system. For every AI system in your portfolio, MAP requires you to document:

• Intended purpose and context: What is the system designed to do? Who are the users? What decisions does it influence?
• Affected stakeholders: Who is affected by the system's outputs, including people who do not directly interact with it?
• Potential harms: What could go wrong? Categorize by type (individual harm, group harm, societal harm, financial, physical, psychological, reputational) and by likelihood and severity.
• Applicable laws and standards: What regulations apply? (EU AI Act, HIPAA, ECOA, NYC LL 144, etc.)
• Assumptions and limitations: What conditions must hold for the system to work as intended? What inputs or contexts will cause failure?

MAP in practice: fill out an AI Risk Registry entry for every new system. Make it a required artifact before any production deployment.

MEASURE is the technical heart of the framework. It requires you to:

• Select metrics that correspond to the risks identified in MAP (fairness metrics, accuracy metrics, robustness metrics, drift metrics)
• Establish baselines and thresholds (what is acceptable? what triggers action?)
• Run assessments on a defined cadence (not just at launch)
• Track metrics over time and document trends
• Test for identified failure modes (adversarial robustness, edge cases)

MEASURE in practice: add fairness metrics, accuracy-by-subgroup, and drift metrics to your ML observability dashboards. Set alert thresholds. Run a quarterly fairness audit. Track the metrics history so you can show regulators a continuous record.

MANAGE is where identified and measured risks get treated. For each significant risk:

• Develop and implement a treatment plan (mitigate, transfer, avoid, or accept with documented rationale)
• Assign accountability for the treatment
• Verify that treatments are effective
• Maintain residual risk documentation
• Escalate risks that exceed tolerance to appropriate decision-makers

MANAGE in practice: create a Risk Register with owner, treatment plan, target date, and status. When mitigation is complete, re-run the MEASURE function to verify the risk is reduced.

#The NIST AI RMF Playbook

NIST publishes a companion Playbook with specific suggested actions for each function. A sampling of the most practically useful:

GOVERN actions

• Document organizational risk tolerance for AI explicitly before building anything
• Create a cross-functional AI review committee with representatives from engineering, legal, ethics, and business
• Establish a process for AI systems to be retired when they are no longer fit for purpose

MAP actions

• Conduct a data provenance audit for every training dataset
• Document all assumptions about deployment environment that the system relies on
• Identify populations at elevated risk of harm from the system's errors

MEASURE actions

• Evaluate performance across demographic groups, not just aggregate metrics
• Test robustness against out-of-distribution inputs
• Measure the gap between development-environment performance and production performance

MANAGE actions

• Implement circuit breakers that automatically restrict or disable AI systems when metrics exceed thresholds
• Maintain rollback capability for model updates
• Document every significant risk that was accepted (not mitigated) and who accepted it

#Model Cards and Datasheets: What Regulators Actually Want to See

Model cards (Mitchell et al., 2019) and datasheets for datasets (Gebru et al., 2018) are documentation formats that have become the de facto standard for AI transparency. Both the EU AI Act (Article 11 technical documentation) and NIST AI RMF (MEASURE function) converge on requiring essentially what a well-written model card contains.

#Model Card Template

## Model Card: [Model Name] v[Version]

### Model Details
- Architecture: [e.g., fine-tuned BERT-base, XGBoost ensemble]
- Training data: [sources, date range, size]
- Model version: [semantic version + git SHA]
- Date trained: [YYYY-MM-DD]
- Authors: [team name + contact]
- License: [license type]

### Intended Use
- Primary intended uses: [specific use cases]
- Primary intended users: [who will use this]
- Out-of-scope use cases: [explicit list of what this model should NOT be used for]

### Factors
- Relevant factors: [demographic groups, environmental conditions, technical variations that affect performance]
- Evaluation factors: [which factors were evaluated]

### Metrics
- Model performance measures: [accuracy, F1, AUC -- overall and by subgroup]
- Decision thresholds: [at what threshold is a classification made?]
- Variation approaches: [confidence intervals, bootstrapped estimates]

| Group | Accuracy | TPR | FPR | Sample Size |
|-------|----------|-----|-----|-------------|
| Overall | 87.2% | 83.1% | 11.4% | 10,000 |
| Group A | 88.6% | 85.2% | 10.1% | 6,200 |
| Group B | 84.1% | 79.3% | 14.2% | 3,800 |

### Training Data
- Data sources: [where data came from]
- Preprocessing: [how it was cleaned, filtered, de-identified]
- Demographics: [representation breakdown by relevant attributes]
- Known biases: [explicitly documented biases in training data]
- Data governance: [consent, licensing, retention]

### Evaluation Data
- Dataset name and source
- Demographic breakdown
- Differences from training data

### Ethical Considerations
- Harms considered: [explicit list of harm types evaluated]
- Mitigations applied: [what was done to reduce bias/risk]
- Fairness metrics: [demographic parity ratio, equalized odds gap]
- Residual risks: [known limitations that were not fully mitigated]

### Caveats and Recommendations
- Known limitations: [specific failure modes observed]
- Deployment recommendations: [human oversight requirements, monitoring cadence]
- Update schedule: [when this card will be reviewed]

#What Makes a Model Card Actually Useful

Most model cards are useless. They contain vague disclaimers ("not intended for high-stakes use") with no supporting data, overall accuracy numbers with no demographic breakdown, and limitations sections that say "may not generalize to all populations" without specifying which populations or why.

A useful model card is specific and honest:

The model card should be updated every time the model is retrained. Treat it as living technical documentation, not a one-time compliance artifact.

#Algorithmic Impact Assessments

An Algorithmic Impact Assessment (AIA) is a structured analysis of an AI system's potential effects on people and society, conducted before and during deployment. It is analogous to an environmental impact assessment for infrastructure projects.

Several US states (Illinois, Maryland, Colorado) now require AIAs for high-risk automated employment and housing decisions. The EU AI Act's fundamental rights impact assessment requirement (Article 27) for deployers of high-risk systems is essentially an AIA.

#How to Conduct an AIA

• What decisions does the system make or influence?
• Who are the affected populations? Who does not interact directly but is still affected?
• What data is used? How was it collected?
• What is the decision authority of the system (fully automated vs. decision support)?

• What are the potential benefits? (efficiency, consistency, accuracy vs. human decision-making)
• What are the potential harms? Categorize by:
  • Type: economic, physical, psychological, social, legal
  • Who: individuals, groups, society
  • Likelihood: rare, possible, likely
  • Severity: minor, moderate, severe, catastrophic
  • Reversibility: correctable, difficult to reverse, irreversible

• Compute fairness metrics across all relevant demographic groups
• Identify and document disparate impacts
• Trace disparities to their source (data, model, features)

• What controls are in place for identified harms?
• What is the residual risk after mitigations?
• Who approved acceptance of residual risks?

• For systems affecting vulnerable populations, have affected communities been consulted?
• What feedback was received and how was it incorporated?

• Who approved deployment?
• What conditions were attached to approval?
• How will mitigations be verified in production?
• When will the AIA be reviewed again?

#AI Incident Report Template

## AI Incident Report

### Incident ID: [YYYY-MM-DD-NNN]
### Severity: [P0 / P1 / P2 / P3]
### Status: [Open / Investigating / Resolved]

### Summary
[One paragraph: what happened, who was affected, when, and current status]

### Detection
- Detected by: [monitoring alert / user report / internal audit / external report]
- Detection time: [YYYY-MM-DD HH:MM UTC]
- Time to detection from incident onset: [hours]

### Impact
- Systems affected: [list AI systems involved]
- Users/people affected: [count and demographic breakdown if available]
- Nature of harm: [discrimination, privacy breach, incorrect output, safety failure, etc.]
- Severity justification: [why this severity level was assigned]

### Timeline
- [YYYY-MM-DD HH:MM] Incident begins
- [YYYY-MM-DD HH:MM] Detection
- [YYYY-MM-DD HH:MM] Triage complete
- [YYYY-MM-DD HH:MM] Containment applied
- [YYYY-MM-DD HH:MM] Resolution

### Root Cause Analysis
- Technical root cause: [model failure, data issue, system bug, configuration error]
- Contributing factors: [monitoring gap, insufficient testing, process failure]
- Why mitigations did not prevent this: [gap analysis]

### Containment and Resolution
- Immediate action taken: [disable feature, apply hotfix, restrict access]
- Permanent fix: [model retrain, data correction, architecture change]

### Regulatory Notification
- Was external notification required? [Yes / No / Under review]
- If yes: [authority notified, date, summary of disclosure]

### Lessons Learned
- What worked in our response
- What should be improved
- Action items with owners and deadlines

### Sign-off
- Incident commander: [name]
- Engineering lead: [name]
- Legal/compliance review: [name + date]

#AI Governance in Practice: Review Boards That Work vs. Theatre

Most AI ethics review boards fail. They are advisory only (leadership can override them), composed entirely of internal employees with P&L pressure, review only finished systems rather than designs, and have no tracking of whether their recommendations were implemented. This is governance theater.

What makes a review board effective

The pattern that produces real governance: embed the AI review process as a sprint gate, not an external audit. Engineers know that any system touching personal decisions will go through the board, so they build with that in mind from the start.

#Sector-Specific Regulations

The EU AI Act is horizontal regulation that applies across sectors. Many sectors have additional, older regulations that specifically apply to algorithmic decision-making.

#Finance: SR 11-7 (Federal Reserve/OCC Model Risk Management)

SR 11-7 (Supervisory Guidance on Model Risk Management) was issued in 2011 and applies to all financial institutions supervised by the Federal Reserve and OCC. It requires:

• Model inventory: Document all models used for business decisions
• Model validation: Independent validation of all models before production use and periodically after
• Challenger models: Benchmark models against alternatives
• Documentation: Development and validation documentation sufficient to allow replication
• Ongoing monitoring: Track model performance against benchmarks
• Change management: Formal process for any model changes

For AI/ML models in banking, SR 11-7 validation requires both quantitative validation (performance metrics, stability analysis, sensitivity analysis) and conceptual soundness review (is the model appropriate for this use case?). The validator must be independent of the model development team. This is the financial industry's equivalent of the EU AI Act for models in credit, trading, and risk management.

#Healthcare: FDA Software as a Medical Device (SaMD)

The FDA regulates AI/ML software that meets the definition of a medical device -- software intended to treat, diagnose, cure, mitigate, or prevent disease. The key regulatory pathway depends on risk:

• Class I (General Controls): Low-risk devices, most exempt from premarket notification
• Class II (510(k)): Moderate risk, requires demonstrating substantial equivalence to a predicate device
• Class III (PMA): High risk (e.g., AI for autonomous diagnosis), requires full premarket approval

Key requirements for AI SaMD

• Performance requirements specified for the intended use
• Training and validation dataset requirements (geographic, demographic, clinical diversity)
• Transparency about model limitations
• Post-market performance monitoring
• Real-world evidence collection

#Employment: NYC Local Law 144

New York City Local Law 144 (effective July 2023) is the first US law requiring independent bias audits of automated employment decision tools (AEDTs) used in hiring or promotion for NYC-based employees.

Requirements

• Annual bias audit by an independent auditor before the tool is used
• Public posting of audit results (impact ratio by gender and race/ethnicity)
• Disclosure to candidates that an AEDT is being used
• Provide candidates with an alternative selection process upon request

NYC LL 144 is a preview of where US employment law is heading. Illinois, Maryland, and Colorado have similar or pending legislation.

#Code Templates

#Quiz

#Key Takeaways

• EU AI Act creates four risk tiers -- unacceptable (banned), high-risk (significant obligations), limited risk (transparency only), and minimal risk (no requirements); employment, credit, healthcare, and law enforcement AI all fall into high-risk with a 2 August 2026 enforcement deadline
• GPAI models have their own compliance track -- all GPAI providers must document training data and implement copyright compliance; systemic risk models (>10^25 FLOPs) additionally face mandatory red-teaming, incident reporting, and enhanced cybersecurity
• NIST AI RMF provides the operational framework -- GOVERN (policies), MAP (per-system risk inventory), MEASURE (metrics and monitoring), MANAGE (treatment and escalation); the most common failure is treating it as a one-time exercise rather than a continuous process
• Model cards are what regulators actually audit -- the EU AI Act Article 11 technical documentation requirements and NIST AI RMF MEASURE function both converge on requiring disaggregated performance metrics, data governance documentation, known limitations, and residual risk acknowledgment
• Sector-specific regulations layer on top -- SR 11-7 for finance, FDA SaMD pathways for healthcare AI, NYC LL 144 and state equivalents for employment AI, ECOA explainability requirements for credit

#Recap