AI Safety as a Career: Where the Hardest Problems Live

The single most-asked question I get from engineers considering an AI safety career is: "Is this just a research job, or is there real engineering?" The answer in 2026 is: it is overwhelmingly engineering. Building circuit-level interpretability tools, scaling evaluation harnesses, red-teaming agents that use tools, and shipping monitoring infrastructure for frontier models — these are some of the most demanding engineering problems in the field. The math you need is "competent at PyTorch and linear algebra." The thing you actually need is conviction and rigor.

#The four subfields of AI safety

AI safety in 2026 is not a single field. It is four overlapping subfields with different cultures, different orgs, and different entry paths. If someone says "I work in AI safety" the right follow-up question is: "alignment, interp, evals, or policy?"

#1. Alignment

The classical problem: how do you train AI systems to robustly do what humans actually want, especially as systems become more capable than the humans supervising them? Concrete research includes RLHF, RLAIF (Constitutional AI), debate, recursive reward modeling, scalable oversight, and reward hacking.

#2. Interpretability (mechanistic interpretability, "mech interp")

The reverse-engineering side of AI safety: opening up a trained model and trying to understand what specific circuits and features inside it actually do. The famous papers — Anthropic's circuits work (Olsson, Elhage, Olah et al.), the "Toy Models of Superposition" paper, "Mathematical Framework for Transformer Circuits" — laid the foundation. In 2024-2026 mech interp has expanded from curiosity-driven research to a tool used for safety eval, debugging, and even feature steering of production models.

#3. Evaluations (evals)

Designing and running the experiments that measure whether a model has a dangerous capability or behavior. This includes capability evals (can the model help with biothreats? can it autonomously make money online?), alignment evals (does the model resist jailbreaks? does it deceive its operator?), and dangerous-capability red-teaming.

#4. Policy

Translating the technical questions into government, industry, and international policy. This is a fast-growing area in 2025-2026 as governments (UK, US, EU, China) build serious AI-policy capacity.

#Compensation reality check (2025-2026 numbers)

Money matters, even when the work matters more. Here is the rough lay of the land for early-to-mid-career researcher and engineer roles:

The pattern: labs pay the most, government and nonprofits pay 30-50% less but offer mission-critical work. For early-career people, the labs are typically the highest-leverage place to learn. For mid-career people with families or geographic constraints, the AISIs are increasingly compelling.

#Entry paths by background

#From an ML engineering background

Your most direct paths are into interpretability engineering or evaluations engineering. Specific moves:

• Study Anthropic's circuits papers and the mech-interp tooling stack (transformer-lens, nnsight, Anthropic's open-source circuits-style notebooks)
• Apply to MATS (ML Alignment & Theory Scholars) — a competitive 12-week program in Berkeley that has placed scholars into Anthropic, OpenAI, METR, Apollo, and AISI
• Apply directly to interp/evals engineering roles at the labs and AISIs — they want experienced engineers and the pipeline is undersaturated relative to research roles
• Build public interpretability artifacts: replicate an attention-head analysis on an open model, publish on a personal blog or LessWrong/Alignment Forum

#From a security background

The fastest pivot in 2025-2026. Red-teaming AI systems requires the same mental muscle as red-teaming traditional software: thinking adversarially about how a system could be subverted. Specific moves:

• Apply to AI red team roles at Anthropic (Frontier Red Team), OpenAI Preparedness, UK AISI Cyber team, Pattern Labs
• Take BlueDot Impact's "Intro to AI Safety" + "AI Security" courses
• Publish a public red-team write-up of a flaw in an open-source model or agent system — this is a high-signal artifact

#From an academic / research background

Your best path is direct application to research scientist roles, especially if you have a relevant ML, neuroscience, or theoretical CS background. Specific moves:

• Apply to MATS as an entry point
• Apply directly to Anthropic, DeepMind Safety, MIRI, Redwood
• Publish on the Alignment Forum or LessWrong — these are read by every hiring manager in the field
• Consider AI Safety Camp (AISC) — a structured remote program that produces concrete research outputs

#From a policy background

Specific moves:

• Apply to Horizon Institute for Public Service (a competitive fellowship that places policy-curious technically-oriented people into US government AI roles)
• Apply to RAND Corporation's tech policy positions
• Open Philanthropy and the Long-Term Future Fund both fund policy-research projects directly
• For UK: the AISI policy team and the Centre for AI Risk in Oxford

#The on-ramp programs you should know by name

These are the named programs that the safety community uses as filters and accelerators. Knowing them by name in an interview signals you have done your homework.

• MATS (ML Alignment & Theory Scholars). 12-week scholar program in Berkeley; competitive entry; produces published research and direct lab placement.
• AISC (AI Safety Camp). Remote, structured, runs in cohorts; lower bar than MATS but useful for first artifacts.
• ARENA (Alignment Research Engineering Accelerator). A curriculum-based bootcamp focused on the engineering side of safety, especially mech interp and evals.
• BlueDot Impact. Runs intro and advanced AI-safety courses online; widely used as a knowledge-leveling on-ramp.
• 80,000 Hours career guide. The canonical career-advice resource for high-impact careers, including AI safety.
• Horizon Institute for Public Service. US government policy fellowship.
• Tarbell Fellowship. Journalism-adjacent fellowship for technically-trained policy writers.
• Open Philanthropy / Long-Term Future Fund (LTFF). Both fund independent researchers; LTFF has lower friction for smaller grants.

If you complete one of these and produce a public artifact, you cross a hiring threshold at most labs. The pattern: program → artifact → application → offer.

#What hiring managers actually look at

When a hiring manager at Anthropic, METR, or Apollo reviews your application, they look at these things — in roughly this order:

1. A public artifact — a blog post, a GitHub repo, an Alignment Forum post — that demonstrates technical taste and serious engagement with the field. This single item is more load-bearing than your resume.
2. Endorsements from people in the field — MATS or AISC mentorship, named recommenders, public collaboration.
3. Resume and ML background — strong industry or academic experience, demonstrated by shipped work.
4. A take-home or interview round — typically a research-engineering exercise on a real safety task (replicating part of a paper, implementing an eval, doing a mini-interp study).

The strategic implication: spend more time producing one strong public artifact than tuning a resume. A single excellent interpretability blog post or eval write-up will outperform a half-year of polishing application materials.

#Hands-on: detecting induction heads in a transformer

This is the canonical exercise that anyone going into mechanistic interpretability does. Induction heads — discovered and described by Olsson, Elhage, Olah et al. at Anthropic — are attention heads that implement an in-context-learning primitive: "look back for the previous occurrence of the current token, and predict what came after it." They are the mechanism behind much of an LLM's in-context-learning behavior.

The scratchpad below sets up a simulated induction-head detection on a small synthetic attention pattern. You run it inline, see the score, and modify the code to detect alternative attention patterns. This is the kind of micro-experiment that a mech-interp interview round will ask you to design from scratch.

#A second scratchpad: a tiny eval harness

The second canonical exercise for safety engineering is building an eval. Below is a minimal harness that scores a hypothetical model's behavior on a "harmful instruction" test set. In a real interview, you would be asked to extend this with proper task batching, error handling, statistical analysis, and a way to interpret refusal versus compliance.

Even this tiny harness reveals the real engineering questions safety evaluators ask: How do we measure refusal vs. false-refusal? What is the statistical significance of the result? How do we extend the eval to cover the long tail of attack variants? Anyone who can build an extended version of this fluently is competitive for an evals-engineering role.

#The cultural rules of the field

The AI safety community has cultural norms that differ from mainstream ML research. Knowing them by reflex will signal that you have been around long enough to be ready to contribute.

• Calibration matters more than confidence. Saying "I am 70% confident" is more respected than "I think this is true." Many people in the field literally make and track public predictions.
• Theoretical results are valued. The community takes formal-ish results seriously — proofs about RL reward hacking, formalisms of "agency," etc. — much more than a typical ML lab.
• Publication norms are different. Many results are posted first on LessWrong or the Alignment Forum, not on arXiv. The community expects engagement with that infrastructure.
• Disagreement is structured. Public, written disagreement (with reasoning, predictions, and citations) is healthier than in many fields. If you have a contrarian take, write it up.
• "Galaxy-brained" is a slur. It refers to reasoning that is internally consistent but disconnected from real-world consequences. Maintain ground-truth contact at all costs.

#Resources to bookmark and use

• 80,000 Hours AI safety career guide. https://80000hours.org/career-guide/ — the canonical career-advice resource
• Alignment Forum. https://alignmentforum.org, where research is discussed
• LessWrong. https://lesswrong.com — broader rationality and AI safety community
• Anthropic Research blog. For papers and the circuits work
• OpenAI Safety blog. Preparedness, superalignment archives
• DeepMind Safety blog. Scalable alignment research
• METR blog and reports. Capability evaluation methodology
• Apollo Research. Agentic-AI evaluations
• BlueDot Impact. https://bluedot.org — entry-level courses
• MATS. https://www.matsprogram.org/ — research scholar program
• ARENA. https://www.arena.education/ — engineering-focused alignment bootcamp
• Transformer Lens. https://github.com/TransformerLensOrg/TransformerLens — the standard mech-interp library