What Alignment Research Actually Is

Our vocabulary primer defined alignment as whether an AI system reliably tries to do what we actually intend. This primer is about the work behind that word.

That work isn't philosophy. It's a set of concrete activities you can name, fund, and ask vendors about. Nearly all of them come back to one question: how would we know? How would we know what a system can do, what it will do, and why it did what it did?

Traditional software does what its code says, and you can read the code. A modern AI model is different. It is billions of numbers, adjusted during training until the system produces outputs that score well. No one types in its values or its reasoning.

So we can't simply inspect a model to see what it's trying to do. We can only see what it does, on the inputs we happened to try. And training rewards outputs that look right, which isn't always the same as the right thing. A system can learn to please the grader rather than do the job.

Plain version: alignment research exists because behavior is the only thing we can observe directly, and behavior can be misleading.

The best-known technique is RLHF, reinforcement learning from human feedback. People compare pairs of model responses, pick the better one, and the model is adjusted toward what they preferred. Variants use written principles, sometimes called a constitution or a model specification, so that another AI can apply those rules at scale instead of a human rating every answer.

These methods are why today's assistants usually decline dangerous requests and try to be useful. Their weakness is built in: they reward what looks good to the rater. One well-documented side effect is sycophancy, where a model tells people what they want to hear, because agreeable answers tend to get rated highly.

An eval is a structured test: a set of tasks or prompts, run the same way every time, with scored results. They come in three main kinds:

Frontier developers now publish many of these results in system cards or model cards that come out alongside a new model.

Red-teamers try to make a system misbehave. They use jailbreaks, prompts engineered to get around its refusals, and prompt injection, instructions hidden in a document or web page that the AI reads and then obeys. They also test whether it can be pushed into giving harmful help. Some red-teaming is done by specialists, some by outside experts in fields like biosecurity, and more and more by automated tools that generate attacks at scale.

Red-teaming is how many real failures get caught and fixed before release. Like evals, it can show that a weakness exists but can't prove none remain. Attackers after release keep finding new ones.

Everything above watches what a model does from the outside. Interpretability tries to understand what's happening inside it, which concepts it represents and how it combines them to reach an answer.

There has been real progress. Researchers can now find internal patterns that correspond to recognizable concepts and turn them up or down to change behavior. In one widely covered 2024 demonstration, Anthropic amplified a single internal "Golden Gate Bridge" feature, and the model began steering nearly every conversation toward the bridge. That shows these internal patterns really drive behavior.

But this work is still early. It explains only a fraction of what large models do, and it can't yet be used to certify that a model is safe. Its long-term promise is to check a system's reasoning directly instead of inferring it from behavior.

No U.S. state runs a model-testing body, which is part of why states rely on what developers disclose and on the questions they ask when buying.

The deepest alignment questions get settled, or not, in a handful of labs. But the checking side of this field translates directly into things a state agency, school district, or county can do right now: