Within Human Feedback
Can AI Principles Improve Human Oversight
Constitutional AI adds written principles and AI critiques to help guide models when direct human review becomes difficult to scale.
On this page
- Why direct feedback becomes harder to scale
- How principle based guidance changes training
- Limits of choosing universal AI principles
Page outline Jump by section
Introduction
Constitutional AI is an approach to AI alignment that tries to make oversight scale by replacing some direct human judgement with a written set of principles that an AI system can use to critique and improve its own behaviour. The central idea is simple: instead of asking humans to label every questionable response, humans define a “constitution” of values and rules, and AI systems use those principles to generate feedback during training.[anthropic.com]anthropic.comConstitutional AI: Harmlessness from AI Feedback \ AnthropicDecember 15, 2022…
This matters because increasingly capable AI systems may produce outputs that are too numerous, complex, or specialised for humans to evaluate one by one. If advanced AI is to support scientific discovery, medicine, education, and other foundations of a more abundant future, alignment methods need to work beyond small-scale human review. Constitutional AI is one attempt to address that challenge: not by removing humans from oversight, but by moving human input towards defining broader goals and principles that can guide large-scale supervision.[Alignment Forum]alignmentforum.orgAlignment Forum Scalable Oversight — AI Alignment ForumAlignment ForumScalable Oversight — AI Alignment ForumApril 17, 2026…
The approach is promising, but it does not solve the deepest alignment questions by itself. A constitution still reflects human choices about values, and an AI system can misunderstand, misapply, or selectively follow principles. The key question is therefore not whether AI can follow written rules, but whether carefully designed principles can help humans maintain meaningful oversight as AI capabilities grow.
Why direct feedback becomes harder to scale
Current AI assistants are often improved through reinforcement learning from human feedback (RLHF), where people compare outputs and reward answers that appear more useful, accurate, or safe. This approach has been important in making AI systems more cooperative, but it depends on a continuing supply of human evaluations. As tasks become harder, the weakness of this approach becomes clearer: humans may struggle to judge outputs in areas where AI systems become faster, more knowledgeable, or more capable than their reviewers.[European Data Protection Supervisor]edps.europa.euOpen source on europa.eu.
The problem is not simply the cost of hiring more evaluators. Some future tasks may be difficult because the evaluator genuinely cannot tell whether an answer is correct. A human may not be able to check a complex scientific argument, a large software system, or a long-term strategic plan produced by a highly capable AI. Scalable oversight research therefore explores methods that allow humans to supervise AI indirectly, often by using AI systems to help evaluate, critique, or compare outputs.[Alignment Forum]alignmentforum.orgAlignment Forum Scalable Oversight — AI Alignment ForumAlignment ForumScalable Oversight — AI Alignment ForumApril 17, 2026…
This creates a shift in the role of human feedback. Instead of humans providing millions of individual judgements, they may need to provide higher-level guidance: principles, standards, examples, and evaluation frameworks that AI systems can apply repeatedly. Constitutional AI is one implementation of this broader idea.
How principle-based guidance changes training
Constitutional AI was introduced by Anthropic as a method for training a harmless AI assistant using written principles rather than relying mainly on human labels identifying harmful outputs. The method uses a constitution: a collection of instructions describing desirable behaviour, such as being helpful, honest, and harmless. The AI then critiques and revises responses according to those principles.[anthropic.com]anthropic.comConstitutional AI: Harmlessness from AI Feedback \ AnthropicDecember 15, 2022…
The process has two main stages.
First, in a supervised learning phase, the model generates responses, evaluates them against constitutional principles, and rewrites them. These revised examples become training data. Instead of thousands of humans manually correcting every harmful or poor response, the AI produces critiques and improvements guided by the written rules.[anthropic.com]anthropic.comConstitutional AI: Harmlessness from AI Feedback \ AnthropicDecember 15, 2022…
Second, in a reinforcement learning phase, the system uses AI-generated feedback rather than only human feedback. The model compares responses and learns from preferences generated according to the constitution, a process known as reinforcement learning from AI feedback (RLAIF).[anthropic.com]anthropic.comConstitutional AI: Harmlessness from AI Feedback \ AnthropicDecember 15, 2022…
The attraction is scalability. A small group of people can spend more effort deciding what principles matter, while AI systems apply those principles across many examples. In theory, this allows oversight to grow with AI capability rather than becoming limited by the number of available human reviewers.
What a constitution adds beyond simple rules
A common misunderstanding is that Constitutional AI is just a list of restrictions. In practice, the goal is closer to creating a framework for judgement. A well-designed constitution does not only say “do not do harmful things”; it attempts to describe why certain behaviours are preferable and how competing goals should be balanced.
This distinction matters because advanced AI systems will encounter situations that developers cannot predict in advance. A long list of narrow rules may fail when a new situation appears. More general principles may allow better transfer to unfamiliar circumstances.
Anthropic’s research has explored whether models can generalise from broad principles rather than requiring a separate rule for every possible failure mode. In experiments, researchers tested whether general guidance such as acting in humanity’s best interests could influence model behaviour beyond specific examples, while also finding that more detailed principles remain useful for controlling particular harms.[anthropic.com]anthropic.comSpecific versus General Principles for Constitutional AI \ AnthropicOctober 24, 2023…
For a future where AI systems contribute to scientific research, governance, or other high-impact domains, this ability to generalise is potentially important. Human values cannot realistically be encoded as an exhaustive instruction manual. Oversight systems need ways to express wider concepts such as honesty, respect for human autonomy, caution under uncertainty, and avoiding unnecessary harm.
The difficult question: who chooses the principles?
The greatest challenge for Constitutional AI is not technical implementation alone. It is deciding what belongs in the constitution.
Human societies disagree about many important values. Different cultures, political systems, and communities may place different emphasis on freedom, equality, authority, privacy, tradition, innovation, or collective welfare. A constitution written by one organisation may reflect the assumptions and priorities of a limited group rather than humanity as a whole.
This creates a governance problem. If AI systems become powerful tools for scientific progress, economic coordination, education, and decision-making, the principles guiding them could influence billions of people. Choosing those principles is therefore not merely a technical decision; it is a question about legitimacy and public accountability.
There is also a more subtle problem: principles written in ordinary language may not translate cleanly into behaviour. Words such as “helpful”, “fair”, or “honest” can have different meanings in different contexts. An AI may appear to follow a principle while interpreting it in a way humans did not intend.
Recent research on creating and evaluating constitutions highlights this challenge. The C3AI framework, for example, examines how principles should be selected and tested, finding that principle design affects whether models actually follow the intended behaviours. The researchers argue that constitutions need systematic evaluation rather than being treated as simple lists of values.[arXiv]arxiv.orgC3AI: Crafting and Evaluating Constitutions for Constitutional AIFebruary 21, 2025…
Where Constitutional AI can fail
Constitutional AI reduces some problems of human feedback, but it introduces new ones.
One risk is that AI-generated oversight may reproduce existing biases. If the system judging outputs has its own limitations, those weaknesses can become part of the training process. The European Data Protection Supervisor notes that scalable oversight methods, including Constitutional AI, may inherit biases from the AI systems producing feedback and still require careful human involvement.[European Data Protection Supervisor]edps.europa.euOpen source on europa.eu.
Another concern is that following a constitution is not the same as understanding human values. A model might learn patterns associated with approved behaviour without reliably grasping the deeper reasons behind those principles. This is particularly important for advanced systems operating in unfamiliar situations where simple imitation may not be enough.
Researchers have also raised concerns about whether scalable oversight methods genuinely improve as AI systems become more capable. A 2025 study on scalable oversight challenges found that human evaluators can develop confirmation biases when relying on AI assistance, highlighting the need to test whether oversight methods remain reliable as capability gaps widen.[AAAI Open Access]ojs.aaai.orgOpen AccessConfirmation Bias: A Challenge for Scalable Oversight | Proceedings of the AAAI Conference on Artificial IntelligenceMarch 14, 2026…
These limitations do not make Constitutional AI useless. They show that it should be viewed as one layer in a broader alignment strategy rather than a complete solution.
Why this matters for an AI-enabled human bloom
The optimistic case for advanced AI depends on more than raw intelligence. Systems capable of accelerating science, improving healthcare, helping solve climate problems, or expanding access to education must remain directed towards outcomes that genuinely benefit people.
Constitutional AI represents one attempt to solve a central problem of that future: how humans can remain involved when AI systems become too capable for ordinary supervision. Instead of trying to inspect every decision individually, humanity may need to create systems that can reason within carefully designed boundaries while remaining open to evaluation and correction.
The approach is still experimental. No constitution can fully capture human flourishing, and no written principles can remove the need for institutions, accountability, and human judgement. But Constitutional AI contributes an important idea to the wider alignment challenge: as intelligence becomes more abundant, oversight may need to become more principled, transparent, and scalable as well.[anthropic.com]anthropic.comConstitutional AI: Harmlessness from AI Feedback \ AnthropicDecember 15, 2022…
For a future where AI helps expand human knowledge, health, creativity, and prosperity, the central question is not only whether machines become more capable. It is whether humanity can build reliable ways to guide that capability towards shared flourishing.
Amazon book picks
Further Reading
Books and field guides related to Can AI Principles Improve Human Oversight. Use these as the next step if you want deeper reading beyond the article.
The Alignment Problem
Finalist for the Los Angeles Times Book Prize A jaw-dropping exploration of everything that goes wrong when we build AI systems and the m...
Human Compatible
A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly...
Superintelligence
This profoundly ambitious and original book picks its way carefully through a vast tract of forbiddingly difficult intellectual terrain.
Artificial Intelligence
Rating: 4.5/5 from 10 Google Books ratings
Artificial intelligence: A Modern Approach, 3e,is ideal for one or two-semester, undergraduate or graduate-level courses in Artificial In...
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Selected fromrobotics poster oneBay.co.uk.
Endnotes
1.
Source: anthropic.com
Title: Constitutional AI: Harmlessness from AI Feedback \ Anthropic
Link:https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback
Source snippet
December 15, 2022...
Published: December 15, 2022
2.
Source: arxiv.org
Title: arXiv Constitutional AI: Harmlessness from AI Feedback
Link:https://arxiv.org/abs/2212.08073
3.
Source: anthropic.com
Title: Specific versus General Principles for Constitutional AI \ Anthropic
Link:https://www.anthropic.com/news/specific-versus-general-principles-for-constitutional-ai
Source snippet
October 24, 2023...
Published: October 24, 2023
4.
Source: arxiv.org
Link:https://arxiv.org/abs/2502.15861
Source snippet
C3AI: Crafting and Evaluating Constitutions for Constitutional AIFebruary 21, 2025...
Published: February 21, 2025
5.
Source: ojs.aaai.org
Title: Open Access
Link:https://ojs.aaai.org/index.php/AAAI/article/view/41124
Source snippet
Confirmation Bias: A Challenge for Scalable Oversight | Proceedings of the AAAI Conference on Artificial IntelligenceMarch 14, 2026...
Published: March 14, 2026
6.
Source: anthropic.com
Link:https://www.anthropic.com/research/collective-constitutional-ai-aligning-a-language-model-with-public-input
7.
Source: anthropic.com
Title: Claude’s Constitution
Link:https://www.anthropic.com/news/claudes-constitution?stream=top
8.
Source: anthropic.com
Title: Constitutional AI: Harmlessness from AI Feedback \ Anthropic
Link:https://www.anthropic.com/news/constitutional-ai-harmlessness-from-ai-feedback?token=91dScZBi-mOLCL2ht2JRXNaLktnej22N
9.
Source: anthropic.com
Title: Measuring Progress on Scalable Oversight for Large Language Models \ Anthropic
Link:https://www.anthropic.com/news/measuring-progress-on-scalable-oversight-for-large-language-models?f=aHR0cHM6Ly93d3cuYW50aHJvcGljLmNvbS9jb25zdGl0dXRpb25hbC5wZGY%3D
10.
Source: edps.europa.eu
Link:https://www.edps.europa.eu/data-protection/technology-monitoring/techsonar/scalable-oversight_en
11.
Source: alignmentforum.org
Title: Alignment Forum Scalable Oversight — AI Alignment Forum
Link:https://www.alignmentforum.org/w/scalable-oversight
Source snippet
Alignment ForumScalable Oversight — AI Alignment ForumApril 17, 2026...
Published: April 17, 2026
12.
Source: aisecurityandsafety.org
Title: scalable oversight
Link:https://aisecurityandsafety.org/en/guides/scalable-oversight/
13.
Source: ai-solutions.daviesmeyer.com
Title: scalable oversight
Link:https://ai-solutions.daviesmeyer.com/en/glossary/scalable-oversight
14.
Source: alignmentforum.org
Title: On scalable oversight with weak LLMs judging strong LLMs — AI Alignment Forum
Link:https://www.alignmentforum.org/posts/Qn3ZDf9WAqGuAjWQe/on-scalable-oversight-with-weak-llms-judging-strong-llms
15.
Source: alignmentforum.org
Link:https://www.alignmentforum.org/posts/hw2tGSsvLLyjFoLFS/scalable-oversight-and-weak-to-strong-generalization
16.
Source: alignmentlibrary.org
Title: Scalable Oversight
Link:https://alignmentlibrary.org/en/problems/other/scalable-oversight
Additional References
17.
Source: planspace.org
Title: Anthropic’s core values for Claude
Link:https://planspace.org/20260710-anthropic_core_values_for_claude/
Source snippet
July 10, 2026 — ANTHROPIC'S CORE VALUES FOR CLAUDE Friday July 10, 2026 * Broadly safe * Broadly ethical * Compliant with Anthropic’s gui...
Published: July 10, 2026
18.
Source: youtube.com
Title: C3AI: Crafting and Evaluating Constitutions for Constitutional AI
Link:https://www.youtube.com/watch?v=BCwNcQTs35A
Source snippet
Zachary Kenton - A Vision for Scalable Oversight [Alignment Workshop]...
19.
Source: algorithmine.com
Title: Most organiza
Link:https://algorithmine.com/research/[alignment-faking
Source snippet
Alignment Faking, Scalable Oversight, and Constitutional AI: The 2026 Landscape of AI Safety ResearchJuly 4, 2026 — PRACTICAL IMPLICATION...
Published: July 4, 2026
20.
Source: techcrunch.com
Link:https://techcrunch.com/2026/01/21/anthropic-revises-claudes-constitution-and-hints-at-chatbot-consciousness/
21.
Source: andrewmaynard.net
Link:https://andrewmaynard.net/constituting-responsibility-what-constitutional-ai-reveals-about-the-[limits
22.
Source: emmanuelgjr.github.io
Link:https://emmanuelgjr.github.io/GenAI-Security-Literature-Review/entry/llmsec-2023-00007/
23.
Source: sandgarden.com
Link:https://www.sandgarden.com/learn/constitutional-ai
24.
Source: apxml.com
Link:https://apxml.com/courses/llm-constitutional-ai-rlaif/chapter-2-constitutional-ai-theory/cai-limitations-critiques
25.
Source: huggingface.co
Link:https://huggingface.co/datasets/rl-llm-wiki/knowledge-base/commit/de612f8ac486014e302ef8a66a21f49a11fc6102
26.
Source: ox.ac.uk
Link:https://www.ox.ac.uk/news/2026-03-27-expert-comment-claude-we-trust-evaluating-new-constitution