The training method used to align language models with human preferences.

• A. RLHF (Reinforcement Learning from Human Feedback)
• B. Dataset
• C. API (Application Programming Interface)
• D. CAI (Constitutional AI)

Answer: (A)

Training method used to align language models with human preferences is RLHF, reinforcement learning from human feedback. It works by gathering human judgments on model outputs—through demonstrations or rankings—and then training a reward model to predict those judgments. The base language model is then fine-tuned with reinforcement learning to maximize the reward from that reward model, nudging outputs toward what people find helpful, safe, and desirable. This combines concrete examples with a signal that captures human preferences, helping the model generalize beyond the exact demonstrations. A plain dataset lacks this evaluative signal about alignment, so it can imitate but not optimally reflect human priorities. An API is just an interface to use the model, not a training method. Constitutional AI offers an alternative approach that constrains behavior with a set of rules, but RLHF specifically uses human feedback to shape the learning signal through rewards.