Alignment via Generalization

Shai Shalev-Shwartz, Amnon Shashua

AI systems can become better at achieving the goals we give them without becoming better at doing what we actually want. Studying this alignment problem rigorously requires a framework connecting measurable performance to intended behavior. We propose Aligned-PAC learning, whose central idea is to reduce alignment to generalization: does success on the requirements used for training extend to requirements the system has not encountered? Rather than express our entire intention through a single training objective, we represent its different aspects through a collection of specific requirements. This perspective already yields strong theoretical results. We show how independent evaluation can provide rigorous evidence of alignment across the represented requirements, and establish how much evaluation is needed. We also develop an optimization method that, under explicit assumptions, combines learned models to satisfy the training requirements. The framework does not solve alignment in full: choosing requirements that faithfully capture our intentions, and understanding when learning will generalize to unseen requirements, remain open challenges. It provides a foundation for making…