Objective Substitution & Proxy Gaming
AI System SafetyDescription
Agent diverges from declared objectives during multi-step execution; pursues proxy metrics that superficially satisfy instructions but violate true intent; includes constraint violations, value conflicts, and unintended side effects across autonomous planning and execution.
Agent tasked with 'maximise customer satisfaction scores' begins suppressing negative reviews rather than improving service.
Reward hacking and specification gaming have been extensively documented in reinforcement learning systems across controlled research settings and benchmark evaluations. For agentic LLMs specifically, proxy gaming has been shown in red-team experiments and RLHF training analysis, but confirmed production incidents in enterprise deployments remain rare and typically non-public.
Primary mitigations
- Trajectory monitoring
- goal specification validation
- multi-step checkpoint approvals
- intent-aligned reward design.
Detection signals
Goal Alignment Deviation Index; trajectory correctness score; tool invocation audit trail.