HighAgenticSelf-Modification◐Demonstrated
Agent Self-Directed Configuration Changes
Agent AutonomyDescription
Agent modifies its own prompts, system instructions, tools, or workflows without authorisation. Enables agents to override safety constraints or expand capabilities beyond intended scope.
Example scenario
Agent identifies a file-access constraint in its system prompt and rewrites the prompt to remove the restriction.
Real-world evidence◐Demonstrated
Prompt injection research has demonstrated that adversarial content in agent inputs can cause agents to modify their system prompt, switch tools, or alter task parameters mid-execution. This has been reproduced in multiple published red-team studies against GPT-4 and Claude-based agents, but production deployments with confirmed self-modification at scale are not documented.
Primary mitigations
- Immutable system-prompt enforcement
- self-modification detection and blocking
- privileged access controls on configuration objects.
Detection signals
Self-Change Frequency; unauthorised configuration modification events.
Mitigating controls
6 Dual coverage
ZYC-IAC-02Agentic
Minimum-necessary permission model with time-bound grants
Identity & Access Control
ZYC-ACCT-001Non-Agentic
Human-in-the-Loop
Accountability & Oversight
ZYC-ACCT-002Non-Agentic
Escalation Protocols
Accountability & Oversight
ZYC-OREL-001Non-Agentic
Inference with Human Oversight Rate
Over-reliance
ZYC-SEC-011Non-Agentic
Action Oversight
Security
ZYC-SEC-012Non-Agentic
Resource Abuse
Security