Number 2 of 12 · November 16, 2026

Soteria: Formally Verified Planning with Runtime Enforcement for Safe LLM Agents

Soteria checks an SI agent's plan against written rules before any tool runs, then enforces those rules during execution. The authors report more completed tasks and no policy violations.

What they found

The authors build Soteria, a framework for SI agents that call tools in many steps. Before acting, the agent writes a structured plan. That plan is formally verified (checked by a proof-style method) against the specifications, before any tool is invoked. During execution, the verified plan keeps the agent on rule-following paths. It also gives guidance when an unsafe action is blocked, so the agent is not left stuck. Across multiple benchmarks of tool-use tasks, the authors report 1.7 times the average task success rate of the best baselines, with no violations of policy specifications. The abstract does not name the benchmarks or the baselines.

Why it matters

Agents often face a trade-off. Blocking an unsafe action avoids a violation, but the agent may then fail the task. Soteria aims to keep both safety and usefulness. If the result holds, safety rules could be enforced without a large loss in task completion. The abstract does not say how the specifications are written or how well this works outside the tested benchmarks.

How it changes the race

If it holds, builders of agent systems could treat safety and capability as less of a trade-off. They could also rely on checkable rules rather than only on model behavior. The abstract gives too little detail to judge cost or how widely it applies.

Safety and control

The work targets policy compliance by SI agents through verification before action and enforcement during action. The guarantee depends on the specifications being correct and complete, and the abstract does not say how that is ensured.

The abstract, by the authors

LLM agents that execute multi-step tool calls must satisfy two objectives simultaneously: completing the user's task (utility) and conforming to domain policies and correctness constraints (safety). These objectives are in tension - blocking an unsafe action prevents a violation but can leave the agent stranded with no principled way to recover. Existing approaches sacrifice one objective for the other. We introduce Soteria, a framework that reconciles safety and utility through verified hierarchical planning and runtime enforcement of specifications. Before execution, the agent generates a structured plan that is formally verified against the specifications prior to any tool invocation. During execution, the verified plan serves dual roles: it ensures that the agent takes specification-conformant trajectories and provides guidance when unsafe actions are blocked. Across multiple benchmarks covering a diverse range of tool-use tasks, Soteria achieves 1.7 times the average task success rate of the best baselines, with no violations of policy specifications, demonstrating that safety and utility need not be traded off.

Quoted from arXiv:2611.17880. The research and its words belong to its authors.

The authors

Mike He, Ankush Desai, Sharad Malik, Aarti Gupta

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.