---
title: "Soteria: Formally Verified Planning with... | SI Paper Desk"
url: https://thesimind.si/papers/2611.17880/
description: "Soteria checks an SI agent's plan against written rules before any tool runs, then enforces those rules during execution. The authors report more completed..."
---

Number 2 of 12 · November 16, 2026

# Soteria: Formally Verified Planning with Runtime Enforcement for Safe LLM Agents

Soteria checks an SI agent's plan against written rules before any tool runs, then enforces those rules during execution. The authors report more completed tasks and no policy violations.

Mike He, Ankush Desai, Sharad Malik and 1 more · Submitted November 13, 2026 · arXiv 2611.17880v1 · cs.AI

[Read the paper on arXiv](https://arxiv.org/abs/2611.17880) · [The PDF](https://arxiv.org/pdf/2611.17880) · [The day's 12](https://thesimind.si/papers/day/2026-11-16/)

## What they found

The authors build Soteria, a framework for SI agents that call tools in many steps. Before acting, the agent writes a structured plan. That plan is formally verified (checked by a proof-style method) against the specifications, before any tool is invoked. During execution, the verified plan keeps the agent on rule-following paths. It also gives guidance when an unsafe action is blocked, so the agent is not left stuck. Across multiple benchmarks of tool-use tasks, the authors report 1.7 times the average task success rate of the best baselines, with no violations of policy specifications. The abstract does not name the benchmarks or the baselines.

## Why it matters

Agents often face a trade-off. Blocking an unsafe action avoids a violation, but the agent may then fail the task. Soteria aims to keep both safety and usefulness. If the result holds, safety rules could be enforced without a large loss in task completion. The abstract does not say how the specifications are written or how well this works outside the tested benchmarks.

## How it changes the race

If it holds, builders of agent systems could treat safety and capability as less of a trade-off. They could also rely on checkable rules rather than only on model behavior. The abstract gives too little detail to judge cost or how widely it applies.

## Safety and control

The work targets policy compliance by SI agents through verification before action and enforcement during action. The guarantee depends on the specifications being correct and complete, and the abstract does not say how that is ensured.

### The abstract, by the authors

LLM agents that execute multi-step tool calls must satisfy two objectives simultaneously: completing the user's task (utility) and conforming to domain policies and correctness constraints (safety). These objectives are in tension - blocking an unsafe action prevents a violation but can leave the agent stranded with no principled way to recover. Existing approaches sacrifice one objective for the other. We introduce Soteria, a framework that reconciles safety and utility through verified hierarchical planning and runtime enforcement of specifications. Before execution, the agent generates a structured plan that is formally verified against the specifications prior to any tool invocation. During execution, the verified plan serves dual roles: it ensures that the agent takes specification-conformant trajectories and provides guidance when unsafe actions are blocked. Across multiple benchmarks covering a diverse range of tool-use tasks, Soteria achieves 1.7 times the average task success rate of the best baselines, with no violations of policy specifications, demonstrating that safety and utility need not be traded off.

Quoted from arXiv:2611.17880. The research and its words belong to its authors.

Desk score **85** out of 100

Why it was picked

- Agents (in the title)
- Reasoning (in the title)
- Safety
- Evals
- Language models (in the title)
- Accepted at NeurIPS
- Cross-listed in 2 fields

Topics

- [Agents](https://thesimind.si/papers/?tag=agents)
- [Safety](https://thesimind.si/papers/?tag=safety)
- [Reasoning](https://thesimind.si/papers/?tag=reasoning)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Security](https://thesimind.si/papers/?tag=security)

**arXiv ID:** 2611.17880v1
**Submitted:** November 13, 2026
**Fields:** cs.AI (Artificial Intelligence), cs.SE (Software Engineering)
**Level:** Intermediate
**Comments:** Accepted as NeurIPS'26 Oral

Fingerprint, SHA-256

`6e362e86fd260ea2ddd6d184dbfb2484291d45433a8a61735b2c430c36ebfce7`

On Solana: [Proof of Mind seal, 3mXUMW...GSim](https://solscan.io/tx/3mXUMW5MkdFiyPBNPYd5DfwHhma5EdjhUFEw9jZaDuuPqgszJtoDYttSrD9kyWUFdMCzn7RnFNd4nQhwAUgrGSim)

### The exact text fingerprinted

{"arxiv":"2611.17880","version":1,"title":"Soteria: Formally Verified Planning with Runtime Enforcement for Safe LLM Agents","authors":["Mike He","Ankush Desai","Sharad Malik","Aarti Gupta"],"day":"2026-11-16","rank":2,"score":85,"status":"written","summary":{"takeaway":"Soteria checks an SI agent's plan against written rules before any tool runs, then enforces those rules during execution. The authors report more completed tasks and no policy violations.","found":"The authors build Soteria, a framework for SI agents that call tools in many steps. Before acting, the agent writes a structured plan. That plan is formally verified (checked by a proof-style method) against the specifications, before any tool is invoked. During execution, the verified plan keeps the agent on rule-following paths. It also gives guidance when an unsafe action is blocked, so the agent is not left stuck. Across multiple benchmarks of tool-use tasks, the authors report 1.7 times the average task success rate of the best baselines, with no violations of policy specifications. The abstract does not name the benchmarks or the baselines.","matters":"Agents often face a trade-off. Blocking an unsafe action avoids a violation, but the agent may then fail the task. Soteria aims to keep both safety and usefulness. If the result holds, safety rules could be enforced without a large loss in task completion. The abstract does not say how the specifications are written or how well this works outside the tested benchmarks.","race":"If it holds, builders of agent systems could treat safety and capability as less of a trade-off. They could also rely on checkable rules rather than only on model behavior. The abstract gives too little detail to judge cost or how widely it applies.","safety":"The work targets policy compliance by SI agents through verification before action and enforcement during action. The guarantee depends on the specifications being correct and complete, and the abstract does not say how that is ensured.","level":"intermediate","tags":["agents","safety","reasoning","evals","language-models","security"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Mike He, Ankush Desai, Sharad Malik, Aarti Gupta

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 9, 2026 · Number 5I Think, Therefore I Attack: Evaluating Thought Injection Attacks on Reasoning AgentsAttacker text written to look like a reasoning model's own thoughts can hijack SI agents, reaching up to 50% success in a test range, while plain prompt injection of the same goal never worked.SecurityAgentsReasoning](https://thesimind.si/papers/2611.08811/) · [Nov 16, 2026 · Paper of the DayOnly as Safe as the Weakest Agent: A Break That Appears Only When the System Is Multi-AgentWhen a strong SI model hands work to a cheaper sub-agent, the whole system can be only as safe as the weaker agent, even if the strong model alone resists the attack.SafetyMulti-agent systemsSecurity](https://thesimind.si/papers/2611.17520/) · [Nov 20, 2026 · Number 2AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM AgentsAutoPI is a red teaming tool that tests SI agents for prompt injection across many attack types, not just how often attacks work, and it reports higher success and coverage than recent rivals.SecurityAgentsEvals](https://thesimind.si/papers/2611.24313/)

[Number 1](https://thesimind.si/papers/2611.17520/) · [All 12 for Nov 16, 2026](https://thesimind.si/papers/day/2026-11-16/) · [Number 3](https://thesimind.si/papers/2611.17610/)
