---
title: "Runtime Verification of Multiple Natural... | SI Paper Desk"
url: https://thesimind.si/papers/2611.23074/
description: "The authors built VFM, a small verifier that checks an SI agent's output against many written rules at once and says which are met, broken, or not relevant."
---

Number 2 of 12 · November 19, 2026

# Runtime Verification of Multiple Natural Language Criteria for Agent Governance

The authors built VFM, a small verifier that checks an SI agent's output against many written rules at once and says which are met, broken, or not relevant.

Silviu Pitis, Parand A. Alamdari, Jessica Tang and 2 more · Submitted November 17, 2026 · arXiv 2611.23074v1 · cs.CL

[Read the paper on arXiv](https://arxiv.org/abs/2611.23074) · [The PDF](https://arxiv.org/pdf/2611.23074) · [The day's 12](https://thesimind.si/papers/day/2026-11-19/)

## What they found

The authors introduce the VFM, a verifier that reads the shared context and the agent's output once, then scores N written criteria in parallel. Each criterion gets its own verdict: satisfied, violated, or inapplicable. It is pretrained on CriteriaBank, a corpus of 394K examples covering over 321K distinct criterion strings. A finetuned VFM-4B outperforms GPT-5.4 on a synthetic governance benchmark. On DynaBench it beats DynaGuard-4B at spotting failures across a whole trace. The abstract says its per-criterion verdicts point to the violated rule and can pass hard cases to a larger chain-of-thought judge. On a single H100, VFM-4B checks 1,000 criteria against a 4K-token context in under 2s using under 11 GB of memory.

## Why it matters

SI agents are often governed by hundreds of rules written in plain language. Checking each rule one by one with a large model is slow and costly. A fast checker that runs while the agent works could make rule-based oversight practical. The abstract also says CriteriaBank pretraining helps transfer to new domains, especially when labeled data is scarce, and matches direct finetuning when data is plentiful.

## How it changes the race

If the results hold, runtime rule checking becomes cheap enough for smaller teams to deploy on one GPU. This is a gain in oversight tooling rather than in raw SI capability. The abstract's benchmarks are one synthetic set and DynaBench, so wider performance is not shown.

## Safety and control

The work targets governance and control of SI agents by checking outputs against written rules as they run. The abstract gives no evidence on how well it resists an agent trying to evade the checks, and it suggests escalating hard cases to a larger judge.

### The abstract, by the authors

AI agents are governed by hundreds of natural language criteria, yet existing verifiers cannot efficiently evaluate an agent's output against a large set of such criteria and report which are satisfied, violated, or inapplicable. We introduce the VFM, a tree-attention verifier that encodes the shared context and target text, and scores N natural language criteria in parallel, returning independent per-criterion ternary verdicts. The VFM is pretrained on CriteriaBank, a corpus of 394K (context, target, criterion, verdict) tuples spanning over 321K distinct criterion strings. A finetuned VFM-4B outperforms GPT-5.4 on a synthetic governance benchmark. On the DynaBench multi-criterion benchmark, it outperforms DynaGuard-4B on trace-level failure detection, and its per-criterion verdicts localize the violated rule and support focused escalation to a larger chain-of-thought judge. We show that CriteriaBank pretraining improves cross-domain transfer, particularly in the low-data regime, while matching a strong direct-finetuning control at full data. On a single H100, VFM-4B evaluates 1,000 criteria against a 4K-token context in under 2s at <11 GB VRAM, making it suitable for runtime verification.

Quoted from arXiv:2611.23074. The research and its words belong to its authors.

Desk score **83** out of 100

Why it was picked

- Agents (in the title)
- Policy (in the title)
- Reasoning
- Evals
- Studies frontier models (GPT-5.4)
- Accepted at NeurIPS
- Cross-listed in 3 fields

Topics

- [Safety](https://thesimind.si/papers/?tag=safety)
- [Agents](https://thesimind.si/papers/?tag=agents)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Efficiency](https://thesimind.si/papers/?tag=efficiency)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Alignment](https://thesimind.si/papers/?tag=alignment)

**arXiv ID:** 2611.23074v1
**Submitted:** November 17, 2026
**Fields:** cs.CL (Computation and Language), cs.AI (Artificial Intelligence), cs.LG (Machine Learning)
**Level:** Intermediate
**Comments:** Accepted at NeurIPS 2026

Fingerprint, SHA-256

`278df47b4b2fef9103617e9bda0d1b340ad97950642161051d3ab15d7c8f5690`

On Solana: [Proof of Mind seal, 3rBYfi...QvEH](https://solscan.io/tx/3rBYfiFP3PxcUbtJzCwpcVq5MngquU5iAtJFeiP7PFVgnsuoTjwwJPjNDYgKxQzUnVq5keQT2KEgX46YTdHqQvEH)

### The exact text fingerprinted

{"arxiv":"2611.23074","version":1,"title":"Runtime Verification of Multiple Natural Language Criteria for Agent Governance","authors":["Silviu Pitis","Parand A. Alamdari","Jessica Tang","Toryn Q. Klassen","Sheila A. McIlraith"],"day":"2026-11-19","rank":2,"score":83,"status":"written","summary":{"takeaway":"The authors built VFM, a small verifier that checks an SI agent's output against many written rules at once and says which are met, broken, or not relevant.","found":"The authors introduce the VFM, a verifier that reads the shared context and the agent's output once, then scores N written criteria in parallel. Each criterion gets its own verdict: satisfied, violated, or inapplicable. It is pretrained on CriteriaBank, a corpus of 394K examples covering over 321K distinct criterion strings. A finetuned VFM-4B outperforms GPT-5.4 on a synthetic governance benchmark. On DynaBench it beats DynaGuard-4B at spotting failures across a whole trace. The abstract says its per-criterion verdicts point to the violated rule and can pass hard cases to a larger chain-of-thought judge. On a single H100, VFM-4B checks 1,000 criteria against a 4K-token context in under 2s using under 11 GB of memory.","matters":"SI agents are often governed by hundreds of rules written in plain language. Checking each rule one by one with a large model is slow and costly. A fast checker that runs while the agent works could make rule-based oversight practical. The abstract also says CriteriaBank pretraining helps transfer to new domains, especially when labeled data is scarce, and matches direct finetuning when data is plentiful.","race":"If the results hold, runtime rule checking becomes cheap enough for smaller teams to deploy on one GPU. This is a gain in oversight tooling rather than in raw SI capability. The abstract's benchmarks are one synthetic set and DynaBench, so wider performance is not shown.","safety":"The work targets governance and control of SI agents by checking outputs against written rules as they run. The abstract gives no evidence on how well it resists an agent trying to evade the checks, and it suggests escalating hard cases to a larger judge.","level":"intermediate","tags":["safety","agents","evals","efficiency","language-models","alignment"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Silviu Pitis, Parand A. Alamdari, Jessica Tang, Toryn Q. Klassen, Sheila A. McIlraith

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 20, 2026 · Number 6Learning to Coordinate: LLM-Guided Workflow Evolution for Token-Efficient Red TeamingL2C is a system that teams several SI agents to red team language models, and it finds more safety failures than EvoSynth while using fewer online tokens.SafetyMulti-agent systemsAgents](https://thesimind.si/papers/2611.24221/) · [Nov 13, 2026 · Number 5When Safety Becomes an Attack Surface: Manipulating Abstention in Tool-Using AI AgentsFake risk warnings hidden in low-trust content can trick SI agents into refusing legitimate tasks, and caution-first prompting makes this worse, so agents need to check who is warning them.SecurityAgentsSafety](https://thesimind.si/papers/2611.14864/) · [Nov 11, 2026 · Paper of the DayMitigating Agentic Deception via Evaluation-Side Partially Observable GamesThe authors treat SI agent deception as a game between an agent and a monitor that checks the whole interaction, and report fewer deceptive behaviors and better task success in tests on small models.SafetyAlignmentAgents](https://thesimind.si/papers/2611.12835/)

[Number 1](https://thesimind.si/papers/2611.22511/) · [All 12 for Nov 19, 2026](https://thesimind.si/papers/day/2026-11-19/) · [Number 3](https://thesimind.si/papers/2611.22971/)
