---
title: "SceneAgent: A Multi-Agent Framework for... | SI Paper Desk"
url: https://thesimind.si/papers/2611.17821/
description: "SceneAgent separates a scene's saved state, each agent's short-term reasoning, and reusable skills, so a game scene generator can move to a new engine..."
---

Number 3 of 12 · November 17, 2026

# SceneAgent: A Multi-Agent Framework for Portable Game Scene Generation

SceneAgent separates a scene's saved state, each agent's short-term reasoning, and reusable skills, so a game scene generator can move to a new engine without starting over.

Xingyi Li, Jiahao Yang, Yiqin Zhu and 3 more · Submitted November 13, 2026 · arXiv 2611.17821v1 · cs.CV

[Read the paper on arXiv](https://arxiv.org/abs/2611.17821) · [The PDF](https://arxiv.org/pdf/2611.17821) · [The day's 12](https://thesimind.si/papers/day/2026-11-17/)

## What they found

The authors built SceneAgent, a system where several SI agents build 3D game scenes. A shared record called SceneContext tracks the scene as it grows. Small backend adapters handle engine details such as coordinates, assets and export. Fresh Builder sessions read this record through structured handoffs, so they do not need the full history of earlier work. Human-approved builds are turned into reusable skills for layout and content. On GameSceneBench, a new set of 69 indoor and outdoor prompts, SceneAgent got human tie-adjusted preference scores of 78.2-89.4% against five baselines. Ablations show the skills and the multi-agent setup mainly improve prompt alignment and scene completeness, at the cost of extra runtime. A second engine reused the same protocol and skill library, with only the backend integration parts replaced.

## Why it matters

Game scene tools are often tied to one engine, so switching can mean rebuilding planning, tools and stored experience. This work suggests a way to keep those parts reusable. The abstract gives no detail on how many engines were tested beyond a second one, or how big the preference gaps are for each baseline.

## How it changes the race

This is a practical engineering step for SI-driven content creation. If it holds, smaller teams could reuse one scene-building agent across engines instead of rebuilding it each time.

### The abstract, by the authors

Moving a scene generator to a new game engine requires more than exporting its final output: planning, construction tools, and stored experience may also depend on the original backend. We present SceneAgent, a multi-agent framework for reusing the construction process and its experience across backends. A canonical SceneContext maintains the evolving world, while backend adapters isolate coordinate conventions, asset bindings, and export. Structured handoffs let fresh Builder sessions query this persistent state without inheriting the full construction transcript. Across generations, human-approved traces are distilled into asset-ID-free Layout skills and semantic Content skills that can be retrieved and recombined. We introduce GameSceneBench, comprising 69 indoor and outdoor prompts. Under a common asset and evaluation protocol, SceneAgent obtains human tie-adjusted preference scores of 78.2-89.4% against five baselines. Ablations show that reusable skills and the multi-agent protocol primarily improve prompt alignment and scene completeness, with additional runtime. Deployment in a second engine preserves the shared construction protocol and skill library while replacing backend integration components. Together, these results support separating persistent scene state, task-local reasoning, and reusable experience for portable scene construction.

Quoted from arXiv:2611.17821. The research and its words belong to its authors.

Desk score **78** out of 100

Why it was picked

- Agents (in the title)
- Alignment
- Reasoning
- Evals
- Multi-agent (in the title)
- Cross-listed in 2 fields

Topics

- [Agents](https://thesimind.si/papers/?tag=agents)
- [Multi-agent systems](https://thesimind.si/papers/?tag=multi-agent)
- [Vision](https://thesimind.si/papers/?tag=vision)
- [Benchmarks](https://thesimind.si/papers/?tag=benchmarks)
- [Memory](https://thesimind.si/papers/?tag=memory)
- [Code](https://thesimind.si/papers/?tag=code)

**arXiv ID:** 2611.17821v1
**Submitted:** November 13, 2026
**Fields:** cs.CV (Computer Vision), cs.AI (Artificial Intelligence)
**Level:** Intermediate
**Comments:** Project page: [https://xingyi-li.github.io/sceneagent/](https://xingyi-li.github.io/sceneagent/)

Fingerprint, SHA-256

`dc6a156807ed9215e9999283abf43770e1387997e806a1381ee3a7b51365cc01`

On Solana: [Proof of Mind seal, 2HNJWs...cNY9](https://solscan.io/tx/2HNJWsLa5Tp3F7wQaNr1CPR71N1aoGNptA7fgCafC5m3BwbmKwhNGTXyaSfyEa9jqug6kM55u5VmDvbSrRepcNY9)

### The exact text fingerprinted

{"arxiv":"2611.17821","version":1,"title":"SceneAgent: A Multi-Agent Framework for Portable Game Scene Generation","authors":["Xingyi Li","Jiahao Yang","Yiqin Zhu","Jingwen Ye","Yang Sun","Weidong Zhang"],"day":"2026-11-17","rank":3,"score":78,"status":"written","summary":{"takeaway":"SceneAgent separates a scene's saved state, each agent's short-term reasoning, and reusable skills, so a game scene generator can move to a new engine without starting over.","found":"The authors built SceneAgent, a system where several SI agents build 3D game scenes. A shared record called SceneContext tracks the scene as it grows. Small backend adapters handle engine details such as coordinates, assets and export. Fresh Builder sessions read this record through structured handoffs, so they do not need the full history of earlier work. Human-approved builds are turned into reusable skills for layout and content. On GameSceneBench, a new set of 69 indoor and outdoor prompts, SceneAgent got human tie-adjusted preference scores of 78.2-89.4% against five baselines. Ablations show the skills and the multi-agent setup mainly improve prompt alignment and scene completeness, at the cost of extra runtime. A second engine reused the same protocol and skill library, with only the backend integration parts replaced.","matters":"Game scene tools are often tied to one engine, so switching can mean rebuilding planning, tools and stored experience. This work suggests a way to keep those parts reusable. The abstract gives no detail on how many engines were tested beyond a second one, or how big the preference gaps are for each baseline.","race":"This is a practical engineering step for SI-driven content creation. If it holds, smaller teams could reuse one scene-building agent across engines instead of rebuilding it each time.","safety":"","level":"intermediate","tags":["agents","multi-agent","vision","benchmarks","memory","code"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Xingyi Li, Jiahao Yang, Yiqin Zhu, Jingwen Ye, Yang Sun, Weidong Zhang

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 20, 2026 · Number 8Semantic Context Handoff with On-Demand Evidence for Long-Horizon AgentsSCH keeps a short summary of what an SI agent currently thinks, stores the raw evidence separately, and brings old evidence back when needed. This improved results and cut cost.AgentsMemoryEfficiency](https://thesimind.si/papers/2611.24357/) · [Nov 13, 2026 · Number 4Coding Agent Memory Post-Training II: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via On-Policy DistillationTeaching SI agents to use plain files as memory, through a teacher that already knows file tools, gave 2.97 times the gain of the strongest baseline on the authors' new test suite.AgentsMemoryReinforcement learning](https://thesimind.si/papers/2611.14009/) · [Nov 18, 2026 · Paper of the DayGRACE: Parallel Best-of-N Multi-Agent Analysis of Closed-Form Questions over Dataframes with Guardrail Selection and Adaptive RetryGRACE runs several SI agents in parallel on spreadsheet-style questions, judges their executed work instead of voting, and can reject all answers and retry, lifting answer match from 39% to 71% on one benchmark.AgentsMulti-agent systemsCode](https://thesimind.si/papers/2611.20983/)

[Number 2](https://thesimind.si/papers/2611.18397/) · [All 12 for Nov 17, 2026](https://thesimind.si/papers/day/2026-11-17/) · [Number 4](https://thesimind.si/papers/2611.18494/)
