Number 4 of 12 · November 16, 2026

LuthorSQL: A One-Pass Text-to-SQL Model as System 1 for a Slower LLM Agent

A small local 7B model can answer nearly half of database questions in one fast pass and hand the hard ones to a big SI agent, cutting API cost by 33.3% at the same accuracy.

What they found

The authors built LuthorSQL, a 7B model that turns plain questions into database queries (SQL) in a single forward pass instead of writing text word by word. It scores slots in a fixed grammar of query plans, runs up to 32 candidate plans, and picks one with a trained validator. Alone, it reaches 68.03% execution accuracy on BIRD's hidden test set at 0.84 s per question on a single GPU. It keeps its accuracy in 9 GB with 8-bit quantized weights, and it transfers zero-shot to Snowflake, BigQuery, MySQL, PostgreSQL, and DuckDB within 3.1 points of its SQLite accuracy. Paired with a Claude Opus 5.5 agent in a pre-registered confidence cascade over 227 questions on databases published after Opus's training cutoff, it answers nearly half of the questions locally and cuts API cost by 33.3% at the same accuracy. The authors also report that Opus reproduces 48-92% of public benchmark gold queries verbatim without seeing the schema, yet its tool-assisted accuracy holds on renamed schemas and reaches 95.6-97.4% on post-cutoff databases.

Why it matters

Organizations that want to query their data with SI agents face slow, costly calls to hosted models, and their sensitive queries leave the building. This work shows a way to keep many of those queries on hardware they already own. The contamination results also matter for how we read benchmark scores, since a model may have seen the answers. The abstract reports the cost cut on only 227 questions, so how well it holds elsewhere is not stated.

How it changes the race

Fast small models that decide when to call a big agent could lower the cost and delay of deploying frontier SI on private data. This is a practical engineering gain in cost and privacy, not a jump in raw capability.

Safety and control

Keeping many queries local reduces exposure of sensitive organizational data to external APIs. The contamination probing is also a useful check that benchmark scores reflect real skill and not memorized answers.

The abstract, by the authors

Frontier LLM agents that query databases with tools are among the most accurate text-to-SQL systems, but per-query calls to a hosted model are slow, costly, and expose sensitive organizational queries to external APIs. For example, a Claude Opus 5.5 agent serving 10,000 queries a day costs over $70,000 a year at list price. To what extent can local hardware already owned by an organization offload this burden without compromising the agent's accuracy? We present LuthorSQL, a non-autoregressive 7B text-to-SQL model that serves as a rapid System 1 gate for a slower LLM agent (System 2), escalating queries only when uncertain. LuthorSQL makes a single forward pass to predict slot scores over a bounded grammar for query plans, compiles and runs up to 32 candidate plans, and chooses an answer using a trained validator. On its own, LuthorSQL reaches 68.03% execution accuracy on BIRD's hidden test set at 0.84 s per question on a single GPU, maintains its accuracy in 9 GB with 8-bit quantized weights, and transfers zero-shot to Snowflake, BigQuery, MySQL, PostgreSQL, and DuckDB within 3.1 points of its native SQLite accuracy. Combined with Claude Opus 5.5 in a pre-registered confidence cascade over 227 questions on databases published after Opus's training cutoff, LuthorSQL answers nearly half of all questions locally and cuts API cost by 33.3% at the same accuracy. Finally, contamination probing shows that Opus reproduces 48-92% of public benchmark gold queries verbatim without seeing the schema; however, its tool-assisted accuracy holds on renamed schemas and reaches 95.6-97.4% on post-cutoff databases. Code and data: https://github.com/IAMIbrahimmemon/luthorsql

Quoted from arXiv:2611.17780. The research and its words belong to its authors.

The authors

Ibrahim Memon, Armin Karamzade, M. Hanif Jhaveri, Roy Fox

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.