---
title: "Correct First, Fast Second: What a Local... | SI Paper Desk"
url: https://thesimind.si/papers/2611.12079/
description: "With a small local 7B code model writing GPU kernels, getting code that works matters more than speed, and extra agent structure only helps when matched to..."
---

Number 4 of 12 · November 11, 2026

# Correct First, Fast Second: What a Local 7B Code Model Can and Cannot Do for GPU Kernel Optimization

With a small local 7B code model writing GPU kernels, getting code that works matters more than speed, and extra agent structure only helps when matched to what the model can do.

Jui-Chien Tsou and Po-Yi Wu · Submitted November 9, 2026 · arXiv 2611.12079v1 · cs.LG

[Read the paper on arXiv](https://arxiv.org/abs/2611.12079) · [The PDF](https://arxiv.org/pdf/2611.12079) · [The day's 12](https://thesimind.si/papers/day/2026-11-11/)

## What they found

The authors tested Qwen2.5-Coder-7B-Instruct on 30 KernelBench tasks, run on a V100 GPU. The model had to replace PyTorch operators with custom CUDA kernels (code that runs on the GPU). They compared four setups: zero-shot prompting, profiling-guided prompting, iterative refinement on execution feedback, and a multi-agent loop. Profiling-guided prompting did best overall, with 43% correct and 27% faster than baseline. Even correct kernels ran at only 0.97-0.99x the speed of eager PyTorch, and the real wins were fused memory-bound epilogues. The agent loop was the only method to solve any level-2 fusion problem (3/10 vs. 0/10 for the others). The first agent scored 3% correct, below zero-shot's 30%. Best-of-n sampling under a real execution check raised correctness from 3% to 33% going from n=1 to n=8.

## Why it matters

Many teams want small, local SI models to help write fast GPU code. This study suggests where such a model helps and where it does not. It also warns that adding agent steps can make results worse if the plan is too ambitious for the model. The test is small, with 30 tasks, one model and one GPU, so the findings may not carry over widely.

## How it changes the race

This is a modest, practical result. It suggests that cheap sampling checked by real execution can help a small model more than extra reasoning steps, which may lower the cost of kernel work for smaller groups. It does not show small models beating tuned vendor libraries.

### The abstract, by the authors

We study how much the interaction strategy around a fixed, locally hosted 7B-parameter code model (Qwen2.5-Coder-7B-Instruct) affects its ability to replace PyTorch operators with custom CUDA kernels. On 30 KernelBench tasks (levels 1-3, first 10 each, evaluated on a V100), we compare four regimes of increasing structure: zero-shot prompting, profiling-guided prompting, iterative refinement on execution feedback, and a multi-agent loop (analysis, retrieval, generation, evaluation, feedback triage) in which the same model fills four roles around a non-LLM evaluator. Three findings stand out. (1) Correctness, not speed, is the binding constraint: even correct generated kernels average 0.97-0.99x the speed of eager PyTorch, because a 7B model cannot out-write cuBLAS/cuDNN on dense GEMM and convolution; the only real wins are fused memory-bound epilogues. (2) The best strategy depends on difficulty: profiling-guided prompting is strongest overall (43% correct, 27% faster than baseline), but the agent loop is the only method that solves any level-2 fusion problem (3/10 vs. 0/10 for all others). (3) Naive agentic scaffolding is actively harmful: our first agent scored 3% correct, below zero-shot's 30%, and the decisive repairs were matching the agent's ambition to the model's competence and spending compute on best-of-n sampling under a real execution oracle (3% to 33% correct from n=1 to n=8), not on more reasoning steps.

Quoted from arXiv:2611.12079. The research and its words belong to its authors.

Desk score **83** out of 100

Why it was picked

- Compute (in the title)
- Safety
- Agents
- Reasoning
- Evals
- Studies frontier models (Qwen2.5)
- Multi-agent

Topics

- [Code](https://thesimind.si/papers/?tag=code)
- [Agents](https://thesimind.si/papers/?tag=agents)
- [Efficiency](https://thesimind.si/papers/?tag=efficiency)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Multi-agent systems](https://thesimind.si/papers/?tag=multi-agent)

**arXiv ID:** 2611.12079v1
**Submitted:** November 9, 2026
**Fields:** cs.LG (Machine Learning)
**Level:** Intermediate

Fingerprint, SHA-256

`9de52746df1e2e96721385e41f51bcf78718b4b7d697d49c2ce26570e73651c8`

On Solana: [Proof of Mind seal, ipj4pc...z6te](https://solscan.io/tx/ipj4pcQsmFjnBmF5dFmUSA15Eex18bkxb83FjQuvHnS4DCz6yJoWcESM8bwWT8cMMh3f4GmDjbebLEAsauqz6te)

### The exact text fingerprinted

{"arxiv":"2611.12079","version":1,"title":"Correct First, Fast Second: What a Local 7B Code Model Can and Cannot Do for GPU Kernel Optimization","authors":["Jui-Chien Tsou","Po-Yi Wu"],"day":"2026-11-11","rank":4,"score":83,"status":"written","summary":{"takeaway":"With a small local 7B code model writing GPU kernels, getting code that works matters more than speed, and extra agent structure only helps when matched to what the model can do.","found":"The authors tested Qwen2.5-Coder-7B-Instruct on 30 KernelBench tasks, run on a V100 GPU. The model had to replace PyTorch operators with custom CUDA kernels (code that runs on the GPU). They compared four setups: zero-shot prompting, profiling-guided prompting, iterative refinement on execution feedback, and a multi-agent loop. Profiling-guided prompting did best overall, with 43% correct and 27% faster than baseline. Even correct kernels ran at only 0.97-0.99x the speed of eager PyTorch, and the real wins were fused memory-bound epilogues. The agent loop was the only method to solve any level-2 fusion problem (3/10 vs. 0/10 for the others). The first agent scored 3% correct, below zero-shot's 30%. Best-of-n sampling under a real execution check raised correctness from 3% to 33% going from n=1 to n=8.","matters":"Many teams want small, local SI models to help write fast GPU code. This study suggests where such a model helps and where it does not. It also warns that adding agent steps can make results worse if the plan is too ambitious for the model. The test is small, with 30 tasks, one model and one GPU, so the findings may not carry over widely.","race":"This is a modest, practical result. It suggests that cheap sampling checked by real execution can help a small model more than extra reasoning steps, which may lower the cost of kernel work for smaller groups. It does not show small models beating tuned vendor libraries.","safety":"","level":"intermediate","tags":["code","agents","efficiency","evals","language-models","multi-agent"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Jui-Chien Tsou, Po-Yi Wu

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 10, 2026 · Number 3PACE: Keeping the Right Tokens in the KV Cache Is Not Enough for Long-Horizon LLM AgentsKeeping the needed tokens in a SI agent's memory cache is not enough. PACE also renumbers their positions so they stay reachable, and it holds accuracy in sessions of up to 2M tokens.AgentsEfficiencyMemory](https://thesimind.si/papers/2611.10240/) · [Nov 11, 2026 · Number 10AMSchem: Automated Schematic Visualization of AMS Circuits via Verifiable Agentic GenerationAMSchem uses an SI agent with strict automatic checks to turn analog circuit netlists into readable schematics, reaching 85.0% validity versus 32.5% for direct generation on a 40-circuit subset.AgentsCodeEvals](https://thesimind.si/papers/2611.12333/) · [Nov 11, 2026 · Number 6Steering Emergent Behavior in Multi-Agent LLM Systems via Information NetworksIn groups of SI agents, showing agents what their peers did does not always help. When agents are tempted to free-ride, more information can make group results worse.Multi-agent systemsSafetyAgents](https://thesimind.si/papers/2611.11714/)

[Number 3](https://thesimind.si/papers/2611.11796/) · [All 12 for Nov 11, 2026](https://thesimind.si/papers/day/2026-11-11/) · [Number 5](https://thesimind.si/papers/2611.12075/)