What they found
The authors built a pipeline called TRACE that screens draft laws. It points to provisions an expert would criticise and names the defect in each. They made an open benchmark from published conclusions of Ukraine's National Agency on Corruption Prevention (NAZK): 132 draft laws with 474 model-extracted findings. On 100 draft laws, the pipeline matches 0.28 of findings in its top ten flags with the local Gemma 4 31B model and 0.30 with Sonnet 5.5. Across all flags it matches 0.43 to 0.46, though this was not blind; the frozen blind baseline scores 0.21. Most misses happen when the system found the right provision but named a different risk. None of the 32 legal-collision findings was matched. On 21 held-out acts, the pipeline beat a single long prompt under the strict measure but not the lenient one, and Opus 5.5 added nothing over Sonnet.
Why it matters
Checking laws for corruption risk takes scarce expert time, so a screening tool could help reviewers decide where to look. The results here show a useful but limited tool, not a replacement for experts. The scores come from an LLM judge and model-extracted findings, with no human expert validation, so they should be read with care.
How it changes the race
This is a narrow, applied result. It suggests that a structured pipeline can run on a local open model with scores close to a stronger hosted model, but the abstract shows only modest gains over a simple prompt.
Safety and control
The design keeps external human oversight, and the authors say outright that the scores lack human expert validation. The complete miss on legal-collision findings shows where human reviewers should not trust the tool.
The abstract, by the authors
We present an AI assistant for automating anti-corruption expertise of draft legislation based on the TRACE architecture, designed to screen bills for corruption risks: pinpoint provisions an expert would criticise and name the regulatory defect in each. The pipeline combines a deterministic parsing core, model-guided single-provision qualification, rule-based factor assignment, and external human oversight. To evaluate the assistant, we constructed an open benchmark from the published conclusions of Ukraine's National Agency on Corruption Prevention (NAZK), comprising 132 draft laws with 474 model-extracted findings. A system flag agrees with a finding when it targets a compatible provision and an LLM judge rates the risk as the same (strict) or partly the same (lenient). On 100 draft laws, the pipeline matches 0.28 of the findings within its top ten flags using the local Gemma 4 31B model, 0.30 with Sonnet 5.5, and 0.43-0.46 across all flags (non-blind; the frozen blind baseline achieves 0.21, 95% CI 0.16-0.26). Across all runs, most misses occur on provisions that the system located but qualified as a different risk, and none of the 32 legal-collision findings was matched. On 21 acts held out from tuning, the pipeline outperforms a monolithic prompt containing the full methodology under the strict measure, though not under the lenient one, and Opus 5.5 adds nothing over Sonnet. The reported scores reflect agreement with model-extracted findings rated by an LLM judge whose two access routes agree on 73 of 74 labels, without human expert validation.
Quoted from arXiv:2611.14244. The research and its words belong to its authors.
