---
title: "Parameter-Efficient Conciseness... | SI Paper Desk"
url: https://thesimind.si/papers/2611.17128/
description: "The authors add one simple term to ordinary fine-tuning so a small medical SI model gives much shorter answers, cutting length by 72.8% while accuracy rose..."
---

Number 5 of 12 · November 14, 2026

# Parameter-Efficient Conciseness Adaptation: Differentiable Length Control for Concise Clinical Reasoning in Smaller Language Models

The authors add one simple term to ordinary fine-tuning so a small medical SI model gives much shorter answers, cutting length by 72.8% while accuracy rose from 40.8% to 44.0%.

Tanzeel Abbas, Ahmad Kamran Malik, Nafees Qamar and 1 more · Submitted November 12, 2026 · arXiv 2611.17128v1 · cs.CL

[Read the paper on arXiv](https://arxiv.org/abs/2611.17128) · [The PDF](https://arxiv.org/pdf/2611.17128) · [The day's 12](https://thesimind.si/papers/day/2026-11-14/)

## What they found

The authors present PECA, a way to make small language models answer medical questions more briefly. Length is a count of tokens (word pieces), which cannot be tuned directly by training. So PECA instead raises the chance of the model emitting its end-of-sequence signal (the "stop" marker) once an answer passes a set length budget. This runs inside normal QLoRA fine-tuning, a low-memory way to adapt a model. It needs no preference data, no reward model, no helper model and only one commodity GPU. On a 1.7-billion-parameter model trained on the medical-o1-reasoning-SFT corpus, mean response length fell by 72.8%, and multiple-choice diagnostic accuracy rose from 40.8% to 44.0%. Changing the strength of the conciseness term traces a trade-off curve between accuracy and length.

## Why it matters

Long answers are hard to use in clinical settings where time is short, even when the reasoning is sound. Many existing ways to control length need extra data, extra models or large proprietary teachers, which cuts against the low cost that makes small models appealing. A method that works within standard fine-tuning on one GPU could make concise, specialised medical SI tools easier for small teams to build. The abstract reports one model and one training corpus, and the accuracy is still low in absolute terms.

## How it changes the race

This is an incremental, practical result. It suggests length control can be cheap and open to anyone with one GPU, but the abstract does not show it on larger models or other domains.

## Safety and control

The abstract tests only multiple-choice accuracy and length. It does not say whether shorter answers drop caveats or safety-relevant details, so clinical use would need careful checks. The sweep over conciseness strength is meant to help balance brevity against diagnostic completeness.

### The abstract, by the authors

Smaller language models (SLMs) make it feasible to deploy clinical natural language processing where large proprietary models are too costly, and parameter-efficient fine-tuning allows them to be specialised on commodity hardware. One obstacle persists after training: on medical tasks these models answer at far greater length than the clinical question requires. In clinical decision support and point-of-care use, where time is short, such verbosity is hard to act on even when the underlying reasoning is sound. Existing length-control methods depend on preference optimisation, reward models, or proprietary teachers that undercut the accessibility making SLMs attractive. We introduce Parameter-Efficient Conciseness Adaptation (PECA), which adds a single differentiable conciseness term to the QLoRA supervised fine-tuning objective. Because the token count is discrete and non-differentiable, PECA instead raises the end-of-sequence probability once generation exceeds a token budget, a smooth signal optimised during ordinary training, with no preference data, no auxiliary model, and a single commodity GPU. On a 1.7-billion-parameter model fine-tuned on the medical-o1-reasoning-SFT corpus, PECA reduces mean response length by 72.8% while improving multiple-choice diagnostic accuracy from 40.8% to 44.0%. A sweep over the conciseness strength traces an interpretable accuracy-to-length frontier for balancing brevity and diagnostic completeness.

Quoted from arXiv:2611.17128. The research and its words belong to its authors.

Desk score **81** out of 100

Why it was picked

- Reasoning (in the title)
- Alignment
- Interpretability
- Compute
- Language models (in the title)
- Studies frontier models (o1)
- Cross-listed in 2 fields

Topics

- [Efficiency](https://thesimind.si/papers/?tag=efficiency)
- [Health](https://thesimind.si/papers/?tag=health)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Training](https://thesimind.si/papers/?tag=training)
- [Reasoning](https://thesimind.si/papers/?tag=reasoning)
- [Evals](https://thesimind.si/papers/?tag=evals)

**arXiv ID:** 2611.17128v1
**Submitted:** November 12, 2026
**Fields:** cs.CL (Computation and Language), cs.AI (Artificial Intelligence)
**Level:** Intermediate

Fingerprint, SHA-256

`4584d58de1c2855d2c36eb9a9b503db921493e6fbbd7b80ab7ce2da32120ec1b`

On Solana: [Proof of Mind seal, 4ut3vv...Y9uP](https://solscan.io/tx/4ut3vv2vTeUs5fAbknPuGnt1i1jJGh18Ry7QYkUAkvvDWCKQk8wrmGNkp4C5Loc937KnsizybKH71BB2w6QAY9uP)

### The exact text fingerprinted

{"arxiv":"2611.17128","version":1,"title":"Parameter-Efficient Conciseness Adaptation: Differentiable Length Control for Concise Clinical Reasoning in Smaller Language Models","authors":["Tanzeel Abbas","Ahmad Kamran Malik","Nafees Qamar","Sana Liaqat"],"day":"2026-11-14","rank":5,"score":81,"status":"written","summary":{"takeaway":"The authors add one simple term to ordinary fine-tuning so a small medical SI model gives much shorter answers, cutting length by 72.8% while accuracy rose from 40.8% to 44.0%.","found":"The authors present PECA, a way to make small language models answer medical questions more briefly. Length is a count of tokens (word pieces), which cannot be tuned directly by training. So PECA instead raises the chance of the model emitting its end-of-sequence signal (the \"stop\" marker) once an answer passes a set length budget. This runs inside normal QLoRA fine-tuning, a low-memory way to adapt a model. It needs no preference data, no reward model, no helper model and only one commodity GPU. On a 1.7-billion-parameter model trained on the medical-o1-reasoning-SFT corpus, mean response length fell by 72.8%, and multiple-choice diagnostic accuracy rose from 40.8% to 44.0%. Changing the strength of the conciseness term traces a trade-off curve between accuracy and length.","matters":"Long answers are hard to use in clinical settings where time is short, even when the reasoning is sound. Many existing ways to control length need extra data, extra models or large proprietary teachers, which cuts against the low cost that makes small models appealing. A method that works within standard fine-tuning on one GPU could make concise, specialised medical SI tools easier for small teams to build. The abstract reports one model and one training corpus, and the accuracy is still low in absolute terms.","race":"This is an incremental, practical result. It suggests length control can be cheap and open to anyone with one GPU, but the abstract does not show it on larger models or other domains.","safety":"The abstract tests only multiple-choice accuracy and length. It does not say whether shorter answers drop caveats or safety-relevant details, so clinical use would need careful checks. The sweep over conciseness strength is meant to help balance brevity against diagnostic completeness.","level":"intermediate","tags":["efficiency","health","language-models","training","reasoning","evals"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Tanzeel Abbas, Ahmad Kamran Malik, Nafees Qamar, Sana Liaqat

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 15, 2026 · Number 10Reasoning Supervision and Capability Retention in LoRA Domain AdaptationIn fine-tuning Qwen3-8B on financial tasks, training on labels alone beat adding teacher reasoning traces for in-domain results, while general abilities stayed close to the original model.TrainingLanguage modelsEvals](https://thesimind.si/papers/2611.17072/) · [Nov 16, 2026 · Number 6Alignment with Reflect-Reasoning: Learning How To Follow PrinciplesReflect-Reasoning trains models to reason about whether they are following a given set of principles, and the authors report it beats system prompting across five models.AlignmentReasoningLanguage models](https://thesimind.si/papers/2611.17398/) · [Nov 20, 2026 · Number 9QDA: Closed-Loop LLM Quantization Design with AgentsQDA uses an SI agent to design the compression recipe for each language model, and the abstract says it beats the strongest public recipes on four reasoning models.EfficiencyAgentsLanguage models](https://thesimind.si/papers/2611.23388/)

[Number 4](https://thesimind.si/papers/2611.16485/) · [All 12 for Nov 14, 2026](https://thesimind.si/papers/day/2026-11-14/) · [Number 6](https://thesimind.si/papers/2611.16591/)
