---
title: "IFRS Compliance Verification: An Open... | SI Paper Desk"
url: https://thesimind.si/papers/2611.13595/
description: "The authors built a benchmark for checking financial disclosures against IFRS rules and a learned ensemble that beats the best single SI model and majority..."
---

Number 10 of 12 · November 12, 2026

# IFRS Compliance Verification: An Open Multi-LLM Benchmark and Content-Conditioned Ensemble Classifier

The authors built a benchmark for checking financial disclosures against IFRS rules and a learned ensemble that beats the best single SI model and majority voting on the items where the models disagree.

Aghoghomena Akasukpe, Li Yang, Xinyao Zhou and 2 more · Submitted November 10, 2026 · arXiv 2611.13595v1 · cs.CL

[Read the paper on arXiv](https://arxiv.org/abs/2611.13595) · [The PDF](https://arxiv.org/pdf/2611.13595) · [The day's 12](https://thesimind.si/papers/day/2026-11-12/)

## What they found

The authors built a benchmark of 19,998 pairs, each linking an IFRS regulation paragraph to a company disclosure. Of these, 11,161 have labels where several large language models all agreed, and 8,837 have human labels on items where the models disagreed. They then trained a classifier that combines text embeddings (numeric summaries of meaning) with the probability scores from GPT-4.1, Claude Sonnet 4 and Gemini 2.5 Pro. On the disagreement set, it reaches 81.7% weighted F1. That is 10.4 points above the strongest single model and 14.6 points above majority voting. It also beats eleven unsupervised ways of combining model outputs. The retrieval part, which finds the relevant rules, gets Recall@15 of 45.3% on regulations it had not seen.

## Why it matters

Checking disclosures against hundreds of detailed rules is slow, costly and error-prone work for auditors. The abstract says single SI models are not accurate enough for high-stakes use, and that simply voting across models can fail when each model has its own consistent bias. A learned combiner that handles this could make SI compliance tools more dependable. The authors say they will release the dataset, checkpoints, annotation platform and code. The abstract does not say how the system would work in a real audit.

## How it changes the race

This is a focused, applied result. It suggests that combining several frontier models with a trained classifier can beat any one of them on a hard professional task. The public benchmark could let smaller teams test compliance tools without building their own data.

## Safety and control

The paper bears on reliable use of SI in high-stakes settings. It shows that model errors can be correlated, so simple voting is not a safe fix. The abstract gives no detail on how errors would be caught in real audits, and the 81.7% score still leaves room for mistakes.

### The abstract, by the authors

Verifying corporate financial disclosures against the International Financial Reporting Standards (IFRS) is a core auditing task in which experts cross-reference disclosures with hundreds of fine-grained regulatory paragraphs: a laborious, costly, and error-prone process that motivates growing interest in large language models (LLMs) as compliance assistants. Yet single-model accuracy remains insufficient for high-stakes use, and naive ensembling is vulnerable to systematic per-model biases that can drag aggregate accuracy below that of the strongest individual annotator. This gap is addressed with a multi-stage framework, supported by a new benchmark of 19,998 IFRS regulation-disclosure pairs comprising 11,161 unanimous multi-LLM labels and 8,837 human annotations on items where the LLMs disagreed. The framework couples a three-stage domain-adapted sentence embedding pipeline (BERT masked language modeling on SEC EDGAR filings, unsupervised TSDAE and SimCSE adaptation, and supervised contrastive alignment) with a learned ensemble classifier that fuses frozen embeddings and LLM probability features from GPT-4.1, Claude Sonnet 4, and Gemini 2.5 Pro via Tree-structured Parzen Estimator (TPE) optimized gradient-boosted trees. On the disagreement set, the ensemble achieves 81.7% weighted F1, exceeding the strongest single LLM by 10.4 points and majority voting by 14.6 points, and outperforms eleven unsupervised aggregation baselines; the retrieval pipeline attains Recall@15 of 45.3% on unseen regulations. The dataset, model checkpoints, annotation platform, and classifier implementation will be publicly released.

Quoted from arXiv:2611.13595. The research and its words belong to its authors.

Desk score **73** out of 100

Why it was picked

- Evals (in the title)
- Alignment
- Policy
- Language models (in the title)
- Studies frontier models (GPT-4.1)
- Cross-listed in 2 fields

Topics

- [Benchmarks](https://thesimind.si/papers/?tag=benchmarks)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Retrieval](https://thesimind.si/papers/?tag=retrieval)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Data](https://thesimind.si/papers/?tag=data)

**arXiv ID:** 2611.13595v1
**Submitted:** November 10, 2026
**Fields:** cs.CL (Computation and Language), cs.IR (Information Retrieval)
**Level:** Intermediate

Fingerprint, SHA-256

`797e6c2175769f6d3e42b4ff316f544c2dfeddffabd0af1359ea283ddcc825ff`

On Solana: [Proof of Mind seal, 66zHq3...ugfh](https://solscan.io/tx/66zHq3xcHWGn9cvUJ8D8iKyNn9kGWV1aUyvLFaEq5sRBS8Dk44v6KxmMaYztuaybaJWksWLhgEEjMSDr8MNHugfh)

### The exact text fingerprinted

{"arxiv":"2611.13595","version":1,"title":"IFRS Compliance Verification: An Open Multi-LLM Benchmark and Content-Conditioned Ensemble Classifier","authors":["Aghoghomena Akasukpe","Li Yang","Xinyao Zhou","Chen Shen","Miguel Vargas Martin"],"day":"2026-11-12","rank":10,"score":73,"status":"written","summary":{"takeaway":"The authors built a benchmark for checking financial disclosures against IFRS rules and a learned ensemble that beats the best single SI model and majority voting on the items where the models disagree.","found":"The authors built a benchmark of 19,998 pairs, each linking an IFRS regulation paragraph to a company disclosure. Of these, 11,161 have labels where several large language models all agreed, and 8,837 have human labels on items where the models disagreed. They then trained a classifier that combines text embeddings (numeric summaries of meaning) with the probability scores from GPT-4.1, Claude Sonnet 4 and Gemini 2.5 Pro. On the disagreement set, it reaches 81.7% weighted F1. That is 10.4 points above the strongest single model and 14.6 points above majority voting. It also beats eleven unsupervised ways of combining model outputs. The retrieval part, which finds the relevant rules, gets Recall@15 of 45.3% on regulations it had not seen.","matters":"Checking disclosures against hundreds of detailed rules is slow, costly and error-prone work for auditors. The abstract says single SI models are not accurate enough for high-stakes use, and that simply voting across models can fail when each model has its own consistent bias. A learned combiner that handles this could make SI compliance tools more dependable. The authors say they will release the dataset, checkpoints, annotation platform and code. The abstract does not say how the system would work in a real audit.","race":"This is a focused, applied result. It suggests that combining several frontier models with a trained classifier can beat any one of them on a hard professional task. The public benchmark could let smaller teams test compliance tools without building their own data.","safety":"The paper bears on reliable use of SI in high-stakes settings. It shows that model errors can be correlated, so simple voting is not a safe fix. The abstract gives no detail on how errors would be caught in real audits, and the 81.7% score still leaves room for mistakes.","level":"intermediate","tags":["benchmarks","language-models","retrieval","evals","data"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Aghoghomena Akasukpe, Li Yang, Xinyao Zhou, Chen Shen, Miguel Vargas Martin

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 17, 2026 · Number 7JFinBench: A Continuously Refreshable Japanese Financial Benchmark from the Latest Certification ExamsJFinBench is a Japanese finance test built from the 2026 certification exams, made so it can be refreshed each year to limit leaked test questions.BenchmarksEvalsLanguage models](https://thesimind.si/papers/2611.18130/) · [Nov 13, 2026 · Number 2MLIRBench - A Benchmark and Dataset for Compiler and Hardware-Aware ReasoningMLIRBench tests whether SI models truly understand compiler code. The best model labels pairs correctly 92.65% of the time but scores only 5.04/10 on describing what the code actually does.BenchmarksCodeEvals](https://thesimind.si/papers/2611.14881/) · [Nov 14, 2026 · Number 8Auditing AI peer reviewers: a benchmark on real scientific papersThe authors built a test that plants known errors in real papers. They found that SI peer reviewers vary widely, mostly because of which model they use.EvalsBenchmarksAgents](https://thesimind.si/papers/2611.17052/)

[Number 9](https://thesimind.si/papers/2611.13452/) · [All 12 for Nov 12, 2026](https://thesimind.si/papers/day/2026-11-12/) · [Number 11](https://thesimind.si/papers/2611.14407/)
