---
title: "EEGProbe: Do Vision-Language Models... | SI Paper Desk"
url: https://thesimind.si/papers/2611.20779/
description: "EEGProbe tests whether general vision-language models can read raw brain-wave signals, and the best of 11 models scored only 41.63%, so they do not yet..."
---

Number 10 of 12 · November 18, 2026

# EEGProbe: Do Vision-Language Models Really Understand EEG? A Benchmark for Signal-Grounded EEG Signals Evaluation

EEGProbe tests whether general vision-language models can read raw brain-wave signals, and the best of 11 models scored only 41.63%, so they do not yet understand EEG well.

Haozhe Zhu, Dan Yuan, Chenghao Gu and 3 more · Submitted November 16, 2026 · arXiv 2611.20779v1 · cs.AI

[Read the paper on arXiv](https://arxiv.org/abs/2611.20779) · [The PDF](https://arxiv.org/pdf/2611.20779) · [The day's 12](https://thesimind.si/papers/day/2026-11-18/)

## What they found

The authors built EEGProbe, a benchmark of 620 questions across 13 subtasks. The questions fall into three groups: EEG Signal Classification, EEG Cross-modal Alignment, and EEG Neurophysiological Reasoning. They tested 11 open and closed vision-language models with no extra training (zero-shot). The strongest model, GPT-5.6 Sol, scored 41.63% overall. Matching EEG with other kinds of data (cross-modal alignment) was a major weak spot. The authors also report that giving models neurophysiological guidance at inference time changed their predictions. It helped them spot patterns in the input and link them to relevant neurological knowledge. The abstract does not say how large that gain was.

## Why it matters

EEG records the brain's electrical activity. Many SI models are now used in science and medicine. This benchmark shows that strong general skill in images and text does not carry over to this kind of signal. The authors tried to limit shortcut answers. They kept the time structure of the recordings, minimized processing, tested that the signal was needed, and had experts verify the questions. That makes the scores a more honest check of what models can do.

## How it changes the race

This is a measuring tool, not a new capability. It shows a gap in general multimodal models and gives developers a public target for improving EEG reading. It may help guide work on models that handle brain-signal data.

## Safety and control

The low scores are a caution against trusting general SI models on EEG evidence in health settings without checks. The abstract does not discuss safety directly.

### The abstract, by the authors

Vision-language models (VLMs) have shown strong multimodal capabilities across scientific and biomedical domains, yet their ability to understand electroencephalography (EEG) signals remains unclear. We introduce EEGProbe, a signal-grounded benchmark for evaluating whether general-purpose VLMs can extract and reason from minimally processed, channel-wise EEG without task-specific feature engineering or EEG-specific adaptation. EEGProbe contains 620 EEG-centered questions across 13 subtasks and three capability categories: EEG Signal Classification, EEG Cross-modal Alignment, and EEG Neurophysiological Reasoning. To reduce shortcut-based answering, EEGProbe preserves the temporal structure of the recordings where possible, minimizes benchmark-induced processing, and applies signal-necessity testing followed by expert verification. We evaluate 11 representative open- and closed-source VLMs under a unified zero-shot protocol. The strongest model, GPT-5.6 Sol, achieves an overall score of only 41.63%, with cross-modal EEG alignment emerging as a major bottleneck. We further examine model behavior from a neurophysiological perspective and analyze how temporal, spectral, spatial, and physiological EEG structure relates to model successes and failures. Inference-time neurophysiological guidance can reshape model predictions by helping models recognize the neurophysiological patterns present in the input and connect them with relevant neurological knowledge, narrowing the gap between general multimodal competence and EEG understanding. EEGProbe provides a testbed for current VLMs and a basis for developing future multimodal models that can interpret EEG evidence more reliably. The EEGProbe benchmark will be publicly available at https://huggingface.co/datasets/2122ZHU/EEGProbe.

Quoted from arXiv:2611.20779. The research and its words belong to its authors.

Desk score **77** out of 100

Why it was picked

- Evals (in the title)
- Alignment
- Reasoning
- Language models (in the title)
- Studies frontier models (GPT-5.6)
- Cross-listed in 3 fields
- Code or data released

Topics

- [Benchmarks](https://thesimind.si/papers/?tag=benchmarks)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Multimodal](https://thesimind.si/papers/?tag=multimodal)
- [Health](https://thesimind.si/papers/?tag=health)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Science](https://thesimind.si/papers/?tag=science)

**arXiv ID:** 2611.20779v1
**Submitted:** November 16, 2026
**Fields:** cs.AI (Artificial Intelligence), cs.CV (Computer Vision), cs.HC (Human-Computer Interaction)
**Level:** Intermediate
**Comments:** 21 pages, 12 figures

Fingerprint, SHA-256

`69d8191c0a5efabd5aa3e2d3e894956defcfc76bd1f482a852e4fe2b56c8407c`

On Solana: [Proof of Mind seal, 5WzZUk...UPMg](https://solscan.io/tx/5WzZUksJeCL3fYgfzcQ6DSdVxipu97PAmY5Y9M59gCMXbJtnyBVsqaoPSDHWrP4AiZAh7k4iVso5ts7WvxdNUPMg)

### The exact text fingerprinted

{"arxiv":"2611.20779","version":1,"title":"EEGProbe: Do Vision-Language Models Really Understand EEG? A Benchmark for Signal-Grounded EEG Signals Evaluation","authors":["Haozhe Zhu","Dan Yuan","Chenghao Gu","Feike Xu","Yue Wen","Shin'ichi Konomi"],"day":"2026-11-18","rank":10,"score":77,"status":"written","summary":{"takeaway":"EEGProbe tests whether general vision-language models can read raw brain-wave signals, and the best of 11 models scored only 41.63%, so they do not yet understand EEG well.","found":"The authors built EEGProbe, a benchmark of 620 questions across 13 subtasks. The questions fall into three groups: EEG Signal Classification, EEG Cross-modal Alignment, and EEG Neurophysiological Reasoning. They tested 11 open and closed vision-language models with no extra training (zero-shot). The strongest model, GPT-5.6 Sol, scored 41.63% overall. Matching EEG with other kinds of data (cross-modal alignment) was a major weak spot. The authors also report that giving models neurophysiological guidance at inference time changed their predictions. It helped them spot patterns in the input and link them to relevant neurological knowledge. The abstract does not say how large that gain was.","matters":"EEG records the brain's electrical activity. Many SI models are now used in science and medicine. This benchmark shows that strong general skill in images and text does not carry over to this kind of signal. The authors tried to limit shortcut answers. They kept the time structure of the recordings, minimized processing, tested that the signal was needed, and had experts verify the questions. That makes the scores a more honest check of what models can do.","race":"This is a measuring tool, not a new capability. It shows a gap in general multimodal models and gives developers a public target for improving EEG reading. It may help guide work on models that handle brain-signal data.","safety":"The low scores are a caution against trusting general SI models on EEG evidence in health settings without checks. The abstract does not discuss safety directly.","level":"intermediate","tags":["benchmarks","evals","multimodal","health","language-models","science"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Haozhe Zhu, Dan Yuan, Chenghao Gu, Feike Xu, Yue Wen, Shin'ichi Konomi

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 18, 2026 · Number 11Razavi-Bench: Tracking AI Progress in Analog Circuit ReasoningOn a 50-question analog circuit benchmark, the top SI model score rose from 28.83% to 98.33% in 30 months, so the test no longer separates the best models.BenchmarksEvalsMultimodal](https://thesimind.si/papers/2611.20935/) · [Nov 11, 2026 · Number 2Benchmarking language models and agents for scientific reasoning in evolving materials laboratoriesMatScope is a new test showing that SI models handle given calculations and physical explanations well, but struggle to tell close material states apart and to reason with numbers.BenchmarksScienceEvals](https://thesimind.si/papers/2611.12014/) · [Nov 12, 2026 · Number 3Beyond Proxy Metrics: MLLMs as Human Surrogates for Explainable AI EvaluationThe authors test whether multimodal SI models can stand in for human participants when judging explanations of SI systems, and find that all models trend positively, with OpenAI and Qwen models aligning best.InterpretabilityEvalsMultimodal](https://thesimind.si/papers/2611.13926/)

[Number 9](https://thesimind.si/papers/2611.21541/) · [All 12 for Nov 18, 2026](https://thesimind.si/papers/day/2026-11-18/) · [Number 11](https://thesimind.si/papers/2611.20935/)
