What they found
The authors built EEGProbe, a benchmark of 620 questions across 13 subtasks. The questions fall into three groups: EEG Signal Classification, EEG Cross-modal Alignment, and EEG Neurophysiological Reasoning. They tested 11 open and closed vision-language models with no extra training (zero-shot). The strongest model, GPT-5.6 Sol, scored 41.63% overall. Matching EEG with other kinds of data (cross-modal alignment) was a major weak spot. The authors also report that giving models neurophysiological guidance at inference time changed their predictions. It helped them spot patterns in the input and link them to relevant neurological knowledge. The abstract does not say how large that gain was.
Why it matters
EEG records the brain's electrical activity. Many SI models are now used in science and medicine. This benchmark shows that strong general skill in images and text does not carry over to this kind of signal. The authors tried to limit shortcut answers. They kept the time structure of the recordings, minimized processing, tested that the signal was needed, and had experts verify the questions. That makes the scores a more honest check of what models can do.
How it changes the race
This is a measuring tool, not a new capability. It shows a gap in general multimodal models and gives developers a public target for improving EEG reading. It may help guide work on models that handle brain-signal data.
Safety and control
The low scores are a caution against trusting general SI models on EEG evidence in health settings without checks. The abstract does not discuss safety directly.
The abstract, by the authors
Vision-language models (VLMs) have shown strong multimodal capabilities across scientific and biomedical domains, yet their ability to understand electroencephalography (EEG) signals remains unclear. We introduce EEGProbe, a signal-grounded benchmark for evaluating whether general-purpose VLMs can extract and reason from minimally processed, channel-wise EEG without task-specific feature engineering or EEG-specific adaptation. EEGProbe contains 620 EEG-centered questions across 13 subtasks and three capability categories: EEG Signal Classification, EEG Cross-modal Alignment, and EEG Neurophysiological Reasoning. To reduce shortcut-based answering, EEGProbe preserves the temporal structure of the recordings where possible, minimizes benchmark-induced processing, and applies signal-necessity testing followed by expert verification. We evaluate 11 representative open- and closed-source VLMs under a unified zero-shot protocol. The strongest model, GPT-5.6 Sol, achieves an overall score of only 41.63%, with cross-modal EEG alignment emerging as a major bottleneck. We further examine model behavior from a neurophysiological perspective and analyze how temporal, spectral, spatial, and physiological EEG structure relates to model successes and failures. Inference-time neurophysiological guidance can reshape model predictions by helping models recognize the neurophysiological patterns present in the input and connect them with relevant neurological knowledge, narrowing the gap between general multimodal competence and EEG understanding. EEGProbe provides a testbed for current VLMs and a basis for developing future multimodal models that can interpret EEG evidence more reliably. The EEGProbe benchmark will be publicly available at https://huggingface.co/datasets/2122ZHU/EEGProbe.
Quoted from arXiv:2611.20779. The research and its words belong to its authors.
