---
title: "Large Language Models as Digital Twins... | SI Paper Desk"
url: https://thesimind.si/papers/2611.16586/
description: "The authors found that today's language models do not reliably answer a personality questionnaire the way a specific person would, even when given that..."
---

Number 10 of 12 · November 14, 2026

# Large Language Models as Digital Twins: a Distribution-Aware Questionnaire-Based Evaluation

The authors found that today's language models do not reliably answer a personality questionnaire the way a specific person would, even when given that person's social-media posts.

Francesco Di Cursi, Chiara Boldrini, Andrea Passarella and 1 more · Submitted November 12, 2026 · arXiv 2611.16586v1 · cs.AI

[Read the paper on arXiv](https://arxiv.org/abs/2611.16586) · [The PDF](https://arxiv.org/pdf/2611.16586) · [The day's 12](https://thesimind.si/papers/day/2026-11-14/)

## What they found

The authors gave SI language models a user's social-media posts and asked them to fill in the BFI-2-60 Big Five personality questionnaire as that user. They tested GPT-4, Gemma, Llama 3.1 and Mistral on the MyPersonality dataset. User text did not consistently improve results compared with answers made without any user text or with values based on general population tendencies. Models did better for people near the middle of a trait range. For people with low or high scores, they often stayed near central values, or did well at one end and poorly at the other. When given only an explicit Big Five profile, GPT-4 most clearly turned it into consistent answers, especially for Extraversion and Neuroticism. Adding more text did not steadily cut error: open-source models fluctuated more, while GPT-4 stayed fairly stable.

## Why it matters

Many people want to use SI models as digital twins, meaning stand-ins that act like a real person, for surveys or social research. This paper tests that idea with a careful check against real questionnaire scores. It suggests that models may look plausible while mostly giving average answers. That matters for anyone who would rely on such stand-ins to represent individuals, especially people at the extremes.

## How it changes the race

This does not change how frontier SI is built. It does set a more careful bar for claims about simulating individual people, and it shows that gains from adding more context are not guaranteed.

## Safety and control

The results warn against trusting SI models to represent specific individuals, for example in research or decisions about people. The abstract does not discuss misuse or control directly.

### The abstract, by the authors

This paper investigates whether Large Language Models can answer a personality questionnaire as a specific individual would. Given a user's social-media posts, the model completes the BFI-2-60 Big Five questionnaire as that user. The predicted responses are converted into trait scores and compared with the user's actual scores. We evaluate GPT-4, Gemma, Llama 3.1, and Mistral on the MyPersonality dataset. To determine whether predictions reflect information from the user's writing, we compare them with answers produced without user text and with values derived from general population tendencies. We measure prediction error across all users and separately for individuals with low, intermediate, and high trait scores. We also test whether an LLM can express an explicit personality profile when given only the target Big Five scores, without user text, and examine how predictions change as more text is provided. The results show that user text does not consistently improve performance. Models perform better near the middle of the trait distribution, while at the lower and upper ends they often remain close to central values or perform well at one end but poorly at the other. When given an explicit profile, GPT-4 most clearly translates trait information into consistent questionnaire responses, particularly for Extraversion and Neuroticism. Increasing context does not systematically reduce error: open-source models fluctuate more, whereas GPT-4 remains comparatively stable. Overall, the evaluated models do not consistently reproduce the questionnaire behavior of specific individuals.

Quoted from arXiv:2611.16586. The research and its words belong to its authors.

Desk score **73** out of 100

Why it was picked

- Frontier lab: FAIR
- Evals (in the title)
- Language models (in the title)
- Studies frontier models (GPT-4)
- Cross-listed in 2 fields

Topics

- [Evals](https://thesimind.si/papers/?tag=evals)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Society](https://thesimind.si/papers/?tag=society)
- [Benchmarks](https://thesimind.si/papers/?tag=benchmarks)
- [Data](https://thesimind.si/papers/?tag=data)

**arXiv ID:** 2611.16586v1
**Submitted:** November 12, 2026
**Fields:** cs.AI (Artificial Intelligence), cs.CY (Computers and Society)
**Level:** Intermediate
**Comments:** This work was partially funded by the European Union within the National Programme 'Research, Innovation and Competitiveness' 2021-2027, project SINTESI (Nr. MI_0000962). F. Di Cursi's, C. Boldrini's, A. Passarella's, and M. Conti's work was also partly funded by the PNRR - M4C2 - Investimento...

Fingerprint, SHA-256

`e75f317cebc420f47cf803e90ad16af63e0f7bc5334141592b434f04c2939192`

On Solana: [Proof of Mind seal, 4ut3vv...Y9uP](https://solscan.io/tx/4ut3vv2vTeUs5fAbknPuGnt1i1jJGh18Ry7QYkUAkvvDWCKQk8wrmGNkp4C5Loc937KnsizybKH71BB2w6QAY9uP)

### The exact text fingerprinted

{"arxiv":"2611.16586","version":1,"title":"Large Language Models as Digital Twins: a Distribution-Aware Questionnaire-Based Evaluation","authors":["Francesco Di Cursi","Chiara Boldrini","Andrea Passarella","Marco Conti"],"day":"2026-11-14","rank":10,"score":73,"status":"written","summary":{"takeaway":"The authors found that today's language models do not reliably answer a personality questionnaire the way a specific person would, even when given that person's social-media posts.","found":"The authors gave SI language models a user's social-media posts and asked them to fill in the BFI-2-60 Big Five personality questionnaire as that user. They tested GPT-4, Gemma, Llama 3.1 and Mistral on the MyPersonality dataset. User text did not consistently improve results compared with answers made without any user text or with values based on general population tendencies. Models did better for people near the middle of a trait range. For people with low or high scores, they often stayed near central values, or did well at one end and poorly at the other. When given only an explicit Big Five profile, GPT-4 most clearly turned it into consistent answers, especially for Extraversion and Neuroticism. Adding more text did not steadily cut error: open-source models fluctuated more, while GPT-4 stayed fairly stable.","matters":"Many people want to use SI models as digital twins, meaning stand-ins that act like a real person, for surveys or social research. This paper tests that idea with a careful check against real questionnaire scores. It suggests that models may look plausible while mostly giving average answers. That matters for anyone who would rely on such stand-ins to represent individuals, especially people at the extremes.","race":"This does not change how frontier SI is built. It does set a more careful bar for claims about simulating individual people, and it shows that gains from adding more context are not guaranteed.","safety":"The results warn against trusting SI models to represent specific individuals, for example in research or decisions about people. The abstract does not discuss misuse or control directly.","level":"intermediate","tags":["evals","language-models","society","benchmarks","data"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Francesco Di Cursi, Chiara Boldrini, Andrea Passarella, Marco Conti

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 13, 2026 · Number 2MLIRBench - A Benchmark and Dataset for Compiler and Hardware-Aware ReasoningMLIRBench tests whether SI models truly understand compiler code. The best model labels pairs correctly 92.65% of the time but scores only 5.04/10 on describing what the code actually does.BenchmarksCodeEvals](https://thesimind.si/papers/2611.14881/) · [Nov 17, 2026 · Number 7JFinBench: A Continuously Refreshable Japanese Financial Benchmark from the Latest Certification ExamsJFinBench is a Japanese finance test built from the 2026 certification exams, made so it can be refreshed each year to limit leaked test questions.BenchmarksEvalsLanguage models](https://thesimind.si/papers/2611.18130/) · [Nov 12, 2026 · Number 10IFRS Compliance Verification: An Open Multi-LLM Benchmark and Content-Conditioned Ensemble ClassifierThe authors built a benchmark for checking financial disclosures against IFRS rules and a learned ensemble that beats the best single SI model and majority voting on the items where the models disagree.BenchmarksLanguage modelsRetrieval](https://thesimind.si/papers/2611.13595/)

[Number 9](https://thesimind.si/papers/2611.16879/) · [All 12 for Nov 14, 2026](https://thesimind.si/papers/day/2026-11-14/) · [Number 11](https://thesimind.si/papers/2611.16816/)
