What they found
The authors describe a way to read per-layer internal activations (the residual stream) from a live GPU process. They use a debugger that does not modify the process. It needs no prior knowledge of the model's architecture parameters or weights. The method is a six-stage process with platform-specific adapters, and it uses one of two "clocks" to find tensor allocations in video memory. They tested it on 33 open-source decoder-only models across nine architecture classes, and the recovered bytes were byte-exact. As a use case, linear probes trained on standard framework activations were applied to the recovered states. They separated refusal from compliance with an AUROC of 0.88 on held-out prompts.
Why it matters
Most memory forensics for machine learning looks at static things, like model weights or prompt logs. This paper looks at the live computation inside a running service that cannot be restarted or changed. That could give investigators a new kind of evidence about how a deployed SI system behaved on a specific request. The abstract reports tests on open-source models only. It does not say how well this works on other setups.
How it changes the race
This is mostly a tool for investigators and operators, not a gain in capability. If it holds up, it could make it easier to check what deployed SI systems did, without the cooperation of the service or any changes to it.
Safety and control
The work gives a way to observe a deployed model's internal states from outside, which could help audits and incident analysis. The same access to hidden states could also raise privacy or misuse concerns. The abstract does not discuss this.
The abstract, by the authors
Existing memory forensics for machine learning systems typically targets static artifacts, such as model weights or prompt logs, rather than the transient computational states of a live inference process. This gap limits an examiner's ability to analyze the specific behavior of a running large language model (LLM) service that cannot be restarted or modified. We present a method for acquiring per-layer residual-stream activations from a live GPU process using a non-modifying debugger, without requiring prior knowledge of the model's architecture parameters or weights. The approach employs a general six-stage process with platform-specific adapters, utilizing either a PyTorch allocator clock or a kernel-launch clock to identify and read tensor allocations from video memory. We validate this acquisition on 33 open-source decoder-only models across nine architecture classes, demonstrating that the recovered bytes are byte-exact and usable for downstream analysis. As a use case, we apply linear probes trained on standard framework activations to the recovered states, successfully distinguishing refusal from compliance with an AUROC of 0.88 on held-out prompts. This work establishes that live, observational access to intermediate hidden states is feasible on named inference platforms, providing a new source of evidence for examining the operational behavior of deployed AI systems.
Quoted from arXiv:2611.18272. The research and its words belong to its authors.
