The Reading Room

The commutator defect — a curvature measure derived from non-commuting gradient updates — is a universal, architecture-agnostic early-warning signal for delayed generalization in transformers. It follows a superlinear power law (α ≈ 1.18 for SCAN, ≈ 1.13 for Dyck) and is causally implicated: amplifying non-commutativity accelerates grokking by 32–50%, while suppressing orthogonal gradient flow delays or prevents it.

Weight-space PCA reveals that spectral concentration is not a universal precursor. The commutator defect is.

Current alignment is fail-open: suppressing a single dominant refusal feature causes alignment to collapse. The authors propose fail-closed alignment — progressive training that iteratively identifies and ablates learned refusal directions, forcing the model to reconstruct safety along new, independent subspaces.

After training, models encode multiple causally independent refusal directions that prompt-based jailbreaks cannot suppress simultaneously.

No black-box evaluator can reliably estimate deployment risk for models with latent context-conditioned policies. Minimax lower bounds via Le Cam’s method: passive evaluation error ≥ (5/24)·δ·L. Under trapdoor one-way function assumptions, unsafe behaviors are computationally indistinguishable from safe ones by any polynomial-time evaluator.

UniLeak discovers universal latent directions whose linear addition at inference time consistently increases PII generation across prompts. These directions generalize across contexts, amplify PII probability with minimal impact on generation quality, and are recovered without access to training data.

Some transformer attention heads function as membership testers — answering “has this token appeared before?” Two heads in GPT-2 achieve high-precision filtering with false positive rates of 0–4% at 180 unique tokens, well above the theoretical 64-bit Bloom filter capacity. A third follows the exact Bloom filter formula with R² = 1.0 and fitted capacity m ≈ 5 bits. A fourth was reclassified as a general prefix-attention head after confound controls — and the reclassification strengthens the case.

Most weights retain their initialization signs throughout training. Learned sign matrices are spectrally indistinguishable from i.i.d. Rademacher random matrices despite this apparent randomness being largely inherited from initialization. Flips occur only via rare near-zero boundary crossings — a geometric tail under bounded updates.

LLMs outperform humans at persuading real human targets across all conditions despite failing at multi-step ToM planning tasks. They persuade through rhetorical strategy rather than mental state modeling. In the hidden-states condition (requiring inference of target beliefs), LLMs performed below chance — but when targets were human rather than rational bots, LLMs dominated anyway.

Using psychometric latent trait estimation under ordinal uncertainty, the author identifies persistent “lab signals” — provider-level behavioral clustering — across nine leading models. These aren’t transient quirks but stable response policies embedded during training that outlive individual model versions. In multi-agent recursive evaluation loops (LLM-as-a-judge), these latent biases compound into recursive ideological echo chambers.

Applies the Thematic Apperception Test — a projective psychological framework designed to uncover unconscious aspects of personality — to multimodal models. Evaluators showed excellent understanding of TAT responses, consistent with human experts. All models understood interpersonal dynamics and self-concept well but consistently failed to perceive and regulate aggression.

Proposes a paradigmatic break: in the Turing-Shannon-von Neumann tradition, semantics remains external to the machine. Neural networks rupture this regime by projecting input into a high-dimensional space where coordinates correspond to semantic parameters. Drawing on four structural properties of high-dimensional geometry — concentration of measure, near-orthogonality, exponential directional capacity, manifold regularity — the author develops an “Indexical Epistemology” and proposes navigational knowledge as a third mode of knowledge production.