The Activation Geometry Program: Twelve Papers on the Mathematical Structure of Neural Network Representations

Executive Summary

Over twelve papers, we developed a systematic account of how neural network activation spaces encode, separate, and mix domain-specific and structural information. The program began with a practical question — can training compute be reduced by exploiting trajectory structure? — and arrived at a theoretical result: the terminal measurement limit, which establishes that linear interventions at the output layer cannot invert nonlinear mixing at intermediate layers.

The Arc

Papers I-II establish the measurement foundations. Training trajectories have detectable stable and chaotic regimes. Independently initialized runs converge to the same activation manifold (cosine similarity >0.999), making the geometry a reproducible object of study.

Papers III-V show this geometry supports practical applications: parameter-level model composition (93.3% cross-domain win rate), domain-structure decomposition via INLP with double dissociation (shape survival 98.7-100%), and distillation detection (AUC 1.000 on systematic attacks at 1.5B+; 0.856-0.905 when randomized attacks are included).

Paper VI proves five sufficient conditions under which stochastic resonance produces net-beneficial noise effects, unifying the SR phenomena observed across the series.

Papers VII-VIII put the forward pass under perturbation. Paper VII finds that RLHF suppresses pragmatic inference at 7B and that stochastic resonance partially rescues it. Paper VIII attempts domain-selective entropy control with noise shaped to INLP directions. The central negative result: cross-domain selectivity is fundamentally limited; the response matrix is invertible, but predicted-vs-actual correlation is approximately zero.

Papers IX-XII characterize why. Layer-resolved injection shows selectivity peaks modestly at layer 10 (mean 0.57), with no layer above 1.0. The layer Jacobian is an isotropic amplifier treating INLP directions identically to random directions. The concentration barrier theorem proves that any k-dimensional subspace captures variance fraction at most k/d_eff. Domain-specific information content is bounded at approximately 2 bits against ~15 bits of domain-agnostic perturbation.

The Collected Finding

A dissociation: classification exploits any separable signal, but intervention requires causal signal specific to the target. INLP finds directions that separate domains in representation space, but those directions are not the causal pathways through which domain-specific computation flows. The geometric footprint of domain-specific directions is small relative to the full activation space (k/d_eff), and the isotropic forward pass provides no mechanism to amplify it. The formal connection from this geometric constraint to a behavioral selectivity ceiling remains an open derivation.

The terminal measurement limit and the concentration barrier are not engineering failures — they are properties of high-dimensional computation.

Program Papers

• I. Leap+Verify: Regime-Adaptive Speculative Weight Prediction
• II. Training Once Is Enough: Activation Fingerprint Convergence
• III. Constellation-Indexed Model Composition
• IV. The Shape of the Problem: Domain-Invariant Structural Signatures
• V. Capability Manifold Surveillance: Distillation Detection
• VI. The Generative Lossy Channel: Five Sufficient Conditions for SR
• VII. GenAI Is Socially Awkward: RLHF Damages Social Cognition
• VIII. Shaped Noise Injection: The Terminal Measurement Limit
• IX. Layer-Resolved Response Tensor
• X. Spectral Geometry of the Forward Pass
• XI. The Concentration Barrier
• XII. Channel Capacity of Domain-Specific Stochastic Resonance