The Concentration Barrier: Effective Dimensionality Bounds Domain Selectivity in Neural Network Activations

Executive Summary

Paper VIII showed that shaped noise injection at terminal transformer layers cannot achieve domain-selective entropy effects. Paper IX mapped the response tensor at each layer. This paper establishes the mathematical explanation: the concentration barrier.

We prove that in an activation space with effective dimensionality d_eff (measured by participation ratio), any k unit directions capture at most a fraction k/d_eff of the total activation variance. The variance bound is a theorem, proved via Cauchy-Schwarz. Its connection to behavioral selectivity (the entropy change ratios measured in Papers VIII and IX) is indirect: it is not derived here, but it is consistent with the low selectivity those papers observe at every layer.

Key Findings

• Concentration barrier theorem: The variance fraction captured by any k directions is bounded by k/d_eff — proved analytically and verified empirically at all 28 layers, where the bound is loose by 11-557x at layers 0-18
• Effective dimensionality measurements: d_eff ranges from 4.7 to 26.0 (mean 19.2) for last-token extraction across Qwen-2.5 7B layers
• Pooling collapse: Mean-pooled d_eff collapses to 1.0 at layers 3-25, revealing previously undocumented representation anisotropy invisible to position-aware measurements
• INLP variance fraction: Increases from 1.3% at early layers to 12.5% at the terminal layer, yet never exceeds the concentration barrier bound

The Core Result

The concentration barrier bounds how much of the activation variance any fixed set of directions can capture, and so limits the domain-specific leverage of perturbations along those directions. The INLP directions genuinely separate domains in classification, but at the terminal layer (d_eff = 22.3) the 36 directions capture 12.46% of the variance, and that variance is domain-entangled rather than domain-selective. The paper is explicit that the barrier is necessary but not sufficient to explain Paper VIII's terminal measurement limit: the additional ingredient is cross-domain entanglement within the INLP subspace. The pooling duality also carries a methodological warning: mean-pooled representations, common in probing, discard the dimensional structure that determines intervention effectiveness, and make the bound vacuous.

Key References

• McEntire (2026) — Shaped Noise Injection: terminal measurement limit (Paper VIII)
• McEntire (2026) — Layer-Resolved Response Tensor: selectivity profile (Paper IX)
• McEntire (2026) — Structural Transfer: source of the 36 INLP domain directions (Paper IV)