Research · Per Ardua

Spectral Geometry of the Forward Pass: How INLP Directions Interact with Layer Jacobians

Why domain directions receive no preferential treatment from the forward pass

AI-10 Activation Geometry DOI

Executive Summary

Paper IX showed that domain selectivity peaks at intermediate layers (7-10) and declines at both input and terminal layers. Paper XI showed that effective dimensionality bounds achievable selectivity. This paper asks why selectivity peaks where it does by measuring the amplification spectrum of INLP domain directions through the layer Jacobian.

Using finite-difference Jacobian-vector products, we compute how perturbations along INLP directions propagate through consecutive layer pairs and compare to random directions and PCA structure. Both hypotheses are cleanly rejected.

Key Findings

  • No preferential amplification: INLP/random amplification ratio is 0.99 +/- 0.05 — the forward pass does not treat domain directions differently from arbitrary directions
  • PCA-INLP alignment is near-random at intermediate layers but increases toward terminal layers where selectivity is lowest — the opposite of what would be needed for selective intervention
  • The forward pass is an isotropic amplifier: It treats INLP directions as generic directions in activation space, consistent with the concentration barrier as the sole constraint

Significance

This result eliminates the possibility that domain selectivity could be rescued by better direction-finding methods. The problem is not that INLP finds the wrong directions — it's that the forward pass does not privilege any fixed directions. The Jacobian amplification spectrum is flat across the subspaces that matter for intervention.

Key References

  • McEntire (2026) — Layer-Resolved Response Tensor: selectivity profile across layers (Paper IX)
  • McEntire (2026) — The Concentration Barrier: dimensionality bounds (Paper XI)
  • Vershynin (2018) — High-dimensional probability and concentration of measure

Download Full Paper

Access the complete research paper with detailed methodology, empirical evidence, and formal proofs.

Download PDF