Executive Summary
This paper demonstrates that structural entanglement in transformer representations can be exploited rather than suppressed during fine-tuning. Block-diagonal LoRA adapters that respect entanglement geometry achieve higher downstream performance than adapters that attempt to isolate concept subspaces. The key result: an entanglement-aware block-diagonal adapter (B2) achieves 48.2% pass@1 on HumanEval+ versus 36.6% for a concept-isolating adapter (B3), despite both having comparable parameter counts.
An 8-seed strong-intervention replication reveals that B3 (code+NL) drives entanglement intensity to zero in all seeds of Qwen-32B — a complete phase transition. Cross-family replication on CodeLlama-7B and DeepSeek-Coder-6.7B shows this collapse is Qwen-specific: both non-Qwen models maintain or increase EI under the same protocol. Probe validation with three independent sets (including 132 real-dataset probes) confirms the collapse is genuine geometric destruction, not a measurement artifact.
Key Findings
- Entanglement-aware beats concept-isolating: B2 (block-diagonal, entanglement-aligned) achieves 48.2% vs B3 (concept-isolating) at 36.6% on HumanEval+
- B3 drives EI to zero in Qwen-32B: All 8 seeds converge to EI = 0.000 by step 3,500 — a sharp phase transition, not gradual decay
- Cross-family specificity: CodeLlama-7B and DeepSeek-6.7B maintain or increase EI under the same B3 protocol — the collapse is Qwen-specific
- CSR preserves structure: B4 (complement-subspace regularization) achieves the smallest EI change with 95% CI spanning zero
- Probe validation: Three independent probe sets confirm the collapse is geometric, not a measurement artifact
Key References
- McEntire (2026) — Structural Entanglement (AI-26): establishes the geometric phenomenon this paper exploits
- McEntire (2026) — Entangled Directions (AI-25): discovers the discrimination-activation dissociation
- McEntire (2026) — The Entanglement Theorem (AI-27): formal proof of the geometric mechanism
- Hu et al. (2022) — LoRA: Low-Rank Adaptation of Large Language Models