Algebraic Dead Directions and the Critical Line: An Exploratory Assessment

Abstract

This essay examines the structural correspondence between algebraic dead directions in LayerNorm transformers and spectral properties of the Riemann zeta function.

Algebraic Dead Directions in Neural Networks

The source paper (Shirodkar & Narayanan, 2026) studies dead directions in the parameter space of pretrained transformers—directions along which the Fisher information metric degenerates. For LayerNorm transformers, they identify a remarkable algebraic structure: the inverse-scale direction γ⁻¹ is an exact kernel of the post-final-norm centred activation covariance A_ℓ, independent of the input distribution. This direction is computable from the scale parameter γ alone, without forward passes. The paper introduces the Schur ratio R(h) := λ_min(A_ℓ)/(A_ℓ)_u,u to measure how "deep" the dead direction becomes during training, observing a ~10³× deepening from random initialization to trained checkpoints.

The Spectral Analogy to Zeta

The Riemann Hypothesis concerns the zeros of ζ(s) lying on the critical line Re(s) = 1/2. The Hilbert-Pólya conjecture posits that these zeros correspond to eigenvalues of a self-adjoint operator. This suggests a vague parallel: the "dead direction" (zero eigenvalue of Fisher information) might analogize to the "critical line" (locus of non-trivial zeros), and the Schur ratio might analogize to a measure of spectral rigidity or zero repulsion. Both fields also invoke random matrix theory—Marchenko-Pastur for random network initialization and GUE for zeta zero statistics.

Assessment of the Correspondence

We rate this analogy a suggestive metaphor only. While both domains study spectra, the specific algebraic predictivity of the LayerNorm dead direction (deriving from the mean-subtraction projector and KFAC factorization) has no known counterpart in the zeta function's analytic theory. The Marchenko-Pastur versus GUE appearance is superficial: they are different ensembles, and the "trained" network's deviation from random initialization lacks a clear zeta-theoretic parallel. The essay proposes computational experiments to test the general random-matrix universality, but emphasizes that these do not validate the specific structural bridge.

Why the Bridge Fails

The failure is fundamental: the source structure is a finite-dimensional parametric algebraic identity tied to neural network architecture, while the Riemann zeta function is an infinite-dimensional analytic object. The "dead direction" is a vector in parameter space; the critical line is a line in the complex plane. Without a statistical model interpretation of ζ(s) possessing a Fisher metric that degenerates precisely on Re(s) = 1/2, the analogy remains metaphorical and does not suggest productive computational approaches to RH.

This essay was produced by an automated research pipeline and has not been peer reviewed; conjectures herein are unproven.