Spectral Geometry

Shows that optimizers can determine how much nominal FFN width becomes realized spectral capacity, even when validation loss is matched.

Introduces eigenspectrum-based tools for tracking how nonlinearities reshape FFN representation geometry across layers and model scales.

Studies how effectively LLM feed-forward networks use latent width through soft- and hard-spectral-rank scaling laws.

Uses random-matrix tools to analyze how multi-head latent attention evolves during training, revealing capacity bottlenecks and representation-geometry shifts.