Feed-Forward Networks

NerVE: Nonlinear Eigenspectrum Dynamics in LLM Feed-Forward Networks
Introduces eigenspectrum-based tools for tracking how nonlinearities reshape FFN representation geometry across layers and model scales.
Nandan Kumar Jha, Brandon Reagen
PDF Cite Code Project
Spectral Scaling Laws in Language Models: How Effectively Do Feed-Forward Networks Use Their Latent Space?
Studies how effectively LLM feed-forward networks use latent width through soft- and hard-spectral-rank scaling laws.
Nandan Kumar Jha, Brandon Reagen
PDF Cite ICML 2025 AIW Related code