Representation Learning

Shows that optimizers can determine how much nominal FFN width becomes realized spectral capacity, even when validation loss is matched.

Introduces eigenspectrum-based tools for tracking how nonlinearities reshape FFN representation geometry across layers and model scales.

Studies how effectively LLM feed-forward networks use latent width through soft- and hard-spectral-rank scaling laws.

Studies entropy regularization for self-attention as a soft inductive bias in large language models.