Large Language Models

Shows that optimizers can determine how much nominal FFN width becomes realized spectral capacity, even when validation loss is matched.

Introduces eigenspectrum-based tools for tracking how nonlinearities reshape FFN representation geometry across layers and model scales.

Studies how effectively LLM feed-forward networks use latent width through soft- and hard-spectral-rank scaling laws.

Uses random-matrix tools to analyze how multi-head latent attention evolves during training, revealing capacity bottlenecks and representation-geometry shifts.

Develops entropy-guided attention and hierarchical entropy regularization for efficient private LLM inference with reduced nonlinearities.

Studies entropy regularization for self-attention as a soft inductive bias in large language models.

Panel discussion on the infrastructure and systems behind modern AI.

Fireside chat on entropy-guided attention mechanisms for efficient private LLM inference.