Entropy-Guided Attention for Private LLMs
Abstract
Presented our research on entropy-guided attention mechanisms for private large language models, exploring how attention patterns can be optimized while maintaining privacy guarantees.
Date
Jul 15, 2024 7:00 PM — 8:00 PM
Event
AI Fireside Chat
Location
Virtual Event
We presented our work on entropy-guided attention for private LLMs at the AI Fireside Chat, exploring novel attention mechanisms that enhance privacy while maintaining model performance and interpretability.
Nandan Kumar Jha
PhD student at NYU CCS
The broader goal is to guide LLM design from first principles, enabling more efficient pre-training, faster (private) inference, and robust generalization.