Nandan Kumar Jha

PhD student at NYU CCS

New York University

About me

I am a PhD candidate at the Center for Cybersecurity, New York University (NYU), advised by Prof. Brandon Reagen. I’m broadly interested in cryptographically secure privacy-preserving machine learning (PPML) and work at the intersection of deep learning and applied cryptography (homomorphic encryption and multiparty computation) as a part of DPRIVE projects. My primary research focuses on the design and optimization of neural networks for efficient processing of encrypted inputs in Private Inference.

In the first half of my PhD, I worked on nonlinear-efficient CNNs and developed ReLU-optimization techniques (DeepReDuce, ICML'21), and proposed methods for redesigning existing CNNs (DeepReShape, TMLR'24) for end-to-end private inference efficiency.

Currently, my research focuses on the privacy and security of large language models (LLMs). Specifically, I am investigating the role of nonlinearities (self-attention, GELU, and LayerNorm) in GPT models, aiming to develop innovative methods for designing GPT models with fewer nonlinearities for efficient private inference.

I have also served as an invited reviewer for NeurIPS'23, ICLR'24, CVPR'24, and ICML'24. If you are interested in collaborating, please feel free to email me!

Interests

Privacy-preserving Machine Learning (PPML)
Efficient Design of LLMs for Privacy and Security
Cryptographic Methods for Secure Neural Network Computation

Education

Ph.D. in Privacy-preserving Deep Learning, 2020 - present
New York University
M.Tech. (Research Assistant) in Computer Science and Engineering, 2017 - 2020
Indian Institute of Technology Hyderabad
B.Tech. in Electronics and Communication Engineering, 2009 - 2013
National Institute of Technology Surat

DeepReShape accepted to TMLR!

We introduced a novel approach (ReLU-equalization) for redesigning network architectures for processing encrypted data efficiently, and proposed HybReNets

Have a look at the paper!

Our work on end-to-end system-level optimizations for private inference is accepted to ASPLOS 2023!

Have a look at the paper!

Circa accepted to NeurIPS 2021!

We co-design Garbled circuit and stochastic ReLUs exploiting the fault tolerance of DNNs for fast private inference

Have a look at the paper!

Our work CryptoNite is out!

We showed that current practices in private inference optimization primarily work with zero arrival requests but tend to fail even at moderate request rates.

Read the paper!

Sisyphus accepted to ACM CCS PPML 2021!

We investigate the “escaping activations” problem in low-degree polynomial substitution of ReLUs for faster private inference and proposed Quadratic Imitation Learning

Have a look at the paper and code!

DeepReDuce accepted to ICML 2021 [Spotlight]!

DeepReDuce is the first work which introduces criticality based ReLU dropping for fast private inference

Have a look at the paper!

Started PhD at ECE, NYU!

Paper on arithmetic intensity for DNNs accepted to IEEE TC 2020!

We proposed a data-type aware arithmetic intensity for estimating energy efficiency of DNNs at design time

Read the paper!

Paper on DNN security accepted to ACM JETC 2020!

We demonstrated how side-channel attacks can be used to decipher the architecture of basic building blocks in DNNs

Read the paper!

Presented our work at ISVLSI 2020!

We proposed DNN co-optimization techniques to circumvent the low PE utilization in depthwise convolution

Read the paper!

Presented our work at WACV 2020!

We proposed light-weight subspace attention module for compact DNNs

Read the paper!

Presented our work at VLSID 2020!

We proposed group convolution techniques for energy-efficient and accurate DNNs

Read the paper!

Presented poster at ICCD 2019!

Received certificate of appreciation for research in CSE IIT Hyderabad!

My first paper accepted at VLSID 2019!

We showed the design ramifications of FLOPs and parameter optimized DNNs

Read the paper!

Featured Publications

Nandan Kumar Jha, Brandon Reagen

April 2024 In TMLR 2024

DeepReShape: Redesigning Neural Networks for Efficient Private Inference

DeepReShape is the first work to conduct a rigorous characterization of desirable neural network attributes for efficient Private Inference (PI). We discovered that distinct network attributes are required for different ReLU counts; in particular, wider networks are beneficial only for higher ReLU counts, whereas networks with a greater proportion of least-critical ReLU are desirable for lower ReLU counts. Further, we introduced a novel network design principle called “ReLU-equalization” to strategically allocate channels within the network to optimize ReLUs and FLOPs efficiency simultaneously. DeepReShape outperforms the current SOTA (SENets, ICLR'23) by achieving a 2.1% increase in accuracy and a 5.2x faster runtime at iso-ReLU counts on CIFAR-100, and an 8.7x faster runtime at iso-accuracy on the TinyImageNet dataset.

Nandan Kumar Jha, Zahra Ghodsi, Siddharth Garg, Brandon Reagen

July 2021 In ICML 2021 (Spotlight)

DeepReDuce: ReLU Reduction for Fast Private Inference

DeepReDuce is a set of optimizations for the judicious removal of ReLUs to reduce private inference latency by leveraging the ReLUs heterogeneity in classical networks. DeepReDuce strategically drops ReLUs upto 4.9x (on CIFAR-100) and 5.7x (on TinyImageNet) for ResNet18 with no loss in accuracy. Compared to the state-of-the-art for private inference DeepReDuce improves accuracy and reduces ReLU count by up to 3.5% (iso-ReLU) and 3.5×(iso-accuracy), respectively.

Recent Publications

Quickly discover relevant content by filtering publications.

Karthik Garimella, Zahra Ghodsi, Nandan Kumar Jha, Siddharth Garg, Brandon Reagen (2023). Characterizing and Optimizing End-to-End Systems for Private Inference. In ASPLOS 2023.

PDF Cite Code Poster

Karthik Garimella, Nandan Kumar Jha, Zahra Ghodsi, Siddharth Garg, Brandon Reagen (2021). CryptoNite: Revealing the Pitfalls of End-to-End Private Inference at Scale. In Arxiv Preprint.

PDF Cite

Zahra Ghodsi, Nandan Kumar Jha, Brandon Reagen, Siddharth Garg (2021). Circa: Stochastic ReLUs for Private Deep Learning. In NeurIPS 2021.

PDF Cite Poster

Karthik Garimella, Nandan Kumar Jha, Brandon Reagen (2021). Sisyphus: A Cautionary Tale of Using Low-Degree Polynomial Activations in Privacy-Preserving Deep Learning. In (ACM CCS) PPML 2021.

PDF Cite Code Poster PPML Proceeding