报告题目：Heavy-Tailed Methods in Machine Learning: Algorithmic Stability, Generalization and Differential Privacy
时 间：2025年6月18日（星期三）10:30
地 点：科研楼18号楼1102
主 办：数学与统计学院
参加对象：感兴趣的老师和学生
报告摘要：Heavy-tail phenomena in stochastic gradient descent (SGD) have been reported in several empirical studies. Experimental evidence in previous works suggests a strong interplay between the heaviness of the tails and generalization behavior of SGD. We develop generalization bounds for a general class of objective functions, which includes non-convex functions. Our approach is based on developing Wasserstein stability bounds for heavy-tailed SDEs and their discretizations, which we then convert to generalization bounds. Our theoretical results are in line with the observations from the experiments on neural networks. The injection of heavy-tailed noise into the iterates of stochastic gradient descent (SGD) has garnered growing interest in recent years due to its theoretical and empirical benefits for optimization and generalization. However, its implications for privacy preservation remain largely unexplored. Aiming to bridge this gap, we provide differential privacy (DP) guarantees for noisy SGD, when the injected noise follows an alpha stable distribution, for a broad class of loss functions which can be non-convex. Contrary to prior work that necessitates bounded sensitivity for the gradients or clipping the iterates, our theory can handle unbounded gradients without clipping, and reveals that under mild assumptions, such a projection step is not actually necessary. Our results suggest that, given other benefits of heavy-tails in optimization, heavy-tailed noising schemes can be a viable alternative to their light-tailed counterparts.
报告人简介：朱凌炯，佛罗里达州立大学教授。朱凌炯教授于2008年获剑桥大学学士学位，2013年获纽约大学Courant数学研究所博士学位，师从著名数学家S.R.S. Varadhan。他毕业后曾任职于摩根士丹利与明尼苏达大学，2015年加入佛罗里达州立大学任助理教授，现为该校数学系教授，思考机器杰出学者。朱教授的研究领域涵盖应用概率论、数据科学、金融工程与运筹学，曾在各领域顶级期刊和会议Ann Appl Probab, Bernoulli, Financ Stoch, ICML, INFORMS J Comput, J Mach Learn Res, NeurIPS, Oper Res, Prod Oper Manag, SIAM J Financ Math, Stoch Proc Appl, Rev Econ Stat等发表论文，并获Courant研究所2013年度Kurt O. Friedrichs杰出博士论文奖、佛罗里达州立大学2022年发展学者奖、2023年研究生导师奖，以及2023年MSOM iFORM SIG最佳论文奖。