BEGIN:VCALENDAR
PRODID:-//eluceo/ical//2.0/EN
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VEVENT
UID:www.tcs.tifr.res.in/event/1350
DTSTAMP:20231027T065509Z
SUMMARY:SGD from the Lens of Markov process: An Algorithmic Stability Persp
 ective
DESCRIPTION:Speaker: Anant Raj (University of Illinois at Urbana-Champaign)
 \n\nAbstract: \nIn this talk\, we delve into the intricate relationships b
 etween heavy-tailed distributions\, generalization error\, and algorithmic
  stability in the realm of noisy stochastic gradient descent. Recent resea
 rch has illustrated the emergence of heavy tails in stochastic optimizatio
 n and their intriguing links to generalization error. However\, these stud
 ies often relied on challenging topological and statistical assumptions. E
 mpirical evidence has further challenged existing theory\, suggesting that
  the relationship between heavy tails and generalization is not always mon
 otonic. In response\, we introduce novel insights\, exploring the relation
 ship between tail behavior and generalization properties through the lens 
 of algorithmic stability. Our analysis reveals that the stability of stoch
 astic gradient descent (SGD) varies based on how we measure it\, leading t
 o interesting conclusions about its behavior.Expanding upon these findings
 \, we extend the scope to a broader class of objective functions\, includi
 ng non-convex ones. Leveraging Wasserstein stability bounds for heavy-tail
 ed stochastic processes\, our research sheds light on the non-monotonic co
 nnection between generalization error and heavy tails\, offering a more co
 mprehensive perspective.Additionally\, we introduce a unified approach for
  proving Wasserstein stability bounds in stochastic optimization\, emphasi
 zing time-uniform stability and its role in various scenarios\, including 
 convex and non-convex losses. Our approach is versatile and applicable to 
 popular optimizers\, highlighting the importance of ergodicity.\n
URL:https://www.tcs.tifr.res.in/web/events/1350
DTSTART;TZID=Asia/Kolkata:20231101T203000
DTEND;TZID=Asia/Kolkata:20231101T220000
LOCATION:via Zoom in A201
END:VEVENT
END:VCALENDAR