BEGIN:VCALENDAR
PRODID:-//eluceo/ical//2.0/EN
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VEVENT
UID:www.tcs.tifr.res.in/event/930
DTSTAMP:20230914T125943Z
SUMMARY:Information Theoretic Perspectives on Learning Algorithms
DESCRIPTION:Speaker: Varun Jog (University of Wisconsin-Madison)\n\nAbstrac
 t: \nAbstract:  In statistical learning theory\, generalization error is 
 used to quantify the degree to which a supervised machine learning algorit
 hm may overfit to training data. We overview some recent work [Xu and Ragi
 nsky (2017)] that bounds generalization error of empirical risk minimizati
 on based on the mutual information I(S\;W) between the algorithm input S a
 nd the algorithm output W. We leverage these results to derive generalizat
 ion error bounds for a broad class of iterative algorithms that are charac
 terized by bounded\, noisy updates with Markovian structure\, such as stoc
 hastic gradient Langevin dynamics (SGLD). We describe certain shortcomings
  of mutual information-based bounds\, and propose alternate bounds that em
 ploy the Wasserstein metric from optimal transport theory. We compare the 
 Wasserstein metric-based bounds with the mutual information-based bounds a
 nd show that for a class of data generating distributions\, the former lea
 ds to stronger bounds on the generalization error\n
URL:https://www.tcs.tifr.res.in/web/events/930
DTSTART;TZID=Asia/Kolkata:20190102T160000
DTEND;TZID=Asia/Kolkata:20190102T170000
LOCATION:Homi Bhabha Auditorium
END:VEVENT
END:VCALENDAR