BEGIN:VCALENDAR
PRODID:-//eluceo/ical//2.0/EN
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VEVENT
UID:www.tcs.tifr.res.in/event/930
DTSTAMP:20230914T125943Z
SUMMARY:Information Theoretic Perspectives on Learning Algorithms
DESCRIPTION:Speaker: Varun Jog (University of Wisconsin-Madison)\n\nAbstrac
t: \nAbstract: In statistical learning theory\, generalization error is
used to quantify the degree to which a supervised machine learning algorit
hm may overfit to training data. We overview some recent work [Xu and Ragi
nsky (2017)] that bounds generalization error of empirical risk minimizati
on based on the mutual information I(S\;W) between the algorithm input S a
nd the algorithm output W. We leverage these results to derive generalizat
ion error bounds for a broad class of iterative algorithms that are charac
terized by bounded\, noisy updates with Markovian structure\, such as stoc
hastic gradient Langevin dynamics (SGLD). We describe certain shortcomings
of mutual information-based bounds\, and propose alternate bounds that em
ploy the Wasserstein metric from optimal transport theory. We compare the
Wasserstein metric-based bounds with the mutual information-based bounds a
nd show that for a class of data generating distributions\, the former lea
ds to stronger bounds on the generalization error\n
URL:https://www.tcs.tifr.res.in/web/events/930
DTSTART;TZID=Asia/Kolkata:20190102T160000
DTEND;TZID=Asia/Kolkata:20190102T170000
LOCATION:Homi Bhabha Auditorium
END:VEVENT
END:VCALENDAR