BEGIN:VCALENDAR
PRODID:-//eluceo/ical//2.0/EN
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VEVENT
UID:www.tcs.tifr.res.in/event/1044
DTSTAMP:20230914T125948Z
SUMMARY:Learning in Gated Neural Networks
DESCRIPTION:Speaker: Makkuva Ashok Vardhan (Department of Electrical & Comp
 uter Engineering\nUniversity of Illinois at Urbana-Champaign (UIUC)\nUnite
 d States)\n\nAbstract: \nAbstract: Gating is a key feature in modern neur
 al networks including LSTMs\, GRUs and sparsely-gated deep neural networks
 . The backbone of such gated networks is a mixture-of-experts layer\, wher
 e several experts make regression decisions and gating controls how to wei
 gh the decisions in an input-dependent manner. Despite having such a promi
 nent role in both modern and classical machine learning\, very little is u
 nderstood about parameter recovery of mixture-of-experts since gradient de
 scent and EM algorithms are known to be stuck in local optima in such mode
 ls. In this work\, we perform a careful analysis of the optimization lands
 cape and show that with appropriately designed loss functions\, gradient d
 escent can indeed learn the parameters accurately. A key idea underpinning
  our results is the design of two distinct loss functions\, one for recove
 ring the expert parameters and another for recovering the gating parameter
 s. We demonstrate the first sample complexity results for parameter recove
 ry in this model for any algorithm and demonstrate significant performance
  gains over standard loss functions in numerical experiments. \n\nBio: A
 shok is a 5th year graduate student in the ECE department at UIUC\, advise
 d by Prof. Pramod Viswanath. He obtained his Masters in ECE (advised by Pr
 of. Yihong Wu) from UIUC in 2017 and Bachelors in EE (advised by Prof. Viv
 ek Borkar) with a minor in Mathematics from IIT Bombay in 2015. His curren
 t research interests are theoretical and algorithmic aspects of machine le
 arning and information theory. He is a recipient of Best Paper Award from 
 ACM MobiHoc 2019. He has won several graduate student awards and fellowshi
 ps including Joan and Lalit Bahl Fellowship\, Sundaram Seshu International
  Student Fellowship\, and was also a finalist for the Qualcomm Innovation 
 Fellowship 2018. Outside the convex hull of research activities\, he likes
  to learn new languages\, watch and read about international films\, readi
 ng history\, and remembering trivia. For more details\, please visit: http
 ://makkuva2.web.engr.illinois.edu/\n
URL:https://www.tcs.tifr.res.in/web/events/1044
DTSTART;TZID=Asia/Kolkata:20200123T143000
DTEND;TZID=Asia/Kolkata:20200123T153000
LOCATION:A-201 (STCS Seminar Room)
END:VEVENT
END:VCALENDAR