BEGIN:VCALENDAR
PRODID:-//eluceo/ical//2.0/EN
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VEVENT
UID:www.tcs.tifr.res.in/event/976
DTSTAMP:20230914T125945Z
SUMMARY:Finite-Time Analysis of Q-Learning with Linear Function Approximati
on
DESCRIPTION:Speaker: Siva Theja Maguluri (Georgia Tech\nUnited States of Am
erica)\n\nAbstract: \nAbstract: We consider the model-free reinforcement l
earning problem and study the popular Q-learning algorithm with linear fun
ction approximation for finding the optimal policy. We provide a finite-ti
me error bounds for the convergence of Q-learning with linear function app
roximation under an assumption on the sampling policy. Unlike some prior w
ork in the literature\, we do not need to make the unnatural assumption th
at the samples are i.i.d. (since they are Markovian)\, and do not require
an additional projection step in the algorithm. To show this result\, we f
irst consider a more general nonlinear stochastic approximation algorithm
under Markovian noise\, and derive a finite-time bound on the mean-square
error\, which we believe is of independent interest. The key idea of our p
roof is to use Lyapunov drift arguments and exploit the geometric mixing p
roperty of the underlying Markov chain.\n\nBio: Siva Theja Maguluri is an
Assistant Professor in the School of Industrial and Systems Engineering a
t Georgia Tech. Before that\, he was a Research Staff Member in the Mathem
atical Sciences Department at IBM T. J. Watson Research Center. He obtaine
d his Ph.D. and MS in ECE as well as MS in Applied Math from UIUC\, and B.
Tech in Electrical Engineering from IIT Madras. His research interests are
broadly in Applied Probability\, Optimization and Reinforcement Learning\
, and include Scheduling\, Resource Allocation and Revenue Optimization in
a variety of systems including Data Centers\, Cloud Computing\, Wireless
Networks\, Block Chains\, Ride hailing systems etc. He is a co-recipient o
f the “Best Publication in Applied Probability” award\, presented by I
NFORMS Applied probability society every two years.\n
URL:https://www.tcs.tifr.res.in/web/events/976
DTSTART;TZID=Asia/Kolkata:20190723T143000
DTEND;TZID=Asia/Kolkata:20190723T153000
LOCATION:A-201 (STCS Seminar Room)
END:VEVENT
END:VCALENDAR