Information Theory

This course on basic information theory will cover fundamental concepts in data compression
and reliable communication over noisy channels.

Hours: (36-42 hours) 3 hours per week.
Prerequisites: None

Syllabus:
Data compression: lossless data compression, entropy and typical sequences, universal
lossless source coding.
Channel coding: mutual information and channel capacity, Gaussian channels.
Source coding (lossy): quantization, rate-distortion function.
Information theory and statistics: large deviation theory, hypothesis testing, Fisher information.
Network information theory: Slepian-Wolf coding, multiple access channels.
Additional topics based on the interest of students (time permitting).

Textbook:
T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley, 2006.

Additional references:
R. Gallager, Information Theory and Reliable Communication, Wiley, 1968.
J. M. Wozencraft and I. M. Jacobs, Principles of Communication Engineering, Wiley, 1965.
C. E. Shannon, The Mathematical Theory of Communication, University of Illinois Press, 1949.
I. Csiszár and J. Körner, Information Theory, Cambridge, 2011.
R. Yeung, Information Theory and Network Coding, Springer, 2008.