Information Theory



  • 2016 Spring/Summer (Jan - May)

Data compression: lossless data compression, entropy and typical sequences, universal

lossless source coding.

Channel coding: mutual information and channel capacity, Gaussian channels.

Source coding (lossy): quantization, rate-distortion function.

Information theory and statistics: large deviation theory, hypothesis testing, Fisher information.

Network information theory: Slepian-Wolf coding, multiple access channels.

Additional topics based on the interest of students (time permitting).


T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley, 2006.

Additional references:

R. Gallager, Information Theory and Reliable Communication, Wiley, 1968.

J. M. Wozencraft and I. M. Jacobs, Principles of Communication Engineering, Wiley, 1965.

C. E. Shannon, The Mathematical Theory of Communication, University of Illinois Press, 1949.

I. Csiszár and J. Körner, Information Theory, Cambridge, 2011.

R. Yeung, Information Theory and Network Coding, Springer, 2008.