Information Theory

Course Objectives: Understanding the basics of information theory, the subject that deals with answering questions of the kind, how much data can be transmitted over a channel reliably, or how much data can be compressed that can be retrieved reliably. The course will require some probability theory material, such as law of large numbers etc, that will be developed in the class itself.

Syllabus:

• Entropy and Mutual Information
• Assymptotic Equipartition property
• Data Compression Limits
• Channel Capacity
• Gaussian Channel
• Rate-Distortion Theory
• Network Information Theory

Textbook: We will follow "Elements of Information Theory" Ed. 1 by Thomas Cover and Joy Thomas.

Lectures: Two lectures per week.

Grading: There will be 1 mid-term test and a final (one in-class and the other take home) with equal weightage of 30%: The rest 30% will comprise of homeworks. Rest 10% will be based on in-class performance.

Auditing Policy: Students planning to audit the course will be required to turn in all the homeworks, and exams.