One-day Short-Course
on
Information Theory and Statistics
Instructors: Mark Hansen (UCLA) and Bin Yu (UC
Berkeley)
June 1, 2005
Clark Building A202
Colorado State University, Fort Collins, CO
80523
Registration/Check-in 8:30-9:00am
Juice, fruit & bagels will be available during registration.
(Weber 206 is a computer laboratory to be used following presentations made in
the Clark Building)
Information Theory deals with a basic challenge in communication: How do we
transmit information efficiently? In addressing that issue, Information
Theorists have created a rich mathematical framework to describe communication
processes with tools to characterize so-called fundamental limits of data
compression and transmission.
What might Statisticians learn from Information Theory? Basic concepts like
entropy and Kullback-Leibler divergence have certainly played a role in
statistics. But so too have estimation frameworks like the Maximum Entropy
principle; novel decompositions like ICA; and even model selection methodologies
like AIC and the Principle of Minimum Description Length. In this course we will
illustrate how the basic questions and tools of Information Theory relate to
statistical practice and theory.
