Vierailijana Jyvaskylassa 18.-19.5. : Jorma Rissanen
Teemu Antti-Poika
Department of Mathematics
Department of Statistics
Department of Mathematical Information Technology
----------------------------------------------------------------------
Jyväskylä Center for for Mathematical and Computational Modeling
proudly presents
Statistical Modeling Based on Information Theory
Jorma Rissanen, IBM Research ARC, San Jose, CA
----------------------------------------------------------------------
+ Tuesday 18.5. 14-16 Mattilanniemi Campus MaA 211, Basics of Coding
+ Wednesday 19.5. 9-11 Mattilanniemi Campus MaA 211, Modeling Theory
This two-part lecture series discusses statistical modeling based
on the coding theoretic idea of the MDL (Minimum Description Length)
principle, which also may be viewed as a global maximum likelihood
principle, where any two models may be compared regardless of their
structures and numbers of parameters. This is accomplished by assessing
the goodness of a model by the number of bits it allows us to encode
the observed data with, including the code length required to specify
the model itself in a class of suggested ones. The shortest code
length found gives a decomposition of the data into the information
bearing part defined by the optimal model and the rest, which is just
noise having no useful information that can be described by the
suggested model class. In this view no impossible to verify assumption
that the data form a sample from a distribution is needed.
The lectures cover the following topics to the extent time permits:
+ Basics of Coding Theory
- prefix codes and Kraft-inequality - Shannon's Noiseless Coding
Theorem
- coding of random processes
+Universal Coding
- general - Lempel-Ziv algorithm - Algorithm Context
+ Kolmogorov Complexity
- universal algorithmic model - sufficient statistics decomposition
+ Stochastic Complexity
- models - universal models and the MDL principle
+ Three Universal Density Functions
- normalized 2-part density function - NML-density function - mixture
density function - universal sufficient statistics decomposition
+ Applications
- linear LS regression - MDL denoising
Welcome!