Seminar Details
2013-09-11 (14.00) : Rare probability estimation and discrete heavy tails
At Euler building
Organized by Mathematical Engineering
Speaker :
Mesrob I. Ohannessian (Université Paris-Sud)
Abstract :
Despite the increasing volume of data in modern statistical applications, critical patterns and events have often very little,
if any, representation. This isn't unreasonable, given that such variables are critical precisely because they are rare. The natural question is then:
when can we infer something meaningful in such contexts? As a concrete instance of this problem, think of estimating the probability of symbols that
occur very rarely in samples drawn independently from an unknown discrete distribution. The most extreme case is: what is the probability of all
symbols we have never observed? This seems to be ill-posed, yet the classical Good-Turing estimator appears to always work. I will show that, in fact,
if we adopt a multiplicative/relative error notion of consistency, the Good-Turing may fail to be consistent. By taking inspiration from extreme value
theory, I will then demonstrate how a notion of heavy tail is sufficient for consistency. At the core of this result is a new multiplicative
concentration inequality. In the heavy-tailed regime, I will propose new consistent estimators that justify and go beyond some heuristic variants
of the Good-Turing estimator that are used in natural language modeling.
A link to the paper may be found at: http://web.mit.edu/mesrob/www/OhannessianDahleh-COLT2012.pdf