Supplementary material to the paper:

        Grendár, M. and Špitalský, V., Multinomial and Empirical Likelihood under convex constraints: directions of recession, Fenchel duality, perturbations. arXiv:1408.5621 [math.ST], Aug 2014.


The primal problem of multinomial likelihood maximization restricted to a convex closed subset of the probability simplex is studied. Contrary to widely held belief, a solution of this problem may assign a positive mass to an outcome with zero count. Related flaws in the simplified Lagrange and Fenchel dual problems, which arise because the recession directions are ignored, are identified and corrected.

A solution of the primal problem can be obtained by the PP (perturbed primal) algorithm, that is, as the limit of a sequence of solutions of perturbed primal problems. The PP algorithm may be implemented by the simplified Fenchel dual.

The results permit us to specify linear sets and data such that the empirical likelihood-maximizing distribution exists and is the same as the multinomial likelihood-maximizing distribution. The multinomial likelihood ratio reaches, in general, a different conclusion than the empirical likelihood ratio.

Implications for minimum discrimination information, compositional data analysis, Lindsay geometry, bootstrap with auxiliary information, and Lagrange multiplier tests are discussed. 

R code
An R code and data to reproduce the numerical examples can be found in here


V. Špitalský, Multinomial likelihood: the recession cone view, a talk at Mathematical Methods in Economy and Industry, Smolenice, Slovakia September 8-12, 2014.

M. Grendár, Multinomial and empirical likelihood under convex constraints, a talk at Andrej Pazman's seminar, FMPH Comenius University, Bratislava, September 29, and October 13, 2014.

M. Grendár, Contingency tables and the PP algorithm, a talk at the Institute of Mathematics, UPJS, Kosice, June 3, 2015. -- Valuable feedback from participants of the seminar is gratefully acknowledged.

Value added

The paper is devoted to the problem P of maximization of the closed multinomial likelihood restricted to a convex, closed set C. Presence of zero counts is the main source of challenges; though their nature is here rather different than in the case of log-linear models.

The work
  1.  recognizes and proves that a solution of the problem P may assign a positive mass to an outcome with zero count;

  2. identifies related flaws in the simplified Lagrange and Fenchel dual formulations of the problem, which have been used in the statistics literature for decades;
  3. determines under what conditions the simplified Lagrange and Fenchel duals lead to a solution of P;
  4. develops the correct form of the Fenchel (and Lagrange) dual to P. The notion of recession direction, a fundamental concept in convex analysis, plays a key role;
  5. establishes the convergence (epi-convergence for a general, convex, closed C; pointwise convergence for a linear C) of solutions of perturbed problems to a solution of P . This suggests that the common practice of replacing the zero counts by a small, arbitrary value, can be replaced by the PP algorithm, i.e., by a sequence of perturbed problems;
  6. discusses the implications for the empirical likelihood (EL) method. Besides recovering the convex hull restriction and the zero-likelihood problem, our results suggest another fact, unfavorable to EL: the EL-maximizing distribution, if it exists at all, may be different than the multinomial likelihood-maximizing distribution. Consequently, the multinomial likelihood ratio may lead to different inferential and evidential conclusions than the EL ratio. Thus, at least in the discrete iid setting, the price for intentionally restricting attention to observed outcomes may be higher than previously thought;
  7. briefly discusses how the findings affect also other methods, such as the minimum discrimination information, compositional data analysis, Lindsay geometry of multinomial mixtures, bootstrap with auxiliary information, and Lagrange multiplier test, which, in the studied context, explicitly or implicitly ignore information about the support and are restricted to the observed outcomes