91ɬÂþ

Abstract

Learning a categorical distribution from its samples underlies many machine learning tasks, including categorical generative modeling and classification.

A critical property of a learned distribution is its calibration which measures how well the predicted probabilities reflect the true distribution. Proper calibration is essential both for generative tasks, where underconfidence or overconfidence can lead to undesirable effects, and for classification with efficient inference, where reliable confidence estimates are needed for cost-aware adaptive computation. However, evaluating and learning calibrated models in the categorical setting is challenging: the support space is typically prohibitively large, ground-truth probabilities are inaccessible, and access to samples from the ground truth is usually very limited.

This thesis investigates the problem of learning and evaluating calibrated models for categorical data, addressing both generation and the setting of adaptive computation for classification.

First, I propose a novel categorical generative model that combines a diffusion process with a structured, sphere-packed encoding and Gaussian mixture denoising, which improves calibration, sample quality, and efficiency.

Second, I develop a principled evaluation framework for categorical generative models based on synthetic distribution coarsening, which provides interpretable diagnostics and statistical guarantees in high-dimensional, purely nominal categorical spaces. This contribution addresses the current gap in the literature on evaluation for categorical generative models, as existing evaluation methods focus on continuous settings. Third, I formulate a unified framework for adaptive computation with fixed classifiers, showing that optimal resource allocation in this setting reduces to accurately predicting per-sample error probabilities. Lastly, building on this, I introduce a two-stage classification formulation in which multiple classifiers and the resource allocation module are trained jointly, and provide a practical surrogate loss that is consistent with a principled cost-aware target loss.

Together, these contributions advance the understanding and design of calibrated probabilistic models and highlight the relationship between calibration, evaluation, and adaptive computation in categorical settings.