Synthesis and Analysis of Data as Probability Measures with Entropy-Regularized Optimal Transport

The Big Picture

Imagine a collection of photographs of handwritten letters: not the pixel arrays, but the ink distribution on each page. Each “A” smears ink differently, some bold and angular, others loopy and soft. Can you build a Platonic ideal of the letter A by averaging these distributions intelligently? And given an unknown ink blob, can you determine how much it resembles each letter in your alphabet?

That’s the problem a team from Tufts University takes on in this paper. Instead of ink blobs, they work with any data modeled as a probability measure: a point cloud, a distribution of particle energies, a histogram of features. The challenge is designing principled tools for synthesis (building new distributions from references) and analysis (decomposing a distribution into its components).

Their approach uses entropy-regularized optimal transport, a framework for measuring geometric distances between distributions. They prove it works efficiently, with stable guarantees, and even beats neural networks when labeled data is scarce.

Key Insight: Adding an entropy regularization term to classical optimal transport yields dimension-free convergence rates and fast computation, turning a computationally expensive problem into a practical data analysis tool with strong theoretical backing.

How It Works

Classical optimal transport (OT) measures the “distance” between two probability distributions: the minimum cost to rearrange one pile of dirt into the shape of another. The Wasserstein-2 distance squares the cost of moving each grain, making it sensitive to distributional geometry. But exact computation scales cubically in the number of samples, and in high dimensions, estimation requires exponentially many of them.

Entropy regularization fixes this. The entropy-regularized Wasserstein-2 cost adds a penalty proportional to the Kullback-Leibler divergence between the transport plan and the baseline independent coupling. The practical payoff is large: the problem becomes parallelizable via the Sinkhorn-Knopp algorithm, and the cost can be estimated from data at rates that do not depend on dimension. The Sinkhorn divergence is an unbiased variant that corrects for self-transport costs.

Two linked problems sit at the center of the paper:

1. Synthesis (the barycenter problem): Given reference measures and a weight vector summing to one, compute their geometric “average,” a new measure that lies between the references in Wasserstein space.
2. Analysis (the inverse problem): Given an unknown measure, find the weights that make the barycenter of your references as close as possible to it.

The main theoretical result is a fixed-point characterization of entropy-regularized barycenters. A measure is a barycenter if and only if it is a fixed point of a weighted average of entropy-regularized displacement maps. Think of it as an equilibrium condition: let each reference optimally push mass toward the barycenter, take the weighted average of those pushes, and it doesn’t move.

This leads to a clean practical payoff. When the target measure is itself a barycenter of the references, recovering its coefficients reduces to a finite-dimensional, convex quadratic program with a guaranteed unique global solution. No gradient descent, no iterative black-box optimization. Just a convex problem you can solve exactly.

The recovered coefficients are stable: small perturbations in the Wasserstein-2 metric produce small changes in the output. Convergence rates don’t depend on dimension, so the method holds up as data dimensionality grows. For scientific applications where distributions live in high-dimensional spaces, this is a critical property.

Why It Matters

Point clouds (collections of 3D coordinates representing object surfaces) show up everywhere in robotics, autonomous vehicles, and scientific simulation. Classifying corrupted or occluded point clouds is hard. Deep learning attacks the problem with complex architectures that need large labeled datasets.

The Tufts team applied their barycentric coefficient framework to corrupted point cloud classification. Using the coefficients as features fed into a standard classifier, they outperformed neural network baselines when training data was limited. With only a handful of labeled examples, the geometric structure captured by optimal transport provides a stronger inductive bias than a network can learn from scratch.

The applications go well beyond point clouds. Any domain where data is naturally modeled as distributions stands to benefit: particle physics (energy depositions in detectors), cosmology (galaxy surveys as point processes), materials science (atomic configurations). Because the convergence rates are dimension-free, the framework scales to high-dimensional probability measures without the curse of dimensionality dragging performance down.

Open questions remain. The analysis result currently requires knowing the target measure is itself a barycenter of the references. Extending recovery to approximate barycenters, or to settings where the reference set is learned jointly, would open up many more use cases.

Bottom Line: By combining entropy regularization with a new fixed-point characterization of barycenters, this work delivers a rigorous and computationally tractable framework for distribution-valued data analysis that beats deep learning when labels are scarce.

---