Frequentist Uncertainties on Neural Density Ratios with wifi Ensembles

The Big Picture

A detective at a crime scene finds two sets of footprints, one from the suspect and one from a bystander, but doesn’t know the exact shoe sizes. Comparing the prints can suggest which belongs to whom. But how confident should the detective be, and can that confidence be quantified?

Physicists at the Large Hadron Collider deal with a version of this problem constantly. Billions of proton collisions spray out cascades of particles called jets, and sorting those jets into types (quarks vs. gluons, signal vs. background) means comparing probability distributions that nobody can write down in closed form.

The standard approach, density ratio estimation (DRE), trains neural networks to approximate the ratio between two distributions. It answers the question: “How much more likely is this particle pattern under one scenario than the other?” But until recently, there was no principled way to put reliable uncertainty bars on that ratio.

Sean Benevedes and Jesse Thaler at MIT’s Center for Theoretical Physics take this on with a framework they call wifi ensembles. The method produces statistically guaranteed error bars on neural density ratios, no expensive repeated retraining required.

Key Insight: By modeling a density ratio as a weighted sum of neural network basis functions, wifi ensembles convert unquantifiable model error into quantifiable statistical uncertainty, giving physicists honest error bars on machine-learned density ratios.

How It Works

Instead of training a single neural network and hoping it’s right, wifi ensembles split the job into two stages.

Stage 1: Train an ensemble of basis functions. You train several neural networks f₁(x), f₂(x), …, fₙ(x), each a candidate approximation of the log-density-ratio. Think of these as multiple detectives, each with their own theory of how the footprints differ.

Stage 2: Fit the weights statistically. Rather than averaging outputs naively, wifi ensembles introduce scalar weights w₁, w₂, …, wₙ, one per basis function, fit using the training data:

log r̃(x|w) = Σ wᵢ fᵢ(x)

The wᵢ are treated as M-estimators, a class of estimators with well-established mathematical guarantees. The authors then derive asymptotic confidence intervals directly from classical statistics. No retraining. No bootstrapping. Just matrix algebra.

Once you have uncertainties on the weights, you propagate them forward. If the density ratio is a likelihood ratio conditioned on some physics parameter (say, the quark fraction in a sample), the Gong-Samaniego theorem translates weight uncertainties into parameter uncertainties. The whole pipeline is computationally cheap.

There’s a useful distinction here between mismodeling and uncertainty. Mismodeling happens when no combination of weights can reproduce the true distribution. More data won’t fix it, and you can’t bound it rigorously. Uncertainty is different: it shrinks as data grows and can be bounded mathematically.

wifi ensembles convert one into the other by design. Adding basis functions reduces mismodeling, in exchange for a larger but quantifiable uncertainty budget.

Validation. The team first tested the method on a Gaussian example where the true density ratio is known analytically. The confidence intervals landed where they should: a 68% interval contained the true value 68% of the time.

Then came the harder test: quark/gluon jet discrimination using QCD simulations. Quark and gluon jets look similar but differ in subtle ways. Gluons spray more particles; quarks are more collimated. The team trained wifi ensembles on simulated jet data, learned the likelihood ratio between jet types, and inferred the quark fraction in a synthetic mixed sample.

The inferred fractions matched ground truth, and uncertainty intervals showed proper frequentist coverage across a range of true quark fractions. All of this without the computationally expensive Neyman construction that traditional bootstrapping requires.

• Faster: No retraining needed for uncertainty quantification once basis functions are fixed.
• Principled: Uncertainties are asymptotically correct by construction, not empirically tuned.
• Propagable: Parameter uncertainties flow from density ratio uncertainties via established theorems.

Why It Matters

This goes well beyond quark-gluon sorting. Simulation-based inference (SBI) has become a central tool of modern particle physics. Measurements of the strong coupling constant, the top quark mass, and dozens of other fundamental parameters all rest on density ratio estimates. The uncertainty on the ratio itself has been a blind spot for years, handled with expensive heuristics or just ignored.

wifi ensembles address that directly. Physicists get a computationally cheap way to put rigorous error bars on neural density ratios, making the entire SBI pipeline more trustworthy. The approach should extend to detector unfolding, simulation reweighting, anomaly detection, and other DRE applications.

Open questions remain. The method assumes the model is well-specified, meaning some combination of basis functions can actually represent the true ratio. Diagnosing violations of that assumption is still an active area of research, and extending wifi ensembles to handle genuine model misspecification would be the obvious next step.

Bottom Line: wifi ensembles give high-energy physicists statistically rigorous, frequentist uncertainty bars on machine-learned density ratios without the computational cost of bootstrapping, demonstrated on simulated quark/gluon jet data for LHC analysis.