Thesis

What we are betting on

01

Prediction was the wrong objective

Machine learning spent a decade minimising prediction error, and it succeeded. Forecasts of demand, failure, price and arrival are better than they have ever been. The systems those forecasts feed did not improve at the same rate, and the reason is structural rather than incidental.

Prediction error and decision regret are different objectives, and optimising the first does not optimise the second. A model can reduce mean squared error while making the downstream decision worse, because the loss it minimises is symmetric and the cost of being wrong is not. Being ten units short and ten units long are the same number to a regression and very different numbers to the operation paying for it.

The field that takes this seriously is decision focused learning: training the model on the quality of the decision it induces rather than the accuracy of the number it emits. Elmachtoub and Grigas formalised it as Smart Predict then Optimise. Bertsimas and Kallus framed the prescriptive version. Donti, Amos and Kolter showed the gradient can be pushed through the optimisation layer itself, which is the hard part, because argmax is not differentiable and neither is a solver.

Almost nobody trains this way. The industry optimises prediction because prediction is easy to score and decisions are not. We think that is the single largest misallocation of effort in applied machine learning, and we are building on the other side of it.

02

The specification bottleneck dissolved

Constrained optimisation is fifty years old and works. Structural causal models have a Turing Award behind them. Simulation based inference, conformal prediction, off policy evaluation, stochastic programming, all published, all mature, all rigorous, all largely unused in the places where decisions actually get made.

They went unused for one reason, and it was never mathematical. Somebody had to write the formal specification by hand. A domain expert who understood the system and a modeller who could encode it, in a room together, for months, once per problem. The encoding cost more than the answer was worth almost everywhere, so the methods stayed in journals and the decisions stayed in spreadsheets.

Language models are specification writers. They read unstructured descriptions of a system, its documents, its schemas, its exception cases, the reasons people give when they override it, and emit a first draft of a formal model. The literature is early and moving fast: autoformalization in theorem proving, NL4Opt as the benchmark for natural language to optimisation model, OptiMUS for LLM generated mixed integer programmes. None of it is finished. All of it works well enough to change the economics.

This is the observation the company is built on. An entire class of rigorous methods just had its only blocker removed, and most of the field has not noticed, because it is busy asking language models to produce the answer instead of asking them to produce the formalism. The model should write the specification. Something else should solve it.

03

The simulator is mechanistic, not generative

A language model is the wrong instrument for evaluating an option space. It costs cents and seconds per call, and it returns a different answer to the same question twice. Neither property survives the volume required when the space is swept continuously rather than queried on demand. Non determinism is disqualifying on its own: two identical states must produce identical evaluations or nothing downstream can be audited.

What replaces it is a mechanism. A deterministic forward model that takes a state and an action and returns a distribution over outcomes, compiled, versioned, testable, and cheap enough to evaluate a million times before lunch. Where the structure is known it is written as code or as a constraint model and solved with MILP or CP SAT. Where the structure is partially unknown, the gap between mechanism and reality is learned.

The research lineage is model based reinforcement learning and world models: Sutton's Dyna, then PlaNet, the Dreamer line, MuZero learning the model and the search jointly, PETS for learned dynamics with explicit uncertainty in the small data regime. That work was aimed at control and robotics. The methods transfer. The domain has barely been touched.

One correction that matters. This is not a reinforcement learning company. RL is the right tool where an action changes the state you face next over a long horizon, and that describes a narrow slice of real decisions. Most consequential decisions are single shot or two step, and reaching for a policy network on a single shot decision is how strong teams spend two years losing to a solver and a surrogate. We use sequential methods only where the sequence is real.

04

The sweep is a numerics problem

Ten levers at ten settings is ten billion combinations. That space is not conquered by running faster. It is conquered by not running most of it, and the technique stack for doing so is well developed and rarely assembled.

Global sensitivity analysis comes first. Sobol indices and Morris screening reduce a thousand nominal parameters to the handful that actually move the outcome, which collapses the problem before any serious compute is spent. Then surrogates: Gaussian processes, polynomial chaos expansions, or neural emulators fitted to the exact mechanism and swept in its place, three to six orders of magnitude cheaper per evaluation. Then variance reduction: quasi Monte Carlo with Sobol sequences instead of pseudorandom sampling, control variates, Giles style multilevel Monte Carlo putting most of the work at the cheapest fidelity. Then adaptive sampling, Bayesian optimisation to place evaluations where uncertainty or the optimum lives rather than uniformly. Then incremental recomputation, so a small change in state propagates through a dependency graph instead of triggering a full re-sweep. Then batched vectorisation, because a hundred thousand scenarios evaluated as one tensor operation is where the microseconds actually come from.

Stacked, a sweep across thousands of parameters becomes a few hundred true mechanism evaluations and a surrogate query. That is the difference between a system that answers when asked and a system that is always already answering.

The hard part is not speed. It is the error bound on the surrogate. A fast answer with unknown error is worse than a slow one, and the discipline is adaptive refinement where the bound is too wide and automatic fallback to the exact mechanism when it cannot be tightened. Precision budgets differ by decision type. Some invert on two percent error. Some tolerate fifteen. The system has to know which it is looking at.

05

An estimate without an interval is not an input

A point estimate is not a decision input. It is a number with unstated confidence, and acting on it is indistinguishable from guessing with extra steps.

Conformal prediction is the most practical route to honest coverage: distribution free, finite sample, model agnostic, and it wraps anything. The complication is that it assumes exchangeability, and real systems violate that constantly through seasonality, regime change, and policy edits made midway through a quarter. Weighted and adaptive conformal under covariate shift is the version that survives contact. Deep ensembles and calibrated regression carry part of the load. Gaussian process surrogates hand you posterior variance for free, which is one argument for preferring them where the dimension permits.

Fitting the mechanism to observed data is simulation based inference. Cranmer, Brehmer and Louppe mapped the field. It is exactly the problem of a mechanistic simulator with parameters you cannot observe directly and outcomes you can, and it is the formal version of what calibration means here.

The behaviour that follows is a design commitment rather than a technique. When the interval is too wide to separate the options, the system says so and stays silent. A confident wrong answer is more expensive than no answer, and a system that interrupts people with unearned certainty gets ignored within a month.

06

The counterfactual problem is the hard one

Everything above is engineering. This section is the research.

A system of consequential decisions produces a few hundred observations a year, not a few hundred thousand. Randomisation is mostly unavailable, because nobody permits experiments on live operations to satisfy an estimator. And a log of decisions taken contains almost no information about the decisions not taken, which is precisely the information required to know whether the choice was good.

The tools exist and none of them is sufficient alone. Off policy evaluation with doubly robust estimation, following Dudík, Langford and Li, recovers what an alternative policy would have returned from data a different policy generated. Counterfactual risk minimisation in the Swaminathan and Joachims formulation learns directly from logged bandit feedback. Bottou and colleagues built the production version of this inside computational advertising in 2013, which remains the clearest demonstration that counterfactual estimation survives contact with a real system. Structural causal models and do calculus give the identification conditions. Double machine learning and causal forests handle heterogeneous effects. Synthetic controls and natural experiments mined from the log recover comparisons the system never deliberately ran.

Interference makes it worse. Multiple decisions in flight, outcomes entangled, SUTVA violated by default. Attributing an outcome to a single decision when several were active is unsolved in general and has to be handled case by case.

Then there is exploration, which is the part we think is genuinely underexplored. If a system may deviate from its own recommendation on some small fraction of decisions, which fraction, and chosen how? Contextual bandits give the machinery, LinUCB and Thompson sampling among them, but the standard formulations assume abundance. The regime here is scarcity, where each exploratory decision is expensive and individually visible to the person accountable for it. Experimental design under a hard regret budget, maximising information per unit of expected loss, is the specific problem, and it determines how quickly a record of decisions becomes worth anything. We think it is the highest leverage open question in the entire stack.

07

What we are betting on

Five bets, stated plainly.

That decision regret is the correct objective and prediction error is a proxy the field mistook for the target.

That the specification bottleneck was the only thing keeping fifty years of rigorous method out of practice, and that its removal is a larger event than it currently looks.

That mechanistic simulation with learned correction beats generative approximation for anything evaluated at volume, and that determinism is a feature rather than a limitation.

That calibration compounds. A record of predictions against realised outcomes, accumulated across many decisions, is an asset that cannot be purchased, scraped, or hired, because the outcomes of unchosen options do not exist in any dataset.

And that the shapes recur. Elasticity, saturation, lag, congestion. A simulator calibrated on those shapes in one setting starts the next one less wrong, which means the record improves faster than it accumulates.

If the last bet is wrong, this is a very good tool for one domain at a time. If it is right, it is infrastructure.

Curious enough to talk?

laman[at]avirhq[dot]com