The Oracle Could Choose, but the Online Planner Could Not
Exact-grid action values exposed a large information gap, while causal self-rollout became inaccurate and prohibitively expensive once planning work was counted.
Series: VBF Experiments, August 2026
Contents
The encouraging result from controlled belief projection was that an exact-grid oracle could choose projection actions much better than a locked fixed policy. The uncomfortable result was that this advantage depended on information the online filter did not have.
We tested the obvious bridge: plan from the bounded approximate belief, generate causal futures under the known model, and score the projection actions through self-rollout. It is a deployable information pattern. It was also the wrong planner for this benchmark.
On the same decision states, full-information rollout was much better than causal rollout. Once planning computation was charged, the causal planner failed decisively.
This is not a generic argument against model-based planning. It is a measured failure of one planner under a strict online contract, and it clarifies what an oracle result does—and does not—buy us.
The complete experiment and claim ledger are in
dwrtz/ml-examples, summarized at
source commit ddc9215.
Four policies that sound more similar than they are#
The experiment used four decision concepts:
- A fixed policy always applies the same projection operator.
- A full-information one-step oracle scores every action against the exact grid belief and chooses the smallest immediate distortion-plus-compute cost.
- A full-information four-step rollout uses exact-grid action scoring for a one-step deviation, then follows a named continuation policy.
- A causal self-rollout begins from the bounded Gaussian-mixture belief, samples possible future observations, recursively applies approximate filtering actions, and pays for the planning work it performs.
Only the last policy has the intended online information contract. It may use the current approximate predictive mixture, current observation and causal context, model parameters, bounded memory, and remaining compute budget. It may not read the exact posterior, latent state, future observations, or an undeclared future channel schedule.
The distinction is easy to blur because all four policies return the same kind of action. The path used to choose that action is the experiment.
Same state, different information#
To isolate information quality from differences in physical states, we ran the full-information and causal planners on the same saved decision states. The four-step result was large:
| Same-state comparison | States | Full-information cost | Causal carried cost | Relative gap | 95% CI for absolute gap |
|---|---|---|---|---|---|
| Known schedule | 16 | 1.3320 | 3.4580 | 61.5% | [0.5296, 3.7224] |
| Stochastic hazards | 10 | 1.4853 | 2.9925 | 50.4% | [0.5718, 2.4426] |
The action choices disagreed in 8 of 16 known-schedule states and 8 of 10 stochastic-hazard states. These were not merely small numerical changes around the same decisions.
“Carried cost” counts the filtering loss and execution cost incurred by the selected actions. It deliberately omits the internal work used to search for them. Even on that favorable measure, the causal planner lagged far behind the oracle.
Why? A self-rollout planner does not get to sample futures from the exact filtering distribution. It generates them from the bounded belief produced by earlier approximations. When that belief merges, shifts, or underweights a plausible mode, the error changes both the simulated observations and the later belief states used to score actions. The planner evaluates its own lossy counterfactual world.
This is a form of causal-state aliasing. Two exact beliefs can map to the same bounded mixture summary while assigning different value to preserving a mode. No amount of tree search from the aliased state can recover information that the state no longer contains.
The carried-cost improvement was small and unresolved#
The causal four-step planner was compared with repeated causal greedy action selection. Before charging for planning, the result looked mildly positive:
| Information setting | Causal H4 carried cost | Repeated causal greedy | Relative gain | 95% CI for absolute gain | |
|---|---|---|---|---|---|
| Known schedule | 3.4580 | 3.5195 | 1.75% | [-0.0825, 0.2055] | |
| Stochastic hazards | 2.9925 | 3.0244 | 1.05% | [-0.2106, 0.2743] |
Both intervals cross zero. Even if rollouts were free, the measured action improvement would be small and unresolved at this screening scale.
Rollouts were not free.
Planning work reversed the comparison#
The mean planning charge was 25.8079 in the known-schedule setting and 15.6937 under stochastic hazards. Adding it to carried cost changed the result:
| Information setting | Causal H4 total cost | Repeated causal greedy | Relative gain |
|---|---|---|---|
| Known schedule | 29.2659 | 4.4477 | -558.0% |
| Stochastic hazards | 18.6862 | 3.5858 | -421.1% |
The paired 95% intervals for the total-cost difference were entirely negative:
[-33.7632, -15.8732] and [-17.5583, -12.6425]. Every evaluated state lost
to repeated causal greedy once planning work was included.
The one-step version failed too. Against the best fixed action, total causal H1 cost was 217.8% worse in the known-schedule screen and 47.7% worse in the stochastic screen, with unresolved intervals at those smaller state counts.
Simple causal rules made the scale problem even clearer. Their mean costs were 2.8047 for known schedules and 2.5781 for stochastic hazards. A planner that spends an order of magnitude more work to produce a less accurate action is not a plausible deployment candidate.
Why excluding overhead would be misleading#
There are legitimate experiments where planner time is reported separately from task loss. This was not one of them. Projection actions were explicitly chosen under a compute budget, and the objective already penalized operator work. Ignoring search cost would let the controller spend unbounded computation to shave a small amount from a computation-aware objective.
The accounting therefore separated three quantities:
- Carried filtering cost: distortion plus the work of the selected projection actions.
- Planning cost: all simulated filtering and action evaluation used to make the choice.
- Total cost: the sum paid by a real online system.
That separation is what prevents a 1–2% unresolved carried-cost improvement from being presented as a planning win.
What the negative result teaches#
The failure narrows the problem in a useful way.
First, the exact-grid one-step result still shows that action choice is valuable. The causal planner does not erase that mechanism; it fails to access it cheaply and accurately.
Second, the same-state gap says that scaling this particular rollout tree is unlikely to be enough. More samples would attack estimator noise and cost more, but would not repair state aliasing.
Third, the bounded belief is not necessarily useless. It may contain predictive features that a direct controller can learn without regenerating a future tree for every action. The next study therefore changed the question from “can the filter plan its way to the oracle?” to “how much of the oracle’s one-step value is predictable from a bounded causal state?”
That route produced substantial offline signal—and then failed under repeated online control. The distinction between those two statements is the subject of Useful Offline, Harmful Online.