Iteration 3: a second zig helps everyone, classical most
Iteration 3 changed the problem, not the models. After the owner's realism check in the viewer, ownship now zigs twice, and both turns cross the line of sight. The 2c models were trained from scratch on the new engagements. Speed improved by almost half, but not enough: the iteration stopped at validation under its pre-registered rule, and the test split was not scored.
- Speed fails the rule. The instantaneous LSTM reaches 1.10 kt [1.06, 1.16] at zig + 600 s on the σ = 1° slice. The rule needed an interval reaching below 1 kt. The forecast, recorded before training, was 1.1–1.3 kt.
- The BiLSTM gets closest of any model so far: 1.02 kt [0.98, 1.05]. It is reported, not gated.
- Range and course pass easily: 8.8% and 6.2° for the LSTM, 7.2% and 4.4° for the BiLSTM.
- Classical gains more. On the same scenarios it reaches 0.34 kt, so the LSTM is now 3.3 × classical on speed, against 2.4 × in iteration 2c.
Record: experiments/20261021-i3a-val/.
Figures: iteration-3 results.
What changed
The owner watched iteration-2c engagements in the viewer and judged that, with both platforms on steady legs, there was too little information. An operator without a steady solution would zig again, traditionally across towards the contact. The seed-203 gate data agreed in part: turns that crossed the line of sight gave the best classical solutions, and an earlier zig mainly shortened the first leg (spec 007 research R1).
- Generator 0.3.0. A second zig 3–5 minutes after leg 2 starts. Each turn puts the target 30–90° on the other bow, by the shorter turn. The first zig keeps its timing.
- Models. The 2c configuration (two stacked 128-unit recurrent layers, speed weighted 4× in the loss), from scratch on a fresh dataset, seed 204.
- Pre-registered rule. The decision record was committed before training. The pair would go to the test split only if the LSTM's speed interval on validation reached below 1 kt.
Seed-204 validation split (σ = 1°)
| Estimator | Range, zig+300 s | Course, zig+300 s | Speed, zig+600 s |
|---|---|---|---|
| G2 threshold | < 20% | < 10° | < 1 kt |
| lstm-instantaneous | 8.8% | 6.19° | 1.10 kt [1.06, 1.16] |
| bilstm-time-late | 7.2% | 4.36° | 1.02 kt [0.98, 1.05] |
| 2c models (out of distribution) | 11.4% / 17.7% | 6.53° / 7.72° | 2.01 / 1.93 kt |
| classical | 3.4% | 1.43° | 0.34 kt |
| naive | 40.2% | 89.7° | 4.23 kt |
Reading it critically
- The lever worked on the problem, not on the gap. Every estimator improved. Classical halved its speed error (0.61 → 0.34 kt), as the forecast said it would. The models improved too, but the ratio to classical grew on every measure: 3.3 × on speed, 2.6 × on range and 4.3 × on course. The extra information went mostly to the estimator that already uses it well.
- The "improvement" against 2c is weak evidence. The 2c models were trained on the two-leg problem; on these engagements they degrade to about 2 kt. Beating them shows that training on the new data helped, nothing more. The rule said so in advance.
- This is an easier problem. Large turns (median 143°) and longer windows make TMA easier for everyone. A pass here would have been a pass on the three-leg problem, not on iteration 2's.
- The BiLSTM near-miss costs. It took 35 runner-hours over three dispatches to stop by patience. Its interval straddles the threshold, but it is not the gated model, and the pair moves together.
What next
Four attempts have now failed on speed, with the gap to classical between 2.4 × and 3.3 ×. Changing the engagements again looks unpromising: it helps classical at least as much. The remaining Annex A levers change the models instead: output parameterisation, derived features, training changes or capacity (B.2.2–B.2.5). The next one is the owner's choice, pre-registered with a fresh seed.