Phase 4: an end-to-end learned tracker wins overall, loses on straight legs

Phase 4 asked whether a learned tracker, trained end to end, could beat a tuned IMM tracker on targets that manoeuvre up to three times. Two trackers were trained from scratch and compared with the frozen reference on the same validation split:

• T1, a learned-gain filter: Kalman-style physics with a gain computed by a small recurrent network;
• T2, a pure recurrent tracker (2 × 128 LSTM) that also states its own uncertainty.

Neither reached the test. The pre-registered rule needs every gated check to pass on validation.

Median error, ratio to the IMM	Limit	T2	T1
Range, all gate rows	< 1	0.85	2.18
Course, all gate rows	< 1	0.84	1.32
Range, settled rows	< 1.1	0.97	2.80
Course, settled rows	< 1.1	1.44	2.12
Speed, settled rows	< 1.1	1.66	2.30

"Settled" rows are at least five minutes after the target's last manoeuvre. Both of T2's failures have their whole 97.5% interval above the limit, so the fresh test split (seed 212) was never scored.

Record: the validation record.

What was tested

• Data. Generator 0.5.0 gives the target 0–3 course changes and, in 30% of scenarios, a speed change, in 20–60 min windows.
• Reference. An IMM (constant velocity plus coordinated turn), tuned on validation over all gate rows and frozen. Its median range error there is 9.2%.
• Gate rows. σ = 1° bearings at 5, 10, 15 and 20 min after ownship's zig.
• Forecast, recorded before training: T1 carried with probability about 35%, T2 about 10%, G5 passed about 20%.

What happened

T2 is the best estimator yet on manoeuvring targets.

• Range. 7.8% median against the IMM's 9.2%. The lead grows with time and with each turn: at 20 min after the zig, 10.8% against 16.6%.
• Course. After three turns, 34° against the IMM's 46°.
• Gross errors. 3.5 percentage points fewer than the IMM.
• Its stated uncertainty is honest. 71% of its errors fall within one stated standard deviation (68% expected) and 95% within two (95% expected).

On straight legs it still loses. On settled rows its speed error is 2.16 kt against the IMM's 1.30 kt, and its course error 10.7° against 7.4°. Before the first manoeuvre the gap is widest: 2.0 kt against 0.7 kt.

T1 never learned range. Its range error is 19–22% at every gate time and does not fall after ownship's zig, as a working tracker's does.

• Why. Its gain is expressed along and across the line of sight. The cross term is kept positive, so the bearing loop is stable at any range. The range term has to change sign with the geometry, and the network did not learn that from what it saw.
• The first run diverged in epoch 0. It was fixed before any result was seen, and the fix is recorded as an amendment to the decision.

Reading it critically

• This is the third time the same pattern has appeared. Phases 3, 3b and now 4 each found a network that beats a tuned tracker after manoeuvres and gives accuracy back when the target holds course. More turns per scenario moved the overall checks in the network's favour; they did not touch the straight-leg deficit.
• The settled gap is mostly speed. A Kalman filter on a straight leg gets velocity almost for free from a long, consistent bearing history. The networks have the same history but do not match it, even with a speed loss weight of 4.
• A stronger straight-leg reference exists. Batch least squares reaches 6.7% settled range, better than both the IMM and T2. The settled checks compare against a reference that is not the best achievable there.
• The gate asked for dominance, not a trade. On a mix of manoeuvring and straight targets, T2 is better overall and worse when settled. Whether that trade is useful is an operational question that G5 deliberately did not answer.
• Training was slower and less robust than planned. There were four restarts: a diverging filter, a 2 GiB file limit, a refused cross-commit resume and a lost CI runner. None changed a result. T2's fresh run reproduced its first run's epochs to the last digit.

What next

SRD Annex C leaves the follow-on to the owner if G5 is not passed. Options include:

• A hybrid. T2's manoeuvre advantage switched in only when it reports high uncertainty or a manoeuvre, with the tracker otherwise, as G3's hybrid did for classical estimates.
• A narrower question. Manoeuvre detection and recovery only, where the network already wins.
• Stopping here and writing up the programme.

Real or recorded data stay deferred.