Results: iteration 2c (G2, final attempt)
Question: do the two levers that each moved speed on validation reach the G2 speed threshold together? Iteration 2c (spec 005) combines the speed-weighted loss of iteration 2b with the capacity the scan pointed to (two stacked 128-unit recurrent layers, from scratch) and judges G2 on a third fresh test split (seed 203). It is the last iteration the SRD allows under G2 (§7, amended 2026-10-15 before 2b's test scoring).
The carried set is the 2c models, declared before training; the 2b models are scored beside them on the same split and reported, never gated. Thresholds, slice, coverage and the pass / fail / inconclusive rule are as on the iteration-2 page.
Speed fails for both models, by less again. The instantaneous model is at 1.47 kt [1.37, 1.52] and the time-late model at 1.53 kt [1.47, 1.61], against 1 kt; the 2b models on the same scenarios are at 1.78 kt and 1.77 kt, and classical at 0.61 kt, at the Cramér–Rao bound.
Range and course are the best of the project and pass for both models: 13.5% / 7.36° instantaneous and 12.1% / 6.12° time-late. The time-late model is now better than the instantaneous one on range and course, and worse on speed.
Full record:
review.md,
the validation note
validation.md
and the no-go write-up.
Experiment 20261016-i2d-g2 (attempt 3; earlier attempts below): machine verdict G2 fail (agent review panel, round 2 (no rejections; no open blocker or major findings): G2 FAIL (SRD §5); speed fails for both variants (CI wholly above 1 kt: 1.47 kt [1.37, 1.52] and 1.53 kt [1.47, 1.61]); range and course pass for both; iteration 2c is the last allowed under G2 (SRD §7): no-go; see review.md).
| Check | Result | Detail |
|---|---|---|
g2-instantaneous-range | pass | instantaneous lstm-instantaneous median range error 0.1348 vs threshold 0.2 on sigma=1deg at zig+300s; 95% CI [0.1295, 0.1385] (bootstrap, 2000 resamples, n 4689); model n 4689, excluded 339 (93.3% of the slice reach zig+300s; decision 13); naive 0.3718 (n 4689, excluded 339), classical 0.0876 (n 4689, excluded 339) |
g2-instantaneous-course | pass | instantaneous lstm-instantaneous median course error 7.3552° vs threshold 10° on sigma=1deg at zig+300s; 95% CI [7.0453, 7.7018] (bootstrap, 2000 resamples, n 4689); model n 4689, excluded 339 (93.3% of the slice reach zig+300s; decision 13); naive 90.4661 (n 4689, excluded 339), classical 3.4095 (n 4689, excluded 339) |
g2-instantaneous-speed | fail | instantaneous lstm-instantaneous median speed error 1.4654 kt vs threshold 1 kt on sigma=1deg at zig+600s; 95% CI [1.3731, 1.5225] (bootstrap, 2000 resamples, n 3038); model n 3038, excluded 1990 (60.4% of the slice reach zig+600s; decision 13); naive 4.4366 (n 3038, excluded 1990), classical 0.6132 (n 3038, excluded 1990) |
g2-time-late-range | pass | time-late bilstm-time-late median range error 0.1210 vs threshold 0.2 on sigma=1deg at zig+300s; 95% CI [0.1169, 0.1260] (bootstrap, 2000 resamples, n 4689); model n 4689, excluded 339 (93.3% of the slice reach zig+300s; decision 13); naive 0.3718 (n 4689, excluded 339), classical 0.0876 (n 4689, excluded 339) |
g2-time-late-course | pass | time-late bilstm-time-late median course error 6.1210° vs threshold 10° on sigma=1deg at zig+300s; 95% CI [5.8716, 6.4076] (bootstrap, 2000 resamples, n 4689); model n 4689, excluded 339 (93.3% of the slice reach zig+300s; decision 13); naive 90.4661 (n 4689, excluded 339), classical 3.4095 (n 4689, excluded 339) |
g2-time-late-speed | fail | time-late bilstm-time-late median speed error 1.5298 kt vs threshold 1 kt on sigma=1deg at zig+600s; 95% CI [1.4680, 1.6117] (bootstrap, 2000 resamples, n 3038); model n 3038, excluded 1990 (60.4% of the slice reach zig+600s; decision 13); naive 4.4366 (n 3038, excluded 1990), classical 0.6132 (n 3038, excluded 1990) |
g1-instantaneous | pass | lstm-instantaneous median range error 0.1455 vs limit 0.1881 (0.5 × naive 0.3761) at zig+300s; n 9340 (naive 9340), excluded 660 (naive 660); model/naive 0.387, 95% CI [0.375, 0.397] (paired bootstrap, 2000 resamples, n 9340) |
g1-time-late | pass | bilstm-time-late median range error 0.1306 vs limit 0.1881 (0.5 × naive 0.3761) at zig+300s; n 9340 (naive 9340), excluded 660 (naive 660); model/naive 0.347, 95% CI [0.336, 0.357] (paired bootstrap, 2000 resamples, n 9340) |
time-late-not-worse | fail | 3/19 points worse: t=1020s 0.1313 > 0.1195; t=1080s 0.1325 > 0.1236; t=1140s 0.1361 > 0.1284 (4 later points with n < 500 reported, not gated) |
instantaneous-beats-control | pass | lstm-instantaneous vs cnn-causal-control median range error — zig+300s: 0.1455 vs 0.2838, final: 0.1372 vs 0.2904 |
time-late-beats-control | pass | bilstm-time-late vs mlp-control median range error — zig+300s: 0.1306 vs 0.2733, final: 0.1402 vs 0.2861 |
shuffled-labels-at-naive | pass | lstm-instantaneous-shuffled median range error 0.3863 = 1.027 × naive 0.3761 (allowed 1 ± 0.1) at zig+300s; course error median 77.6° vs naive 90.4° (reported, not gated) |
Model / naive at the G1 point
Median range error of each model divided by naive's, 5 minutes after the zig, with a paired bootstrap 95% interval. G1 passes only if the whole interval is below 0.5; an interval that straddles 0.5 is inconclusive.
| Model | Ratio | 95% interval | n |
|---|---|---|---|
classical | 0.328 | [0.314, 0.343] | 9340 |
bilstm-time-late | 0.347 | [0.336, 0.357] | 9340 |
cnn-causal-control | 0.755 | [0.738, 0.772] | 9340 |
lstm-instantaneous | 0.387 | [0.375, 0.397] | 9340 |
lstm-instantaneous-shuffled | 1.027 | [1.012, 1.042] | 9340 |
mlp-control | 0.727 | [0.709, 0.745] | 9340 |
Error against time since the zig
Dashed lines are the naive and classical baselines; test split, all scenarios.
- ━ naive
- ━ classical
- ━ bilstm-time-late
- ━ cnn-causal-control
- ━ lstm-instantaneous
- ━ lstm-instantaneous-shuffled
- ━ mlp-control
Metrics by noise slice and point
Brackets: bootstrap 95% interval of the median (σ and overall slices at zig + 300 s and zig + 600 s).
| Estimator | n | Median range error | P90 range error | Median course error | Median speed error | Gross-error rate |
|---|---|---|---|---|---|---|
naive | 4689 | 37.2% [36.3%, 38.3%] | 145.4% | 90.5° [87.5°, 92.9°] | 4.36 kt [4.22 kt, 4.47 kt] | 33.7% |
classical | 4689 | 8.8% [8.3%, 9.1%] | 54.7% | 3.4° [3.2°, 3.6°] | 1.22 kt [1.16 kt, 1.29 kt] | 11.2% |
bilstm-time-late | 4689 | 12.1% [11.7%, 12.6%] | 35.6% | 6.1° [5.9°, 6.4°] | 1.67 kt [1.63 kt, 1.74 kt] | 4.7% |
cnn-causal-control | 4689 | 27.8% [27.0%, 28.4%] | 83.5% | 22.4° [21.6°, 23.1°] | 3.76 kt [3.64 kt, 3.88 kt] | 20.7% |
lstm-instantaneous | 4689 | 13.5% [12.9%, 13.9%] | 36.6% | 7.4° [7.0°, 7.7°] | 1.80 kt [1.74 kt, 1.87 kt] | 4.8% |
lstm-instantaneous-shuffled | 4689 | 38.3% [37.3%, 39.2%] | 116.5% | 77.5° [74.7°, 80.4°] | 4.34 kt [4.23 kt, 4.46 kt] | 32.8% |
mlp-control | 4689 | 27.1% [26.4%, 27.8%] | 65.8% | 17.0° [16.5°, 17.7°] | 3.28 kt [3.20 kt, 3.37 kt] | 18.8% |
By slice
Median range error by slice:
| Estimator | 0-5pct | 5-10pct |
|---|---|---|
naive | 37.7% n 4814 | 37.6% n 4526 |
classical | 12.2% n 4814 | 12.5% n 4526 |
bilstm-time-late | 13.2% n 4814 | 12.9% n 4526 |
cnn-causal-control | 28.6% n 4814 | 28.1% n 4526 |
lstm-instantaneous | 14.9% n 4814 | 14.2% n 4526 |
lstm-instantaneous-shuffled | 38.5% n 4814 | 38.7% n 4526 |
mlp-control | 27.5% n 4814 | 27.1% n 4526 |
Held-out geometry (reported, not gated)
| Estimator | n | Median range error | P90 range error | Median course error | Median speed error | Gross-error rate |
|---|---|---|---|---|---|---|
naive | 9303 | 59.6% | 69.9% | 90.0° | 4.24 kt | 76.2% |
classical | 9303 | 48.8% | ∞ | 50.9° | 9.21 kt | 49.3% |
bilstm-time-late | 9303 | 33.8% | 53.0% | 26.6° | 3.49 kt | 14.6% |
cnn-causal-control | 9303 | 52.7% | 64.6% | 53.2° | 4.07 kt | 59.4% |
lstm-instantaneous | 9303 | 35.4% | 55.3% | 29.7° | 3.52 kt | 17.9% |
lstm-instantaneous-shuffled | 9303 | 64.6% | 73.8% | 87.1° | 4.26 kt | 84.0% |
mlp-control | 9303 | 51.4% | 66.8% | 45.4° | 3.94 kt | 53.6% |
Earlier attempts
Kept on record; the latest attempt is shown above.
| Experiment | Machine verdict | Decided by | Model / naive, 95% interval |
|---|---|---|---|
20261015-i2c-g2 (review) | fail | agent review panel, round 2 (no rejections; no open blocker or major findings): G2 FAIL (SRD §5); speed fails for both variants (CI wholly above 1 kt: 1.77 kt [1.68, 1.83] and 1.77 kt [1.67, 1.86]); range and course pass for both; see review.md | classical 0.316 [0.303, 0.331]bilstm-time-late 0.430 [0.419, 0.443]cnn-causal-control 0.739 [0.721, 0.758]lstm-instantaneous 0.418 [0.405, 0.432]lstm-instantaneous-shuffled 1.000 [0.989, 1.016]mlp-control 0.687 [0.672, 0.707] |
20261014-i2-g2 (review) | fail | agent review panel, round 2 (no rejections; no open blocker or major findings): G2 FAIL (SRD §5); speed fails for both variants (CI wholly above 1 kt), time-late course and g1-time-late also fail; see review.md | classical 0.330 [0.316, 0.344]bilstm-time-late 0.514 [0.500, 0.529]cnn-causal-control 0.747 [0.732, 0.766]lstm-instantaneous 0.472 [0.458, 0.484]lstm-instantaneous-shuffled 1.015 [1.001, 1.029]mlp-control 0.703 [0.686, 0.720] |