Skip to main content

Results: iteration 2c (G2, final attempt)

Question: do the two levers that each moved speed on validation reach the G2 speed threshold together? Iteration 2c (spec 005) combines the speed-weighted loss of iteration 2b with the capacity the scan pointed to (two stacked 128-unit recurrent layers, from scratch) and judges G2 on a third fresh test split (seed 203). It is the last iteration the SRD allows under G2 (§7, amended 2026-10-15 before 2b's test scoring).

The carried set is the 2c models, declared before training; the 2b models are scored beside them on the same split and reported, never gated. Thresholds, slice, coverage and the pass / fail / inconclusive rule are as on the iteration-2 page.

G2 failed; the project stops at G2 (agent review panel, round 2)

Speed fails for both models, by less again. The instantaneous model is at 1.47 kt [1.37, 1.52] and the time-late model at 1.53 kt [1.47, 1.61], against 1 kt; the 2b models on the same scenarios are at 1.78 kt and 1.77 kt, and classical at 0.61 kt, at the Cramér–Rao bound.

Range and course are the best of the project and pass for both models: 13.5% / 7.36° instantaneous and 12.1% / 6.12° time-late. The time-late model is now better than the instantaneous one on range and course, and worse on speed.

Full record: review.md, the validation note validation.md and the no-go write-up.

Experiment 20261016-i2d-g2 (attempt 3; earlier attempts below): machine verdict G2 fail (agent review panel, round 2 (no rejections; no open blocker or major findings): G2 FAIL (SRD §5); speed fails for both variants (CI wholly above 1 kt: 1.47 kt [1.37, 1.52] and 1.53 kt [1.47, 1.61]); range and course pass for both; iteration 2c is the last allowed under G2 (SRD §7): no-go; see review.md).

Decided by agent review panel, round 2 (no rejections; no open blocker or major findings): G2 FAIL (SRD §5); speed fails for both variants (CI wholly above 1 kt: 1.47 kt [1.37, 1.52] and 1.53 kt [1.47, 1.61]); range and course pass for both; iteration 2c is the last allowed under G2 (SRD §7): no-go; see review.md. Panel record: review.md.
CheckResultDetail
g2-instantaneous-rangepassinstantaneous lstm-instantaneous median range error 0.1348 vs threshold 0.2 on sigma=1deg at zig+300s; 95% CI [0.1295, 0.1385] (bootstrap, 2000 resamples, n 4689); model n 4689, excluded 339 (93.3% of the slice reach zig+300s; decision 13); naive 0.3718 (n 4689, excluded 339), classical 0.0876 (n 4689, excluded 339)
g2-instantaneous-coursepassinstantaneous lstm-instantaneous median course error 7.3552° vs threshold 10° on sigma=1deg at zig+300s; 95% CI [7.0453, 7.7018] (bootstrap, 2000 resamples, n 4689); model n 4689, excluded 339 (93.3% of the slice reach zig+300s; decision 13); naive 90.4661 (n 4689, excluded 339), classical 3.4095 (n 4689, excluded 339)
g2-instantaneous-speedfailinstantaneous lstm-instantaneous median speed error 1.4654 kt vs threshold 1 kt on sigma=1deg at zig+600s; 95% CI [1.3731, 1.5225] (bootstrap, 2000 resamples, n 3038); model n 3038, excluded 1990 (60.4% of the slice reach zig+600s; decision 13); naive 4.4366 (n 3038, excluded 1990), classical 0.6132 (n 3038, excluded 1990)
g2-time-late-rangepasstime-late bilstm-time-late median range error 0.1210 vs threshold 0.2 on sigma=1deg at zig+300s; 95% CI [0.1169, 0.1260] (bootstrap, 2000 resamples, n 4689); model n 4689, excluded 339 (93.3% of the slice reach zig+300s; decision 13); naive 0.3718 (n 4689, excluded 339), classical 0.0876 (n 4689, excluded 339)
g2-time-late-coursepasstime-late bilstm-time-late median course error 6.1210° vs threshold 10° on sigma=1deg at zig+300s; 95% CI [5.8716, 6.4076] (bootstrap, 2000 resamples, n 4689); model n 4689, excluded 339 (93.3% of the slice reach zig+300s; decision 13); naive 90.4661 (n 4689, excluded 339), classical 3.4095 (n 4689, excluded 339)
g2-time-late-speedfailtime-late bilstm-time-late median speed error 1.5298 kt vs threshold 1 kt on sigma=1deg at zig+600s; 95% CI [1.4680, 1.6117] (bootstrap, 2000 resamples, n 3038); model n 3038, excluded 1990 (60.4% of the slice reach zig+600s; decision 13); naive 4.4366 (n 3038, excluded 1990), classical 0.6132 (n 3038, excluded 1990)
g1-instantaneouspasslstm-instantaneous median range error 0.1455 vs limit 0.1881 (0.5 × naive 0.3761) at zig+300s; n 9340 (naive 9340), excluded 660 (naive 660); model/naive 0.387, 95% CI [0.375, 0.397] (paired bootstrap, 2000 resamples, n 9340)
g1-time-latepassbilstm-time-late median range error 0.1306 vs limit 0.1881 (0.5 × naive 0.3761) at zig+300s; n 9340 (naive 9340), excluded 660 (naive 660); model/naive 0.347, 95% CI [0.336, 0.357] (paired bootstrap, 2000 resamples, n 9340)
time-late-not-worsefail3/19 points worse: t=1020s 0.1313 > 0.1195; t=1080s 0.1325 > 0.1236; t=1140s 0.1361 > 0.1284 (4 later points with n < 500 reported, not gated)
instantaneous-beats-controlpasslstm-instantaneous vs cnn-causal-control median range error — zig+300s: 0.1455 vs 0.2838, final: 0.1372 vs 0.2904
time-late-beats-controlpassbilstm-time-late vs mlp-control median range error — zig+300s: 0.1306 vs 0.2733, final: 0.1402 vs 0.2861
shuffled-labels-at-naivepasslstm-instantaneous-shuffled median range error 0.3863 = 1.027 × naive 0.3761 (allowed 1 ± 0.1) at zig+300s; course error median 77.6° vs naive 90.4° (reported, not gated)

Model / naive at the G1 point

Median range error of each model divided by naive's, 5 minutes after the zig, with a paired bootstrap 95% interval. G1 passes only if the whole interval is below 0.5; an interval that straddles 0.5 is inconclusive.

ModelRatio95% intervaln
classical0.328[0.314, 0.343]9340
bilstm-time-late0.347[0.336, 0.357]9340
cnn-causal-control0.755[0.738, 0.772]9340
lstm-instantaneous0.387[0.375, 0.397]9340
lstm-instantaneous-shuffled1.027[1.012, 1.042]9340
mlp-control0.727[0.709, 0.745]9340

Error against time since the zig

Dashed lines are the naive and classical baselines; test split, all scenarios.

0.0%13.6%27.1%40.7%54.2%05101520minutes since the zig
  • ━ naive
  • ━ classical
  • ━ bilstm-time-late
  • ━ cnn-causal-control
  • ━ lstm-instantaneous
  • ━ lstm-instantaneous-shuffled
  • ━ mlp-control

Metrics by noise slice and point

Brackets: bootstrap 95% interval of the median (σ and overall slices at zig + 300 s and zig + 600 s).

EstimatornMedian range errorP90 range errorMedian course errorMedian speed errorGross-error rate
naive468937.2% [36.3%, 38.3%]145.4%90.5° [87.5°, 92.9°]4.36 kt [4.22 kt, 4.47 kt]33.7%
classical46898.8% [8.3%, 9.1%]54.7%3.4° [3.2°, 3.6°]1.22 kt [1.16 kt, 1.29 kt]11.2%
bilstm-time-late468912.1% [11.7%, 12.6%]35.6%6.1° [5.9°, 6.4°]1.67 kt [1.63 kt, 1.74 kt]4.7%
cnn-causal-control468927.8% [27.0%, 28.4%]83.5%22.4° [21.6°, 23.1°]3.76 kt [3.64 kt, 3.88 kt]20.7%
lstm-instantaneous468913.5% [12.9%, 13.9%]36.6%7.4° [7.0°, 7.7°]1.80 kt [1.74 kt, 1.87 kt]4.8%
lstm-instantaneous-shuffled468938.3% [37.3%, 39.2%]116.5%77.5° [74.7°, 80.4°]4.34 kt [4.23 kt, 4.46 kt]32.8%
mlp-control468927.1% [26.4%, 27.8%]65.8%17.0° [16.5°, 17.7°]3.28 kt [3.20 kt, 3.37 kt]18.8%

By slice

Median range error by slice:

Estimator0-5pct5-10pct
naive37.7% n 481437.6% n 4526
classical12.2% n 481412.5% n 4526
bilstm-time-late13.2% n 481412.9% n 4526
cnn-causal-control28.6% n 481428.1% n 4526
lstm-instantaneous14.9% n 481414.2% n 4526
lstm-instantaneous-shuffled38.5% n 481438.7% n 4526
mlp-control27.5% n 481427.1% n 4526

Held-out geometry (reported, not gated)

EstimatornMedian range errorP90 range errorMedian course errorMedian speed errorGross-error rate
naive930359.6%69.9%90.0°4.24 kt76.2%
classical930348.8%∞50.9°9.21 kt49.3%
bilstm-time-late930333.8%53.0%26.6°3.49 kt14.6%
cnn-causal-control930352.7%64.6%53.2°4.07 kt59.4%
lstm-instantaneous930335.4%55.3%29.7°3.52 kt17.9%
lstm-instantaneous-shuffled930364.6%73.8%87.1°4.26 kt84.0%
mlp-control930351.4%66.8%45.4°3.94 kt53.6%

Earlier attempts

Kept on record; the latest attempt is shown above.

ExperimentMachine verdictDecided byModel / naive, 95% interval
20261015-i2c-g2 (review)failagent review panel, round 2 (no rejections; no open blocker or major findings): G2 FAIL (SRD §5); speed fails for both variants (CI wholly above 1 kt: 1.77 kt [1.68, 1.83] and 1.77 kt [1.67, 1.86]); range and course pass for both; see review.md
classical 0.316 [0.303, 0.331]
bilstm-time-late 0.430 [0.419, 0.443]
cnn-causal-control 0.739 [0.721, 0.758]
lstm-instantaneous 0.418 [0.405, 0.432]
lstm-instantaneous-shuffled 1.000 [0.989, 1.016]
mlp-control 0.687 [0.672, 0.707]
20261014-i2-g2 (review)failagent review panel, round 2 (no rejections; no open blocker or major findings): G2 FAIL (SRD §5); speed fails for both variants (CI wholly above 1 kt), time-late course and g1-time-late also fail; see review.md
classical 0.330 [0.316, 0.344]
bilstm-time-late 0.514 [0.500, 0.529]
cnn-causal-control 0.747 [0.732, 0.766]
lstm-instantaneous 0.472 [0.458, 0.484]
lstm-instantaneous-shuffled 1.015 [1.001, 1.029]
mlp-control 0.703 [0.686, 0.720]