Results: iteration 1 (G1)

Question: is there a learnable signal? G1 passes if both the instantaneous and time-late models' median range error, 5 minutes after the zig, is below half of the naive baseline's. The models page describes the four architectures; the evaluation page defines the metrics.

With the experiment's paired bootstrap (2,000 resamples of the test scenarios), each G1 check passes only if the whole 95% interval of model/naive is below 0.5, fails if it is wholly above, and is otherwise inconclusive.

Additional checks, recorded in the verdict:

• time-late-not-worse: the time-late model is never more than 5% worse than the instantaneous one at any point on the curve (it sees strictly more data).
• *-beats-control: each recurrent model beats its control (causal CNN, MLP) at 5 minutes after the zig and at the end of the window.
• shuffled-labels-at-naive: a model trained on shuffled labels scores within ±10% of naive.

All numbers below are from the committed experiment folder, generated at site build time. The latest attempt is shown in full; earlier attempts stay on record and are listed at the end. A metric shown as ∞ is a percentile that falls on a non-converged estimate (counted as +∞).

G1 passed (panel round 3) with a caveat

The latest experiment, 20261012-g1-r2, re-scores the time-late model after its one pre-registered continuation to early stopping (50 epochs, best 46). Its machine verdict is inconclusive because time-late-not-worse fails; both ratio intervals lie wholly below 0.5 and the shuffled-label check passes, so the agent review panel passed G1 under pre-registered rule 1 (SRD §5, amended 2026-10-12). Caveat: time-late is not shown better than instantaneous; it is measurably worse (1.075 [1.046, 1.108] × instantaneous at zig + 300 s). Full record: review.md and the write-up. Attempt 1 (20261011-g1-r2, time-late stopped on the wall-clock cap after 25 of 60 epochs) stays on record below.

Experiment 20261012-g1-r2 (attempt 3; earlier attempts below): machine verdict G1 inconclusive (agent review panel, round 3 (no rejections; no open blocker or major findings): G1 PASS under pre-registered rule 1 (SRD §5, amended 2026-10-12); machine result inconclusive from time-late-not-worse, caveat recorded; see review.md).

Decided by agent review panel, round 3 (no rejections; no open blocker or major findings): G1 PASS under pre-registered rule 1 (SRD §5, amended 2026-10-12); machine result inconclusive from time-late-not-worse, caveat recorded; see review.md. Panel record: review.md.

Check	Result	Detail
g1-instantaneous	pass	lstm-instantaneous median range error 0.1609 vs limit 0.1920 (0.5 × naive 0.3840) at zig+300s; n 9784 (naive 9784), excluded 216 (naive 216); model/naive 0.419, 95% CI [0.407, 0.431] (paired bootstrap, 2000 resamples, n 9784)
g1-time-late	pass	bilstm-time-late median range error 0.1730 vs limit 0.1920 (0.5 × naive 0.3840) at zig+300s; n 9784 (naive 9784), excluded 216 (naive 216); model/naive 0.450, 95% CI [0.438, 0.464] (paired bootstrap, 2000 resamples, n 9784)
time-late-not-worse	fail	14/20 points worse: t=120s 0.2111 > 0.1991; t=180s 0.1889 > 0.1787; t=240s 0.1788 > 0.1670; t=300s 0.1730 > 0.1609; t=360s 0.1677 > 0.1570 (3 later points with n < 500 reported, not gated)
instantaneous-beats-control	pass	lstm-instantaneous vs cnn-causal-control median range error — zig+300s: 0.1609 vs 0.2927, final: 0.1642 vs 0.2934
time-late-beats-control	pass	bilstm-time-late vs mlp-control median range error — zig+300s: 0.1730 vs 0.2631, final: 0.1847 vs 0.2823
shuffled-labels-at-naive	pass	lstm-instantaneous-shuffled median range error 0.3860 = 1.005 × naive 0.3840 (allowed 1 ± 0.1) at zig+300s

Model / naive at the G1 point

Median range error of each model divided by naive's, 5 minutes after the zig, with a paired bootstrap 95% interval. G1 passes only if the whole interval is below 0.5; an interval that straddles 0.5 is inconclusive.

Model	Ratio	95% interval	n
bilstm-time-late	0.450	[0.438, 0.464]	9784
lstm-instantaneous	0.419	[0.407, 0.431]	9784

Error against time since the zig

Dashed lines are the naive and classical baselines; test split, all scenarios.

Metric: Median range errorP90 range errorMedian course errorMedian speed errorGross-error rate

• ━ naive
• ━ classical
• ━ bilstm-time-late
• ━ cnn-causal-control
• ━ lstm-instantaneous
• ━ lstm-instantaneous-shuffled
• ━ mlp-control

Overall, 5 minutes after the zig (the G1 point)

Estimator	n	Median range error	P90 range error	Median course error	Median speed error	Gross-error rate
naive	9784	38.4%	161.5%	90.0°	4.30 kt	35.7%
classical	9784	4.6%	29.2%	1.9°	0.64 kt	5.7%
bilstm-time-late	9784	17.3%	43.5%	9.6°	2.39 kt	7.3%
cnn-causal-control	9784	29.3%	69.7%	23.0°	3.81 kt	20.1%
lstm-instantaneous	9784	16.1%	39.6%	9.4°	2.16 kt	5.4%
lstm-instantaneous-shuffled	9784	38.6%	122.6%	82.9°	4.33 kt	34.9%
mlp-control	9784	26.3%	75.7%	16.8°	3.39 kt	18.7%

Overall, end of window

Estimator	n	Median range error	P90 range error	Median course error	Median speed error	Gross-error rate
naive	10000	40.9%	121.1%	89.9°	4.30 kt	38.1%
classical	10000	2.4%	18.7%	0.9°	0.33 kt	3.1%
bilstm-time-late	10000	18.5%	44.2%	8.6°	2.27 kt	7.3%
cnn-causal-control	10000	29.3%	79.0%	20.4°	3.87 kt	21.0%
lstm-instantaneous	10000	16.4%	41.4%	8.3°	2.10 kt	6.2%
lstm-instantaneous-shuffled	10000	37.8%	121.9%	82.9°	4.31 kt	33.1%
mlp-control	10000	28.2%	80.8%	17.3°	3.40 kt	20.2%

By slice

Slice: range_bandsigmazig_band Point: finalzig+300s

Median range error by slice:

Estimator	0-2km	10-20km	2-5km	20-30km	30+km	5-10km
naive	603.7% n 132 (too few to judge)	32.5% n 3741	150.8% n 1640	58.5% n 557	71.8% n 9 (too few to judge)	23.2% n 3705
classical	0.7% n 132 (too few to judge)	7.2% n 3741	2.0% n 1640	10.4% n 557	8.2% n 9 (too few to judge)	3.8% n 3705
bilstm-time-late	46.2% n 132 (too few to judge)	15.6% n 3741	21.2% n 1640	27.5% n 557	30.3% n 9 (too few to judge)	15.8% n 3705
cnn-causal-control	76.5% n 132 (too few to judge)	27.4% n 3741	46.5% n 1640	48.3% n 557	60.2% n 9 (too few to judge)	22.9% n 3705
lstm-instantaneous	29.9% n 132 (too few to judge)	16.2% n 3741	17.2% n 1640	28.7% n 557	27.0% n 9 (too few to judge)	13.6% n 3705
lstm-instantaneous-shuffled	457.2% n 132 (too few to judge)	39.9% n 3741	112.7% n 1640	62.6% n 557	73.2% n 9 (too few to judge)	18.8% n 3705
mlp-control	92.0% n 132 (too few to judge)	22.4% n 3741	45.5% n 1640	38.8% n 557	43.3% n 9 (too few to judge)	24.0% n 3705

Held-out geometry (reported, not gated)

Estimator	n	Median range error	P90 range error	Median course error	Median speed error	Gross-error rate
naive	9769	59.7%	70.2%	89.9°	4.25 kt	77.6%
classical	9769	18.8%	∞	16.3°	3.95 kt	23.9%
bilstm-time-late	9769	43.9%	58.5%	34.5°	3.51 kt	31.1%
cnn-causal-control	9769	55.0%	67.0%	52.1°	4.12 kt	65.5%
lstm-instantaneous	9769	42.4%	57.6%	34.9°	3.56 kt	28.5%
lstm-instantaneous-shuffled	9769	65.3%	74.5%	88.9°	4.25 kt	86.3%
mlp-control	9769	48.7%	62.5%	46.6°	4.07 kt	46.2%

Earlier attempts

Kept on record; the latest attempt is shown above.

Experiment	Machine verdict	Decided by	Model / naive, 95% interval
20261012-g1	fail	informational only: the training run's own scoring (train/i1 code, no bootstrap); verdict of record is ../20261012-g1-r2 (agent review panel, round 3: G1 pass)	–
20261011-g1-r2 (review)	fail	agent review panel, rounds 1-2: fail as run (time-late undertrained, 25 of 60 epochs); one continuation pre-registered under SRD §5 (amended 2026-10-12), decided in ../20261012-g1-r2/review.md	bilstm-time-late 0.551 [0.538, 0.565] lstm-instantaneous 0.419 [0.407, 0.431]