← Research

Shrinkage-constant reliability study

Empirical replacement for the ad-hoc pseudo-observation constants in project_rosters_2026_27.py (PSEUDO_GP=25, PSEUDO_STARTS=80) and the Chunk-D shooter-GAx shrink. Two independent methods; the walk-forward predictive sweep is the decider. All rates are shrunk as sum / (n + k) -- k is "pseudo-observations of league-average (zero)".

Data: relwpa_v3_by_game*.csv (16 seasons, skaters only -- goalies removed via player_bio.csv pos=G plus a TOI/GP>2700s fallback), shooters_gax_by_season_v2.csv, goalies_gsax_by_season_v2.csv. relwpa is in win-probability percentage points (/100 = wins).

Method 1 -- split-half reliability (skater relWPA)

Player-seasons with >=40 GP, games split odd/even by game_id, per-game mean each half, correlated across 8,805 player-seasons:

quantityvalue
half-vs-half Pearson r0.577
(Spearman)0.537
median GP (full season)67
Spearman-Brown full-season reliability0.732
implied k = m(1-r)/r pseudo-GP24.5

Cross-check for the shot metrics (adjacent-season correlation is an upper bound on k because it also contains true talent drift): GSAx raw per-shot YoY weighted r = 0.112 at median 820 shots -> implied k ~ 6491 shots (231 starts); GAx raw per-SOG YoY weighted r = 0.149 at median 83 SOG -> implied k ~ 474 SOG.

Method 2 -- walk-forward predictive k sweep (the decider)

For each season s >= 2012: predict next-season rate as sum_s/(n_s + k), score vs realized s+1 rate, weighted by s+1 sample size, pooled across all seasons.

Skater relWPA per GP -- single-season predictor (9,775 pairs)

kweighted corrweighted MAE
50.54751.6858
100.55201.6299
150.55371.5957
200.55451.5739
250.55491.5602
300.55501.5518
350.5551 (max)1.5469
400.55501.5447
450.55501.5442 (min)
500.55491.5449
600.55461.5494
750.55421.5593

argmax corr k=35, argmin MAE k=45.

Skater relWPA per GP -- 3-season decayed predictor (decay 0.55, 10,180 pairs)

This variant matches how the projection actually feeds the shrink (recency-decayed multi-season sums), so this k is the one that replaces PSEUDO_GP.

kweighted corrweighted MAE
50.56391.5852
100.56841.5590
150.57011.5432
200.57091.5327
250.57131.5258
300.57141.5214
350.5714 (max)1.5190
400.57141.5180 (min)
450.57131.5180
500.57121.5188
600.57091.5220
750.57031.5293

argmax corr k=35, argmin MAE k=40.

Shooter GAx per SOG (9,574 pairs; MAE per 100 SOG)

kweighted corrweighted MAE
500.21902.516
1000.22392.390
2000.22752.311
3000.22892.292
4000.22972.289 (min)
5000.23012.291
6000.23032.294
8000.2306 (max)2.301

argmax corr k=800, argmin MAE k=400.

Goalie GSAx per shot faced (999 pairs; MAE per 100 shots)

League shots per start (Faced/Games): 28.1.

kweighted corrweighted MAE
5000.14930.948
10000.15160.896
20000.15340.864
30000.15420.855
50000.15500.853 (min)
80000.15540.855
120000.1557 (max)0.857

argmax corr k=12000 shots (= 427 starts), argmin MAE k=5000 shots (= 178 starts).

Goalie rebound control (RebPer100 deviation from season mean, 999 pairs; MAE per 100 shots)

kweighted corrweighted MAE
1000.26970.918
3000.27400.848
5000.2746 (max)0.815
10000.27420.778
20000.27250.765 (min)
30000.27120.767

argmax corr k=500, argmin MAE k=2000.

Recommendations

constantcurrentrecommendedbasis
PSEUDO_GP (skater relWPA)2535decayed-predictor sweep: corr max at k=35, MAE min at k=40; plateau ~25-50. Split-half implies 25. Current 25 sits on the plateau edge -- cost of staying is ~0.0002 corr / +0.007 pct-pt MAE; 35 is the center of the plateau on both criteria.
GAx pseudo-SOG (Chunk D)--500MAE min k=400, corr max k=800 but nearly flat 400-800; the independent YoY-implied k = 474 SOG lands on the same value. Plateau 400-600.
PSEUDO_STARTS (goalie GSAx)80180 (= 5000 shots / 28.1 shots-per-start)MAE min at k=5000 shots; corr still creeping up at 12000 but +0.0007 from 5000; YoY-implied ~6491 shots agrees. Current 80 starts = ~2250 shots under-shrinks: MAE 0.00864 vs 0.00853 per shot, corr 0.1534 vs 0.1550.
RebPer100 pseudo-SOG--500corr max at k=500 (plateau 300-1000). MAE keeps falling toward heavier shrink (deviation target is near zero-mean, so MAE alone rewards over-shrinking); corr is the discriminating criterion here. Note rebound control needs ~10x less shrink than GSAx -- per-shot rebound skill is far more reliable than per-shot save skill, consistent with the xg_v2 finding that goalie persistence concentrates in the rebound game.

Shape of the optima: flat. Every sweep is a broad plateau (relWPA 25-50, GAx 400-600, GSAx 3000-8000 shots, rebounds 300-1000), so the exact integer matters little; what matters is order of magnitude, and the ad-hoc PSEUDO_GP=25 was essentially right while PSEUDO_STARTS=80 was ~2x too trusting of goalie samples.

Cross-method agreement (split-half / YoY-implied vs predictive sweep): skater 25 vs 35, GAx 474 vs 500, GSAx 6491 vs 5000 shots -- two independent derivations land within a factor of ~1.4 everywhere, which is strong evidence the constants are real properties of the metrics, not artifacts of the tuning procedure.

Multi-season note: the 3-season decayed predictor wants the same k as the single-season one (both plateau at 35-50); the decayed effective sample (den = sum of decayed GP, ~1.9x a single season for a full-time player) is what the shrink divides into, so no separate constant is needed -- just keep applying num/(den+k) to the decayed sums as the projection already does.