{"as_of":"2026-09-07","combiner":"isotonic(sigmoid(0.6*logit(shutdown_risk_v4_prob) + 0.40*logit(honest_forecast_v1_prob)))","full_panel_auc":0.8921832670927028,"generated_at":"2026-09-08T06:30:21.437527+00:00","honest_caveats":["FORWARD-VALIDATED (the real product): in a strict past->future temporal holdout the CROSS-COUNTRY ranking holds \u2014 full-panel AUC ~0.87, country-ranking AUC ~0.90, 7.8x PR-lift over base rate. The model reliably tells you WHICH countries are heading into danger, forward in time. BUT the within-country median AUC is CROSS-SECTIONAL (full panel, in-sample isotonic) \u2014 it measures day-ranking when the model has seen the whole period, NOT forward skill. In a strict past->future temporal holdout, within-country timing AUC is ~0.38 (below chance), for BOTH v5 and v9, driven by non-stationarity (shutdowns cluster in time). USE THIS MODEL FOR *WHICH* COUNTRY IS AT RISK (cross-country full-panel AUC 0.90, which IS forward-meaningful), NOT as a within-country *WHEN* oracle.","v9 replaces v5's multiplicative product with a logit-additive blend (alpha=0.6 on structural risk). The product collapsed to ~0 when honest_forecast was near-zero for a country, annihilating a good structural signal; the logit blend cannot zero out either factor.","Within-country median AUC 0.705 vs v5 0.729. Improvement confirmed by leave-one-country-out alpha selection (all 23 folds chose alpha=0.60; held-out median +0.0095, mean +0.0131, 18/23 countries up).","Still country-level; sub-national shutdowns (~half of KeepItOn events) remain harder. Low-N countries are sample-size limited, not model-limited.","Same two honestly-validated base models as v5 (v4 structural + honest_forecast_v1). No new external features (unlike the gated v7 CF-Radar and v8 GDELT attempts) \u2014 purely a better combiner."],"honest_validation":{"forward_country_ranking_auc":0.9037267080745341,"forward_temporal_crosscountry_auc":0.8642660481128321,"forward_temporal_crosscountry_pr_lift":9.658019017613755,"forward_temporal_n_countries":14,"forward_temporal_within_median":0.38297032087354665,"full_panel_auc":0.8921832670927028,"full_panel_pr_auc":0.19837636440892042,"method":"full-panel scoring; within-country AUCs for countries with >=5 KeepItOn positive days + both classes; alpha chosen by leave-one-country-out","n_countries_scored_within":23,"n_countries_within_auc_ge_065":13,"n_countries_within_auc_ge_080":1,"per_country":[{"country":"MM","ensemble_auc":0.7905743110225633,"hf_only_auc":0.4435145003152009,"n":1704,"pos":341,"sr_only_auc":0.8095476813911007},{"country":"IN","ensemble_auc":0.7742851235465354,"hf_only_auc":0.3451295195649534,"n":1704,"pos":203,"sr_only_auc":0.8090895068312421},{"country":"RU","ensemble_auc":0.7482419091274545,"hf_only_auc":0.6305633721931889,"n":1704,"pos":121,"sr_only_auc":0.732297708608511},{"country":"PK","ensemble_auc":0.7054094727337401,"hf_only_auc":0.6082031356882918,"n":1704,"pos":115,"sr_only_auc":0.6888390291952827},{"country":"TZ","ensemble_auc":0.5722867513611616,"hf_only_auc":0.5670780399274048,"n":1703,"pos":50,"sr_only_auc":0.5647973381730188},{"country":"IQ","ensemble_auc":0.7825530774897864,"hf_only_auc":0.6930949032214855,"n":1704,"pos":45,"sr_only_auc":0.7299243185319135},{"country":"IR","ensemble_auc":0.727145784596413,"hf_only_auc":0.6239051194167891,"n":1704,"pos":33,"sr_only_auc":0.6095787316613168},{"country":"JO","ensemble_auc":0.47541979202182066,"hf_only_auc":0.5561498465734742,"n":1704,"pos":28,"sr_only_auc":0.4220614558472554},{"country":"TR","ensemble_auc":0.7487534094783498,"hf_only_auc":0.736937436072281,"n":1704,"pos":28,"sr_only_auc":0.641706443914081},{"country":"KE","ensemble_auc":0.7482851473922902,"hf_only_auc":0.560827664399093,"n":1701,"pos":21,"sr_only_auc":0.6642006802721088},{"country":"SD","ensemble_auc":0.400376312141018,"hf_only_auc":0.2358034122740005,"n":1704,"pos":21,"sr_only_auc":0.6511190334719745},{"country":"SY","ensemble_auc":0.5437335148472731,"hf_only_auc":0.440171303780595,"n":1700,"pos":21,"sr_only_auc":0.5829859043081198},{"country":"VE","ensemble_auc":0.7874888921800948,"hf_only_auc":0.9328717417061612,"n":1704,"pos":16,"sr_only_auc":0.4846156694312796},{"country":"BY","ensemble_auc":0.593385460693153,"hf_only_auc":0.4904902789518174,"n":1704,"pos":14,"sr_only_auc":0.5727810650887575},{"country":"GN","ensemble_auc":0.7691481583036286,"hf_only_auc":0.7867653779362032,"n":1577,"pos":14,"sr_only_auc":0.639955214331414},{"country":"ET","ensemble_auc":0.2587759912450543,"hf_only_auc":0.2649212896708477,"n":1704,"pos":7,"sr_only_auc":0.4979375368296995},{"country":"ID","ensemble_auc":0.29552150854449033,"hf_only_auc":0.476807812105396,"n":1704,"pos":7,"sr_only_auc":0.3918680023571008},{"country":"KG","ensemble_auc":0.4699469652327637,"hf_only_auc":0.7038471251788871,"n":1704,"pos":7,"sr_only_auc":0.2451384796700059},{"country":"NG","ensemble_auc":0.7678676656284199,"hf_only_auc":0.78238908999074,"n":1704,"pos":7,"sr_only_auc":0.4979375368296995},{"country":"OM","ensemble_auc":0.03322357311719014,"hf_only_auc":0.021276595744680864,"n":1699,"pos":7,"sr_only_auc":0.49793144208037826},{"country":"TG","ensemble_auc":0.6839814032121725,"hf_only_auc":0.7859678782755706,"n":1697,"pos":7,"sr_only_auc":0.37248520710059174},{"country":"US","ensemble_auc":0.8850913376546847,"hf_only_auc":0.9668322249347588,"n":1704,"pos":7,"sr_only_auc":0.32616381850324105},{"country":"VN","ensemble_auc":0.3462412660998401,"hf_only_auc":0.44911187810421754,"n":1704,"pos":7,"sr_only_auc":0.3635827931644078}],"within_country_auc_mean":0.6046842098986913,"within_country_auc_median":0.7054094727337401,"within_country_is_cross_sectional":true},"label_source":"Access Now #KeepItOn STOP dataset (journalist-verified)","loaded":true,"method":"trained AND validated on real, verified shutdown labels; see scripts/train-shutdown-risk-v4.py for the trainer and scripts/validate-against-keepiton.py for the prior honest cross-target validation","model_version":"shutdown_risk_v9","n_countries":60,"recommended_threshold":0.14,"schema":"voidly-shutdown-risk-info/v4","score_formula":"isotonic(sigmoid(0.6*logit(shutdown_risk_v4_prob) + 0.40*logit(honest_forecast_v1_prob)))","within_country_median":0.7054094727337401}