{"explainer":"running_accountability tracks every archived shutdown_risk prediction (one per country per day, from archive-shutdown-risk-history.py) against the actual KeepItOn-documented shutdowns that followed in the 7 days after. early days have small sample size; validated_backtest contains the v4 trainer's static AUC-0.90 result (full-panel).","generated_at":"2026-10-07T06:31:06.891301+00:00","running_accountability":{"all_time":{"flag_rate":0.0432,"n_actual_shutdowns_following":0,"n_false_negatives":0,"n_false_positives":249,"n_flagged":249,"n_predictions":5764,"n_true_positives":0,"precision":0.0,"recall":null},"first_prediction_date":"2025-09-26","last_30_days":{"flag_rate":0.0369,"n_actual_shutdowns_following":0,"n_false_negatives":0,"n_false_positives":44,"n_flagged":44,"n_predictions":1192,"n_true_positives":0,"precision":0.0,"recall":null},"last_7_days":{"flag_rate":0.0396,"n_actual_shutdowns_following":0,"n_false_negatives":0,"n_false_positives":9,"n_flagged":9,"n_predictions":227,"n_true_positives":0,"precision":0.0,"recall":null},"latest_prediction_date":"2026-10-05","n_distinct_countries":60},"schema":"voidly-shutdown-risk-accountability/v1","validated_backtest":{"features":["kio_hist_5y","kio_recent_7d"],"honest_caveats":["Real KeepItOn-trained predictor. Country-level only.","Differentiates BETWEEN countries (the bulk of the AUC) much more than WITHIN a country day-to-day.","Score grows over time as new shutdowns are documented; the model is most reliable for countries with established history.","City-level / sub-national shutdowns (~half of KeepItOn events) are not strongly predictable from country aggregates.","Three prior versions (HistGB, LR, isotonic w/ temporal CV) lost this 0.89 signal to overfitting / non-stationary clipping; v4 is the minimal scorer that preserves it."],"honest_validation":{"auc_full_panel":0.8995877860609838,"combined":{"auc":0.8995877860609838,"pr_auc":0.1724620453384445},"comment":"These are full-panel AUCs because the predictor is essentially a country base-rate that grows over time as new shutdowns are documented. The 0.89 expresses how well the score RANKS country-days by true future shutdown probability. The model differentiates BETWEEN countries far more than WITHIN a country.","method":"score formula evaluated on the full labeled panel (no fold splits \u2014 the feature is non-stationary; rolling-origin clipping pathology made v1/v2/v3 underperform their own input)","pr_auc_full_panel":0.1724620453384445,"raw_kio_hist_5y":{"auc":0.8980404532784527,"pr_auc":0.16164008309977493},"raw_kio_recent_7d":{"auc":0.6828148774707113,"pr_auc":0.19469618195045582}},"model_kind":"isotonic_calibrated_score","n_countries":60,"n_train":103670,"recommended_threshold":0.15022908041650987,"score_formula":"combined_h5_plus_2x_r7","threshold_metrics":{"f1":0.36,"precision":0.35294117647058826,"recall":0.3673469387755102},"train_positive_rate":0.01087103308575287,"trained_at":"2026-10-07T06:30:20.783723+00:00","version":"shutdown_risk_v4"}}