# Binzagr PoV: model review, before and after (06_Model_Review)

*Version 1, 23 Oct 2026. Site read on 23 Oct 2026 (binzagr.snop.live, Value cockpit to Value roll-up). Data: 02_Cleaned v3, the same files the site uses.*

**Start with `Binzagr_Model_Review_Before_After.pdf`.** Section 2 is the before/after table: every figure to change on the site, page by page, with the text shown now and the text to replace it with. The same table is `data/before_after_figures.csv` / `.json`.

## The answer in five lines
- **Churn:** Extra trees instead of LightGBM. AUC 0.772 to 0.797 on unseen outlets (90% interval +0.011 to +0.040); riskiest 20% catches 59% of quiet outlets instead of 52%.
- **Next-best SKU:** a LightGBM ranker over EASE and RP3beta scores. Picks bought 11.6% (our copy of the site's 12.0%) to 12.4% on the site's own test (+0.5 to +0.9 points at 90%).
- **Cluster adoption:** a logistic regression + XGBoost blend. Within-cluster AUC 0.71 to 0.75 in our copy; the site's 0.90 could not be reproduced with a time-honest split, so check it before the demo.
- **Demand forecast:** one equal blend for all SKUs is 1.4 points better at the same bias (66.9% to 65.5%); keep the site's van, principal and cluster blends; show bias next to every error.
- **Value:** all model upgrades add about SAR 12K a year to the SAR 3.45M (under 1%). The big levers need better data, not better models.

## Types in the before/after table
- **Replace:** measured on the same test after switching the model; change it once the new model is live.
- **Fix:** a wrong or mislabelled figure; change it now.
- **Add:** keep the figure and add the note.
- **Verify first:** lower in our time-honest re-test; re-check the site test before the demo.
- **Recompute:** needs the site pipeline re-run.
- **Keep:** no change.

## Files
| File | What it holds |
|---|---|
| `Binzagr_Model_Review_Before_After.pdf` | The report (9 pages); section 2 = the before/after table |
| `data/before_after_figures.csv`, `data/before_after_figures.json` | 46 rows: page, place on the page, text on the site now, replacement text, type (Replace / Fix / Add / Verify first / Recompute / Keep), basis, numeric before and after |
| `data/forecast_by_level.csv` | Per level: site result and bias, our baselines, our copy of the site rule, the picked rule, error at the site's bias |
| `data/forecast_all_rules.csv` | Every model and combination rule, validation and hold-out error, per level |
| `data/next_best_sku_models.csv` | Every recommender on the site's test (4,097 outlets, picks on 25 Aug) |
| `data/churn_models.csv` | Every churn model and blend: two validation folds, test AUC, riskiest-20% capture |
| `data/adoption_models.csv` | Every adoption model and blend: pooled and within-cluster AUC, top-10% capture |
| `data/next_best_sku_picks_v2.csv` | Ten picks per outlet as of 20 Oct 2026 from the new ranker (4,961 outlets; only SKUs sold somewhere in the last 4 weeks) |
| `data/churn_scores_v2.csv` | Churn risk per outlet as of 20 Oct 2026: Extra trees (v2) beside LightGBM (v1), with High / Medium / Low tiers |
| `scripts/` | The code, in run order: fc_race, fc_eval, reco_race, reco_final_v2, reco_picks_v2, churn_race, churn_final_v2, adoption_race, charts, data_changes, replacements, build_report, check_report |

## How the after figures were reached
- Same data, series, test windows and baselines as the site. Our copy reproduces the site's hold-out sales to the cent and its baselines to within 0.7 points.
- Every model and combination was chosen on validation windows that end before the test; the test was run once.
- Forecasts are compared at the same bias, because weekly error rewards forecasting low.
- The site's foundation models (Chronos-2, TimesFM 2.5, TiRex) could not be run here, so forecast "after" figures use LightGBM and statistical models only.

IDs are text: `outlet_id` = `dist_cd-cust_cd`, `sku` = product code, `main_van` = van code (no rep names). UTF-8 with BOM.

Confidential: contains Binzagr data.