# Payments Stat Hub — plan

Indian digital payments statistics hub. Official sources only (NPCI + RBI), one
canonical DuckDB, published as a zo.space site + GitHub repo. Consumer-desk angle:
not just headline volumes — make bank-wise shares, reliability, and cross-rail
trends legible.

**Decision log**
- 2026-09-18: Plan approved by user. ThinkPad browser available as Indian-vantage
  fallback; used only where sandbox is blocked (CIMS). NPCI + RBI legacy pages work
  from sandbox via agent-browser.
- Sources verified live 2026-09-18 — see `SOURCES.md`.

**North star**: `/payments` on zo.space + open-data repo. Every number on the page
traceable to source URL + extraction date.

## Phase 0 — Verification sprint (DONE 2026-09-18)
- [x] NPCI SPA → found underlying Strapi JSON API; tab-detail endpoint returns
      self-describing schema; full UPI history pulls per-FY (verified 2016-17).
- [x] Tab inventory captured: 52 tabs / 9 products.
- [x] RBI legacy .aspx sources live, curl 200; data behind postback → browser fetch.
- [x] CIMS geo-fenced (sandbox refused, ThinkPad 200). DBIE curl-200.
- [x] NPCI ecosystem page maps bank-wise sources → RBI .aspx endpoints.

## Phase 1 — Ingestion skill `Skills/payments-fetch/`
- [ ] `scripts/npci_stats.ts` (Bun): boots agent-browser to acquire session, then
      in-page `fetch()` of `/api/product-statistic/tabs` + per-tab detail across FY
      ranges + page_size paging. Writes raw JSON to `Projects/payments-stat-hub/raw/npci/`.
- [ ] `scripts/rbi_aspx.py`: agent-browser flow for PSIUserView / NEFTView / ATMView
      (select month → export), writes raw HTML/XLSX to `raw/rbi/`.
- [ ] `scripts/parse_*.py`: normalize to long-form CSV: `series, month, metric, value, unit, source_id, extracted_at`.
- [ ] `scripts/build.duckdb`: load CSVs into `data/payments.duckdb` (tables: npc_monthly, npc_daily, rbi_psi, bankwise, meta_sources).
- [ ] `scripts/verify.py`: hermetic checks — row counts, latest month, value sanity
      (UPI monthly value in ₹ cr between plausible bounds), FY iteration coverage.
- [ ] Backfill first: UPI 2016→now, IMPS 2013→now, then expand to all 9 products.

## Phase 2 — Warehouse + derived views
- [ ] DuckDB views: monthly cross-rail table (UPI/IMPS/NACH/AePS/NETC/CTS/NFS/BHIM),
      YoY/MoM growth, bank-wise share of UPI volumes, per-capita proxy (vol/adult pop),
      uptime/incidents rollup.
- [ ] `meta_sources` table: every series ↔ URL, extraction date, unit, caveats.
- [ ] Sanity cross-check: NPCI UPI monthly vs RBI PSI UPI row — must reconcile.

## Phase 3 — Publish
- [ ] zo.space `/payments` — headline tiles (latest month per rail), cross-rail chart,
      YoY table, bank-wise top-10, reliability strip. Reads `/api/payments` route.
- [ ] `/api/payments` — Hono route querying DuckDB (or pre-rendered JSON for speed).
- [ ] GitHub repo `CashlessConsumer/payments-stat-hub`: data/ (CSV exports), scripts/,
      README with provenance. OG metadata per identity-project rule (default CC card).
- [ ] Note page linking from related projects (upi-cost-paper, upi-alternatives).

## Phase 4 — Freshness automation
- [ ] Monthly agent (day 8 ~11 AM IST, after NPCI posts prior month): run fetch →
      verify → rebuild views → update site → post diff to Discord #datasets.
- [ ] Alert path: if verify fails, post to #zoops, do not update site.
- [ ] IdleMax fallback for RBI .aspx changes (fragile pages).

## Deliverables
- `Projects/payments-stat-hub/` — raw/, data/payments.duckdb, notes/
- `Skills/payments-fetch/` — repeatable ingestion + verify
- zo.space `/payments` + `/api/payments`
- GitHub repo with open data exports