STStratosGovernance diagnostic

Methodology changelog

Substantive changes to the methodology described in seed/02_methodology_specification.md are recorded here with date, change, reasoning, and version stamp.

The methodology version applied to any dashboard output is recorded in data/outputs/methodology_version.json so historical outputs remain reproducible.

Methodology version 0.4.0 — 2026-05-15

WHO Global Health Observatory integrated as the third source feeding conversion efficiency. Source diversity now meets the spec target across all five dimensions.

Changes from 0.3.0 to 0.4.0

  • WHO GHO source added. Six indicators wired into conversion efficiency: UHC Service Coverage Index, healthy life expectancy at birth, life expectancy at birth, under-5 mortality, maternal mortality, physicians per 10,000 population.
  • Conversion efficiency now three-source. WDI / UNDP / WHO weighting keeps any single source below 50% within the dimension. WDI weights rebalanced down (life expectancy 2.0->1.5, maternal mortality 1.5->1.0).
  • Source-diversity property test fully restored. The 85% per-dimension exemption for conversion efficiency is gone; the test now runs at the default 60% tolerance for every dimension.
  • Cross-source validation expanded to four pairs: WDI tax vs IMF revenue, WDI vs UNDP life expectancy, WDI vs WHO life expectancy, WHO vs WDI maternal mortality. Per-pair scores expose where sources agree and where they disagree.

Methodology unchanged from 0.3.0 to 0.4.0

  • Five-dimensional scoring formula
  • Regime classifier (V-Dem-driven with bootstrap fallback)
  • 80% confidence-band derivation
  • Missingness suppression threshold (40%)
  • Known-falsification flag registry
  • Cluster-specific narrative templates

Licence impact in 0.4.0

WHO GHO data is CC-BY-NC-SA-3.0-IGO: non-commercial use with attribution and ShareAlike on derived works. This propagates a non-commercial constraint to any derived data product that incorporates WHO indicators. Commercial downstream users must substitute equivalents (likely from WDI alone) or partition WHO-dependent calculations into an isolated module.

Effect on country scores in 0.4.0

Most affected: countries where WHO/WDI/UNDP agree closely on health outcomes — cross-source agreement scores go up, data-quality composites rise materially. Australia's data-quality composite moved from ~83 to 91.8. Countries with limited WHO coverage are not penalised; the missingness suppression rule still applies.

Methodology version 0.3.0 — 2026-05-15

V-Dem v16 integrated as both regime classifier and accountability source.

Changes from 0.2.0

  • Regime classifier is now V-Dem-driven (spec §3). v2x_regime provides the primary cluster mapping; the Cluster A seed list remains the curatorial override; the rule-based heuristic survives as a bootstrap fallback for countries V-Dem doesn't cover. Fragility detection on government effectiveness < -1.0 runs before any regime assignment.
  • Accountability dimension expanded (spec §2.4 and §5.1). Six V-Dem indicators added; weights rebalanced so neither WGI nor V-Dem exceeds 50% of the dimension. Resolves the methodology debt logged in 0.1.0 where accountability was 100% WGI-derived.
  • Adaptive capacity rebalanced. IMF gross-debt indicator added; cross-listed WDI internet use weighted at half (it also enters execution capacity). WDI/IMF now near 50/50.
  • Source-diversity property test tightened. Default tolerance is 60%; per-dimension exceptions in SINGLE_SOURCE_TOLERANCE are now documented in code. Conversion efficiency retains the 85% tolerance pending WHO integration.

Methodology unchanged from 0.2.0

  • Five-dimensional scoring formula, weights are unchanged within their dimensions
  • 80% confidence-band derivation
  • Missingness suppression threshold (40%)
  • Known-falsification flag registry (six entries)

Effect on country scores

The pipeline does not back-fill old methodology versions. Country profiles generated under 0.3.0 carry the version stamp; comparing them to 0.2.0 outputs requires the methodology-version stamp on each output JSON.

Most affected countries are those where V-Dem disagrees with WGI on accountability — notably Cluster A and B countries that V-Dem rates lower than WGI's expert surveys do. USA's accountability score is the clearest example: positive under WGI-only scoring, negative once V-Dem signals enter.

Methodology version 0.2.0 — 2026-05-15

Phase 2 backend methodology features. No change to the dimensional scoring formula or to the regime classifier; this version adds methodology features the spec already required but Phase 1 deferred.

Changes from 0.1.0

  • Known-falsification flagging operational (spec §7.2). Six canonical entries (Argentina 2007-2015, Venezuela post-2014, China provincial GDP, DPRK, Eritrea). Observations flagged exclude are removed from scoring; mark observations stay but the country profile carries the flag with its citation. See pipeline/src/stratos_pipeline/scoring/falsification.py.
  • Cluster-specific diagnostic templates (spec §8.3). Headline and Constraints sections now adapt framing to the country's regime cluster. Closed-autocracy and electoral-autocracy profiles carry an explicit note that comparison to democratic clusters reads as "different operating models," not "different performance levels."
  • Cross-source validation expanded. Two pairs are now wired: WDI tax-revenue vs IMF government-revenue (existing), and WDI vs UNDP life-expectancy (new). Per-pair breakdown is surfaced on every country profile.

Methodology unchanged from 0.1.0

  • Five-dimensional scoring formula, weights, and direction encoding
  • Bootstrap regime classifier (V-Dem-driven replacement still pending)
  • 80% confidence-band derivation (coverage + within-dimension dispersion)
  • Missingness suppression threshold (40%)

Phase 2 work still deferred

  • V-Dem-driven regime classifier (waiting on operator-curated CSV path)
  • WHO Global Health Observatory for conversion-efficiency source diversification
  • Translated narratives (needs human language experts)
  • Structural-constraint conditioning beyond cluster (geography, demography)

Methodology version 0.1.0 — 2026-05-14

Initial methodology as specified in seed/02_methodology_specification.md. Five-dimensional scoring (execution capacity, conversion efficiency, distributional reach, accountability infrastructure, adaptive capacity), six-cluster regime classification, hierarchical structural-constraint conditioning, PCR-based uncertainty quantification on conversion efficiency, template-based diagnostic narratives.

Phase 1 implementation decisions (deliberate departures from the full methodology, documented so they can be tightened later):

  • Regime classifier: rules-based bootstrap using WGI government effectiveness, voice/accountability, and UNDP HDI rather than V-Dem. Cluster A uses a hand-curated seed list. To be replaced with a V-Dem-driven classifier before public launch.
  • Structural-constraint conditioning: implemented as within-cluster comparison only. Geography/demography/resource endowment conditioning is deferred to Phase 2.
  • Uncertainty quantification: 80% confidence bands derived from indicator coverage and within-dimension dispersion. PCR-based uncertainty on conversion efficiency is deferred to Phase 3 per the seed roadmap.
  • Indicator weighting: starts at the theory-driven weights in pipeline/src/stratos_pipeline/scoring/catalogue.py. Reliability/coverage-based adjustment is deferred.

No prior versions.