Methodology changelog
Substantive changes to the methodology described in seed/02_methodology_specification.md are recorded here with date, change, reasoning, and version stamp.
The methodology version applied to any dashboard output is recorded in data/outputs/methodology_version.json so historical outputs remain reproducible.
Methodology version 0.4.0 — 2026-05-15
WHO Global Health Observatory integrated as the third source feeding conversion efficiency. Source diversity now meets the spec target across all five dimensions.
Changes from 0.3.0 to 0.4.0
- WHO GHO source added. Six indicators wired into conversion efficiency: UHC Service Coverage Index, healthy life expectancy at birth, life expectancy at birth, under-5 mortality, maternal mortality, physicians per 10,000 population.
- Conversion efficiency now three-source. WDI / UNDP / WHO weighting keeps any single source below 50% within the dimension. WDI weights rebalanced down (life expectancy 2.0->1.5, maternal mortality 1.5->1.0).
- Source-diversity property test fully restored. The 85% per-dimension exemption for conversion efficiency is gone; the test now runs at the default 60% tolerance for every dimension.
- Cross-source validation expanded to four pairs: WDI tax vs IMF revenue, WDI vs UNDP life expectancy, WDI vs WHO life expectancy, WHO vs WDI maternal mortality. Per-pair scores expose where sources agree and where they disagree.
Methodology unchanged from 0.3.0 to 0.4.0
- Five-dimensional scoring formula
- Regime classifier (V-Dem-driven with bootstrap fallback)
- 80% confidence-band derivation
- Missingness suppression threshold (40%)
- Known-falsification flag registry
- Cluster-specific narrative templates
Licence impact in 0.4.0
WHO GHO data is CC-BY-NC-SA-3.0-IGO: non-commercial use with attribution and ShareAlike on derived works. This propagates a non-commercial constraint to any derived data product that incorporates WHO indicators. Commercial downstream users must substitute equivalents (likely from WDI alone) or partition WHO-dependent calculations into an isolated module.
Effect on country scores in 0.4.0
Most affected: countries where WHO/WDI/UNDP agree closely on health outcomes — cross-source agreement scores go up, data-quality composites rise materially. Australia's data-quality composite moved from ~83 to 91.8. Countries with limited WHO coverage are not penalised; the missingness suppression rule still applies.
Methodology version 0.3.0 — 2026-05-15
V-Dem v16 integrated as both regime classifier and accountability source.
Changes from 0.2.0
- Regime classifier is now V-Dem-driven (spec §3).
v2x_regimeprovides the primary cluster mapping; the Cluster A seed list remains the curatorial override; the rule-based heuristic survives as a bootstrap fallback for countries V-Dem doesn't cover. Fragility detection on government effectiveness < -1.0 runs before any regime assignment. - Accountability dimension expanded (spec §2.4 and §5.1). Six V-Dem indicators added; weights rebalanced so neither WGI nor V-Dem exceeds 50% of the dimension. Resolves the methodology debt logged in 0.1.0 where accountability was 100% WGI-derived.
- Adaptive capacity rebalanced. IMF gross-debt indicator added; cross-listed WDI internet use weighted at half (it also enters execution capacity). WDI/IMF now near 50/50.
- Source-diversity property test tightened. Default tolerance is 60%; per-dimension exceptions in
SINGLE_SOURCE_TOLERANCEare now documented in code. Conversion efficiency retains the 85% tolerance pending WHO integration.
Methodology unchanged from 0.2.0
- Five-dimensional scoring formula, weights are unchanged within their dimensions
- 80% confidence-band derivation
- Missingness suppression threshold (40%)
- Known-falsification flag registry (six entries)
Effect on country scores
The pipeline does not back-fill old methodology versions. Country profiles generated under 0.3.0 carry the version stamp; comparing them to 0.2.0 outputs requires the methodology-version stamp on each output JSON.
Most affected countries are those where V-Dem disagrees with WGI on accountability — notably Cluster A and B countries that V-Dem rates lower than WGI's expert surveys do. USA's accountability score is the clearest example: positive under WGI-only scoring, negative once V-Dem signals enter.
Methodology version 0.2.0 — 2026-05-15
Phase 2 backend methodology features. No change to the dimensional scoring formula or to the regime classifier; this version adds methodology features the spec already required but Phase 1 deferred.
Changes from 0.1.0
- Known-falsification flagging operational (spec §7.2). Six canonical entries (Argentina 2007-2015, Venezuela post-2014, China provincial GDP, DPRK, Eritrea). Observations flagged
excludeare removed from scoring;markobservations stay but the country profile carries the flag with its citation. See pipeline/src/stratos_pipeline/scoring/falsification.py. - Cluster-specific diagnostic templates (spec §8.3). Headline and Constraints sections now adapt framing to the country's regime cluster. Closed-autocracy and electoral-autocracy profiles carry an explicit note that comparison to democratic clusters reads as "different operating models," not "different performance levels."
- Cross-source validation expanded. Two pairs are now wired: WDI tax-revenue vs IMF government-revenue (existing), and WDI vs UNDP life-expectancy (new). Per-pair breakdown is surfaced on every country profile.
Methodology unchanged from 0.1.0
- Five-dimensional scoring formula, weights, and direction encoding
- Bootstrap regime classifier (V-Dem-driven replacement still pending)
- 80% confidence-band derivation (coverage + within-dimension dispersion)
- Missingness suppression threshold (40%)
Phase 2 work still deferred
- V-Dem-driven regime classifier (waiting on operator-curated CSV path)
- WHO Global Health Observatory for conversion-efficiency source diversification
- Translated narratives (needs human language experts)
- Structural-constraint conditioning beyond cluster (geography, demography)
Methodology version 0.1.0 — 2026-05-14
Initial methodology as specified in seed/02_methodology_specification.md. Five-dimensional scoring (execution capacity, conversion efficiency, distributional reach, accountability infrastructure, adaptive capacity), six-cluster regime classification, hierarchical structural-constraint conditioning, PCR-based uncertainty quantification on conversion efficiency, template-based diagnostic narratives.
Phase 1 implementation decisions (deliberate departures from the full methodology, documented so they can be tightened later):
- Regime classifier: rules-based bootstrap using WGI government effectiveness, voice/accountability, and UNDP HDI rather than V-Dem. Cluster A uses a hand-curated seed list. To be replaced with a V-Dem-driven classifier before public launch.
- Structural-constraint conditioning: implemented as within-cluster comparison only. Geography/demography/resource endowment conditioning is deferred to Phase 2.
- Uncertainty quantification: 80% confidence bands derived from indicator coverage and within-dimension dispersion. PCR-based uncertainty on conversion efficiency is deferred to Phase 3 per the seed roadmap.
- Indicator weighting: starts at the theory-driven weights in pipeline/src/stratos_pipeline/scoring/catalogue.py. Reliability/coverage-based adjustment is deferred.
No prior versions.