Validation

How the framework was validated against held-out BMKG stations.

The training reference is CPC-UNI. Independent validation uses BMKG weather stations whose records are not in the CPC pipeline: 180 are archived, of which 171 pass the land-mask and minimum-paired-days filters and enter validation.

Validation protocol

Per-station extraction. For each BMKG station, find the nearest 0.1 deg pixel of the corrected field and extract the daily time series.
Pairwise filtering. Days where the station value is missing (BMKG sentinel 8888.0) or where the corrected field is NaN (ocean / outside AOI) are dropped.
Per-station metrics. Computed only for stations with at least min_valid_days = 30 valid paired days (WMO guidance for reliable verification statistics).
Regional aggregation. Stations are grouped by region_mapping (Sumatra, Jawa, Kalimantan, Sulawesi, Bali Nusa Tenggara, Maluku, Papua) for regional Taylor diagrams.

What is reported

Per station and per region, the following are computed for each correction stage and for IMERG-L (raw) and IMERG-F (baseline):

RMSE, MAE, ME (mean error / bias)
Pearson r, Spearman rho
Nash-Sutcliffe Efficiency, Kling-Gupta Efficiency
POD, FAR, CSI (at wet_day_threshold = 1.0 mm/day)
Percentile errors at 50, 75, 90, 95, 99
Standard deviation ratio (corrected / observed)

Dry-day agreement

Dry days are about half the validated record, so dry-day skill is a first-order property. The EQM stage matches the corrected wet-day frequency to the gauge-based reference, which improves agreement on no-rain days:

Metric	LS	LSEQM+DL	Target	Basis
Wet-day frequency ratio	1.21	0.95	1.0	171 stations
Correct-dry rate (gauge dry -> product dry)	~56%	~69%	higher	171 stations
False-alarm rate on dry days	~44%	~31%	lower	complement
Probability of detection (cost)	0.78	0.65	higher	171 stations

The first three rows are the intended gain; the last is the honest cost - suppressing over-detected drizzle reclassifies some light-rain hits as dry. The gain is the EQM stage; the CNN refinement leaves dry days untouched. See Methodology > EQM for the mechanism.

Reading the Taylor diagrams

Each station produces one point on the Taylor diagram, encoding correlation (angle), standard deviation ratio (radius), and centred RMSE (distance to the reference point). The desired pattern is movement toward the reference point as you go from IMERG-L -> LS -> LSEQM -> LSEQM+DL.

Headline results

Note

Quantitative headline numbers from the full-Indonesia run will be inserted here once the companion paper is finalised. The Bali tutorial bundle produces 4-station numbers that are directional but not statistically reliable on their own.

For the paper’s full station validation, see the companion paper (Remote Sens. 2026, 18, 2298; https://doi.org/10.3390/rs18142298).

For the robustness of these results under variation of the three exposed parameters (blend_alpha, gpd_threshold_percentile, saturation_count), see Sensitivity Analysis. The sweep demonstrates that none of the fifteen tested settings reverses the headline pattern, and that the Pearson correlation stays in the narrow band $[0.332, 0.348]$ across all parameter variations.

For the engineering-side verification (continuous integration and the synthetic-data smoke suite), see Testing.

--- title: "Validation" description: "How the framework was validated against held-out BMKG stations." --- The training reference is CPC-UNI. Independent validation uses BMKG weather stations whose records are not in the CPC pipeline: 180 are archived, of which 171 pass the land-mask and minimum-paired-days filters and enter validation. ## Validation protocol 1. **Per-station extraction.** For each BMKG station, find the nearest 0.1 deg pixel of the corrected field and extract the daily time series. 2. **Pairwise filtering.** Days where the station value is missing (BMKG sentinel `8888.0`) or where the corrected field is NaN (ocean / outside AOI) are dropped. 3. **Per-station metrics.** Computed only for stations with at least `min_valid_days = 30` valid paired days (WMO guidance for reliable verification statistics). 4. **Regional aggregation.** Stations are grouped by `region_mapping` (Sumatra, Jawa, Kalimantan, Sulawesi, Bali Nusa Tenggara, Maluku, Papua) for regional Taylor diagrams. ## What is reported Per station and per region, the following are computed for each correction stage and for IMERG-L (raw) and IMERG-F (baseline): - RMSE, MAE, ME (mean error / bias) - Pearson r, Spearman rho - Nash-Sutcliffe Efficiency, Kling-Gupta Efficiency - POD, FAR, CSI (at `wet_day_threshold = 1.0` mm/day) - Percentile errors at 50, 75, 90, 95, 99 - Standard deviation ratio (corrected / observed) ## Dry-day agreement Dry days are about half the validated record, so dry-day skill is a first-order property. The EQM stage matches the corrected wet-day frequency to the gauge-based reference, which improves agreement on no-rain days: | Metric | LS | LSEQM+DL | Target | Basis | |--------|---:|---------:|:------:|-------| | Wet-day frequency ratio | 1.21 | 0.95 | 1.0 | 171 stations | | Correct-dry rate (gauge dry -> product dry) | ~56% | ~69% | higher | 171 stations | | False-alarm rate on dry days | ~44% | ~31% | lower | complement | | Probability of detection (cost) | 0.78 | 0.65 | higher | 171 stations | The first three rows are the intended gain; the last is the honest cost - suppressing over-detected drizzle reclassifies some light-rain hits as dry. The gain is the EQM stage; the CNN refinement leaves dry days untouched. See [Methodology > EQM](../methodology/eqm.qmd#wet-dry-handling) for the mechanism. ## Reading the Taylor diagrams Each station produces one point on the Taylor diagram, encoding correlation (angle), standard deviation ratio (radius), and centred RMSE (distance to the reference point). The desired pattern is movement toward the reference point as you go from IMERG-L -> LS -> LSEQM -> LSEQM+DL. ## Headline results ::: {.callout-note} Quantitative headline numbers from the full-Indonesia run will be inserted here once the companion paper is finalised. The Bali tutorial bundle produces 4-station numbers that are directional but not statistically reliable on their own. ::: For the paper's full station validation, see the companion paper (*Remote Sens.* 2026, 18, 2298; https://doi.org/10.3390/rs18142298). ## See also For the robustness of these results under variation of the three exposed parameters (`blend_alpha`, `gpd_threshold_percentile`, `saturation_count`), see [Sensitivity Analysis](sensitivity-analysis.qmd). The sweep demonstrates that none of the fifteen tested settings reverses the headline pattern, and that the Pearson correlation stays in the narrow band $[0.332, 0.348]$ across all parameter variations. For the engineering-side verification (continuous integration and the synthetic-data smoke suite), see [Testing](testing.qmd).