Issued 1 November 2023: the next four weeks

Weekly subseasonal rainfall forecasts for Indonesia from one Wednesday’s GEFSv12 run, calibrated with CCA and PCR

Case 06 · Subseasonal · Weekly rainfall · Full Indonesia

On Wednesday 1 November 2023 the GEFSv12 run started at 00 UTC gives a forecaster the next four weeks: week 1 (2-8 November), week 2 (9-15), week 3 (16-22), week 4 (23-29), and weeks 3-4 together (16-29 November), the period of NOAA CPC’s week 3-4 outlook. This case calibrates that run with CCA and PCR through CPT, shows what it said against what happened, and measures how far each lead can be trusted.

It is the same moment as the monthly and seasonal cases, one step closer in time. Every map works at CHIRPS v3.0’s native 0.05° grid: 61,190 pixels over Indonesia. All numbers come from the executed notebook.

Hover, tap or use the arrow keys on any map or chart to read values; each view can be downloaded as it is set. Maps load when you choose them, one small file per map.

NoteThe short result
  • The forecast of 1 November leaned dry, and the weeks were dry. Weeks 1 to 3 came in at 45-47 mm over Indonesia against normals of 63-70 mm. Week 1 was forecast at 48.9 mm (CCA); weeks 2 and 3 were forecast closer to normal, at 53.1 and 61.3 mm.
  • Placing the categories was harder: for every lead the hit rate of the most likely category was below that of always forecasting the category that turned out most common, picked with hindsight, by 0.8 to 6.9 points.
  • How far to trust each lead, from all October-December weeks of 2000-2018: the area-mean CCA correlation is 0.35 for week 1, 0.29 for week 2, 0.26 for week 3, 0.17 for week 4, and 0.26 for weeks 3-4 together. The raw GEFSv12 correlation is higher for weeks 1 and 2 (0.50 and 0.34) and level from week 3.

The forecast issued on 1 November 2023

A forecast is issued at 00 UTC on a Wednesday. Model day d covers roughly the same 24 hours as observed day issue + d, so each lead is a window of days after the issue date. The menus start at this forecast and add the five later Wednesdays, up to 6 December 2023, the last whose five leads all fall in October-December, the period forecast and verified here.

Each map is one lead against its own target, week 1 at the top. Switch between the forecast, the observation, their difference and the categories, or pick another issue date.

CCA forecast issued 1 November 2023, area-weighted over Indonesia. Source: issued-2023-11-01.csv, which also has PCR.
Lead Target Forecast (mm) Observed (mm) 2000-2018 mean (mm) Observed below / near / above normal (% of area) Hit rate (%) Reference (%)
Week 1 2-8 Nov 48.9 46.2 63.0 52.4 / 40.0 / 7.6 51.0 52.4
Week 2 9-15 Nov 53.1 44.9 65.7 58.6 / 31.7 / 9.7 57.8 58.6
Week 3 16-22 Nov 61.3 47.0 69.8 68.3 / 20.5 / 11.2 63.8 68.3
Week 4 23-29 Nov 67.4 66.8 69.7 35.2 / 36.4 / 28.3 34.0 36.4
Weeks 3-4 16-29 Nov 131.2 113.8 139.6 54.5 / 31.8 / 13.7 47.6 54.5
  • Amounts. The first three weeks of November were dry: 45-47 mm against normals of 63-70 mm. The forecast for week 1 was close (48.9 against 46.2 mm). Further ahead it moved towards the normal (53.1 mm for week 2, 61.3 mm for week 3) and so missed more of the dry spell. A regression forecast moves towards the normal as its skill falls, which is what the skill section below measures. Week 4 returned to near normal (66.8 mm) and was forecast almost exactly (67.4 mm).
  • Categories. For weeks 1-3 the forecast made below normal the most likely category over 84% to 96% of the area, and below normal was observed over 52% to 68%. Its hit rate (51% to 64%) stayed under the reference, always forecasting the category that turned out most common, chosen with hindsight, by 0.8 to 4.5 points; for weeks 3-4 together the gap was 6.9 points (47.6% against 54.5%). For week 4 the forecast favoured above normal over 55% of the area in a week that was mixed; its hit rate was 34% against 36%.

One issue date is one outcome per lead. The next sections show how well each lead does on average.

How far each lead can be trusted

The calibration of each lead is trained and scored on every week starting on a Thursday in October-December 2000-2018, 249 targets per lead. GEFSv12’s hindcasts start on Wednesdays, like the real-time forecast, so training uses exactly the same leads as real use. CCA (3 predictor and 3 predictand EOFs, 2 CCA modes) and PCR (3 predictor EOFs) are fitted through cptcore, the layer PyCPT’s evaluate_models calls, with cross-validation leaving out 27 targets at a time (the held-out week and 13 on each side, about a season). At 0.05° one fit took 76 to 90 minutes for CCA and 23 to 26 minutes for PCR, so the ten fits ran in parallel. For every fit, CPT’s correlation map equals the correlation recomputed from its own cross-validated hindcasts to within 0.000001.

Pearson correlation is 0 for no linear relationship; 2AFC is the chance that the forecast ranks two different weeks in the observed order, 50% for no skill (Mason and Weigel 2009).

Area-weighted means over 61,190 pixels, 249 targets of 2000-2018. Source: skill-by-lead.csv.
Lead CCA Pearson PCR Pearson Raw GEFS Pearson CCA 2AFC (%) PCR 2AFC (%)
Week 1 0.35 0.36 0.50 62.8 63.1
Week 2 0.29 0.28 0.34 60.3 60.3
Week 3 0.26 0.26 0.25 59.6 59.6
Week 4 0.17 0.18 0.18 56.7 56.8
Weeks 3-4 0.26 0.26 0.25 59.6 59.7
  1. Skill falls with lead, most between weeks 3 and 4. CCA’s mean correlation drops by 0.06 from week 1 to 2, 0.02 from week 2 to 3, and 0.09 from week 3 to 4.
  2. Two weeks together hold up better than the second of them. Weeks 3-4 score 0.26, close to week 3 and well above week 4 alone (0.17).
  3. Calibration does not beat the raw model. GEFSv12’s own anomaly correlates better in weeks 1 and 2 (by 0.15 and 0.05 over CCA) and about equally from week 3. The calibrated scores pay a cross-validation cost (Barnston and van den Dool 1993) and see the model only through 3 EOF patterns; neither effect was measured here.
  4. CCA and PCR are nearly identical, within 0.01 at every lead.

Where the skill is, pixel by pixel

The whole season: October-December 2023

Every week of October-December 2023 is forecast at every lead, so the scores below also use the Wednesdays of September and December whose forecasts reach into the season only in part: sixteen issue dates in all, from 13 September to 27 December. Over the 13 targets of each lead, scored over every pixel and target (area-weighted):

October-December 2023. Source: ond-2023-weekly-forecasts.csv.
Lead Method Hit rate (%) Reference (%) RPSS 2023 RMSE 2023 (mm) Mean forecast (mm) Observed mean (mm)
Week 1 CCA 50.4 49.4 0.12 28.6 55.7 51.1
Week 1 PCR 49.5 49.4 0.12 28.8 55.7 51.1
Week 2 CCA 50.2 49.4 0.12 28.5 56.0 51.1
Week 2 PCR 49.6 49.4 0.11 28.7 55.7 51.1
Week 3 CCA 46.8 49.4 0.07 29.7 58.0 51.1
Week 3 PCR 45.9 49.4 0.07 29.7 57.8 51.1
Week 4 CCA 45.6 49.4 0.08 29.8 57.8 51.1
Week 4 PCR 44.8 49.4 0.07 30.0 57.7 51.1
Weeks 3-4 CCA 53.2 52.2 0.14 48.3 117.5 106.5
Weeks 3-4 PCR 51.1 52.2 0.13 48.7 117.4 106.5
  • Hit rate against the reference, always forecasting the most frequent observed category (below normal, 49.4% of one-week pixel-targets and 52.2% for weeks 3-4): weeks 1, 2 and 3-4 matched it to within about a point; weeks 3 and 4 fell 2.5 to 4.5 points short.
  • RPSS 2023 compares the forecast probabilities with climatological odds (33/34/33%) using the ranked probability score (Epstein 1969): positive at every lead, highest for weeks 3-4 and weeks 1-2. One season of 13 weeks rewards leaning the right way in 2023, not skill in general.
  • The totals were too wet on average: 55.7 to 58.0 mm a week against an observed 51.1 mm, although the most likely category was below normal over 58% to 73% of the area.

Observations and anomalies

The observations are daily CHIRPS v3.0 (the rnl variant), exported for Indonesia from Earth Engine and converted to yearly NetCDF files by the tutorial’s gee_daily_to_netcdf.py. Their sums over three dekads (2005, 2012 and 2023) equal the dekadal archive used by the other cases to within 0.0001 mm at every Indonesian pixel, with the same missing pixels. A weekly total is the sum of its seven days (14 for weeks 3-4).

WarningDaily CHIRPS is not an independent daily product

CHC builds daily CHIRPS v3.0 from the pentad product: each pentad’s amount is shared between its days using the timing of ERA5 (the rnl variant used here) or IMERG, with no separate daily station blending, and CHC recommends pentadal or longer totals wherever the science allows (CHIRPS daily readme). These weeks run Thursday to Wednesday and so cut across pentads: how much of a pentad’s rain falls inside a week comes from ERA5’s timing, not from stations. The weekly totals, and the skill measured against them, carry that dependence; how much it matters was not measured here. The same readme notes that the daily dates follow ERA5’s UTC days (00 to 00 UTC), while the stations behind the pentad totals do not all report on UTC days, so the match with GEFSv12’s model days is approximate. The pentad case uses the pentad product itself.

Rainfall rises through October-December, so model and observations are turned into anomalies before fitting. A target’s mean is the average of 19 training targets of the same lead, one from each year 2000-2018: the one starting closest to the same day of the season (on average 1.7 days away, at most 6). Each forecast total is that mean for the observations plus the calibrated anomaly, set to zero where the sum is negative. The 2000-2018 mean over Indonesian pixels is 64.1 mm a week (129.2 mm for weeks 3-4).

What this does and does not show

This is one model, one set of fixed settings, 19 training years and one verifying season, with the headline on one issue date. It shows that, for GEFSv12 weekly rainfall over Indonesia in October-December, skill falls from 0.35 to 0.17 over four weeks, that combining weeks 3 and 4 keeps more skill than week 4 alone, and that calibration with these settings did not add to the raw model’s correlation. The observations are retrospective CHIRPS v3.0, not what was available when each forecast was issued, and the 2023 forecasts average 31 GEFSv12 members while the hindcasts have 11.

The dekad case asks the same question by dekad, issued on 31 October 2023.

Files

References

  1. IRI Data Library. SubX EMC GEFSv12_CPC (hindcast and forecast precipitation). https://iridl.ldeo.columbia.edu/SOURCES/.Models/.SubX/.EMC/.GEFSv12_CPC/ Browsing Data Library pages requires signing in (checked 10 November 2026); PyCPT’s data requests in this tutorial succeeded without a key.
  2. Guan, H., Zhu, Y., Sinsky, E., et al. (2022). GEFSv12 reforecast dataset for supporting subseasonal and hydrometeorological applications. Monthly Weather Review, 150(3), 647-665. https://doi.org/10.1175/MWR-D-21-0245.1
  3. NOAA Climate Prediction Center. Week 3-4 Outlooks (issued on Fridays; the outlook shown on 11 November 2026 was issued Friday 6 November 2026 for 21 November to 4 December, days 15 to 28 after issue). https://www.cpc.ncep.noaa.gov/products/predictions/WK34/
  4. Climate Hazards Center, UC Santa Barbara. CHIRPS: Rainfall Estimates from Rain Gauge and Satellite Observations (CHIRPS version 3.0 page). https://www.chc.ucsb.edu/data/chirps3
  5. Funk, C., Peterson, P., Landsfeld, M., et al. (2014). A quasi-global precipitation time series for drought monitoring. U.S. Geological Survey Data Series 832. https://doi.org/10.3133/ds832
  6. Climate Hazards Center. Information about CHIRPS Daily Precipitation (CHIRPS v3.0 daily readme). https://data.chc.ucsb.edu/products/CHIRPS/v3.0/daily/readme.txt
  7. IRI. PyCPT 2.10.4 source: pycpt/src/pycpt/notebook.py (setup, download_data, summarize_available_years, evaluate_models and the plotting functions). Tag pycpt-2.10.4, commit 084bb5c. https://github.com/iri-pycpt/pycpt/blob/084bb5c380c200b5f65f11ba224a965547b91da1/pycpt/src/pycpt/notebook.py
  8. IRI. PyCPT 2.10.4 source: pycpt/src/cptcore/functional/cca.py (canonical_correlation_analysis). Tag pycpt-2.10.4, commit 084bb5c. https://github.com/iri-pycpt/pycpt/blob/084bb5c380c200b5f65f11ba224a965547b91da1/pycpt/src/cptcore/functional/cca.py
  9. IRI. PyCPT 2.10.4 source: pycpt/src/cptextras/redate.py (redate). Tag pycpt-2.10.4, commit 084bb5c. https://github.com/iri-pycpt/pycpt/blob/084bb5c380c200b5f65f11ba224a965547b91da1/pycpt/src/cptextras/redate.py
  10. Mason, S. J., and Weigel, A. P. (2009). A generic forecast verification framework for administrative purposes. Monthly Weather Review, 137(1), 331-349. https://doi.org/10.1175/2008MWR2553.1
  11. Epstein, E. S. (1969). A scoring system for probability forecasts of ranked categories. Journal of Applied Meteorology, 8(6), 985-987. https://doi.org/10.1175/1520-0450(1969)008%3C0985:ASSFPF%3E2.0.CO;2
  12. Barnston, A. G., and van den Dool, H. M. (1993). A degeneracy in cross-validated skill in regression-based forecasts. Journal of Climate, 6(5), 963-977. https://doi.org/10.1175/1520-0442(1993)006%3C0963:ADICVS%3E2.0.CO;2
Back to top