Issued 1 November 2023: the next six seasons

Three-month rainfall forecasts for Indonesia, December-February to May-July, from one forecast date

import {selectControl, loadWeekData, issueTargets, skillByLead, weekSkillMaps, sameTargetSkill, compareScales, weekIssueMaps, scoresByLead} from "../assets/leads/leads.js"
leadData = loadWeekData("../assets/seasonal-leads-2023/")
geo = ({indonesia: await FileAttachment("../data/boundaries/indonesia.geojson").json(),
  provinces: await FileAttachment("../data/boundaries/provinces.geojson").json(),
  neighbours: await FileAttachment("../data/boundaries/neighbours.geojson").json()})
leadChoices = leadData.targets.map(t => [String(t.lead), `Lead ${t.lead}: ${t.label}`])

Case 04 · Seasonal · Three-month rainfall · Full Indonesia

A seasonal outlook issued on 1 November 2023 covers the next six overlapping three-month seasons: December-February (DJF), then January-March (JFM) and so on to May-July (MJJ) 2024. This case calibrates the CFSv2 run of that day for each season, scores each calibration over 30 years, and checks every forecast against what happened.

The monthly case does the same for single months from the same run. Everything else is shared: CFSv2, CHIRPS v3.0 at its native 0.05° grid (61,190 pixels over Indonesia), the CCA and PCR settings and the 30 training years (1991-2020). All numbers on this page come from the executed notebook.

Hover, tap or use the arrow keys on any map or chart to read values; each view can be downloaded as it is set. Maps load when you choose them, one small file per map.

The short result

Issued on 1 November, only DJF has some cross-validated skill: an area-mean CCA correlation of 0.16 (PCR 0.18). From JFM to MJJ it lies between -0.10 and 0.05, which is no skill.
Calibration beats the raw CFSv2 correlation for DJF only. For the later seasons the raw model is higher, at 0.12 to 0.23, which is still small.
The target season matters more than the lead. October-December forecast from earlier starts keeps a CCA correlation of 0.33 to 0.48 at the same six leads.
The seasons after DJF were wetter than normal over most of Indonesia. The forecasts leaned dry for DJF to AMJ and, from JFM on, hit less area than always forecasting “above normal” would have; for MJJ they leaned wet.

One issue date, six target seasons

The lead is the number of whole months between the issue date and the first day of the season: DJF is lead 1, MJJ lead 6. PyCPT gives a lead for each month of a season, counted to the middle of that month, so a season spans three PyCPT leads.

Lead	Target	PyCPT leads	Hindcast starts	Training targets
1	DJF 2023-24	1.5 to 3.5	1 Nov, 1990-2019	Dec 1990-Feb 1991 to Dec 2019-Feb 2020
2	JFM 2024	2.5 to 4.5	1 Nov, 1990-2019	JFM 1991-2020
3	FMA 2024	3.5 to 5.5	1 Nov, 1990-2019	FMA 1991-2020
4	MAM 2024	4.5 to 6.5	1 Nov, 1990-2019	MAM 1991-2020
5	AMJ 2024	5.5 to 7.5	1 Nov, 1990-2019	AMJ 1991-2020
6	MJJ 2024	6.5 to 8.5	1 Nov, 1990-2019	MJJ 1991-2020

Training uses the target years 1991-2020 as PyCPT defines them; a season’s year is the year of its middle month, so the first training DJF runs from December 1990 to February 1991. The CFSv2 runs used here reach 9.5 months ahead, enough for MJJ.

Each lead has its own PyCPT case folder, downloaded with download_data and fitted with evaluate_models, using CCA (3 predictor and 3 predictand EOFs, 2 CCA modes) and PCR (3 predictor EOFs), cross-validation with a window of 5 years. These settings are the ones used in the Fit & diagnose case, fixed before any skill was computed. At 0.05° each fit took 15 to 17 minutes, most of it spent writing and reading CPT’s files for 61,190 pixels, so the eighteen fits ran side by side, each in its own process.

viewof timelineLead = selectControl("Highlight", leadChoices, "1")
issueTargets(leadData, timelineLead)

Seasonal rainfall from dekads, at 0.05°

The observations are CHIRPS v3.0 totals built from the local dekadal archive, as in the monthly case: for each training target the notebook reads the first and last day from the hindcast file and sums the nine dekads covering exactly those days, refusing to add them unless they run contiguously. DJF built this way matches the archive’s own 3-month total to within 0.0005 mm for 1990-91 and 2023-24, with the same missing pixels. The predictand is every 0.05° pixel whose centre lies inside Indonesia: 61,190 pixels with data in all 30 training years of every target.

The CFSv2 requests multiply the daily mean by the days in the season, with February counted as 28.25 days in every year (PyCPT’s monthlengths table): 90.25 days for DJF and JFM, 89.25 for FMA. Calibration regresses onto the observed totals, which absorbs a constant factor, and correlation ignores it.

Skill: DJF only

The chart shows the area-weighted mean over Indonesia of two cross-validated scores for each lead. Pearson correlation is 0 for no linear relationship. 2AFC is the chance that the forecast ranks two different years in the observed order, 50% for no skill (Mason and Weigel 2009).

viewof skillMetric = selectControl("Score", [["Pearson", "Pearson correlation"], ["2AFC (%)", "2AFC"]], "Pearson")
skillByLead(leadData, skillMetric)

Area-weighted means over 61,190 pixels, 1991-2020. Source: skill-by-lead.csv.
Lead	Target	CCA Pearson	PCR Pearson	Raw CFSv2 Pearson	CCA 2AFC (%)	PCR 2AFC (%)
1	DJF 2023-24	0.16	0.18	0.13	55.6	56.6
2	JFM 2024	0.03	0.05	0.16	51.6	51.6
3	FMA 2024	0.04	0.05	0.19	52.0	52.5
4	MAM 2024	0.01	0.01	0.19	50.9	50.8
5	AMJ 2024	-0.10	-0.05	0.12	47.4	48.1
6	MJJ 2024	-0.01	0.04	0.23	49.7	51.3

DJF is the only season with calibrated skill: a mean correlation of 0.16 to 0.18 and a 2AFC of 56-57%. It is also the one season where CCA and PCR beat the raw model.
From JFM to MJJ the calibrated forecasts have none: mean correlations between -0.10 and 0.05, 2AFC between 47% and 53%. When there is little real skill, cross-validated correlations of regression forecasts are biased low (Barnston and van den Dool 1993), so values near or below zero mean “no skill”, not “reliably wrong”.
The raw model keeps a small correlation at every lead, 0.12 to 0.23, above the calibrated forecasts from JFM on. It has nothing fitted and so pays no cross-validation cost; whether calibration with other settings would keep that signal was not tested.

Lead or target season?

The notebook also fits the opposite design with the same CCA settings: one target, October-December 2023, forecast from six earlier starts (1 September back to 1 April). There only the lead changes.

sameTargetSkill(leadData)

Area-weighted mean correlation, cross-validated for CCA, 1991-2020. Source: same-target-skill.csv.
Lead	Issued 1 Nov: target	CCA Pearson	Raw CFSv2	OND 2023: start	CCA Pearson	Raw CFSv2
1	DJF 2023-24	0.16	0.13	1 Sep 2023	0.47	0.39
2	JFM 2024	0.03	0.16	1 Aug 2023	0.48	0.30
3	FMA 2024	0.04	0.19	1 Jul 2023	0.45	0.26
4	MAM 2024	0.01	0.19	1 Jun 2023	0.48	0.25
5	AMJ 2024	-0.10	0.12	1 May 2023	0.44	0.27
6	MJJ 2024	-0.01	0.23	1 Apr 2023	0.33	0.15

OND keeps a CCA correlation of 0.44 to 0.48 for five months of lead and 0.33 at six. The seasons the 1 November run forecasts have 0.16 at best for CCA (0.18 for PCR). So, at least for CFSv2 over Indonesia with these settings, how well a season can be forecast depends far more on which season it is than on how far ahead the forecast is made. Why OND is so much more predictable than DJF to MJJ is not tested here.

The season against its first month

The monthly case made a forecast for the first month of each of these seasons from the same run: December for DJF, January for JFM, and so on.

compareScales(leadData)

Area-weighted mean Pearson correlation. Source: seasonal-vs-monthly.csv.
Lead	Season	Month	CCA, season	CCA, month	Raw, season	Raw, month
1	DJF 2023-24	Dec 2023	0.16	0.10	0.13	0.18
2	JFM 2024	Jan 2024	0.03	-0.11	0.16	-0.02
3	FMA 2024	Feb 2024	0.04	-0.06	0.19	0.09
4	MAM 2024	Mar 2024	0.01	0.03	0.19	0.21
5	AMJ 2024	Apr 2024	-0.10	-0.11	0.12	0.12
6	MJJ 2024	May 2024	-0.01	-0.15	0.23	0.05

For CCA the season scores higher than its first month at five of six leads (by 0.01 to 0.14) and lower for MAM (0.008 against 0.031). For the raw model the season is higher at three leads, lower at two and level for AMJ. A season also covers two months beyond the first, so this compares two different questions from the same run, not the same question at two time scales.

Where the skill is

The maps show the cross-validated correlation (or 2AFC) at every 0.05° pixel for each target, lead 1 at the top to lead 6 at the bottom.

viewof mapMethod = selectControl("Forecast", [["CCA", "CCA"], ["PCR", "PCR"], ["raw", "Raw CFSv2"]], "CCA")
viewof mapMetric = selectControl("Score", mapMethod === "raw" ? [["pearson", "Pearson correlation"]] : [["pearson", "Pearson correlation"], ["twoafc", "2AFC"]], "pearson")
weekSkillMaps(leadData, geo, mapMethod, mapMethod === "raw" ? "pearson" : mapMetric)

The forecasts as issued, and what happened

Each forecast is checked against its own season. Categories use the 33rd and 67th percentiles of that season’s 1991-2020 totals at each pixel.

Area-weighted over the 61,190 pixels. Source: issued-2023-11-observed.csv.
Lead	Target	Observed mean (mm)	1991-2020 mean (mm)	Below normal (% of area)	Near normal	Above normal
1	DJF 2023-24	842	842	33.7	33.1	33.2
2	JFM 2024	871	831	23.6	34.6	41.8
3	FMA 2024	875	831	21.8	33.2	45.0
4	MAM 2024	920	816	12.4	32.5	55.1
5	AMJ 2024	881	734	4.6	28.6	66.8
6	MJJ 2024	734	632	6.7	31.9	61.4

DJF 2023-24 was normal on average, with the three categories covering a third of the area each. Every later season was wetter than normal, increasingly so into the middle of 2024: above normal over 55% of the area in MAM and 67% in AMJ.

The CCA forecasts favoured below normal over 66% to 79% of the area for DJF to AMJ, and above normal over 89% for MJJ. The map column shows each lead’s target.

viewof forecastMethod = selectControl("Calibration", [["CCA", "CCA"], ["PCR", "PCR"]], "CCA")
viewof forecastView = selectControl("Show", [["category", "Most likely category"], ["observedCategory", "Observed category"], ["total", "Forecast total"], ["observed", "Observed total"], ["difference", "Forecast minus observed"]], "category")
weekIssueMaps(leadData, geo, leadData.issue.date, forecastMethod, forecastView)

viewof scoreMetric = selectControl("Score", [["hit", "Hit rate"], ["rpss", "RPSS"], ["mean", "Mean forecast total"]], "hit")
scoresByLead(leadData, scoreMetric)

Each target scored against its own observation. Sources: issued-2023-11-forecasts.csv, issued-2023-11-observed.csv.
Lead	Target	Method	Hit rate (%)	Reference hit rate (%)	RPSS	RMSE (mm)	Mean forecast (mm)
1	DJF 2023-24	CCA	46.3	33.7	0.13	163.5	775
1	DJF 2023-24	PCR	47.9	33.7	0.12	163.8	753
2	JFM 2024	CCA	35.7	41.8	-0.01	170.4	792
2	JFM 2024	PCR	40.2	41.8	0.03	164.9	801
3	FMA 2024	CCA	34.8	45.0	-0.00	176.1	771
3	FMA 2024	PCR	37.4	45.0	0.04	171.8	781
4	MAM 2024	CCA	24.7	55.1	-0.16	225.3	760
4	MAM 2024	PCR	28.3	55.1	-0.15	228.8	771
5	AMJ 2024	CCA	21.2	66.8	-0.15	247.2	698
5	AMJ 2024	PCR	38.7	66.8	-0.05	240.1	728
6	MJJ 2024	CCA	56.9	61.4	0.16	157.6	717
6	MJJ 2024	PCR	56.0	61.4	0.15	180.7	723

Hit rate is the share of area where the most likely category was the one observed. The reference is always forecasting the category that covered the most area that season, picked with hindsight. DJF’s forecasts beat it (46-48% against 34%). From JFM on they fell short, by most in AMJ (21% for CCA against 67%).
RPSS compares the forecast probabilities with climatological odds (33/34/33%) using the ranked probability score (Epstein 1969). It was positive for DJF (0.12 to 0.13) and MJJ (0.15 to 0.16), near zero for JFM and FMA, and clearly negative for MAM and for CCA in AMJ, where the forecasts leaned dry in very wet seasons. One outcome per lead rewards being right about 2024, not skill in general.
The deterministic totals were below the observed ones at every lead: by 67 mm for DJF and 17 mm for MJJ (CCA), and by 79 to 183 mm in between.

What this does and does not show

This is one issue date, one model, one set of fixed settings and one verifying year per target. It shows that from a 1 November CFSv2 start, with these settings, calibrated seasonal forecasts for Indonesia carry some skill for DJF and none for the five later seasons, while October-December keeps much higher skill at the same leads. It does not show how other start months, models or settings behave, and the 2023-24 scores describe single outcomes.

The subseasonal cases look at the weeks after the same issue date with GEFSv12.

Files

Executed notebook: every number and figure on this page
Skill by lead (CSV) · Same target from earlier starts (CSV) · Season against month (CSV) · Forecasts as issued, scored (CSV) · Observed targets (CSV) · Run summary (JSON)
Maps at 0.05° (NetCDF): for every lead, the observed total and category, and each method’s correlation, forecast total and most likely category

References

IRI. PyCPT 2.10.4 source: pycpt/src/pycpt/notebook.py (setup, download_data, summarize_available_years, evaluate_models and the plotting functions). Tag pycpt-2.10.4, commit 084bb5c. https://github.com/iri-pycpt/pycpt/blob/084bb5c380c200b5f65f11ba224a965547b91da1/pycpt/src/pycpt/notebook.py
IRI. PyCPT 2.10.4 source: cptdl (data catalog, leads_from_target, evaluate_url, download, simple_download). Tag pycpt-2.10.4, commit 084bb5c. https://github.com/iri-pycpt/pycpt/tree/084bb5c380c200b5f65f11ba224a965547b91da1/pycpt/src/cptdl
IRI. CPT 17.8.3 source code archive. Columbia University Academic Commons. https://doi.org/10.7916/fdp6-v391
CPT+. Forecast Probabilities (CPT help). https://cptplus.org/Help/Tools_Forecast_Probs.html Same page name as on the former cpthelp.iri.columbia.edu, which did not resolve when checked on 10 November 2026.
IRI Data Library. NMME NCEP-CFSv2 hindcasts, PENTAD_SAMPLES_FULL, precipitation. https://iridl.ldeo.columbia.edu/SOURCES/.Models/.NMME/.NCEP-CFSv2/.HINDCAST/.PENTAD_SAMPLES_FULL/.prec/ Browsing Data Library pages requires signing in (checked 10 November 2026); PyCPT’s data requests in this tutorial succeeded without a key.
Saha, S., Moorthi, S., Wu, X., et al. (2014). The NCEP Climate Forecast System Version 2. Journal of Climate, 27(6), 2185-2208. https://doi.org/10.1175/JCLI-D-12-00823.1
Kirtman, B., Min, D., Infanti, J., et al. (2014). The North American Multimodel Ensemble: Phase-1 seasonal-to-interannual prediction; Phase-2 toward developing intraseasonal prediction. Bulletin of the American Meteorological Society, 95(4), 585-601. https://doi.org/10.1175/BAMS-D-12-00050.1
Climate Hazards Center, UC Santa Barbara. CHIRPS: Rainfall Estimates from Rain Gauge and Satellite Observations (CHIRPS version 3.0 page). https://www.chc.ucsb.edu/data/chirps3
Funk, C., Peterson, P., Landsfeld, M., et al. (2014). A quasi-global precipitation time series for drought monitoring. U.S. Geological Survey Data Series 832. https://doi.org/10.3133/ds832
Mason, S. J., and Weigel, A. P. (2009). A generic forecast verification framework for administrative purposes. Monthly Weather Review, 137(1), 331-349. https://doi.org/10.1175/2008MWR2553.1
Epstein, E. S. (1969). A scoring system for probability forecasts of ranked categories. Journal of Applied Meteorology, 8(6), 985-987. https://doi.org/10.1175/1520-0450(1969)008%3C0985:ASSFPF%3E2.0.CO;2
Barnston, A. G., and van den Dool, H. M. (1993). A degeneracy in cross-validated skill in regression-based forecasts. Journal of Climate, 6(5), 963-977. https://doi.org/10.1175/1520-0442(1993)006%3C0963:ADICVS%3E2.0.CO;2