Using reliability diagrams to interpret the ‘signal-to-noise paradox’ in seasonal forecasts of the winter North Atlantic Oscilation

• Main Authors: Strommen, Kristian, MacRae, Molly, Christensen, Hannah
• Format: Other/Unknown Material
• Language: unknown
• Published: Authorea, Inc. 2023
• Subjects: North Atlantic
• Online Access: http://dx.doi.org/10.22541/essoar.167898485.56269215/v1

The ‘signal-to-noise paradox’ for seasonal forecasts of the winter NAO is often described as an ‘underconfident’ forecast and measured using the ratio-of-predictable components metric (RPC). However, comparison of RPC with other measures of forecast confidence, such as spread-error ratios, can give conflicting impressions, challenging this informal description. We show, using a linear statistical model, that the ‘paradox’ is equivalent to a situation where the reliability diagram of any percentile forecast has a slope exceeding 1. The relationship with spread-error ratios is shown to be far less direct. We furthermore compute reliability diagrams of winter NAO forecasts using seasonal hindcasts from the European Centre for Medium-range Weather Forecasts and the UK Meteorological Office. While these broadly exhibit slopes exceeding 1, there is evidence of asymmetry between upper and lower terciles, indicating a potential violation of linearity/Gaussianity. The limitations and benefits of reliability diagrams as a diagnostic tool are discussed.