We use the MHW definition, which describes a MHW as a “prolonged period of anomalously warm water”. Contrary to heat stress indices tailored for coral bleaching risk in summer – such as Degree Heating Weeks ) – this definition is general and includes winter MHWs. We chose the hindcast period (1993–2018) as the baseline and kept a fixed baseline rather than removing the decadal warming trend. While using a detrended baseline (i.e., a baseline computed from a timeseries where the long-term lienar trend has been removed) is appropriate for certain applications where discriminating between short term variability and climate change is necessary, here we wish to include any warm episode that could impact socio-ecosystems. It should also be noted that keeping the warming trend may artificially inflate the skill reported here, which may not accurately reflect the ability of the model to capture seasonal variability . The 90th percentile is computed using a temporal window of 11 d.

Based on this definition, we create a daily gridded field with two variables:

The severity index is a daily index of temperature anomalies, scaled to the local variability severity_index=SST-SSTclimSSTPC90-SSTclim: a value of 0 represents the climatological average (SST^clim), a value of 1 represents the 90th percentile (SST^PC90), a value of 2 represent a SST anomaly twice as large as the 90th percentile, etc. Note that this metrics is continuous and can reach negative values. Contrary to using categories based on fixed percentile (e.g. 90, 95, 98, etc.), this approach is unbounded and categories, as defined below, can be added as temperatures warm even with a fixed baseline. A MHW event is defined by the severity index exceeding 1 for at least 5 d; in addition MHWs separated by less than 2 d are merged into a single event.

The computation of the baseline and MHW category was performed using the “marineHeatWaves” python package written by Eric C. J. Oliver: https://github.com/ecjoliver/marineHeatWaves (last access: October 2023) and all further computations were done using the “xarray” python package .

The severity index and the category indicate the intensity of thermal stress at different temporal scales, which may have consequences for predictability. The severity reflects the SST anomaly at a daily scale while the category reflects the peak SST anomaly at the scale of the MHW. On the one hand, the severity index, being a continuous variable, gives more precise information and should thus be more difficult to predict accurately. On the other hand, the use of thresholds for the category can amplify small differences and result in inflated errors (for instance, a severity of respectively 1.99 and 2.01, despite being very close, corresponds to different categories). In addition, the computation of forecasted MHWs does not include information prior to the start date, which means that if a forecast starts during a MHWs but after its peak, the forecasted category will only correspond to the decline phase and will be lower than if the entire MHW is included.

The MHW category and severity are then used to derive additional metrics that synthesize information at different spatio-temporal scales.

the number of MHW days per month (category 1 or higher) is computed from the daily category.

These metrics are potentially easier to predict than the category or severity at each day and each location, since they provide integrated information on the exposure to MHWs during a certain period or region. The quantification of their predictability is our focus, and the eventual choice of metrics by stakeholders will represent a trade-off between spatio-temporal resolution and forecast accuracy.

The metrics are computed for all members of the forecast and the reanalysis (light grey lines and dashed blue line on Fig. ). The ensemble mean (black line on Fig. ) is then computed as the average of the metric value of each member (for instance, the member average of the severity).

The verification of a forecast consists in the quantification of the differences between the forecast and an observational reference (in our case the GLORYS reanalysis) by various metrics for skills. In addition, the added-value of the forecast is evaluated by comparing it to a simplified reference forecast: here we use a persistence forecast, which is the initial value kept constant for the 7 months. Since the MHW metrics are defined as anomalies, the persistence is in our case equivalent to more complex references such as the climatological seasonal cycle or the persistence of anomalies. Thus outperforming the persistence is not trivial and requires a high level of skill. We performed the verification with the “climpred” python package .

We use both a deterministic and a probabilistic approach, summarized in Fig. .

Overall, the waters around New Caledonia experience MHWs about 10 % of the time (Fig. c). This is consistent with the statistical definition of MHWs as days where the SST is above the 90th percentile, or in other words the 10 % warmest days. A spatial gradient emerges, with MHWs more frequent at low latitudes (15 %) than high latitudes (5 %). This may be caused by the minimum 5 d duration criteria of MHWs that is less frequently met at higher latitudes due to higher variability as also reported in or because of spatial differences in the recent warming trend (i.e., SST has increased faster in that region between 2018–2024, since the end of the baseline period).

The exposure to MHWs of the New Caledonia EEZ exhibits seasonal differences (Fig. a, b). The cold season (April–September) experiences two regimes: prolonged and wide MHWs for a minority of years (1998, 2010 and 2022) and almost no MHWs for the rest of the years. Interestingly, these 3 “unusual” years coincide with particularly strong La Niña events , although some other strong La Niña years show no particularly intense MHWs (such as 2000, 2008, 2011 or 2021). On the contrary, the warm season (October–March) generally experiences shorter MHWs every year that can episodically cover a large portion of the area.

We computed the forecast skill of two forecast systems (Météo-France and ECMWF) for 5 deterministic metrics, first by including the entire hindcast period (orange lines in Figs.  and ) and then by excluding the 3 “unusual” years identified above (1998–2010–2022, blue lines in Figs.  and ). This allows us to evaluate how much of the skill is contained in these 3 years, and the quality of the forecast that can be expected in a “normal” year. We also investigate whether skill can be improved by looking at metrics aggregated over a certain region (such as the EEZ of New Caledonia) and thus ignoring spatial variability within this region. We computed MHW metrics based on the spatial average of the SST in this region, but this approach only resulted in a very slight improvement in skill compared to the computation of the skill metric in each grid point (Fig. ). Thus we present here the skill computed in each grid cell and then spatially averaged in the region of interest. Skill maps show that there is very little spatial variation within the domain (Fig. ).

Several conclusions can be drawn from this analysis:

The skill of the Météo-France system is poor, rarely exceeding a Pearson coefficient of 0.7 even at short leadtimes and falling below 0.5 after the first month (Fig. d). However, at longer leadtimes it is generally able to beat the persistence forecast; but given the weak correlation at these lead times, these forecasts have an overall limited usefulness. The ECMWF system performs better, with very strong correlations during the first month and a higher skill than persistence (Fig. ), but the skill is also very low at longer lead times.

Here we evaluate the ability of the two forecast systems to predict the probability of different levels of exposure (dark and light purple lines), defined in terms of intensity (first two columns in Figs.  and ), duration (third column in Figs.  and ) and spatial extent (fourth column in Figs.  and ).

Both forecast systems perform well with the probabilistic approach, although the skill of ECMWF (Fig. ) is again higher than Météo-France (Fig. ). Reliability diagrams show that the forecasts are well-calibrated, although both systems tend to overpredict MHWs (Figs. –). For the most intense events in particular (dark purple lines in Figs.  and ), the Area Under the Curve (AUC) remains above 0.8 for at least 4 months in the Météo-France forecasts and 6 months in the ECMWF forecasts. In addition, probabilistic forecasts tend to be better than persistence, even at very short leadtimes, while the deterministic forecasts only start to outperform the persistence after the first few weeks or even months (bold lines in Figs. –). While probabilistic and deterministic skill are impossible to compare directly since they are quantified with different scores (the Area Under the Curve and the Pearson correlation coefficient, respectively), these results suggest that the probabilistic approach improves the quality of the forecast. Indeed, forecasts are generally considered to have no skill when the Pearson coefficient and the AUC are below 0.5, and to have “good” skill when the Pearson coefficient is above 0.7 or the AUC is above 0.8 (Horizontal dashed lines in Figs.  and ). Even if these reference values for what constitutes “good skill” are subjective and could be debated, there is a clear contrast between the deterministic and probabilistic scores: deterministic scores fall under these thresholds much quicker than probabilistic scores. For instance, all AUC values are above 0.5 for the entire forecast (7 months), while only some metrics have a Pearson coefficient above 0.5. Pearson coefficients only exceed 0.7 at very short leadtimes, while AUC commonly exceeds 0.8 for many months. As an illustration, Fig.  shows how a MHW in February 2016 can be detected 1 month in advance by a probabilistic forecast (red line) but completely missed by a deterministic forecast (black line).

The bottom panels of Figs.  and show the ROC curves that the AUC scores (top panels) are computed from and guide the interpretation of AUC values. For instance, for the severity index, the most extreme exposure (severity≥1.5) has a AUC that varies from 0.95 to 0.75, averaging 0.8 (Fig. a, dark purple line). The corresponding ROC curve (Fig. e, crosses) is well above the diagonal, and is closest to the top left corner with a decision threshold of 20 % (red cross). With this optimal threshold, the forecast will achieve an 80 % hit rate and a 40 % false alarm rate. Depending on the user, this might be considered useful or not (see the Discussion section).

The shift between the dark and light purple lines (Figs.  and ) show that the level of exposure significantly influences the skill. Thus exposure to MHWs of high intensity (defined with the severity index above 1.5 or the category at MHW scale above 2) is more predictable than MHWs of moderate intensity; exposure to longer MHWs (more than 15 d per month) is more predictable than shorter; exposure to wider MHWs (covering more than 50 % of the EEZ) is more predictable than narrower MHWs. On the other hand, the skill is weakly influenced by the initial metric (i.e. the category, or severity, or number of MHW days, or surface fraction) from which the level of exposure is derived, especially for severe exposure. For moderate exposure, the category has the lowest skill, for both forecasting systems. This implies that while predicting all properties of the most extreme MHWs is straightforward, the category of weaker MHWs is more unpredictable than their severity, spatial extent or duration.

We also investigated the influence on the probabilistic skill of the 3 “unusual” Niña years identified earlier (1998–2010–2022), which contain very long and wide MHWs and coincide with strong La Niña events . We find that, similar to the deterministic skill, the probabilistic skill is lower when these 3 years are excluded from the computation (Fig. ). However the results concerning the various levels of MHW exposure are qualitatively identical with and without the 3 La Niña years.

Finally, we found that, again similar to the deterministic skill, probabilistic skill is not improved when metrics are computed from the spatial average of SST (Fig. ).

Here we evaluate the seasonality of the forecast skill in the Météo-france forecast System 8, and thus compute it for each start month separately: for instance for january the skill is computed based on the 29 forecasts (1993–2018 + 2022–2024) initialized on 1 January of each year. We perform this computation for both deterministic (Fig. ) and probabilistic (Fig. ) skill, as well as an additional computation with the 3 “unusual years” (1998–2010–2022) removed (right columns in Figs.  and ). As in the previous section, the skill is computed as a fonction of leadtime, and can here be interpreted as the forecast target date: all forecasts of the month of august, for instance, are aligned vertically, whether they started 1 month earlier (i.e. with a leadtime of 1 month) or 6 months earlier (i.e. with a leadtime of 6 months).

These results show that the previous sections, where all starting months were aggregated, hide strong variability. For both the probabilistic and deterministic approaches, forecasts targeting the cold season (June–September) generally have a much higher skill than forecasts targeting the warm season (October–March), even at very long lead times (up to 7 months, see for instance forecasts targeting August and starting in February in Figs. a and a–c). This patterns also illustrates the counterintuitive result that skill can sometimes increase with leadtime. For instance for a forecast starting in February the skill is very low for the first few months, which target the warm season (leadtime 1–3 months, i.e. February–April), but becomes higher at longer leadtimes that target the cold season (leadtime 6–7 months, i.e. July–August). The opposite occurs for forecasts that start during the cold season, with initially very high skill that decreases sharply after September/October. Importantly, the forecasts are generally better than the reference persistence forecast (indicated by grey areas in Figs.  and ).

This seasonal difference in skill is likely caused by the higher MHW variability in the warm season, which typically has shorter MHWs every year, in contrast with the cold season. Indeed, these seasonal patterns in skill occur every year, even when the 3 atypical years with very long MHWs (in 1998, 2020 and 2022) are removed from the skill assessment (right column in Figs.  and ).

We now investigate whether the Météo-france forecast System 8 is able to predict the transitions from normal conditions to a MHW (the triggering of the next MHW) and from an ongoing MHW to normal conditions (the end of that MHW). Here MHWs are detected within a probabilistic framework: the forecast is considered in a MHW state if more than 20 % of the members are in a MHW state (category≥1), which is the optimal decision threshold identified with the ROC curve (Fig.  and discussion above). We separate the forecasts that start in normal conditions (Fig. a) and the forecasts that start in a MHW (Fig. b), and for each case we compute the duration (number of days) until the transition to another MHW state occurs. We then evaluate the correlation between the observations (x-axis) and in the forecast (y-axis). The position of points above or below the diagonal indicates whether the forecast has overestimated or underestimated the date of this transition.

We find that the forecast system is generally not very skilled at predicting these transitions, even at short leadtimes. The Pearson coefficient indicates weak correlations between forecast and observations for predictions of the start of the next MHW (0.19) and of the end of the ongoing MHW (0.41). When the forecast starts in normal conditions, it tends to predict that the next MHW will occur too early (majority of points below the diagonal in Fig. a), and there are many cases where it predicts a MHW that never occurs (points aligned vertically at 0), or completely misses the next MHW (points aligned horizontally at 0). Conversely, when the forecast starts in a MHW state, it tends to predict that the current MHW will last longer than observed (all points above the diagonal in Fig. b). However, it should be noted that some transitions are rather well forecasted many months in advance, particularly the end of MHWs during La Niña phases (blue points in Fig. ).

Finally, we investigate if the conclusions reached in the previous sections for the relatively small area of the New Caledonia EEZ are also valid for the wider Southwest Pacific region, and find that this is the case (Figs.  and ). We find (1) a decrease of the skill with leadtime, (2) an important role of the 3 key La Niña years for improving the forecast skill, (3) a better skill with a probabilistic approach compared to a deterministic approach, and (4) better predictability for intense, large-scale and longer MHWs. The results on the seasonality of skill (higher for the cold season) are also valid for this wider region (not shown). In addition, we found that the forecast skill is generally spatially uniform in the Southwest Pacific (Figs. –). A weak spatial gradient in the skill does emerge for some metrics and leadtimes, with higher skill in the northwest part of the domain (Figs. b–d and d). Thus, we conclude that the application of this framework to other close Pacific island nations (such as Fiji, Vanuatu, or Wallis and Futuna) would be worth exploring in a future study.

We investigated three factors that could influence forecast skill: the spatio-temporal scale (MHWs defined at a specific day and point or for an entire region/period), the properties of MHWs (extreme or weak, in terms of intensity, duration and spatial extent) and the timing of MHWs (season and ENSO phase).

Previous studies have found that MHWs are more predictable in regions strongly affected by ENSO, like the tropical eastern Pacific or by ENSO teleconnections, like the Arabian sea . Here, we show that ENSO also affects the predictability of MHWs in the Southwest Pacific, where the influence of ENSO is more ambiguous as it combines with other modes of variability like the MJO and smaller-scale intrinsic ocean variability .

Here in our historical record (1993–2024), we identified that long duration and large-scale MHWs around New Caledonia are associated with strong La Niña events during the austral winter (Fig. ), consistent with previous global studies of MHW drivers . Since ENSO is the climate mode with the highest predictability at seasonal timescales , this suggests that these particular types of MHWs are easier to predict. Indeed, our probabilistic approach, which allowed us to differentiate between long, intense, large-scale MHWs and smaller events, concluded that the predictability of the former is higher (Fig. ). This suggests that the high predictability of ENSO is responsible for the high predictability of these long, intense, large-scale MHWs , likely through anomalous large-scale oceanic and atmospheric conditions influencing air-sea fluxes . However, it should be noted that while forecast skill is generally higher in years affected by La Niña events (compare Figs.  and ), the difference in predictability between the two types of events occurs in other years as well, suggesting that it is generated by a mechanism independent of ENSO. Similarly, the MJO is another climate mode of variability with some predictability at subseasonal timescales known to influence MHWs . However its influence on MHW predictability has received little attention so far .

These particular types of MHWs should not overshadow the much more numerous smaller scale MHWs, which may have a significant cumulated impact despite being individually weaker . While anomalous air-sea fluxes linked to synoptic atmospheric events may play a role in the generation of such smaller MHWs, their poor predictability is consistent with ocean advection as their main driver, as previous studies have suggested for other regions . Indeed, ocean advection is largely chaotic in the study region and thus would be more difficult to forecast.

However, we also found that the scale at which MHWs are defined does not influence skill: contrary to our expectations, there was no improvement in skill when we defined metrics aggregated over a certain region or time period, thus ignoring information about the exact location of a MHW within this region or the exact date of occurrence within that period. This could be explained in two ways: either the small scales and the large scales are equally well-predicted, or MHWs generation is dominated by slower processes because of the thermal inertia of the ocean . The latter interpretation was favored by , who noted that “The consistency between the monthly and daily forecast skill is not surprising given that MHWs defined with daily data are still strongly driven by low frequency variability.” (see their supplementary section) We recommend that future research should focus on identifying the drivers of different types of MHWs in order to understand their sources of predictability.

proposed a three-step framework to help transfer MHW forecast information from researchers to stakeholders, which include ecosystem managers, marine industries like aquaculture or tourism, and indirect users like supermarkets or insurance companies. The first step is the usefulness of the forecast (having the appropriate model type, resolution, and domain, with model skill verified against observations), the second step is the useability (is the skill on timescales relevant to decision making, interpretable by end users, and delivered in a practical format?) and the third step is ensuring that the forecast is used (communication and user engagement). Here we discuss some aspects of our results relevant to the first two questions that can serve as the basis for co-constructing operational bulletins with marine stakeholders.

We performed our analysis with two seasonal forecasting systems (Météo-France System 8 MF8 and ECMWF SEAS5) that have similar configurations, especially for the representation of the ocean: both use the NEMO model at an eddy-permitting resolution (1/4°), although the products are distributed by Copernicus interpolated on a coarser 1° grid. However, NEMO is coupled with a different atmospheric model for each system and the initialization schemes are different. We focused on the scale of an EEZ, choosing New Caledonia as an example, because it represents a coherent unit, administered by a relatively small number of institutions (the New Caledonia government and three provinces) and consistent with the area of operations for many stakeholders (such as the local longline tuna fishery, tourist operators and conservation managers). While a higher resolution would result in a better representation of physical processes, particularly near the coast, there are currently no such operational products available, although Mercator Ocean International is in the process of expanding their 10 d forecast to subseasonal timescales (pers. comm.). The skill was evaluated by comparing the hindcasts against reanalyses that use the same ocean model (NEMO) at different resolutions: 1/12° for Glorys, used to evaluate MF8 and 1/4° for ORAS5, used to evaluate SEAS5. The discrepancy in resolutions could explain why SEAS5 performs better than MF8: the former resolves the same physical processes as its reference ORAS5 while the latter is lacking finescale processes that are resolved by the reference Glorys. Indeed, model resolution was shown to strongly influence MHW characteristics , especially in highly dynamic region like the South West Pacific .

Regarding the composition of bulletins, we first recommend using probabilistic metrics because they tend to be more skilled. However, this increased skill will need to be balanced with their relative complexity, compared with the more simple deterministic metrics, in order to create a forecast suitable for their intended audience and their level of familiarity with such forecasts. Previously, probabilistic metrics have been used to detect the occurrence of MHWs , while deterministic metrics have been used to describe their properties in more detail . Conversely, and assessed both types of metrics simultaneously and found that they have equivalent levels of skill. They concluded that an operational forecast should supply them both, as they provide complementary information. Here, through the use of multiple metrics and value thresholds, we were able to extract probabilistic information on the occurence of MHWs with different properties and at different spatial and temporal scales. Among the deterministic metrics, we found varying levels of skill: the monthly maximum of severity performs best, and the category worse, with other metrics (daily severity, number of MHW days per month and surface fraction in MHW state) falling in between. Among the probabilistic metrics (based on severity, category, number of MHW days per month and surface fraction) we found comparable levels of skill. These metrics may be suitable for different users depending on their exact needs. Since the forecast skill of the most extreme MHWs is higher than that of milder MHWs (covering a smaller area, shorter and less intense), an early warning system based on these more extreme MHWs would be both more reliable and more relevant, since they usually have the largest impacts.

Second, we recommend raising a “MHW alert” when an optimal decision threshold of 20 % is reached (i.e., when more than 20 % of the members predict a MHW) in order to maximize hits and minimize false alarms. This choice is supported by the shape of the ROC curves (Fig. ), and is also intuitive: 20 % is a round number and corresponds to a doubling of the climatological probability of a MHW (which the 90th percentile constrains at 10 %). The climatological probabilities of all the MHW events considered (for instance a MHW of category 2, or covering more than 50 % of the EEZ) needs to be made available along with the forecasts to provide context to guide their interpretation (Tables A1–A4). Also, for simplicity, we recommend a single value for all metrics and leadtimes in both forecast systems, even though the 20 % threshold will perform better in some cases than others. In practice, this means that an alert should be raised if more than 20 % of members predict a MHW. This value may seem low but it should be remembered that MHWs are rare events. Any probability higher than the climatological probability (about 10 % by design with the choice of a 90th percentile threshold, see Fig. e) can be considered an increase in risk. We used this decision threshold of 20 % to define an additional metric of direct relevance to stakeholders: the date of the next MHW when the forecast is started in normal conditions, which could be used in early warning systems, and the end date when the forecast is started during a MHW, which could be used to plan recovery efforts. Unfortunately, we found a low skill for both these metrics in the Météo-France forecast System 8 (Fig. ). The apparent discrepancy between this low skill for the start date and the high probabilistic skill can be explained by the fact that the start date metric gives an equal weight to all MHWs. On the contrary, a long MHW represents many time steps and will weigh more in the computation of probabilistic skill metrics than a short MHW. Thus, the few very long and very large MHWs (occurring in the austral winter during La Niña years) that are well predicted drive an increase in probabilistic skill, while the many ill-predicted shorter summer MHWs have a low influence on the AUC. If used for management, the use of these metrics about the start/end of MHWs will overestimate the risks, predicting that the next MHW will arrive too soon, or that the current MHW will last too long. This could cause many false alarms and result in a loss of credibility in the forecasts. Thus the usefulness of the forecast for a particular user will depend on the application and on the sensibility to false alarms. For instance if the forecast predicts a very long summer MHW, reef managers might be able to plan for continued monitoring of bleaching by requesting resources such as ships, divers or planes, and then cancel these efforts if the MHW stops earlier than expected . On the other hand, repeated false alarms might endanger the profitability of marine industries such as aquaculture if they are associated with investments that never pay off .

Third, since we found similar levels of skill when we performed the verification for each grid-point vs for averages in the EEZ, the presentation of the forecasts (in maps or timeseries) can be decided according to the preferences of stakeholders.

Regarding the interpretation of bulletins, we found that the skill of forecasts is highly variable in time at both seasonal and interannual scales. First, MHW forecasts targeting the warm season have lower skill than forecasts targeting the cold season. This seasonal pattern of skill has not previously been described, although some studies revealed that skill tends to be better in winter . Unfortunately, summer MHWs predictions would be more useful for marine stakeholders, because they are associated with the largest impacts as temperatures can exceed the upper thermal tolerance limit of organisms . For instance coral bleaching, one of the most visible impacts of MHWs, strictly occurs during summer. For this reason, many skill evaluation focus on the summer MHWs or on bleaching metrics . However it should be kept in mind that winter MHWs also have impacts and could benefit from reliable forecasting, for instance for monitoring invasive species or HAB (harmful algal bloom). Second, forecasts coinciding with strong La Niña have more skill than forecasts made during neutral or El Niño phases. Thus, the leadtime at which a forecast can be considered skillful is highly variable, depending on the metric of interest, but also the season and the ENSO phase, ranging from virtually no skill even at short leadtimes (e.g. for deterministic metrics in Austral summer) to sustained skill 6 months in advance (e.g. probabilistic skill in Austral winter). The level of interannual variability could also inform the choice of metrics, as it might be easier to communicate forecasts based on metrics that have the same level of skill every year (such as the severity monthly maximum) We stress that information on skill should be an integral part of bulletins, as it can influence whether the forecast is acted upon or not, in accordance with the risk-profile of each stakeholder. For instance some stakeholders may want to adopt a precautionary position and take action even if the forecast has a high uncertainty, while others may want to minimize false alarms by only acting on skilled predictions (i.e., for winter MHWs during La Niña).