Operational ocean forecasting systems integrate remote observations, in situ measurements and modelling systems, and have been widely recognized as important assets for ocean state monitoring (von Schuckmann et al., 2016) and the development of the blue economy (She et al., 2016). In such a framework, the operational monitoring and forecasting of marine biogeochemistry and ecosystem dynamics is based on biogeochemical models designed to represent the low-trophic-level ecosystem (i.e. from phytoplankton to zooplankton). The improvement of their predictive capability on weekly and seasonal timescales mostly required by users is strongly related to the development of data assimilation capacity, while their quality assessment is constrained by the availability of reference data, both remote and in situ (Gehlen et al., 2015), and possibly independent (i.e. not assimilated; Gregg et al., 2009). In this perspective, efforts to establish a stronger link between operational biogeochemistry products and potential users from the fisheries and environmental science communities are constantly increasing (Berx et al., 2011; Payne et al., 2017).

The European Copernicus Marine Environment Monitoring Service (CMEMS)

http://marine.copernicus.eu/, last access: 25 July 2019.

The Mediterranean Sea Monitoring and Forecasting Centre (Med-MFC; Tonani et al., 2013) is one of the regional systems and is composed of the physical system “Med-PHY” (Tonani et al., 2008; Oddo et al., 2009), which drives the biogeochemical system “Med-BIO” (Lazzari et al., 2010, 2012) and the wave system “Med-WAV” (Ravdas et al., 2018). In recent years, following the CMEMS requirements, the Med-MFC has been consistently upgraded in the physical (Oddo et al., 2014; Clementi et al., 2017; Pistoia et al., 2018), wave (Zacharioudaki et al., 2018), and biogeochemical components (Cossarini et al., 2015; Lazzari et al., 2016), including also data assimilation (Teruzzi et al., 2014; Storto et al., 2015).

More specifically, the last major upgrade of Med-BIO focused on the increase in horizontal resolution from 1/16 to 1/24∘. The upgrade also involved different aspects of the forecasting system aimed to improve the alignment with the physical component: the new non-linear free-surface curvilinear z*-coordinate configuration used in NEMO3.6 (see Madec, 2016, for the NEMO implementation and further details) and the terrestrial input boundary conditions layout, now including 39 rivers. Moreover, Med-BIO improved the former data assimilation scheme (Teruzzi et al., 2014), extending the assimilation of surface chlorophyll concentration to coastal areas (Teruzzi et al., 2018) and reducing the time to solution through a parallelization of the cost function solver (Teruzzi et al., 2019a).

Both historical and near-real-time data of observation systems are strategic to evaluate the quality of operational oceanography products (She et al., 2016). However, although operational ocean models are designed to span the whole water column from the surface to the bottom and are now reaching the sub-mesoscale description, deeper ocean and mesoscale remain still not adequately sampled by operational observation systems (Bell et al., 2015; Hernandez et al., 2018). The assessment of the operational ocean products accuracy already benefits from international intercomparison initiatives (e.g. the GOV Task Team for Intercomparison and Validation, ICV-TT; Bell et al., 2015), which also define specific protocols to quantify the quality level of core variables delivered to users (Hernandez et al., 2015; Ryan et al., 2015). This is applicable to the Mediterranean Sea operational systems which include, besides CMEMS Med-MFC, also the Poseidon operational system built on the HYBRID-POM-ERSEM model coupling (Tsiaras et al., 2017; Petihakis et al., 2018).

More in general and concerning biogeochemical applications in the Mediterranean Sea, the limited availability of observational reference data often hinders the validation assessment of model products. The most common approach is based on contrasting model outputs with satellite-derived surface chlorophyll (Tsiaras et al., 2017, for year 2000; Mattia et al., 2013, Macias et al., 2014, Guyennon et al., 2015, Richon et al., 2017, for a portion or the whole investigated multi-year periods). In situ measurements from vessels and scientific cruises, also used in Richon et al. (2017) and Guyennon et al. (2015), can only validate limited temporal and spatial subsets of the simulations (i.e. time series of fixed stations or single transects in a very confined time range). On the other hand, a few basin-wide validation frameworks, especially for nutrients, are based on comparison with climatology, e.g. Tsiaras et al. (2017) used a seasonally aggregated reference for the whole Mediterranean Sea built on 1990–1999 data from SeaDataNet

https://www.seadatanet.org/, last access: 25 July 2019.

In the present paper, we focus on the CMEMS Mediterranean biogeochemical analysis and forecast system products (delivered from April 2018) and we introduce novel skill metrics based on the comparison between model products and the BGC-Argo floats data. According to the definition adopted within the Copernicus community (Hernandez et al., 2018), our model validation follows two main tasks:

The pre-operational qualification, which is performed when a new version of the system is developed and a full range of validation metrics is applied to provide an evaluation of the skill performance of the model. The qualification is carried out over a short reanalysis run (e.g. a couple of years) which then provides the initial conditions for the operational analysis and forecast run.

Chlorophyll data are derived by the multi-sensor (MODIS-AQUA and NPP-VIIRS) CMEMS daily product

OCEANCOLOUR_MED_CHL_L3_NRT_OBSERVATIONS_009_040.

In situ observations of chlorophyll, nitrate, and oxygen concentrations are derived by the BGC-Argo floats dataset whose records start from 2013. BGC-Argo floats data are downloaded from the Argo Global Data Assembly Centre web portal and processed following the advanced product quality procedure by Schmechtig and Thierry (2016).

BGC-Argo chlorophyll (Chl) adjusted data are derived from real time (RT) data with a series of corrections: the quenching correction (Xing et al., 2012), a recalibration at depth (i.e. by imposing zero for Chl values below 600 m), and a tuning correction (i.e. data are further divided by a factor of 2) due to the detection of an error in the manufacturer calibration of the Chl fluorometer (Roesler et al., 2017). BGC-Argo nitrate concentrations (NO3) were obtained by using the Johnson and Coletti (2002) algorithm on the raw UV absorption spectrum, then corrected with quality control procedures described in Pasqueron de Fommervault et al. (2015). BGC-Argo oxygen data (O2) are estimated after the application of a quality protocol based on a linear regression constrained to pass through the origin between percent oxygen solubility values derived from the float profiles of O2 and climatological values from the World Ocean Atlas Climatology (Takeshita, 2013; Schmechtig and Thierry, 2016). For the pre-operational period 2016–2017, the total amount of floats and profiles for each variable is given in Table 1.

In situ observations of nitrate, phosphate, and oxygen derived by the National Oceanographic Data Centre of OGS (NODC-OGS) dataset covering the period 1999–2013 (the list of cruises and datasets is in Lazzari et al., 2016) are used to compute reference climatological profiles for the sub-basins of Fig. 2. In situ observations of DIC, ALK, and pH (the list of cruises and dataset sources is in Cossarini et al., 2015) are used to compute reference climatological annual profiles in the sub-basins of Fig. 2. Literature data of net primary production are based on multi-annual simulation (Lazzari et al., 2012), satellite model (Colella, 2006), and in situ estimates (Siokou-Frangou et al., 2010), here used to validate the basin-scale consistency of the corresponding model product.

The assessment of the CMEMS Mediterranean Sea biogeochemical model system follows two tasks: pre-operational qualification of the model system and routine (or NRT) validation of forecast products. The aim of the pre-operational assessment is to verify the model consistency, which is its capability of reproducing the salient characteristics of the Mediterranean Sea ecosystem, by comparing a short reanalysis run with historical datasets, climatology, and literature estimates. The timescale of the comparison ranges from daily to seasonal. On the other hand, the operational assessment relies on the NRT observation availability and aims to evaluate the forecast skills with a temporal scale of days.

This work presents the recent achievements in the operational biogeochemical component for the Mediterranean Sea delivered by CMEMS. The MedBFM model system has been integrated with the last scientific achievements of the BFM model (Cossarini et al., 2015; Lazzari et al., 2016), the 3DVarBio assimilation scheme (Teruzzi et al., 2018, 2019a), and the non-linear free surface and volume vertical layer parameterization of the transport operator of the OGSTM model (Salon et al., 2018).

The Med-BIO system has followed the developments of the EU operational marine services (Le Traon et al., 2017), starting from its first version (Lazzari et al., 2010) deployed within the MERSEA project (2004–2008; GMES implementation phase), becoming pre-operational during the MyOcean projects series (2009–2015; GMES demonstration and pre-operational phase), and finally establishing a regular and validated operational product delivery in CMEMS (GMES operational phase). Across this 10-year period, the quality of the Med-BIO products has significantly increased (Fig. 14, quality assessed by the RMSD of the surface chlorophyll concentration, the only product variable that has been consistently validated since the beginning of the Med-BIO activity), with a continuous improvement which has taken advantage of the implementation of the data assimilation, the increased spatial resolution, and the evolutions in the physical component of the Med-MFC system.

The Med-BIO off-line coupling with Med-PHY was outlined since the preliminary work of Lazzari et al. (2010) and has allowed for distinctive developments of the different components. Further, the alignment between physical and biogeochemical models in terms of same horizontal resolution, bathymetry, boundaries (number and position of rivers), and surface forcing (e.g. z* parameterization) – a requisite of the CMEMS framework – guarantees the consistency of the results (as shown by the recent improvement of the performance after April 2018, Fig. 14). Other studies demonstrated that off-line coupling does not affect the transport of biogeochemical tracers when the sub-mesoscale physics is degraded to mesoscale (Lévy et al., 2012). Further improvements of the Med-BIO biogeochemical model system in terms of physical–biogeochemical consistency at local scale are expected with the foreseen implementation of the assimilation of the BGC-Argo floats data (Cossarini et al., 2019), which has shown improvement in the model solution due to the increased consistency of vertical dynamics by the assimilation of the physical and biogeochemical profiles at the same time and position. This result highlights the importance of the joint physical and biogeochemical assimilation, which has been recently demonstrated in a twin experiment to provide superior results with respect to any uncoupled assimilation configuration (Yu et al., 2018).

Communicating the uncertainty is a critical point: it helps the users to properly interpret the validity of the forecast products, even when the forecast actually fails, and to minimize any problem created by the misuse (and misinterpretation) of them (Stow et al., 2009; Payne et al., 2017). The communication of the level of uncertainty in the biogeochemical products remains an open issue for the scarcity of reference NRT biogeochemical observations available and for the complexity of biogeochemical models, which may have tens of variables but only a few can be validated. Further, regional operational models have reached the limit of the sub-mesoscale, which is not adequately sampled by observational systems (Hernandez et al., 2018). As an example, the number of dissolved oxygen observations used to build Fig. 6 is almost one-fifth of those available for phosphate (Teruzzi et al., 2019b); therefore, the reliability of validation using the climatological profiles might be lower and even less for the surface values, since dissolved oxygen exhibits significant seasonal and high-frequency cycles due to the air–sea exchanges mediated by solubility.

We show that, depending on the variables, different uncertainty levels can be provided on the basis of the availability of reference data. In this context, the validation analysis provides a “degree of confirmation” (Oreskes et al., 1994) with respect to the different scales of variability derived from the available observations. GODAE Class 1 metrics show that the model is consistent (in terms of chlorophyll, nitrate, phosphate, oxygen, dissolved inorganic carbon, alkalinity and pH) in reproducing the vertical profile climatology at sub-basin scale (Figs. 3 to 8). The comparison of model primary production with available basin-wide estimates and literature collection can only provide a consistency confirmation of the model estimates at the basin and annual scales. Then, we also demonstrate that GODAE Class 4 metrics are feasible and provide more rigorous skill performance down to the scale of a week and mesoscale, but only for a limited number of variables (see Figs. 9 to 13). Regarding data availability, satellite chlorophyll estimates represent the most reliable source of NRT data, which, however, allow us to investigate only the cloud-free surface of the ocean. The novel BGC-Argo floats dataset empowered us to design new skill metrics, showing the capability of the MedBFM model to reproduce the temporal evolution of the vertical dynamics of the phytoplankton, nitrate, and oxygen, and to assess key ecosystem processes in the Mediterranean Sea. The novel metrics based on BGC-Argo data disclose new and important perspectives for the model validation in the Mediterranean Sea, also considering its very high spatial heterogeneity and the seasonal variability of the coupled physical–biogeochemical processes. However, some caution should be taken before generalizing the conclusions, since the relatively poor BGC-Argo floats coverage in some areas and the on-going improvement of product quality procedures of the BGC-Argo data (Johnson et al., 2017). Moreover, the relevance of the representation error (Hernandez et al., 2018) becomes stricter with BGC-Argo data, since the skill performance analysis is based on comparing a model output with grid cells 3–4 km wide and on the order of tens of metres thick with point-like profiles of a few metres of resolution, thus the model may miss part of the spatio-temporal scales present in the observations (Oke and Sakov, 2008).

Considering the NRT evaluation, Figs. 12 and 13 show that the uncertainty of the forecast products is of the same order (and in some occasions even lower) than the pre-operational run, and that the performance decreases slightly from the first day of forecast to the following ones. This is related to both the uncertainty due to the intrinsic error of the biogeochemical model and the decrease in the performance of the physical variable forecasts driving the biogeochemical ones (see as reference Clementi at al., 2018, and the CMEMS quarterly validation statistics

Available at http://marine.copernicus.eu/services-portfolio/scientific-quality/, last access: 25 July 2019.

We show that the error statistics, in terms of RMSD, are proportional to the variability of the variables. It is shown for surface chlorophyll (Fig. 8) and it can be also noted from BGC-Argo-derived statistics: sub-basins characterized by higher variability have higher error (Table 3). As a result, the performance analysis shows that the western regions have, in general, largest variability and lower performance, specifically during the winter season. Thus, to rationalize the costs of observing systems (Cristini et al., 2016), it may be more efficient to sustain the observing systems with high-frequency observations in high-variability areas. On the other hand, given that field variability may be related to local physical and biogeochemical processes (e.g. vertical mixing, coastal effects due to strong topographic gradients, or terrestrial inputs), the reduction of the model representativeness error can benefit from a more collaborative evolution of the coupled physical–biogeochemical systems, both in terms of process modelling or coupled data assimilation (Fennel et al., 2019).

The present validation framework uses an a priori subdivision which considers the biogeographic approach of D'Ortenzio and Ribera d'Alcalà (2009), and the subsequent refinement proposed by Lazzari et al. (2012) which showed different characterizations according to the Longhurst paradigm. The recent review of Ayata et al. (2018) discusses the variations in the Mediterranean Sea subdivision found in literature, highlighting regions with relatively homogeneous conditions and some heterogeneous regions featuring significant mesoscale activity. Our validation approach demonstrates the importance to provide model uncertainty estimation at different spatial and temporal scales, emphasizing the model capability to reproduce specific processes and their intensity in different areas, while computing metrics over the sub-basins allows us to synthesize the heterogeneity of the Mediterranean Sea, justifying a posteriori our sensible definition of the 16 sub-basins. In fact, the comparison of nutrient profiles in Fig. 5 highlights the satisfactory model performance in reproducing the mean spatial gradients and the possible anomalies, such as the underestimation of upper-layer nitrate in the nwm sub-basin. Moreover, the use of simple indexes, such as means and standard deviation, and of functional spatio-temporal subdivisions increases the readability of the uncertainty communication, which responds also to the request for a user-oriented evolution of the validation framework in operational systems (Hernandez et al., 2018).

Our validation results point out a number of strengths and weaknesses of the CMEMS Mediterranean forecasting biogeochemical system. A strength is that the MedBFM is operationally in place and provides validated and reliable ecosystem products consistently with the physical ones (Clementi et al., 2018).

The system can also provide important feedbacks to the observing autonomous systems. Indeed, the NRT comparison of BGC-Argo floats data with the forecast outputs with respect to the seasonal benchmarking might be beneficial for an additional quality control procedure for detecting anomalous observations that the present quality control fails to detect, as proposed for physical variables measured by Argo systems (Ingleby and Huddleston, 2007). As an example, the Class 4 metrics applied to BGC-Argo oxygen data (Fig. 13, third column) shows a systematic bias which does not appear when contrasted with the Class 1 validation (Figs. 5 and 7), thus pointing out the opportunity of a possible revision of some model formulations or product quality procedure of BGC-Argo oxygen data in the Mediterranean Sea. A specific investigation focused on the oxygen validation framework and the analysis of the oxygen variability simulated by the MedBFM model is in preparation.

Another positive aspect of our work is that, to the best of our knowledge, this is one of the first times that a consistent validation procedure provides sustainable guidelines following GODAE metrics for operational marine biogeochemistry exploiting the BGC-Argo floats data (Hernandez et al., 2018). Our novel metrics (Figs. 9 and 10) provided indications of the model skill performance on some key biogeochemical processes (DCM, nutricline depth), thus setting an advancement to what is described in Hernandez et al. (2015) for NRT assessment of biogeochemical operational forecast and maximizing the values of the available NRT biogeochemical observations (She et al., 2016). In perspective, the integration of BGC-Argo within operational ocean forecasting systems in terms of data assimilation (see Cossarini et al., 2019) becomes strategic for an in-depth study of the interior of the sea and its dynamics. Moreover, considering that BGC-Argo floats also provide profiles of physical quantities (i.e. radiometric quantities – PAR – and temperature), an analysis of specific physical (e.g. mixed layer depth, euphotic zone depth) and biogeochemical (e.g. NITRCL, MWB, DCM) indexes that can reveal relationships between the shape and/or intensity of the profiles and the underlying dynamics would allow us to further delve into coupled vertical physical–biogeochemical processes. In such a view, our work provides a first step to identify and quantify several functional biogeochemical indexes. Nonetheless, the availability of a sufficient amount of profiles for variables like nitrate and oxygen remains a critical point, which may allow for statistically significant analysis.

Concerning the weaknesses of MedBFM, we may point out the reduction in performance close to the domain boundaries at Gibraltar and Dardanelles straits and in the coastal areas. The observed overestimation of chlorophyll, and thus of productivity and phytoplankton biomass, in the Alboran Sea (see Figs. 3 and 8) can be related to an incorrect parameterization of the biogeochemical fluxes through the Strait of Gibraltar or to the effect of vertical mixing. Inconsistent physical–biogeochemical data assimilation might generate incompatible density and nutrient profiles that may generate an extra amount of vertical flux of nutrients in this highly dynamical area, thus enhancing its productivity. Increase in nutrient availability along isocline surfaces has been observed by Raghukumar et al. (2015) suggesting this as a possible cause of an increase in productivity in oligotrophic areas. The upgrade of MedBFM boundary conditions (at the Strait of Gibraltar, Med-PHY is coupled with the CMEMS global product while Med-BIO uses climatological biogeochemical value) with high-frequency values, and the extension of the Atlantic buffer zone, could improve the model performance in this area.

For the coastal areas, the increased resolution to 1/24∘ cannot fully balance the use of low-frequency data of biogeochemical terrestrial inputs (i.e. nutrients and carbonate system estimates from climatological databases). Thus, some quality decrease is observed even though data assimilation of coastal chlorophyll from satellite can partly reduce this deficiency (Teruzzi et al., 2018). Operational or at least higher-frequency coastal data for rivers and the inclusion of the Dardanelles as an open boundary condition are requested to account for the user needs of reliable products in coastal areas.

The present work evaluates the skill performance of the CMEMS Mediterranean biogeochemistry component (Med-BIO) determining the quality of the CMEMS biogeochemical products on the basis of two complementary phases: (1) the pre-operational qualification run (2016–2017) and (2) the operational workflow (started in April 2018 for MedBFMv2.1).

Using different observation reference datasets (from satellite, literature, climatology, BGC-Argo floats), GODAE Class 1 and Class 4 metrics have been applied to the MedBFM model system in order to quantify its consistency in simulating the key features of the Mediterranean biogeochemistry, and its accuracy to routinely reproduce the observations at their specific time and locations. New metrics specifically designed to exploit the richness of the BGC-Argo floats dataset and to evaluate the model capability to reproduce the key elements of the vertical profiles of chlorophyll and nitrate have been proposed. Main results can be here summarized:

MedBFM is consistent in reproducing the general characteristics of biogeochemistry in Mediterranean Sea, and the CMEMS Med-BIO products are well within the climatological variability; quantified correlation values are higher than 0.9 and 0.7 for nutrients and carbonate system products, respectively.

Finally, the validation metrics here presented provides indications of some weaknesses of the Med-BIO (e.g. limited dynamics in coastal areas, Gibraltar boundary, and sub-mesoscale effects on phytoplankton dynamics in the western area) that will lead to future developments. Nevertheless, the validation results support not only the accuracy of the CMEMS Med-BIO products, but also the consistency of the MedBFM model system to simulate the fundamental coupled physical–biogeochemical processes, which is corroborated at the mesoscale and weekly scales.