The global ocean is a major sink for anthropogenic carbon and therefore an important contributor to slowing down the human-induced global warming (Stocker et al., 2013). To calculate the oceanic CO2 uptake, various models have been used to interpolate scarce CO2 measurements in the surface ocean spatially and temporarily to obtain basin-wide (e.g., Zeng et al., 2002; Lefèvre et al., 2005; Chierici et al., 2006; Sarma et al., 2006; Jamet et al., 2007; Friedrich and Oschlies, 2009; Telszewski et al., 2009; Takamura et al., 2010; Landschützer et al., 2013; Nakaoka et al., 2013; Iida et al., 2015; Goddijn-Murphy et al, 2015) and global ocean CO2 maps (Takahashi et al., 2002, 2009, 2014; Park et al., 2010; Rödenbeck et al., 2013; Sasse et al., 2013; Jones et al., 2015; Zeng et al., 2015). The Surface Ocean CO2 Mapping (SOCOM) inter-comparison initiative revealed varying degrees of differences among 14 models (Rödenbeck et al., 2015), of which 5 used neural networks. They include self-organizing maps (SOMs) and feedforward neural networks (FNNs). The SOM has a long history in CO2 mapping (Lefèvre et al., 2005; Friedrich and Oschlies, 2009; Telszewski et al., 2009; Nakaoka et al., 2013). Recently, the FNN has been gaining popularity in this field (Landschützer et al., 2015; Zeng et al., 2014, 2015). In this investigation we introduce a machine learning model called a support vector machine (SVM) for ocean CO2 mapping and compare the SVM with the SOM and FNN. We intend to provide a practical guide for using these machine learning models.

The machine learning models included in this study cannot directly model the long-term trend of CO2. Therefore, we express the dependence of CO2 fugacity (fCO2) on year (YR), month (MON), latitude (LAT), and longitude (LON) as the sum of a nonlinear static component and a linear trend component: fCO2=FstaticLAT,LON,MON+Ftrend(YR). As available observations are scarce with respect to the biogeochemical properties of the surface ocean, we used sea surface temperature (SST), sea surface salinity (SSS), chlorophyll-a concentration (CHL), and mixed layer depth (MLD) as the proxy variables of space and time. These proxy variables were commonly used by models included in the SOCOM. The model equation becomes fCO2=FstaticLAT,SST,SSS,CHL,MLD,dSST+FtrendYR where dSST denotes the difference between the monthly and annual means of SST. Here we excluded LON and MON. They have a circular property and therefore cannot be used directly. For instance, longitude -180∘ is geographically connected to longitude 180∘, but numerically they appear to be two extreme longitude values to the models. Zeng et al. (2014, 2015) circumvented this problem by using sine and cosine transformed components. Their approach could unintentionally enhance the influence of LON and MON on fCO2 as one more derived variable from each of them was added to the model. We excluded LON in the belief that the combination of SST, SSS, CHL, and MLD contains sufficient spatial information, but retained LAT for its different seasonal and geophysical meanings in the Northern and Southern hemispheres. Replacing MON with dSST also improves the expression of the effect of season geographically.

We extracted monthly fCO2 from the track-gridded database of the Surface Ocean CO2 Atlas (SOCAT) version 3.0

http://www.socat.info/

The monthly SST data of 1990 to 2015 were extracted from the Optimum Interpolation (OI) V2 product

http://www.esrl.noaa.gov/psd/data/gridded/data.noaa.oisst.v2.html

The Appendix and Table 1 summarize the algorithms of the three models. Here we focus on discussing their usage in CO2 mapping.

The trend in Eq. (2) cannot be modeled directly by the models. One approach to dealing with the problem is to normalize the measurements to a reference year using a global rate and to only model the nonlinear component. Zeng et al. (2014) presented a method to model the linear component in Eq. (2). Instead of repeating the process, we used their annual rate of 1.5 µatm to remove the trend from fCO2 to normalize it to the reference year 2005, i.e., fCO2normalized=fCO2-1.5⋅(YR-2005). Although Takahashi et al. (2014) obtained a global mean rate of 1.9 µatm yr-1, we used 1.5 µatm yr-1 as this rate was obtained by using the gridded fCO2 of SOCAT version 2. The normalized fCO2 was used to model the nonlinear component in Eq. (2). In later discussions, fCO2 means the normalized fCO2 unless explicitly stated. Similarly, we applied the log transform of Zeng et al. (2014) to CHL prior to data scaling discussed below, i.e., CHL=log⁡10(1.0+CHL).

We examined the goodness of fit by randomly selecting 10 to 50 % of the data points to train the FNN and SVM, and to label the SOM; and then calculated the correlation coefficient between modeled and observed CO2 of the selected data points.

The SOM yields the best correlation in the case of 10 % of randomly selected data points and the correlation decreases with the number of data points (Fig. 1). The reason is that for a given number of neuron cells, the fewer the data points, the less likely a neuron cell will be labeled by multiple measurements and the more likely that the prediction will find the same CO2 value used for labeling. Therefore, the goodness of fit does not necessary mean good SOM modeling.

The correlations obtained by the SVM and FNN do not vary much with the number of data points. While the SVM's correlation decreases monotonically, even though by only a little, with the number of data points, the FNN's correlation obtained with 75 000 data points is larger than that with 60 000 data points. The FNN is known for not being able to find the global optimum in training. This case could be an indication of an imperfect training. The FNN appears inferior to SVM in all cases. However, imperfect training does not account for all the differences. If we use the number of model parameters to be determined by the training as the indicator of the dimension of the model space, the FNN's dimension is far smaller than that of the SVM. The former is determined by the number of hidden neurons and input variables, whereas the latter is determined by the number of training data. For 6 input variables, 15 000 training data, and 64 hidden neurons, the number of parameters is 509 for the FNN and 15001 for the SVM.

A better indicator of the performance of the models would be the goodness of prediction. To emulate the situation that the sampled area was only a small portion of the global ocean, we evaluated the goodness of prediction by training FNN and SVM and labeling SOM with 10 % of randomly selected data to make a prediction for the rest of the data. Figure 2 shows that the SVM yielded the best correlation (R2= 0.72), the FNN fell behind (R2= 0.67), and the SOM performed the worst (R2= 0.54). The differences between predicted and observed fCO2 are 0.1 ± 17.4 µatm for SVM, 0.1 ± 18.9 µatm for FNN, and 0.2 ± 23.3 µatm for SOM. Compared to the variation of fCO2 measurements, these differences are small and their uncertainties are on the same order of magnitude as the variation of measurements. Let us examine the standard deviation (SD) of fCO2 in those grids with at least three data points. The track-gridded fCO2 in SOCAT version 3.0 includes an SD ranging from 0.1 to 71.2 µatm and the mean is 5.2 µatm. Calculating the SD of normalized fCO2 in the same grids and in the same months of all years yielded a mean of 12.5 µatm in the range of 0.1 to 103.1 µatm. The normalization had little effect on the SD as the calculation for non-normalized fCO2 gives a mean SD of 14.6 µatm in the range of 0.1 to 107.5 µatm.

From the algorithm of SOM in the Appendix, it is not difficult to see that the SOM does not make extrapolation – the model always approximates new inputs by the measurements used for training and approximates fCO2 by the measurements used for labeling; therefore, the predicted fCO2 values are within the observed fCO2 range (Fig. 2a). Figure 2c shows that the extrapolated fCO2 by the SVM, if any, did not exceed the observed range. To investigate the extrapolation risk, we used 200 000 data points randomly generated for SST, dSST, SSS, MLD, and CHL in the range of (0, 40 ∘C), (-20, 20 ∘C), (20, 50), (1, 1500 m), and (0 log(mg m-3), 2 log(mg m-3)), respectively. These ranges are larger than the corresponding observed ranges of (0, 34 ∘C), (-13, 16 ∘C), (24, 40), (1, 1000 m), and (0 log(mg m-3), 1.2 log(mg m-3)). The SVM and FNN produced fCO2 in the range of (267, 468 µatm) and (199, 596 µatm), respectively, for the simulated samples. Compared to the observed fCO2 range of (240, 560 µatm), the experiment indicates that the over-extrapolation risk of the SVM is low.

Figure 3 shows fCO2 maps in February and July 2005, which is the reference year for normalization. In the mapping, we randomly selected 50 % of the data to train the FNN and SVM and to label the SOM. All models captured the major features of observed fCO2 distribution. The SOM exhibits obvious discontinuity because of its discrete characteristics of picking up fCO2 values from the labeled SOM. For year 2005, the mean fCO2 difference is -0.05 ± 12.73 µatm for FNN-SVM and -0.6 ± 18.80 for SOM-SVM. The uncertainty is the standard deviation of the mean difference between predicted and observed values. The statistics indicates that FNN agrees better with SVM than SOM does.

Although the differences among models might be on the order of 10 to 20 µatm, the effect on the global ocean CO2 flux estimate is small (Fig. 4). The fluxes are calculated using the wind speed from ECMWF's interim product (Dee et al., 2011). Our estimate for the oceanic uptake is on the higher end among those in Wanninkhof et al. (2013) and Le Quéré et al. (2015). For example, Wanninkhof et al. (2013) reported that the median sea–air anthropogenic CO2 fluxes centered on year 2000 ranged from 1.9 to 2.5 PgC yr-1 among the seven models. In comparison, our estimates by the three models are about 2.4 PgC yr-1. The mean difference of CO2 flux is 0.02 PgC yr-1 between the FNN and the SVM (FNN-SVM) and 0.06 PgC yr-1 between the SOM and the SVM (SOM-SVM). They are small in comparison with those differences among the models in Wanninkhof et al. (2013) and Le Quéré et al. (2015). Note that the flux estimate is highly dependent on wind products as shown by Wanninkhof et al. (2013) and Zeng et al. (2014).

On the spatial scale of tens of degrees, the three models show good mutual agreement for modeled fCO2 distributions among them. However, each model shows distinguished fine structures, which are determined by the biogeochemical processes in the ocean, by model parameters obtained from training, and by the characteristics of the models. We believe that the modeled monthly fCO2 distributions are true to the degree given by the model validations.

The main features of the three machine models are listed in Table 1. The SVM is recommended when the computer has enough memory to store the matrix in Eq. (A18), which is proportional to the square of the number of training data. The SVM performs the best, but the training time could become very long when the number of training data is too large to be handled by a computer without using virtual memory. For any given dataset, using the SVM requires a prior step to find the optimal value for the parameter σ in Eq. (A10) and the parameter γ in Eq. (A11).

The FNN model does not perform as well as the SVM, but the number of training data does not affect its training as much as the SVM's. The training time can become long when a large number of hidden neurons are used and many iterations are needed to achieve convergence. It takes a longer time to train the FNN than the SVM for a small number of data points. However, the FNN is simpler to use as it requires no prior step. However, it may have the risk of over-extrapolation.

The SOM is recommended only when the other two models have over-fitting or over-interpolation problems. The SOM performs the worst and is not as straightforward as the others as its result depends too much on data scaling and the number of neurons. An advantage of the SOM is that once trained, re-labeling the SOM with new CO2 measurements and making a new prediction is fast. Although the SOM does not have the over-extrapolation problem of the FNN, it may produce nonsense predictions due to its strong dependence on data scaling.

In areas where there was no measurement on a large scale, predictions made by the models must be treated conservatively, as SVM and FNN may produce extrapolated results and SOM may extract CO2 from unexpected provinces. Figure 3 shows that the modeled CO2 east of the African coast near the Equator in July 2005 (Fig. 3) appeared much higher than the nearby measurements, which were made in July 1995 and adjusted to 2005 using the global rate of 1.5 µatm yr-1. However, considering the large variations of the rate from region to region (Takahashi et al., 2014) and of the repeated measurements discussed in Sect. 5, the measurements were not sufficient to support rejecting the modeled CO2. Similar CO2 hotspots occurred in the Southern Ocean west of South America in February 2005, around the latitudinal zone of 50∘ S. The modeled CO2 distributions by Takahashi et al. (2014) also showed CO2 hotspots around the latitudinal zone of 30∘ S in the same month and region. Their model used a completely different interpolation scheme based on a diffusion–advection transport model for surface waters. In principle, this hotspot CO2 was produced by our models using measurements somewhere else where the biogeochemical properties were similar to those in the hotspot areas. As the SOM does not make extrapolation, the SVM has a low possibility of over-extrapolation, and the hotspots appeared in all models, the risk of accepting them would not be high.