Data assimilation (DA) strives to improve the forecast skill of a numerical model by combining the model with observations. Observations are incorporated into the model by applying a series of corrections to the internal state of the model. As the state variables of a numerical model are usually not observed directly, this procedure requires an observation operator (OO) to project the model state variables onto the variable that is observed. The difference between the observation and the model prediction, the so-called innovation, forms the basis for calculating the correction to the model state. The accuracy of the OO is paramount in this process: any bias in the projection will lead to a bias in the innovation and therefore result in a biased correction to the model state. For this reason, bias correction procedures have been built considering not only systematic errors in observations but also in observation operators (see e.g. , for satellite radiance data).

Many different types of OO exist. In its simplest form, an OO could just select one of the state variables in a point near the observation or, perhaps, perform an interpolation. More complex OOs may include corrections for processes that influence the observation but are not modelled or are insufficiently modelled. Ultimately, one could even consider a dynamical OO that wraps a second numerical model to locally refine the results of the parent model. The latter solution may very well provide the most accurate results, but the vast number of observations that need to be assimilated in a typical atmospheric or oceanographic model means that this approach would require a prohibitive amount of computing resources. This limits OOs in most practical applications to relatively simple parameterisations in terms of the model state variables. Moreover, variational data assimilation requires observation operators to be linearised around the background within the inner loops (tangent-linear approximation). This translates into a need to construct OOs that can be formally and practically differentiated.

This paper presents a method of parameterising the results of a specialised model in such a way that it can be efficiently used within an OO. The parameterisation is based on canonical correlation analysis (CCA), a well-established mathematical method for finding cross-correlations between datasets. A new pseudo-dynamical OO is generated using the canonical correlation between the inputs and outputs of the specialised model on a large and representative dataset. Once this correlation has been calculated, the application of the pseudo-dynamical OO involves only a matrix multiplication that can be performed at a fraction of the computational cost of the dynamical OO. A similar method has been used previously to build reduced-order OOs in atmospheric data assimilation .

This work is part of the SOSSTA (Statistical-dynamical observation Operator for SST data Assimilation) project, funded by the EU Copernicus Marine Environment Monitoring Service (CMEMS) through the Service Evolution grants. The aim of SOSSTA is to formulate an efficient OO for sea surface temperature (SST) DA that accounts for the diurnal variability of the ocean skin temperature. The results of the project are presented in multiple publications. The modelling of the diurnal cycle of SST is described in , while the current paper focuses on the method for constructing the OO. The project includes pilot studies in the Mediterranean Sea and the Aegean Sea that will be described in forthcoming publications.

The paper is organised as follows: Sect.  provides a quick review of CCA; Sect.  discusses how CCA can be used to construct the OO matrix; Sect.  applies the CCA OO to the modelling of satellite sea surface temperature (SST) measurements in oceanographic models; and Sect.  discusses the performance of the method and other possible applications. Conclusions are presented in Sect. .

CCA is a method to find cross-correlations between two datasets X and Y. The datasets are considered to be matrices structured such that the columns represent different variables and the rows represent the measurements of these variables. CCA then aims to find transformation matrices A and B that transform the anomaly of the variables of X and Y, denoted X′ and Y′, into the set of canonical variables F and G: 1F=X′AG=Y′B.

The structure of F and G matches that of X and Y. The canonical variables are constructed such that the variable Fi is maximally correlated with the variable Gi. At the same time, both Fi and Gi are uncorrelated with Fj and Gj for i≠j; therefore, each additional canonical variable describes the maximal remaining correlation between the two datasets. The number of canonical variables that can be obtained with this procedure is limited to the smallest number of variables in X or Y.

The calculation of the matrices A and B is relatively straightforward using the algorithm of . Writing the requirements outlined above in equation form yields 2aFTF=GTG=I,2bFTG=D, with I the unit matrix and D a diagonal matrix. The algorithm uses a QR decomposition to decompose both X′ and Y′ into an orthogonal matrix Q and an upper-triangular matrix R: 3X′=QxRxY′=QyRy. The algorithm proceeds by applying a singular value decomposition (SVD) on the product QxTQy: 4QxTQy=USVT. By trying the ansatz, 5A≡Rx-1UB≡Ry-1V, the orthonormality requirement of Eq. () becomes 6FTF=ATX′TX′A=UTRx-1TRxTQxTQxRxRx-1U=I, and an analogous result follows for GTG.

The orthogonality requirement of Eq. () becomes 7D=FTG=ATX′TY′B=UTRx-1TRxTQxTQyRyRy-1V=UTUSVTV=S. Therefore, the ansatz of Eq. () is a valid solution for the matrices A and B. Moreover, by counting the number of degrees of freedom in these matrices and the number of constraints provided by Eq. (), it can be shown that all solutions are permutations of Eq. () . The canonical basis is therefore uniquely defined. In the case that X and Y contain different numbers of variables Nx and Ny, the SVD of Eq. () selects the N largest correlations, with N=min⁡(Nx,Ny).

As QR decomposition and SVD are common matrix operations that are efficiently implemented in most numerical libraries, this algorithm is straightforward to implement in most programming languages.

The CCA method can be used to construct an OO. Let X be a set of (possibly) relevant model state variables and Y the corresponding observation values. Here Y could be obtained from a specialised model but also from a historical dataset of real observations. Applying the algorithm of Sect.  yields the matrices A, B, and D. The first two convert the mean subtracted model states X′ and observation values Y′ into their canonical counterparts F and G. The diagonal matrix D holds for each pair of canonical variables i the best fit to the slope of the correlation: Dii=dGi/dFi.

Assuming that Nx≥Ny  – i.e. the number of model state variables is at least equal to the number of observed variables – it is possible to calculate Y′ from X′ by passing through canonical space and applying the fitted slope D, 8Y′=X′ADB-1≡X′M, defining the CCA OO matrix, 9M≡ADB-1, of size Nx×Ny. As the CCA considers only the anomaly of X and Y, an additional offset term needs to be considered to accommodate the mean values of X and Y in the input dataset. However, the mean values of X and Y can be combined by applying the matrix M: 10Y-Y‾=X-X‾MY=XM+K, with 11K≡Y‾-X‾M, a combined offset vector of length Ny.

During the training phase of the CCA OO, the datasets X and Y are used to calculate the matrix M and the offset K. Once computed, they can be used to form an observation operator H that transforms a state x as 12H(x)=xM+K. Furthermore, the tangent-linear approximation used in variational DA schemes requires that 13H(x)∼H(xb)+H′dx, where H′ is the tangent-linear version of the OO, xb the background state, and dx the deviation from the background. The CCA OO is straightforward to implement in this scheme, since for H′ and its adjoint H′T it follows that 14H′=MTH′T=M.

One possible application of the new CCA OO is the assimilation of SST in ocean general circulation models (OGCMs). In recent years OGCMs have seen significant improvements in vertical resolution, particularly near the surface, where the first layer has been reduced to a thickness of the order of 1 m or less. At this resolution, the diurnal cycle of SST should be taken into account. Although diurnal variability is included to some extent , the vertical resolution of OGCMs is still insufficient to fully resolve the variability of the skin and subskin ocean temperature.

This issue becomes particularly evident when assimilating satellite SST observations. The different types of sensors used on satellites probe the ocean temperature at different depths. Infrared (IR) sensors measure the temperature at about 10 µm, a layer that is referred to as the ocean skin. Microwave (MW) sensors, on the other hand, measure the temperature of the layer below that, the subskin, with a depth of about 1 mm. This is much shallower than the vertical resolution of a typical OGCM, while these layers are strongly affected by the atmospheric conditions. The ocean skin cools due to thermodynamic processes at the air–sea interface, while the absorption of solar heat causes a warming of the subskin. At the same time, wind can mix the skin and subskin with the water below, smoothing the temperature variations. During days of low wind and/or high insolation conditions the amplitude of the SST diurnal cycle can be larger than the combined accuracy of the model and observations, causing a straightforward assimilation of SST to degrade the performance of the model . Under favourable conditions this amplitude is typically of the order of a few degrees (see e.g. ), but values as high as 6 ∘C have been observed .

Representation errors have been extensively discussed within ocean applications  and generally include errors due to e.g. limited spatial resolution or unrepresented processes. However, the diurnal variability of skin SST represents a potentially systematic error that requires a proper treatment rather than just increasing the representation component of the observational error.

An important source of SST observational data is the Spinning Enhanced Visible and Infrared Imager (SEVIRI) instrument onboard the Meteosat satellites of the second generation. As these are geostationary satellites, SEVIRI can provide continuous measurements of the same area with a 15 min temporal resolution. Although the IR imager is sensitive to skin temperature, the calibration algorithm of SEVIRI corrects for the cool-skin bias, and the resulting SST products should be considered the subskin temperature . For wind speeds greater than 6 m s-1 the skin temperature may be calculated as Tskin=Tsubskin-0.17 ∘C , but this is only an approximation.

This section will discuss how to use the output of a water column model specifically tuned for modelling the diurnal cycle of SST together with the CCA OO to build an observation operator for SST that accounts for the diurnal variability.

The performance of the GOTM-based CCA OO for SST is compared to other commonly used methods. For this comparison the GOTM dataset is again split along the zonal direction using every other profile to calculate the CCA OO. The remaining profiles are matched to SEVIRI subskin retrievals using only profiles matched to a measurement with an acceptable (4) or good (5) quality control level. The performance can be conveniently expressed in terms of the skill score (SS), defined by  as 15SS=1-MSEmodelMSEreference. The skill score is based on the mean square error (MSE) of the model under testing and of a reference model. Specifically, it expresses the difference in MSE as a fraction of the reference MSE. The skill score is straightforward to interpret: a perfect model (MSE=0) results in a skill score of 1, while a model that shows no improvement over the reference model receives a skill score of 0. Negative skill scores indicate that the model performs worse and its MSE has increased with respect to the reference. In this case the CCA OO will be used as the model and the reference will be another commonly used OO. The MSE is calculated with respect to the SEVIRI subskin temperature.

The simplest method of assimilating satellite SST observations in a model that insufficiently describes the diurnal cycle of SST is to assimilate only at night or during high wind; see, for example, . During the night the cycle of SST is close to its minimum value and the temperature of the upper layer of an OGCM forms a reasonable approximation for the skin temperature. In this situation the assimilation is performed without additional corrections. Figure a shows the skill score of the CCA OO at midnight local time using the temperature of the OGCM upper layer as a reference method. Figure b shows the same situation, but in the afternoon. For high wind and low insolation the CCA OO performs, as expected, similarly to using the upper OGCM layer. However, for low wind speeds and high insolation the CCA OO shows a clear improvement, even at midnight. This can be explained by the fact that at midnight some diurnal signal still remains and, even using the wind and insolation values of the next day, this is correctly modelled by the CCA OO.

A more advanced solution is the parameterisation of , which estimates the diurnal signal as a function of wind, insolation, and time. This is a commonly used parameterisation; for example, it is included with the NEMO ocean model . Figure  shows the skill score for the CCA OO compared to the parameterisation of at the peak of the diurnal cycle (a) and in the early evening (b). It can be seen that for high insolation and low wind, conditions for which the diurnal warming is largest, both methods perform similarly. However, the CCA OO is better at accommodating different atmospheric conditions and shows significant improvements for the intermediate insolation and wind categories. Moreover, Fig. b shows that the CCA OO is able to better parameterise the cooling of the subskin in the late afternoon–evening after the peak of the diurnal warming has passed.

Using the CCA OO to improve the description of SST has many potential applications. For example, the CCA OO could be used as a parameterisation of diurnally varying skin SST within an OGCM as part of the air–sea flux calculations. The skin SST is the true interface temperature for air–sea fluxes, so this approach should result in improved air–sea heat transfer in OGCMs and coupled ocean–atmosphere models. See, for example, . Another possibility would be the use of the CCA OO as a parameterisation of diurnally varying SST within a climate model. The diurnal cycle is a fundamental signal of the climate system, yet for climate models the lack of vertical structure (and temporal resolution) is even more critical. See, for example, .

Due to the way in which it is constructed, the CCA OO is an inherently linear operator. This makes it straightforward to implement in DA schemes that require linearised and differentiable OOs. However, non-linear effects can be accommodated to some extent by constructing a series of CCA OOs conditioned on such a non-linear dependency. For example, in the case of SST, this method has been used to condition the CCA OO on insolation, wind, and time. The only requirement in this case is that the datasets X and Y of Sect.  are sufficiently large to divide them by such a dependent variable.

The minimum size of the input dataset required ultimately depends on the number of model variables used (Nx) and the number of observation variables to predict (Ny). The number of free parameters in the CCA OO matrix M and the offset K equals (Nx+1)Ny. As each entry in the input dataset also provides Ny observation values, Eq. () requires a minimum of Nx+1 entries to be mathematically solvable. However, at this point the CCA OO will be overfitted. It will simply be able to memorise the input datasets rather than being based on general characteristics of the data. Care has to be taken to avoid this situation, making sure the input dataset contains a number of entries n with n>>Nx. Whether a given size n is sufficient should be tested using independent data. One possible method for this test is to withhold part of the input dataset from the CCA OO calculation and then use this subset to calculate the CCA OO performance.

Observation operators (OOs) form a central component in any data assimilation (DA) system, as they transform the state variables of a numerical model into real-world observable variables. Often, an OO also needs to correct for processes that are not fully described by the parent model. Such processes may be best modelled by interfacing the OO to a specialised model, but this is generally not feasible due to computational constraints.

The assimilation of satellite sea surface temperature (SST) in ocean general circulation models (OGCMs) is a prime example of a situation in which insufficiently modelled processes play an important role. The diurnal cycle of SST causes a discrepancy in the temperature of the very thin upper layer measured by a satellite and the rather coarse upper layer in a typical OGCM. On a clear summer day with low wind, this discrepancy can amount to as much as 2 ∘C or more .

The current paper presented a method, based on canonical correlation analysis (CCA), to build parameterisations based on an output dataset of a specialised model. These parameterisations, referred to as the CCA OO, can provide an efficient approximation to the results of the specialised model and are therefore well-suited for use in DA systems.

The case of SST assimilation has been used to demonstrate the new CCA OO. Using an output dataset of the General Ocean Turbulence Model (GOTM), a high-resolution water column model specifically tuned for modelling the diurnal cycle of SST, a new CCA OO has been derived. Subsequently, the operator has been applied to reduced-resolution temperature profiles from the GOTM to simulate its use in a DA system. The approximations provided by the CCA OO are found to be in good agreement with the GOTM at various times of the day and across all atmospheric conditions. The results indicate that the CCA OO could be used to enable the assimilation of SST in conditions under which this was previously not possible. Moreover, the atmospheric categories that were introduced in the construction of the CCA OO for SST show that the linear assumption implicit in CCA can be partially relaxed. This makes the CCA OO versatile for any condition. Compared to commonly used methods for SST assimilation, the CCA OO can provide substantial improvements. This is especially true for measurements of the skin SST, since the CCA OO profits from the modelling of the cool-skin effect that is included in the GOTM.

The ability of the CCA OO to handle complicated physical models in a relatively simple way is attractive for a large number of problems in DA, for which reduced-order OOs are desirable due to computational constraints. Remotely sensed data are the obvious target given the complexity of their relationships with state variables. Observations in coupled assimilations (e.g. ocean–atmosphere, ocean–sea ice, or ocean–biogeochemistry) are examples of challenging problems that could be investigated in the future with the CCA OO.