The skill of current predictions of the warm phase of the El Niño Southern Oscillation (ENSO) reduces significantly beyond a lag time of 6 months. In this paper, we aim to increase this prediction skill at lag times of up to 1 year. The new method combines a classical autoregressive integrated moving average technique with a modern machine learning approach (through an artificial neural network). The attributes in such a neural network are derived from knowledge of physical processes and topological properties of climate networks, and they are tested using a Zebiak–Cane-type model and observations. For predictions up to 6 months ahead, the results of the hybrid model give a slightly better skill than the CFSv2 ensemble prediction by the National Centers for Environmental Prediction (NCEP). Interestingly, results for a 12-month lead time prediction have a similar skill as the shorter lead time predictions.

Approximately every 4 years, the sea surface temperature (SST) is higher
than average in the eastern equatorial Pacific

So far, both statistical and dynamical models are used to predict ENSO

Recently, attempts have been made to improve the ENSO prediction skill beyond
this spring predictability boundary, for example by using machine learning
(ML;

Briefly, ANN is a system of linked neurons that describes, after optimization, a function from one or more input variables (or attributes) to the output variable(s). Generally, one has to choose how large and complicated the ANN structure is. The more complicated an ANN, the more it will filter the important information from the attributes itself, but it will require more input data and is computationally intensive. Therefore, simpler ANN structures are used in this article. However, techniques will have to be applied in order to reduce the amount of input variables and select the important ones, to make the problem appropriate for the simpler ANN. This reduction and selection problem can be tackled in many ways, which are crucial for the prediction. The main issue in these methods, however, is what attributes to use for ENSO prediction.

Complex networks turn out to be an efficient way to represent spatiotemporal
information in climate systems

In this paper, a hybrid model is introduced for ENSO prediction. The model
combines the classical linear statistical method of autoregressive integrated
moving average (ARIMA) and an ANN method. ANN is applied to predict the
residual, due to the nonlinear processes, that is left after the ARIMA
forecast

Section 2 briefly describes the ZC model, the methods considering both the climate networks and ML, and the data from observations. In Sect. 3, the network methods are first applied to the ZC model. Second, the attributes selected for observations are presented. These attributes, among which there is a network variable, are applied in the hybrid prediction model in Sect. 4, which discusses the skill of this model to predict El Niño. The paper concludes with a summary and discussion in Sect. 5.

As observational data, we use the sea surface height (SSH) from the weekly
ORAP5.0 (Ocean ReAnalysis Pilot 5.0) reanalysed dataset of ECMWF from 1979 to
2014 between 140 to

For recent predictions, the SSALTO/DUACS altimeter products are used for the
same spatial domain, since the SSH is available from 1993 up to the present
in this dataset. The SSALTO/DUACS altimeter products were produced and
distributed by the Copernicus Marine and Environment Monitoring Service
(

In addition, the HadISST dataset of the Hadley Centre has been used for the SST and the
NCEP/NCAR Reanalysis dataset for the wind stress from 1980 to the present

To quantify ENSO, the NINO3.4 index is used, i.e. the 3-month running
mean of the average SST anomaly in the extended reconstructed SST dataset
between 170 to

The warm water volume (WWV), being the integrated volume above the

The ZC model

Pacific area (red rectangle) from 140–280

In the ZC model, a shallow-water ocean component is coupled to a steady
shallow-water Gill atmosphere model

As shown in

Three positive feedbacks related to the thermocline depth, upwelling and
zonal advection can cause the amplification of SST anomalies

Apart from the coupled ocean–atmosphere processes, ENSO is also affected by
fast processes in the atmosphere, which are considered as noise in the ZC
model. An important example of atmospheric noise are the so-called westerly
wind bursts (WWB). These are related to the Madden–Julian oscillation

The atmospheric noise in the model is represented by obtaining a residual of
the wind stress from observations as in

Here we explain the methods to calculate a property of a climate network
which is tested in the ZC model and observations and will be used in the
hybrid model. From the network analysis we found several climate network
quantities with interesting properties for prediction, but which are not used
in the hybrid model of the next section. The methods to calculate these
properties can be found in Appendix

An undirected and unweighted network is constructed making use of the Pearson
correlation of climate variables related to ENSO (e.g. SST, thermocline depth
or zonal wind stress). Network nodes are model or observation grid positions

Percolation theory is then considered, describing the connectivity of
different clusters in a network. It has been found that the connectivity of
some climate networks increases just before an El Niño and decreases
afterwards

A hybrid model

This scheme describes a “supervised” model, implying that the predictant is
“known”. This known quantity is the NINO3.4 index. The standard procedure
for supervised learning is to optimize the ML method on a “training set” to
define an optimal model, which predicts ENSO with a certain time ahead. This
function will then be tested on a test set. Here a training set of

First, the training set is used to optimize an ARIMA(

The eventual ARIMA equation results in a prediction

After

Moreover, at every lead time an optimal attribute must be selected. Hence the
final prediction model is tuned for a specific lead time and will not be a
step by step prediction forward in time. Apart from considering the physical
mechanisms the variables represent, two methods will help to decide which
variables can improve the prediction. First, correlation between the
predictor and predictant is a commonly used measure for attribute selection

However, the effect of a variable on ENSO at a short lead time increases the
cross-correlation at a longer lead time, due to the effect of autocorrelation

Finally, the

In this paper, only a feed-forward ANN is applied, having a structure without
loops. The input variables are linearly combined and projected to the first
layer neurons according to

These

To summarize the tuning of the hybrid model: the ARIMA order and the hyperparameters controlling the ANN structure are tuned on the data, i.e. such that the prediction result is optimal. However, we will consider whether some set of different parameter values converges to similar predictions, which can show whether the hyperparameter tuning was a one lucky shot or not. The choice of the attributes is based on the ZC model giving a more physical basis for the information needed for a good prediction. To select them at a specific lag their cross-correlation and Wiener–Granger causality with the ENSO index and performance are also considered, which could lead to the replacement of an attribute with another attribute which is physically related.

In this section, topological properties of climate networks are analysed within the ZC model and observations, which lead to specific choices of attributes in the hybrid prediction model.

Weekly spatiotemporal data
on a

The network variable

The network variable of interest is

The ZC model results have given an indication of the network variables that could be used as attributes in the hybrid model to predict El Niño. Although the network variables show interesting behaviour in the ZC model for prediction, this is not always the case in observations. This section describes which variables, including a network variable, are implemented in the hybrid model and the selection of these attributes at different lead times. Notice that only anomalies of the time series in observations are considered.

The WWV,

First, from the recharge/discharge oscillator point of view, the WWV shows
great potential for the prediction of ENSO

To determine at which lead time the different attributes should be applied,
the cross-correlation and the

To summarize, we are interested in the variables that represent specific
physical characteristics related to the prediction of ENSO, to select the
attributes. Both

This section presents the predictions of the hybrid model, as compared with
observations and with alternative predictions from the CFSv2 model ensemble
of NCEP. The skill with ANN structures up to three hidden layers is
investigated. First, a comparison between both predictions is made for the
year 2010 (Fig.

From now on, the normalized root mean squared error (NRMSE) is used to
indicate the skill of prediction within the test set:

The 9-month ahead prediction starting from every month in the year 2010.
Blue is the hybrid model prediction with ARIMA(12,1,1),

The year 2010 is a recent example of an under-performing CFSv2 ensemble.
Especially in January, all members of the ensemble overestimate the NINO3.4
index, resulting in an overestimation of the ensemble mean (see
Fig.

NINO3.4 predictions of the CFSv2 ensemble mean (red)
and the hybrid model with ARIMA(12,1,0) (blue), compared to the observed index (black).
For the hybrid model predictions, from an ensemble of 84 different
ANN structures, structures resulting in a low NRMSE are presented.

Considering the 3-, 6- and 12-month lead time predictions, both the
3- and 6-month lead time prediction of the CFSv2 ensemble show some lag
and amplification of the real NINO3.4 index (Fig.

Comparing the 3-month lead prediction of the CFSv2 ensemble with the 4-month lead prediction of the hybrid model, both the amplification and the lag of the hybrid model prediction are smaller. While the lead time of the hybrid model is 1 month longer, the prediction skill is better in terms of NRMSE. The prediction skill of the hybrid model decreases at a 6-month lead compared to the 4-month lead time prediction. Thereby the lag and amplification of the CFSv2 prediction increase. Although the hybrid model does not suffer as much from the lag, it underestimates the El Niño event of 2010. In terms of NRMSE the hybrid model still obtains a better prediction skill.

Although the shorter lead time predictions show slightly better results than
the conventional models, most important is a good prediction skill for larger
lead times that appears to overcome the spring predictability barrier. To
perform a 12-month lead prediction which could overcome this barrier, the
attributes from the shorter lead time predictions are found to be
insufficient. However,

Spread and mean (blue line) of ensembles of hybrid model predictions
with different hyperparameter values. The nine optimal (in terms of NRMSE)
predictions from the 84 different ANN structures at the

Cross-validation results of the

The hyperparameter values (i.e. the ARIMA order and the ANN structure) of the
predictions in Fig.

NINO3.4 prediction from May 2017. In black the observed index until May 2017. Red is the CFSv2 ensemble prediction mean and the shaded area is the spread of the ensemble. The hybrid model prediction in blue is given by predictions from hybrid models found to be most optimal at the different lead times with ARIMA(12,1,0). The dashed blue line is the running 12-month lead time prediction.

To test the robustness of these results, a series of cross-validations has
been performed on the prediction models of Fig.

Finally, a prediction is made for the coming year in Fig.

Interestingly, as can be seen in Fig.

A successful attempt was made in this paper to use machine learning (ML)
techniques in a hybrid model to improve the skill of El Niño predictions.
Crucial for the success of this hybrid model is the choice of the attributes
applied to the artificial neural network. Here, we have explored the use of
network variables as additional attributes to several physical ones. Results
of the ZC model provided several interesting network variables. Of these
network variables,

By including the network variable

Although the results of the methods are promising, some adaptations to the methods which select attributes could still improve predictions. Several network variables resulted in a clear signal in the ZC model, but not necessarily for the observations. Perhaps the cross-correlation and a Granger causality test are not enough to determine the suitability of a variable in the observations. Testing all possible attribute sets in the prediction scheme and comparing results costs time. As a solution, the nonlinear methods “lagged mutual information” and “transfer entropy” can be techniques to select variables at different lead times. After all, the attributes are applied in the nonlinear part of the prediction scheme. Consequently, more variables might be found to increase the prediction skill.

Even though the currently applied network measures showed interesting
properties, different climate network construction methods can still be
interesting to apply. The Pearson correlation is a simple, effective method
to define links between nodes. However, different properties of climate
networks could be found when using mutual information instead. Moreover, the
effect of spatial distance between nodes can be investigated and corrected
for

By applying the ARIMA as a simple yet effective statistical method to apply in the first step of the scheme, the hybrid model shows promising results. However, the exact reason for how this model works remains a topic of investigation. The ARIMA prediction could be related to the linear wave dynamics. It can be interesting to replace the ARIMA part of the scheme by a dynamical model accounting for these linear wave dynamics. For the same reason, vector autoregression can be used instead of ARIMA. Being a multivariate generalization of an autoregressive model, this can implement the linear effect of other variables on ENSO.

Next to investigation of the exact reason the hybrid model works, some
adaptations could still improve the prediction scheme. For example, it is
assumed the linear and nonlinear part of the model are additive (see
Eq.

A general difficulty in El Niño prediction is the short available
observational time series, also in other statistical prediction models

Although the hybrid model and the attribute selection can clearly be improved, the results here have shown the potential for ML methods, in particular with network attributes, for El Niño prediction. The underlying reason for this success is likely that through the network attributes, more global correlations are taken into account which are needed to be able to overcome the spring predictability barrier.

From the unweighted network we
compute the local degree

The spatial symmetry of the degree distribution is of interest, since it informs where most links of the network are located. More specifically, our interest will be in the symmetry in the zonal direction in a network. Therefore, the skewness of the meridional mean of the degree in the network is calculated. This defines the zonal skewness of the degree distribution in a network.

The following two climate network properties are derived from a so-called
NetOfNet approach. This is a network constructed with the same methods as
previously, but using multiple variables at each grid point (as specified in
Appendix

The second NetOfNet property is the algebraic connectivity. This is the
second smallest eigenvalue (

A final network property

To calculate the property

Global cross clustering between the SST and wind-stress network
in blue and its variance in green in the ZC model. The coupling strength

Zonal skewness of the degree field of the thermocline network with

Determining how strong noise can excite the ENSO mechanisms in the
subcritical case, or determining whether the feedbacks sustain an
oscillation in the supercritical state, could provide information to increase
the prediction skill.

Here, we introduce a NetOfNet variable which may represent properties of the
stability of the background state: the global cross clustering (

Second, from the classical view of the oscillatory behaviour of ENSO, waves in
the thermocline should contain memory of the system, because of their
negative delayed feedback. The changing structure of the thermocline network
is therefore of interest when predicting ENSO. Calculating this network with
threshold

To capture this zonal asymmetry around the equator with a variable, the zonal
skewness of the degree field will be used between

Third, the quantity

Finally, the algebraic connectivity (

All used observational data are from third parties and are either cited or can be found by URL as specified in Sect. 2.1.

The authors declare that they have no conflict of interest.

Peter D. Nooteboom would like to thank the Instituto de Física Interdisciplinar y Sistemas Complejos (IFISC), for hosting his stay in Mallorca during part of 2017.

Cristóbal López and Emilio Hernández-García acknowledge support from Ministerio de Economia y Competitividad and Fondo Europeo de Desarrollo Regional through the LAOP project (CTM2015-66407-P, MINECO/FEDER) Edited by: Ben Kravitz Reviewed by: Robert Link and one anonymous referee