| Summary: | Poster presented at the Northern Lights Deep Learning Workshop (NLDL) 2020, 19.01.20 - 21.01.20, UiT The Arctic University of Norway, Tromsø Forecasting the dynamics of time-varying systems is essential to maintaining the sustainability of the systems. Recent studies have discovered that Recurrent Neural Networks(RNN) applied in the forecasting tasks outperform conventional models that include AutoRegressive Integrated Moving Average(ARIMA). However, due to the structural limitation of vanilla RNN which holds unit-length internal connections, learning the representation of time series with missing data can be severely biased. The goal of this paper is to provide a robust RNN architecture against the bias from missing data. We propose Dilated Recurrent Attention Networks (DRAN). The proposed model has a stacked structure of multiple RNNs which layer of each having a different length of internal connections. This structure allows incorporating previous information at different time scales. DRAN updates its state by a weighted average of the layers. In order to focus more on the layer that carries reliable information against bias from missing data, it leverages attention mechanism which learns the distribution of attention weights among the layers. We report that our model outperforms conventional ones with respect to the forecast accuracy from two benchmark datasets, including a real-world electricity load dataset.
|
|---|