Improved Generalization in Discrete Univariate Time Series Forecasting using Visual Transformers GAN - An Empirical Study

カテゴリ: 研究会(論文単位)

論文No: CMN23023

グループ名: 【C】電子・情報・システム部門 通信研究会

発行日: 2023/03/26

タイトル(英語): Improved Generalization in Discrete Univariate Time Series Forecasting using Visual Transformers GAN - An Empirical Study

著者名: Ravikiran Manikandan(Hitachi India Pvt Ltd),Ganesh Ananth(Hitachi India Pvt Ltd),Yuichi Nonaka(Hitachi India Pvt Ltd)

著者名(英語): Manikandan Ravikiran(Hitachi India Pvt Ltd),Ananth Ganesh(Hitachi India Pvt Ltd),Nonaka Yuichi(Hitachi India Pvt Ltd)

キーワード: Deep Learning, |Generative Adversarial Networks|Forecasting|Sensor data|Data Generation|Computer Vision|Deep Learning, |Generative Adversarial Networks|Forecasting|Sensor data|Data Generation|Computer Vision

要約(日本語): Discrete univariate time series forecasting problem focuses on prediction of expected forecast based on input data points separated by time intervals that are greater than one second. Often such forecasting is used widely in all form of demand forecasting including inventory optimization in manufacturing, water allocation in distribution networks, customer churn and ATM replenishment in banking sector. Due to its dynamic nature, discrete univariate time series forecasting often encounters generalization errors, needing accurate forecasting suitable for practical real-world applications._x000D_ _x000D_ Increasing robustness to generalization errors usually requires access to large amount of discrete time series data which is currently handled by generating synthetic data either using simple augmentation methods or Generative Adversarial Networks (GAN) that are specifically developed for different time series data domains. However, these methods suffer from multiple drawbacks namely (a) failure in capturing trends and seasonality effectively, (b) generation of data set that are biased and may not be useful for generalization and (c) ability to simultaneously capture properties from synthetically generated data along with the original data. Though time series GAN’s alleviate some of these previous problems, they are often specific to different time series data domains and are not applicable directly on discrete univariate datasets. _x000D_ _x000D_ In this paper we tackle these problems by proposing (a) Transformer based Visual GAN (TraGAN) architecture to generate time series data with sufficient ability to capture trends and seasonality (b) To support discrete data we introduce positional encoding and (c) Drift based channel augmentation. We empirically study effectiveness of our proposed changes for discrete univariate forecasting using Computational Intelligence in Forecasting (CIF) 2016 benchmark dataset and Water Demand Forecasting datasets with DeepAR probabilistic forecasting model, Deep State model and Temporal Fusion transformer models respectively. _x000D_ _x000D_ _x000D_ Compared to existing methods, the proposed methods reduce sMAPE from 38.21% to 14.11% and 32.30% to 13.40% respectively across the said benchmarks with DeepAR probabilistic forecasting model. Further the proposed methods, also reduce RMSE from 36.96 to 15.53 and 656.4 to 186.4. In the due process, we find that (a) for smaller forecasting windows, the results are significantly higher with drift-based augmentation, indicating that the TraGAN with drift-based augmentation captures short term seasonality better and produced useful data for short term forecasting, (b) with TraGAN generated data, comparing across methods, we can see that for short term forecasting DeepAR tends to outperform the other two and (c) with TraGAN we also see that seasonality error is consistent across different forecasting methods, indicating that GAN model specifically is able to capture seasonality with little impact on overall prediction._x000D_

要約(英語): Discrete univariate time series forecasting problem focuses on prediction of expected forecast based on input data points separated by time intervals that are greater than one second. Often such forecasting is used widely in all form of demand forecasting including inventory optimization in manufacturing, water allocation in distribution networks, customer churn and ATM replenishment in banking sector. Due to its dynamic nature, discrete univariate time series forecasting often encounters generalization errors, needing accurate forecasting suitable for practical real-world applications._x000D_ _x000D_ Increasing robustness to generalization errors usually requires access to large amount of discrete time series data which is currently handled by generating synthetic data either using simple augmentation methods or Generative Adversarial Networks (GAN) that are specifically developed for different time series data domains. However, these methods suffer from multiple drawbacks namely (a) failure in capturing trends and seasonality effectively, (b) generation of data set that are biased and may not be useful for generalization and (c) ability to simultaneously capture properties from synthetically generated data along with the original data. Though time series GAN’s alleviate some of these previous problems, they are often specific to different time series data domains and are not applicable directly on discrete univariate datasets. _x000D_ _x000D_ In this paper we tackle these problems by proposing (a) Transformer based Visual GAN (TraGAN) architecture to generate time series data with sufficient ability to capture trends and seasonality (b) To support discrete data we introduce positional encoding and (c) Drift based channel augmentation. We empirically study effectiveness of our proposed changes for discrete univariate forecasting using Computational Intelligence in Forecasting (CIF) 2016 benchmark dataset and Water Demand Forecasting datasets with DeepAR probabilistic forecasting model, Deep State model and Temporal Fusion transformer models respectively. _x000D_ _x000D_ _x000D_ Compared to existing methods, the proposed methods reduce sMAPE from 38.21% to 14.11% and 32.30% to 13.40% respectively across the said benchmarks with DeepAR probabilistic forecasting model. Further the proposed methods, also reduce RMSE from 36.96 to 15.53 and 656.4 to 186.4. In the due process, we find that (a) for smaller forecasting windows, the results are significantly higher with drift-based augmentation, indicating that the TraGAN with drift-based augmentation captures short term seasonality better and produced useful data for short term forecasting, (b) with TraGAN generated data, comparing across methods, we can see that for short term forecasting DeepAR tends to outperform the other two and (c) with TraGAN we also see that seasonality error is consistent across different forecasting methods, indicating that GAN model specifically is able to capture seasonality with little impact on overall prediction._x000D_

本誌: 2023年3月29日通信研究会

本誌掲載ページ: 13-18 p

原稿種別: 英語

PDFファイルサイズ: 1,105 Kバイト