Investigation of Standardized Groundwater Level Index Prediction Model by Attention Neural Networks

應用注意力神經網路建立地下水位預測指標模型之探討

10.6937/TWC.202306_71(2).0001

TSWN-SYAU TSAY；CHI-MING WENG

standardized groundwater level index ； attention neural network ； irrigation

台灣水利

71卷2期（2023 / 06 / 01）

1 - 15

英文;繁體中文

Because groundwater has stable water quality and quantity, it is often used as a supplementary water resource for various purposes. In agricultural water use, groundwater is often taken to be supplementary irrigation water when surface water sources are insufficient. However, whether the groundwater is in a safe condition and can be supplied as supplementary irrigation water during the rice irrigation season is concerned. Groundwater in the Changhua Irrigation District of Taiwan is still an important supplementary irrigation water resource. In this paper, the irrigation areas of TianZhong, TianWei and XiZhou are selected. standardized groundwater level index (SGI) was taken to be the evaluation index of groundwater resources. For the purpose of estimating the safe and usable amount of groundwater, prediction model of SGI with time is developed for evaluating whether groundwater can be extracted as supplementary irrigation water. This research applied the attention neural network to develop a SGI prediction model. The developed model can be a basis for evaluating groundwater as a supplementary irrigation water source during drought seasons in the studied areas. Since the groundwater level has similar periodic changes, time embedding model was applied for the training model. The trained model showed that the accuracy of the SGI prediction model of the 7 observation wells in the study area is higher than that of the model without time embedding. Subsequent research can be improved by modifying function of time embedding, adjusting attention neural network architecture and hyper-parameters of the model to improve the model accuracy.

1. Chen, P.-Y.(2017)。National Central University。
   連結：
2. Lin, Y.-J.,Chiou, Y.-T.,Chen, C.-T.,Lee, C.-H.(2017).The Effects of Irrigation Management Practice on Growth Period and Irrigation Water Use Efficiency in Paddy Rice.Journal of Taiwan Agricultural Engineering,63(2),35-48.
   連結：
3. Wu, R.-S.,Jheng, A.-C.,Wu, H.-C.(2022).Enhance the Management of Irrigation in CHIA-NAN Area.Journal of Taiwan Agricultural Engineering,68(1),39-55.
   連結：
4. Wu, R.-S.,Liu, I.-H.,Liu, J.-S.(2019)。Applying Mixed Crop Irrigation Water Demand Model on Reasonable Application of Groundwater–A Case study of Changhua Xizhou。Journal of Taiwan Agricultural Engineering，65(4)，62-80。
   連結：
5. (2002).Agricultural Meteorology and Applications and Management of Agricultural Water Resources.
6. Abbasimehr, H.,Paki, R.(2022).Improving time series forecasting using LSTM and attention models.Journal of Ambient Intelligence and Humanized Computing,1-19.
7. Bai, T.,Tsai, W.-P.,Chiang, Y.-M.,Chang, F.-J.,Chang, W.-Y.,Chang, L.-C.,Chang, K.-C.(2019).Modeling and investigating the mechanisms of groundwater level variation in the Jhuoshui River Basin of Central Taiwan.Water,11(8),1554.
8. Berg, A.,O'Connor, M.,Cruz, M. T.(2021).Keyword transformer: A self-attention model for keyword spotting.Proc. Interspeech 2021
9. Bertasius, G.,Wang, H.,Torresani, L.(2021).Is space-time attention all you need for video understanding.Proceedings of the 38th International Conference on Machine Learning
10. Bloomfield, J.,Marchant, B.(2013).Analysis of groundwater drought building on the standardised precipitation index approach.Hydrology and Earth System Sciences,17(12),4769-4787.
11. Bridle, J.(1989).Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters.Advances in neural information processing systems,2
12. Caruana, R.,Lawrence, S.,Giles, C.(2000).Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping.Advances in neural information processing systems,13
13. Chang, F.-J.,Chang, L.-C.,Huang, C.-W.,Kao, I.-F.(2016).Prediction of monthly regional groundwater levels through hybrid soft-computing techniques.Journal of Hydrology,541,965-976.
14. Changhua Management Office, I. A., COA. (2022). “Changchua Management Office, Irrigation Agency, COA (in Chinese).” from https://www.iachu.nat.gov.tw/iachu/index.php
15. Chen, C.-T.,Chang, Y.-C.,Hung, C.-T.(2014).A Study on Rice Yield and Water Saving Efficiency for Irrigation Management Practice.Journal of Taiwan Agricultural Engineering,60(1),81-90.
16. Cho, K.,Van Merriënboer, B.,Bahdanau, D.,Bengio, Y.(2014).On the properties of neural machine translation: Encoder-decoder approaches.Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation
17. Collenteur, R. A.,Bakker, M.,Caljé, R.,Klop, S. A.,Schaars, F.(2019).Pastas: open source software for the analysis of groundwater time series.Groundwater,57(6),877-885.
18. Cui, B.,Li, Y.,Chen, M.,Zhang, Z.(2019).Fine-tune BERT with sparse self-attention mechanism.Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP)
19. ndmc@unl.edu. (2022). “SPI Program.” from https://drought.unl.edu/monitoring/SPI/SPIProgram.aspx
20. Dong, Y.,Cordonnier, J.-B.,Loukas, A.(2021).Attention is not all you need: Pure attention loses rank doubly exponentially with depth.International Conference on Machine Learning
21. Dyer, C.,Ballesteros, M.,Ling, W.,Matthews, A.,Smith, N. A.(2015).Transition-based dependency parsing with stack long short-term memory.Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
22. Elmaz, F.,Eyckerman, R.,Casteels, W.,Latré, S.,Hellinckx, P.(2021).CNN-LSTM architecture for predictive indoor temperature modeling.Building and Environment,206,108327.
23. Galassi, A.,Lippi, M.,Torroni, P.(2020).Attention in natural language processing.IEEE Transactions on Neural Networks and Learning Systems,32(10),4291-4308.
24. Gharehbaghi, A.,Ghasemlounia, R.,Ahmadi, F.,Albaji, M.(2022).Groundwater level prediction with meteorologically sensitive Gated Recurrent Unit (GRU) neural networks.Journal of Hydrology,612,128262.
25. Giuliari, F.,Hasan, I.,Cristani, M.,Galasso, F.(2021).Transformer networks for trajectory forecasting.2020 25th International Conference on Pattern Recognition (ICPR)
26. Greaves, G. E.,Wang, Y.-M.(2017).Yield response, water productivity, and seasonal water production functions for maize under deficit irrigation water management in southern Taiwan.Plant Production Science,20(4),353-365.
27. He, K.,Zhang, X.,Ren, S.,Sun, J.(2016).Deep residual learning for image recognition.Proceedings of the IEEE conference on computer vision and pattern recognition
28. Hochreiter, S.,Schmidhuber, J.(1997).Long short-term memory.Neural computation,9(8),1735-1780.
29. Huang, S.,Wang, D.,Wu, X.,Tang, A.(2019).Dsanet: Dual self-attention network for multivariate time series forecasting.Proceedings of the 28th ACM international conference on information and knowledge management
30. Humphreys, E.,Kukal, S. S.,Christen, E. W.,Hira, G. S.,Sharma, R.(2010).Halting the groundwater decline in north-west India-which crop technologies will be winners?.Advances in agronomy,109,155-217.
31. Jenkins, K.,Warren, R.(2015).Quantifying the impact of climate change on drought regimes using the Standardised Precipitation Index.Theoretical and Applied Climatology,120(1),41-54.
32. Juraska, J.,Walker, M.(2021).Attention is indeed all you need: Semantically attention-guided decoding for data-to-text nlg.INLG 2021
33. Kaiser, R.(2005).Groundwater management in Texas: evolution or intelligent design.Kan. JL & Pub. Pol’y,15,467-488.
34. Kazemi, S. M.,Goel, R.,Eghbali, S.,Ramanan, J.,Sahota, J.,Thakur, S.,Wu, S.,Smyth, C.,Poupart, P.,Brubaker, M.(2019).,未出版
35. Kebede, H.,Fisher, D. K.,Sui, R.,Reddy, K. N.(2014).Irrigation methods and scheduling in the Delta region of Mississippi: Current status and strategies to improve irrigation efficiency.American Journal of Plant Sciences,5(20),2917.
36. Khan, S.,Gabriel, H.,Rana, T.(2008).Standard precipitation index to track drought and assess impact of rainfall on watertables in irrigation areas.Irrigation and Drainage Systems,22(2),159-177.
37. Kuo, S.-F.(2014).Evaluation of irrigation water requirements and crop yields with different irrigation schedules for paddy fields in ChiaNan irrigated area, Taiwan.Paddy and water environment,12(1),71-78.
38. Li, Y.,Yao, T.,Pan, Y.,Mei, T.(2022).Contextual transformer networks for visual recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence
39. McKee, T. B.,Doesken, N. J.,Kleist, J.(1993).the relationship of drought frequency and duration to time scales.Proceedings of the 8th Conference on Applied Climatology,California:
40. Mohammadi Farsani, R.,Pazouki, E.(2021).A transformer self-attention model for time series forecasting.Journal of Electrical and Computer Engineering Innovations,9(1),1-10.
41. Muhammad, T.,Aftab, A. B.,Ahsan, M.,Muhu, M. M.,Ibrahim, M.,Khan, S. I.,Alam, M. S.(2022).,未出版
42. Peñaloza, V.(2020).Time2Vec Embedding on a Seq2Seq Bi-directional LSTM Network for Pedestrian Trajectory Prediction.Res. Comput. Sci.,149(11),249-260.
43. Qaisrani, Z. N.,Nuthammachot, N.,Techato, K.(2021).Drought monitoring based on Standardized Precipitation Index and Standardized Precipitation Evapotranspiration Index in the arid zone of Balochistan province, Pakistan.Arabian Journal of Geosciences,14(1),1-13.
44. Song, H.,Rajan, D.,Thiagarajan, J.,Spanias, A.(2018).Attend and diagnose: Clinical time series analysis using attention models.Proceedings of the AAAI conference on artificial intelligence
45. Song, J.,Tong, X.,Xu, X.,Zhao , K.(2022).A Real-Time Reentry Guidance Method for Hypersonic Vehicles Based on a Time2vec and Transformer Network.Aerospace,9(8),427.
46. Tirivarombo, S.,Osupile, D.,Eliasson, P.(2018).Drought monitoring and analysis: standardised precipitation evapotranspiration index (SPEI) and standardised precipitation index (SPI).Physics and Chemistry of the Earth, Parts A/B/C,106,1-10.
47. Vaswani, A.,Shazeer, N.,Parmar, N.,Uszkoreit, J.,Jones, L.,Gomez, A. N.,Kaiser, Ł.,Polosukhin, I.(2017).Attention is all you need.Advances in neural information processing systems,30
48. Wang, R.,Ao, J.,Zhou, L.,Liu, S.,Wei, Z.,Ko, T.,Li, Q.,Zhang, Y.(2022).Multi-view self-attention based transformer for speaker recognition.ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
49. Wu, H.,Zhao, H.,Zhang, M.(2021).,未出版
50. Wunsch, A.,Liesch, T.,Broda, S.(2021).Groundwater level forecasting with artificial neural networks: a comparison of long short-term memory (LSTM), convolutional neural networks (CNNs), and non-linear autoregressive networks with exogenous input (NARX).Hydrology and Earth System Sciences,25(3),1671-1687.
51. Zha, W.,Liu, Y.,Wan, Y.,Luo, R.,Li, D.,Yang, S.,Xu, Y.(2022).Forecasting monthly gas field production based on the CNN-LSTM model.Energy,124889.
52. Zhang, Y.-g.,Tang, J.,He, Z.-y.,Tan, J.,Li, C.(2021).A novel displacement prediction method using gated recurrent unit model with time series analysis in the Erdaohe landslide.Natural Hazards,105(1),783-813.