1. Home
  2. Vol 9, No 5 (2023)
  3. Eka Setiawan

Indoor Climate Prediction Using Attention-Based Sequence-to-Sequence Neural Network

Karli Eka Setiawan, Gregorius N. Elwirehardja, Bens Pardamean

Abstract


The Solar Dryer Dome (SDD), a solar-powered agronomic facility for drying, retaining, and processing comestible commodities, needs smart systems for optimizing its energy consumption. Therefore, indoor condition variables such as temperature and relative humidity need to be forecasted so that actuators can be scheduled, as the largest energy usage originates from actuator activities such as heaters for increasing indoor temperature and dehumidifiers for maintaining optimal indoor humidity. To build such forecasting systems, prediction models based on deep learning for sequence-to-sequence cases were developed in this research, which may bring future benefits for assisting the SDDs and greenhouses in reducing energy consumption. This research experimented with the complex publicly available indoor climate dataset, the Room Climate dataset, which can be represented as environmental conditions inside an SDD. The main contribution of this research was the implementation of the Luong attention mechanism, which is commonly applied in Natural Language Processing (NLP) research, in time series prediction research by proposing two models with the Luong attention-based sequence-to-sequence (seq2seq) architecture with GRU and LSTM as encoder and decoder layers. The proposed models outperformed the adapted LSTM and GRU baseline models. The implementation of Luong attention had been proven capable of increasing the accuracy of the seq2seq LSTM model by reducing its test MAE by 0.00847 and RMSE by 0.00962 on average for predicting indoor temperature, as well as decreasing 0.068046 MAE and 0.095535 RMSE for predicting indoor humidity. The application of Luong's attention also improved the accuracy of the seq2seq GRU model by reducing the error by 0.01163 in MAE and 0.021996 in RMSE for indoor humidity. However, the implementation of Luong attention in seq2seq GRU for predicting indoor temperature showed inconsistent results by reducing approximately 0.003193 MAE and increasing roughly 0.01049 RMSE.

 

Doi: 10.28991/CEJ-2023-09-05-06

Full Text: PDF


Keywords


Deep Learning; Sequence-to-Sequence; Seq2seq; Encoder-Decoder; Solar Dryer Dome; Indoor Climate Prediction; Luong Attention; Neural Network.

References


Prosekov, A. Y., & Ivanova, S. A. (2018). Food security: The challenge of the present. Geoforum, 91(February), 73–77. doi:10.1016/j.geoforum.2018.02.030.

Budiman, A. S., Gunawan, F., Djuana, E., Pardamean, B., Romeli, S., Putri, D. N. N., Aji, D. P. B., Rahardjo, K., Stevanus, Ilham Wibowo, M., Daffa, N., & Owen, R. (2022). Smart Dome 4.0: Low-Cost, Independent, Automated Energy System for Agricultural Purposes enabled by Machine Learning. Journal of Physics: Conference Series, 2224(1), 0–11. doi:10.1088/1742-6596/2224/1/012118.

Kamarulzaman, A., Hasanuzzaman, M., & Rahim, N. A. (2021). Global advancement of solar drying technologies and its future prospects: A review. Solar Energy, 221(December 2020), 559–582. doi:10.1016/j.solener.2021.04.056.

Gunawan, F. E., Budiman, A. S., Pardamean, B., Djuana, E., Romeli, S., Hananda, N., Harito, C., Aji, D. P. B., Putri, D. N. N., & Stevanus. (2022). Design and energy assessment of a new hybrid solar drying dome - Enabling Low-Cost, Independent and Smart Solar Dryer for Indonesia Agriculture 4.0. IOP Conference Series: Earth and Environmental Science, 998(1), 0–11. doi:10.1088/1755-1315/998/1/012052.

Putri, D. N. N., Djuana, E., Rahardjo, K., Aji, D. P., Gunawan, F. E., Budiman, A. S., Pardamean, B., Stevanus, & Romeli, S. (2022). Power System Design for Solar Dryer Dome in Agriculture. 5th International Conference on Power Engineering and Renewable Energy (ICPERE), (22-23 November 2022), Bandung, Indonesia. doi:10.1109/ICPERE56870.2022.10037364.

Udomkun, P., Romuli, S., Schock, S., Mahayothee, B., Sartas, M., Wossen, T., Njukwe, E., Vanlauwe, B., & Müller, J. (2020). Review of solar dryers for agricultural products in Asia and Africa: An innovation landscape approach. Journal of Environmental Management, 268, 110730. doi:10.1016/j.jenvman.2020.110730.

Ullah, I., Fayaz, M., Aman, M., & Kim, D. H. (2022). An optimization scheme for IoT based smart greenhouse climate control with efficient energy consumption. Computing, 104(2), 433–457. doi:10.1007/s00607-021-00963-5.

Morgner, P. et al. (2017). Privacy Implications of Room Climate Data. Computer Security – ESORICS 2017. Lecture Notes in Computer Science, 10493. Springer, Cham, Switzerland. doi:10.1007/978-3-319-66399-9_18.

Lauzon, F. Q. (2012). An introduction to deep learning. 2012 11th International Conference on Information Science, Signal Processing and Their Applications (ISSPA). doi:10.1109/isspa.2012.6310529.

Hewamalage, H., Bergmeir, C., & Bandara, K. (2021). Recurrent Neural Networks for Time Series Forecasting: Current status and future directions. International Journal of Forecasting, 37(1), 388–427. doi:10.1016/j.ijforecast.2020.06.008.

Yang, S., Yu, X., & Zhou, Y. (2020). LSTM and GRU Neural Network Performance Comparison Study: Taking Yelp Review Dataset as an Example. 2020 International Workshop on Electronic Communication and Artificial Intelligence (IWECAI). doi:10.1109/iwecai50956.2020.00027.

Hwang, S., Jeon, G., Jeong, J., & Lee, J. Y. (2019). A novel time series based Seq2Seq model for temperature prediction in firing furnace process. Procedia Computer Science, 155(2018), 19–26. doi:10.1016/j.procs.2019.08.007.

Yousuf, H., Lahzi, M., Salloum, S. A., & Shaalan, K. (2021). A systematic review on sequence-to-sequence learning with neural network and its models. International Journal of Electrical and Computer Engineering, 11(3), 2315–2326. doi:10.11591/ijece.v11i3.pp2315-2326.

Niu, Z., Zhong, G., & Yu, H. (2021). A review on the attention mechanism of deep learning. Neurocomputing, 452, 48–62. doi:10.1016/j.neucom.2021.03.091.

Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. doi:10.48550/arXiv.1409.0473

Fu, X., Gao, F., Wu, J., Wei, X., & Duan, F. (2019). Spatiotemporal Attention Networks for Wind Power Forecasting. 2019 International Conference on Data Mining Workshops (ICDMW). doi:10.1109/icdmw.2019.00032.

Setiawan, K. E., Elwirehardja, G. N., & Pardamean, B. (2022). Systematic Literature Review on Machine Learning Predictive Models for Indoor Climate in Smart Solar Dryer Dome. 2022 4th International Conference on Cybernetics and Intelligent System (ICORIS). doi:10.1109/icoris56080.2022.10031503.

Gunawan, F. E., Budiman, A. S., Pardamean, B., Djuana, E., Romeli, S., Cenggoro, T. W., ... & Asrol, M. (2021). Multivariate time-series deep learning for joint prediction of temperature and relative humidity in a closed space. International Conference on Computer Science and Computational Intelligence, November 2021, Virtual Conference.

Liu, Y., Li, D., Wan, S., Wang, F., Dou, W., Xu, X., Li, S., Ma, R., & Qi, L. (2022). A long short-term memory-based model for greenhouse climate prediction. International Journal of Intelligent Systems, 37(1), 135–151. doi:10.1002/int.22620.

Ali, A., & Hassanein, H. S. (2020). Time-Series Prediction for Sensing in Smart Greenhouses. GLOBECOM 2020 - 2020 IEEE Global Communications Conference. doi:10.1109/globecom42002.2020.9322549.

Ullah, I., Fayaz, M., Naveed, N., & Kim, D. (2020). ANN Based Learning to Kalman Filter Algorithm for Indoor Environment Prediction in Smart Greenhouse. IEEE Access, 8, 159371–159388. doi:10.1109/ACCESS.2020.3016277.

Allouhi, A., Choab, N., Hamrani, A., & Saadeddine, S. (2021). Machine learning algorithms to assess the thermal behavior of a Moroccan agriculture greenhouse. Cleaner Engineering and Technology, 5, 100346. doi:10.1016/j.clet.2021.100346.

Elhariri, E., & Taie, S. A. (2019). H-Ahead Multivariate microclimate Forecasting System Based on Deep Learning. 2019 International Conference on Innovative Trends in Computer Engineering (ITCE). doi:10.1109/itce.2019.8646540.

Chen, S., Li, B., Cao, J., & Mao, B. (2018). Research on Agricultural Environment Prediction Based on Deep Learning. Procedia Computer Science, 139, 33–40. doi:10.1016/j.procs.2018.10.214.

Fang, Z., Crimier, N., Scanu, L., Midelet, A., Alyafi, A., & Delinchant, B. (2021). Multi-zone indoor temperature prediction with LSTM-based sequence to sequence model. Energy and Buildings, 245, 111053. doi:10.1016/j.enbuild.2021.111053.

Setiawan, K. E., Elwirehardja, G. N., & Pardamean, B. (2022). Sequence to Sequence Deep Learning Architecture for Forecasting Temperature and Humidity inside Closed Space. 2022 10th International Conference on Cyber and IT Service Management (CITSM). doi:10.1109/citsm56380.2022.9936008.

Cenggoro, T. W., Tanzil, F., Aslamiah, A. H., Karuppiah, E. K., & Pardamean, B. (2018). Crowdsourcing annotation system of object counting dataset for deep learning algorithm. IOP Conference Series: Earth and Environmental Science, 195(1), 012063. doi:10.1088/1755-1315/195/1/012063.

Pardamean, B., Muljo, H. H., Cenggoro, T. W., Chandra, B. J., & Rahutomo, R. (2019). Using transfer learning for smart building management system. Journal of Big Data, 6(1), 1-12. doi:10.1186/s40537-019-0272-6.

Jebli, I., Belouadha, F. Z., Kabbaj, M. I., & Tilioua, A. (2021). Prediction of solar energy guided by Pearson correlation using machine learning. Energy, 224, 120109. doi:10.1016/j.energy.2021.120109.

Schober, P., & Schwarte, L. A. (2018). Correlation coefficients: Appropriate use and interpretation. Anesthesia and Analgesia, 126(5), 1763–1768. doi:10.1213/ANE.0000000000002864.

Van Houdt, G., Mosquera, C., & Nápoles, G. (2020). A review on the long short-term memory model. Artificial Intelligence Review, 53(8), 5929–5955. doi:10.1007/s10462-020-09838-1.

Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. doi:10.48550/arXiv.1412.3555.

Dey, R., & Salem, F. M. (2017). Gate-variants of Gated Recurrent Unit (GRU) neural networks. 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS). doi:10.1109/mwscas.2017.8053243.

Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, 1724–1734. doi:10.3115/v1/d14-1179.

Setiawan, K. E., Elwirehardja, G. N., & Pardamean, B. (2022). Comparison of Deep Learning Sequence-To-Sequence Models in Predicting Indoor Temperature and Humidity in Solar Dryer Dome. Communications in Mathematical Biology and Neuroscience, 2022, 1–26. doi:10.28919/cmbn/7655.

Bjorck, N., Gomes, C. P., Selman, B., & Weinberger, K. Q. (2018). Understanding batch normalization. Advances in neural information processing systems, 31.

Luong, M. T., Pham, H., & Manning, C. D. (2015). Effective approaches to attention-based neural machine translation. Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing, 1412–1421. doi:10.18653/v1/d15-1166.

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. doi:10.48550/arXiv.1412.6980

Chicco, D., Warrens, M. J., & Jurman, G. (2021). The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. PeerJ Computer Science, 7, e623. doi:10.7717/peerj-cs.623.

Shcherbakov, M. V., Brebels, A., Shcherbakova, N. L., Tyukov, A. P., Janovsky, T. A., & Kamaev, V. A. evich. (2013). A survey of forecast error measures. World Applied Sciences Journal, 24(24), 171–176. doi:10.5829/idosi.wasj.2013.24.itmies.80032.

Hair, J. F., Ringle, C. M., & Sarstedt, M. (2011). PLS-SEM: Indeed a silver bullet. Journal of Marketing Theory and Practice, 19(2), 139–152. doi:10.2753/MTP1069-6679190202.

Chauhan, A. (2017). Time Series Data Mining for Solar Active Region Classification. 1-7. doi:10.13140/RG.2.2.15327.05283.

Schober, P., & Vetter, T. R. (2020). Nonparametric Statistical Methods in Medical Research. Anesthesia and Analgesia, 131(6), 1862–1863. doi:10.1213/ANE.0000000000005101.

Keysers, C., Gazzola, V., & Wagenmakers, E. J. (2020). Using Bayes factor hypothesis testing in neuroscience to establish evidence of absence. Nature Neuroscience, 23(7), 788–799. doi:10.1038/s41593-020-0660-4.


Full Text: PDF

DOI: 10.28991/CEJ-2023-09-05-06

Refbacks

  • There are currently no refbacks.


Bookmark and Share


Copyright (c) 2023 Karli Eka Setiawan, Gregorius N. Elwirehardja, Bens Pardamean

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
x
Message