Assessment of Multimodal Rainfall Classification Systems Based on an Audio/Video Dataset

Roberta Avanzato, Francesco Beritelli, Antonio Raspanti, Michele Russo


In the past few years, there has been an increase in natural disasters due to hydrogeological instability caused by heavy rain. Therefore, to reduce the risk of an imminent occurrence of a disastrous event and reduce the risk to humans, an accurate estimate of the precipitation levels based on advanced machine learning techniques is necessary. In this paper, a new dataset is proposed containing audio/video data recorded via a multimodal rain gauge created ad hoc. The dataset, denominated AVDB-4RC (Audio/Video Database for Rainfall Classification), contains digital audio/video sequences recorded for seven different levels of precipitation intensity. In particular, the database presents a set of audio sequences containing the acoustic timbre produced by the rain and video sequences containing rain videos, both in seven different intensities, i.e., “No rain,” “Weak rain,” “Moderate rain,” “Heavy rain” and “Very heavy rain,” "Shower rain" and "Cloudburst rain." For the validation of the dataset, the paper proposes a novel rainfall classification approach based on a video pattern recognition system that uses CNN neural networks. The average classification accuracy is approximately 49% and can reach 75% if the adjacent misclassifications are not considered. Presumably, it is the first open dataset from the new generation acoustic/video rain gauges available for evaluating the estimated rainfall performance. We hope that this new open dataset will encourage a comparison of rainfall estimation/classification algorithms on this common database so that the adopted techniques are objectively assessed and improved.


audio/video database; rainfall classification; open dataset; performance evaluation; convolutional neural network.

Full Text:



J.M. Trabal, and D.J. McLaughlin, “Rainfall Estimation and Rain Gauge Comparison For X-Band Polarimetric CASA Radars,” in Proc. IEEE International Geoscience and Remote Sensing Symposium, 2017, pp. 2726-2729.

D. Nagel, “Detection of Rain Areas with Airborne Radar,” in Proc. 18th International Radar Symposium (IRS), 2017, pp. 1-7.

A. K. Shukla, C.S.P. Ojha, and R. D. Garg, “Comparative Study ff TRMM Satellite Predicted Rainfall Data with Rain Gauge Data Over Himalayan Basin,” in Proc. IEEE International Geoscience and Remote Sensing Symposium (IGRSS), 2018, pp. 9347-9350.

A. K. Varma, “Measurement of Precipitation from Satellite Radiometers (Visible, Infrared, and Microwave): Physical Basis, Methods, and Limitations,” Remote Sensing of Aerosols, Clouds, and Precipitation, pp. 223–248, 2018.

F. Beritelli, G. Capizzi, G. Lo Sciuto, F. Scaglione, D. Połap, and M. Woźniak, “A Neural Network Pattern Recognition Approach to Automatic Rainfall Classification by Using Signal Strength in LTE/4G Networks,” in Proc. International Joint Conference on Rough Sets, 2017, pp. 505-512.

F. Beritelli, G. Capizzi, G. Lo Sciuto, C. Napoli, and F. Scaglione, “Rainfall estimation based on the intensity of the received signal in a LTE/4G mobile terminal by using a probabilistic neural network,” IEEE Access, vol. 6, pp. 30865–30873, May 2018.

R. Avanzato, F. Beritelli, F. Di Franco, and V.F. Puglisi, “A Convolutional Neural Networks approach to Audio Classification for Rainfall Estimation,” in Proc. 10th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, 2019, pp. 285-289.

A. Gupta, A. Bansal, R. Gupta, D. Naryani, and A. Sood, “Urban Waterlogging Detection and Severity Prediction Using Artificial Neural Networks,” in Proc. IEEE 19th International Conference on High Performance Computing and Communications; IEEE 15th International Conference on Smart City; IEEE 3rd International Conference on Data Science and Systems (HPCC/SmartCity/DSS), 2017, pp. 42-49.

A. H. Manek, and P.K. Singh, “Comparative Study of Neural Network Architectures for Rainfall Prediction,” in Proc. IEEE Technological Innovations in ICT for Agriculture and Rural Development (TIAR), 2016, pp. 171-174.

F. Beritelli, and A. Spadaccini, “Statistical Approach to Biometric Identity Verification based on Heart Sounds,” in Proc. Fourth International Conference on Emerging Security Information, Systems and Technologies, 2010, pp. 93-96.

F. Beritelli, and A. Spadaccini, “The Role of Voice Activity Detection in Forensic Speaker Verification,” in Proc. 17th IEEE International Conference on Digital Signal Processing (DSP), 2011.

W. Dai, C. Dai, S. Qu, J. Li, and S. Das, “Very Deep Convolutional Neural Network for Raw Waveforms,” in Proc. IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2017.

S. Sawant, and P. A. Ghonge, “Estimation of rain drop analysis using image processing, electronics and telecommunication,” International Journal of Science and Research (IJSR), vol. 4, pp. 1981–1986, Jan. 2015.

P. Bacche, S. Basantani, P. Deshpande, J. Joshi, and R. Bhalwankar, “Measurement of raindrop parameters using image processing,” International Journal of Innovative and Emerging Research in Engineering, vol. 3, pp. 41–46, 2016.

K. H. V. Reddy, S.M. Basha, and J. Srinivasulu, “A simple approach for efficient detection and estimation of drops during the rainfall,” IJISET - International Journal of Innovative Science, Engineering & Technology, vol. 2, no. 10, pp. 203–207, Oct. 2015.

F. Nashashibi, R. De Charette, and A. Lia, “Detection of Unfocused Raindrops on a Windscreen using Low Level Image Processing,” in Proc.11th International Conference on Control Automation Robotics & Vision, 2010.

R. Avanzato, and F. Beritelli, “An Innovative Acoustic Rain Gauge Based on Convolutional Neural Networks,” MDPI Information, vol. 11, no. 4, pp. 183, March 2020.

(2020) Smaniotto. [Online]. Available.

F. Beritelli, A. Gallotta, and C. Rametta “A Dual Streaming Approach for Speech Quality Enhancement of VoIP Service Over 3G Networks,” in Proc. IEEE International Conference on Digital Signal Processing (DSP), 2013.

C. Chen, Y. Tock, and S. Girdzijauskas, “BeaConvey: Co-Design of Overlay and Routing for Topic-based Publish/Subscribe on Small-World Networks,” in Proc. 12th ACM International Conference on Distributed and Event-based Systems, 2018, pp. 64–75.

F.N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-Level Accuracy With 50x Fewer Parameters And <0.5mb Model Size,” Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI), pp. 1-12, Nov. 2016. DOI: arXiv:1602.07360.

A. Krizhevsky, I. Sutskever, and G.E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Advances in Neural Information Processing Systems 25 (NIPS 2012).

A. Gholami, K. Kwon, B. Wu, Z. Tai, X. Yue, P. Jin, S. Zhao, and K. Keutzer, “SqueezeNext: Hardware-Aware Neural Network Design,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018, pp. 1638-1647. DOI: arXiv:1803.10.



  • There are currently no refbacks.

Published by INSIGHT - Indonesian Society for Knowledge and Human Development