Business Category Classification via Indistinctive Satellite Image Analysis Using Deep Learning

Injamul Haque Suvon, Yuen Peng Loh, Noramiza Hashim, Wan Noorshahida Mohd-Isa, Choo-Yee Ting, Khairil Imran Ghauth, Arpita Bhattacharijee, Wan Razali Matsah

Abstract


Satellite image analysis has numerous useful applications in various domains. Extracting their visual information has been made easier using remote sensing and deep learning technologies that intelligently interpret clear visual cues. However, satellite information has the potential for more complex tasks, such as recommending business locations and categories based on the implicit patterns and structures of the regions of interest. Nonetheless, this task is significantly more challenging due to the absence of obvious visual cues and the highly similar appearance of each location. This study aims to analyze satellite image similarity between business class categories and investigate the capabilities of state-of-the-art deep learning models for learning non-obvious visual cues. Specifically, a satellite image dataset is constructed using business locations and annotated with the business categories for image structural similarity analysis, followed by business category classification via fine-tuning of deep learning classifiers. The models are then analyzed by visualizing the features learned to determine if they could capture hidden information for such a task. Experiments show that business locations have significantly high SSIM regardless of categories, and deep learning models only recorded a top accuracy of 60%. However, feature visualization using Grad-CAM shows that the models learn biased features and disregard highly informative details such as roads. It is concluded that typical learning models and strategies are insufficient to effectively solve this complex visual problem; thus, further research should be done to formulate solutions for such non-obvious classifications with the potential to support business recommendation applications.

Keywords


Transfer learning; CNN; recommendation; scene classification; visual cue; road network; Grad-CAM

Full Text:

PDF

References


D. Song, X. Tan, B. Wang, L. Zhang, X. Shan, and J. Cui, “Integration of super-pixel segmentation and deep-learning methods for evaluating earthquake-damaged buildings using single-phase remote sensing imagery,†Int J Remote Sens, vol. 41, no. 3, 2020, doi: 10.1080/01431161.2019.1655175.

H. S. Munawar, F. Ullah, S. Qayyum, S. I. Khan, and M. Mojtahedi, “Uavs in disaster management: Application of integrated aerial imagery and convolutional neural network for flood detection,†Sustainability (Switzerland), vol. 13, no. 14, 2021, doi: 10.3390/su13147547.

Y. Pi, N. D. Nath, and A. H. Behzadan, “Convolutional neural networks for object detection in aerial imagery for disaster response and recovery,†Advanced Engineering Informatics, vol. 43, 2020, doi: 10.1016/j.aei.2019.101009.

D. Q. Tran, M. Park, D. Jung, and S. Park, “Damage-map estimation using uav images and deep learning algorithms for disaster management system,†Remote Sens (Basel), vol. 12, no. 24, 2020, doi: 10.3390/rs12244169.

C. Fan, C. Zhang, A. Yahja, and A. Mostafavi, “Disaster City Digital Twin: A vision for integrating artificial and human intelligence for disaster management,†Int J Inf Manage, vol. 56, 2021, doi: 10.1016/j.ijinfomgt.2019.102049.

H. S. Munawar, A. Hammad, F. Ullah, and ..., “After the flood: A novel application of image processing and machine learning for post-flood disaster management,†Proceedings of the 2nd …, no. December, 2019.

Z. Zheng, Y. Zhong, J. Wang, A. Ma, and L. Zhang, “Building damage assessment for rapid disaster response with a deep object-based semantic change detection framework: From natural disasters to man-made disasters,†Remote Sens Environ, vol. 265, 2021, doi: 10.1016/j.rse.2021.112636.

M. Amani et al., “Application of google earth engine cloud computing platform, sentinel imagery, and neural networks for crop mapping in Canada,†Remote Sens (Basel), vol. 12, no. 21, 2020, doi: 10.3390/rs12213561.

M. Burke, A. Driscoll, D. B. Lobell, and S. Ermon, “Using satellite imagery to understand and promote sustainable development,†Science, vol. 371, no. 6535. 2021. doi: 10.1126/science.abe8628.

J. da R. Miranda, M. de C. Alves, E. A. Pozza, and H. Santos Neto, “Detection of coffee berry necrosis by digital image processing of landsat 8 oli satellite imagery,†International Journal of Applied Earth Observation and Geoinformation, vol. 85, 2020, doi: 10.1016/j.jag.2019.101983.

T. T. Nguyen et al., “Monitoring agriculture areas with satellite images and deep learning,†Applied Soft Computing Journal, vol. 95, 2020, doi: 10.1016/j.asoc.2020.106565.

A. Sharifi, “Yield prediction with machine learning algorithms and satellite images,†J Sci Food Agric, vol. 101, no. 3, 2021, doi: 10.1002/jsfa.10696.

X. Huang, Y. Cao, and J. Li, “An automatic change detection method for monitoring newly constructed building areas using time-series multi-view high-resolution optical satellite images,†Remote Sens Environ, vol. 244, 2020, doi: 10.1016/j.rse.2020.111802.

J. John, G. Bindu, B. Srimuruganandam, A. Wadhwa, and P. Rajan, “Land use/land cover and land surface temperature analysis in Wayanad district, India, using satellite imagery,†Ann GIS, vol. 26, no. 4, 2020, doi: 10.1080/19475683.2020.1733662.

N. KranjÄić, D. Medak, R. Župan, and M. Rezo, “Support Vector Machine accuracy assessment for extracting green urban areas in towns,†Remote Sens (Basel), vol. 11, no. 6, 2019, doi: 10.3390/rs11060655.

Z. Pan, J. Xu, Y. Guo, Y. Hu, and G. Wang, “Deep learning segmentation and classification for urban village using a worldview satellite image based on U-net,†Remote Sens (Basel), vol. 12, no. 10, 2020, doi: 10.3390/rs12101574.

W. Sirko et al., “Continental-Scale Building Detection from High Resolution Satellite Imagery,†Jul. 2021, [Online]. Available: http://arxiv.org/abs/2107.12283

D. Verma, A. Jana, and K. Ramamritham, “Transfer learning approach to map urban slums using high and medium resolution satellite imagery,†Habitat Int, vol. 88, 2019, doi: 10.1016/j.habitatint.2019.04.008.

M. Wurm, T. Stark, X. X. Zhu, M. Weigand, and H. Taubenböck, “Semantic segmentation of slums in satellite images using transfer learning on fully convolutional neural networks,†ISPRS Journal of Photogrammetry and Remote Sensing, vol. 150, 2019, doi: 10.1016/j.isprsjprs.2019.02.006.

M. Wu, C. Zhang, J. Liu, L. Zhou, and X. Li, “Towards Accurate High Resolution Satellite Image Semantic Segmentation,†IEEE Access, vol. 7, 2019, doi: 10.1109/ACCESS.2019.2913442.

T. Zhang, J. Su, Z. Xu, Y. Luo, and J. Li, “Sentinel-2 satellite imagery for urban land cover classification by optimized random forest classifier,†Applied Sciences (Switzerland), vol. 11, no. 2, 2021, doi: 10.3390/app11020543.

Q. Zhu et al., “A Global Context-aware and Batch-independent Network for road extraction from VHR satellite imagery,†ISPRS Journal of Photogrammetry and Remote Sensing, vol. 175. 2021. doi: 10.1016/j.isprsjprs.2021.03.016.

Y. Xu, Y. Shen, Y. Zhu, and J. Yu, “Ar2Net: An attentive neural approach for business location selection with satellite data and urban data,†ACM Trans Knowl Discov Data, vol. 14, no. 2, 2020, doi: 10.1145/3372406.

K. Topouzelis, A. Papakonstantinou, and S. P. Garaba, “Detection of floating plastics from satellite and unmanned aerial systems (Plastic Litter Project 2018),†International Journal of Applied Earth Observation and Geoinformation, vol. 79, 2019, doi: 10.1016/j.jag.2019.03.011.

G. N. Vivekananda, R. Swathi, and A. V. L. N. Sujith, “Multi-temporal image analysis for LULC classification and change detection,†Eur J Remote Sens, vol. 54, no. sup2, 2021, doi: 10.1080/22797254.2020.1771215.

R. Gupta and M. Shah, “RescueNet: Joint building segmentation and damage assessment from satellite imagery,†in Proceedings - International Conference on Pattern Recognition, 2020. doi: 10.1109/ICPR48806.2021.9412295.

H. Li, Y. He, Q. Xu, J. Deng, W. Li, and Y. Wei, “Detection and segmentation of loess landslides via satellite images: a two-phase framework,†Landslides, vol. 19, no. 3, 2022, doi: 10.1007/s10346-021-01789-0.

E. Weber and H. Kané, “Building Disaster Damage Assessment in Satellite Imagery with Multi-Temporal Fusion,†Apr. 2020, [Online]. Available: http://arxiv.org/abs/2004.05525

E. Weber et al., “Detecting Natural Disasters, Damage, and Incidents in the Wild,†in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020. doi: 10.1007/978-3-030-58529-7_20.

Y. Yi and W. Zhang, “A New Deep-Learning-Based Approach for Earthquake-Triggered Landslide Detection from Singleoral RapidEye Satellite Imagery,†IEEE J Sel Top Appl Earth Obs Remote Sens, vol. 13, pp. 6166–6176, 2020, doi: 10.1109/JSTARS.2020.3028855.

J. Aversa, S. Doherty, and T. Hernandez, “Big Data Analytics: The New Boundaries of Retail Location Decision Making,†Papers in Applied Geography, vol. 4, no. 4, 2018, doi: 10.1080/23754931.2018.1527720.

A. M. B. M. Rohani and F. F. Chua, “Location analytics for optimal business retail site selection,†in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2018. doi: 10.1007/978-3-319-95162-1_27.

Z. Y. Poo, C. Y. Ting, Y. P. Loh, and K. I. Ghauth, “Multi-Label Classification with Deep Learning for Retail Recommendation,†Journal of Informatics and Web Engineering, vol. 2, no. 2, 2023, doi: 10.33093/jiwe.2023.2.2.16.

A. Alem and S. Kumar, “Transfer Learning Models for Land Cover and Land Use Classification in Remote Sensing Image,†Applied Artificial Intelligence, vol. 36, no. 1, 2022, doi: 10.1080/08839514.2021.2014192.

A. Betti, L. M. Seijas, R. N. Giraldez, and J. L. Márquez, “High spatial resolution remote sensing image scene classification using CNN with transfer learning,†in 2020 IEEE Congreso Bienal de Argentina, ARGENCON 2020 - 2020 IEEE Biennial Congress of Argentina, ARGENCON 2020, 2020. doi: 10.1109/ARGENCON49523.2020.9505492.

U. Muhammad, W. Wang, S. P. Chattha, and S. Ali, “Pre-trained VGGNet Architecture for Remote-Sensing Image Scene Classification,†in Proceedings - International Conference on Pattern Recognition, 2018. doi: 10.1109/ICPR.2018.8545591.

Q. Wang, W. Huang, Z. Xiong, and X. Li, “Looking Closer at the Scene: Multiscale Representation Learning for Remote Sensing Image Scene Classification,†IEEE Trans Neural Netw Learn Syst, vol. 33, no. 4, 2022, doi: 10.1109/TNNLS.2020.3042276.

R. Naushad, T. Kaur, and E. Ghaderpour, “Deep transfer learning for land use and land cover classification: A comparative study,†Sensors, vol. 21, no. 23, 2021, doi: 10.3390/s21238083.

A. Albert, J. Kaur, and M. C. Gonzalez, “Using convolutional networks and satellite imagery to identify patterns in urban environments at a large scale,†in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017. doi: 10.1145/3097983.3098070.

Y. Li, Y. Zheng, S. Ji, W. Wang, L. H. U, and Z. Gong, “Location selection for ambulance stations: A data-driven approach,†in GIS: Proceedings of the ACM International Symposium on Advances in Geographic Information Systems, 2015. doi: 10.1145/2820783.2820876.

L. Wang, H. Fan, and Y. Wang, “Site selection of retail shops based on spatial accessibility and hybrid BP neural network,†ISPRS Int J Geoinf, vol. 7, no. 6, 2018, doi: 10.3390/ijgi7060202.

Y. Rui, Z. Yang, T. Qian, S. Khalid, N. Xia, and J. Wang, “Network-constrained and category-based point pattern analysis for Suguo retail stores in Nanjing, China,†International Journal of Geographical Information Science, vol. 30, no. 2, 2016, doi: 10.1080/13658816.2015.1080829.

G. Lin, X. Chen, and Y. Liang, “The location of retail stores and street centrality in Guangzhou, China,†Applied Geography, vol. 100, 2018, doi: 10.1016/j.apgeog.2018.08.007.

Q. Li et al., “Warehouse Vis: A Visual Analytics Approach to Facilitating Warehouse Location Selection for Business Districts,†Computer Graphics Forum, vol. 39, no. 3, 2020, doi: 10.1111/cgf.13996.

D. Liu et al., “SmartAdP: Visual Analytics of Large-scale Taxi Trajectories for Selecting Billboard Locations,†IEEE Trans Vis Comput Graph, vol. 23, no. 1, 2017, doi: 10.1109/TVCG.2016.2598432.

Z. Zheng, T. Morimoto, and Y. Murayama, “Optimal location analysis of delivery parcel-pickup points using AHP and network huff model: A case study of shiweitang sub-district in Guangzhou city, China,†ISPRS Int J Geoinf, vol. 9, no. 4, 2020, doi: 10.3390/ijgi9040193.




DOI: http://dx.doi.org/10.18517/ijaseit.13.6.19059

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development