Efficient Supervised Features Learning for Remote Sensing Image Classification
Abstract
The features extracted from the fully connected (FC) layers of a convolutional neural network (ConvNet or CNN) can provide accurate classification results as long as the labelled datasets are large enough. On the other end, high accuracy remote sensing image (RSI) classification is demanded various implementations such as urban planning, environmental monitoring, and geographic image retrieval. Many studies have been presented in this domain; however, satisfactory classification accuracy is yet to be achieved. In this study, the proposed method of fine-tuning the pre-trained ConvNets (GoogleNet, VGG16, and ResNet50) on RSI, extracting features from the last fine-tuned FC layer of these networks and reprocess the extracted features for classification by SVM, produced high classification accuracy. Extensive experiments have been conducted on three RSI datasets: the NWPU, AID, and PatternNet. Comparative results over the selected datasets demonstrate that our method considerably outperforms the state-of-the-art best-stated results. Also, the overall accuracy (OA) and confusion matrix report quantitative evaluation. Our best outcomes from the first part were 99.54%, 94.60%, and 94.83% on the PatternNet, NWPU, and AID datasets, respectively, achieved by fine-tuned ResNet50. Moreover, the best classification accuracies with training ratios 20% and 50% on the AID dataset, 10% and 20% on the NWPU dataset, and the 10%, 20%, 50% and 80% on PatternNet dataset were 95.72%, 97.53%, 96.19%, 96.85%, 99.60%, 99.56%, 99.75% and 99.80% respectively. The classification performance of each class was estimated using a confusion matrix for the three datasets.
Keywords
Full Text:
PDFReferences
G. Cheng, J. Han, and X. Lu, "Remote sensing image scene classification: Benchmark and state of the art," Proceedings of the IEEE, vol. 105, pp. 1865-1883, 2017.
L. Fang, N. He, S. Li, P. Ghamisi, and J. A. Benediktsson, "Extinction profiles fusion for hyperspectral images classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 56, pp. 1803-1815, 2017.
G.-S. Xia, J. Hu, F. Hu, B. Shi, X. Bai, Y. Zhong, et al., "AID: A benchmark data set for performance evaluation of aerial scene classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 55, pp. 3965-3981, 2017.
W. Zhou, S. Newsam, C. Li, and Z. Shao, "PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval," ISPRS journal of photogrammetry and remote sensing, vol. 145, pp. 197-209, 2018.
X. Bian, C. Chen, L. Tian, and Q. Du, "Fusing local and global features for high-resolution scene classification," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 10, pp. 2889-2901, 2017.
G. Cheng, J. Han, L. Guo, and T. Liu, "Learning coarse-to-fine sparselets for efficient object detection and scene classification," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1173-1181.
G. Cheng, J. Han, L. Guo, Z. Liu, S. Bu, and J. Ren, "Effective and efficient midlevel visual elements-oriented land-use classification using VHR remote sensing images," IEEE Transactions on Geoscience and Remote Sensing, vol. 53, pp. 4238-4249, 2015.
L. Huang, C. Chen, W. Li, and Q. Du, "Remote sensing image scene classification using multi-scale completed local binary patterns and fisher vectors," Remote Sensing, vol. 8, p. 483, 2016.
X. Lu, X. Zheng, and Y. Yuan, "Remote sensing scene classification by unsupervised representation learning," IEEE Transactions on Geoscience and Remote Sensing, vol. 55, pp. 5148-5157, 2017.
B. Zhao, Y. Zhong, G.-S. Xia, and L. Zhang, "Dirichlet-derived multiple topic scene classification model for high spatial resolution remote sensing imagery," IEEE Transactions on Geoscience and Remote Sensing, vol. 54, pp. 2108-2123, 2015.
Q. Zhu, Y. Zhong, B. Zhao, G.-S. Xia, and L. Zhang, "Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery," IEEE Geoscience and Remote Sensing Letters, vol. 13, pp. 747-751, 2016.
J. Zou, W. Li, C. Chen, and Q. Du, "Scene classification using local and global features with collaborative representation fusion," Information Sciences, vol. 348, pp. 209-226, 2016.
Z. Chen, Y. Wang, W. Han, R. Feng, and J. Chen, "An Improved Pretraining Strategy-Based Scene Classification With Deep Learning," IEEE Geoscience and Remote Sensing Letters, 2019.
M. M. Al Rahhal, Y. Bazi, T. Abdullah, M. L. Mekhalfi, H. AlHichri, and M. Zuair, "Learning a multi-branch neural network from multiple sources for knowledge adaptation in remote sensing imagery," Remote Sensing, vol. 10, p. 1890, 2018.
O. Sen and H. Y. Keles, "Scene Recognition with Deep Learning Methods Using Aerial Images," in 2019 27th Signal Processing and Communications Applications Conference (SIU), 2019, pp. 1-4.
Y. Yao, H. Zhao, D. Huang, and Q. Tan, " Remote Sensing Scene Classification Using Multiple Pyramid Pooling," International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 2019.
Y. Feng, Y. Yuan, and X. Lu, "Learning deep event models for crowd anomaly detection," Neurocomputing, vol. 219, pp. 548-556, 2017.
X. Lu, B. Wang, X. Zheng, and X. Li, "Exploring models and data for remote sensing image caption generation," IEEE Transactions on Geoscience and Remote Sensing, vol. 56, pp. 2183-2195, 2017.
W. Zhang, X. Lu, and X. Li, "A coarse-to-fine semi-supervised change detection for multispectral images," IEEE Transactions on Geoscience and Remote Sensing, vol. 56, pp. 3587-3599, 2018.
J. Zhu, L. Fang, and P. Ghamisi, "Deformable convolutional neural networks for hyperspectral image classification," IEEE Geoscience and Remote Sensing Letters, vol. 15, pp. 1254-1258, 2018.
M. A. Kadhim and M. H. Abed, "Convolutional Neural Network for Satellite Image Classification," in Asian Conference on Intelligent Information and Database Systems, 2019, pp. 165-178.
Y. Yu and F. Liu, "Dense connectivity based two-stream deep feature fusion framework for aerial scene classification," Remote Sensing, vol. 10, p. 1158, 2018.
F. Hu, G.-S. Xia, J. Hu, and L. Zhang, "Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery," Remote Sensing, vol. 7, pp. 14680-14707, 2015.
G. Cheng, C. Yang, X. Yao, L. Guo, and J. Han, "When deep learning meets metric learning: Remote sensing image scene classification via learning discriminative CNNs," IEEE transactions on geoscience and remote sensing, vol. 56, pp. 2811-2821, 2018.
N. He, L. Fang, S. Li, A. Plaza, and J. Plaza, "Remote sensing scene classification using multilayer stacked covariance pooling," IEEE Transactions on Geoscience and Remote Sensing, vol. 56, pp. 6899-6910, 2018.
J. Xie, N. He, L. Fang, and A. Plaza, "Scale-free convolutional neural network for remote sensing scene classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 57, pp. 6916-6928, 2019.
J. Zhang, C. Lu, X. Li, H.-J. Kim, and J. Wang, "A full convolutional network based on DenseNet for remote sensing scene classification," Math. Biosci. Eng, vol. 16, pp. 3345-3367, 2019.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9.
K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, 2012, pp. 1097-1105.
N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), 2005, pp. 886-893.
M. J. Swain and D. H. Ballard, "Color indexing," International journal of computer vision, vol. 7, pp. 11-32, 1991.
A. Oliva and A. Torralba, "Modeling the shape of the scene: A holistic representation of the spatial envelope," International journal of computer vision, vol. 42, pp. 145-175, 2001.
A. Coates and A. Y. Ng, "Learning feature representations with k-means," in Neural networks: Tricks of the trade, ed: Springer, 2012, pp. 561-580.
G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," science, vol. 313, pp. 504-507, 2006.
L. K. Saul and S. T. Roweis, "An introduction to locally linear embedding," unpublished. Available at: http://www. cs. toronto. edu/~ roweis/lle/publications. html, 2000.
F. Özyurt, "Efficient deep feature selection for remote sensing image recognition with fused deep learning architectures," The Journal of Supercomputing, pp. 1-19, 2019.
R. Zhu, L. Yan, N. Mo, and Y. Liu, "Attention-Based Deep Feature Fusion for the Scene Classification of High-Resolution Remote Sensing Images," Remote Sensing, vol. 11, p. 1996, 2019.
K. Nogueira, O. A. Penatti, and J. A. Dos Santos, "Towards better exploiting convolutional neural networks for remote sensing scene classification," Pattern Recognition, vol. 61, pp. 539-556, 2017.
S. Chaib, H. Liu, Y. Gu, and H. Yao, "Deep feature fusion for VHR remote sensing scene classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 55, pp. 4775-4784, 2017.
S. Hijazi, R. Kumar, and C. Rowen, "Using convolutional neural networks for image recognition," Cadence Design Systems Inc.: San Jose, CA, USA, pp. 1-12, 2015.
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, et al., "Imagenet large scale visual recognition challenge," International journal of computer vision, vol. 115, pp. 211-252, 2015.
O. A. Penatti, K. Nogueira, and J. A. Dos Santos, "Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2015, pp. 44-51.
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, et al., "Caffe: Convolutional architecture for fast feature embedding," in Proceedings of the 22nd ACM international conference on Multimedia, 2014, pp. 675-678.
P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "Overfeat: Integrated recognition, localization and detection using convolutional networks," arXiv preprint arXiv:1312.6229, 2013.
C. Chen, B. Zhang, H. Su, W. Li, and L. Wang, "Land-use scene classification using multi-scale completed local binary patterns," Signal, image and video processing, vol. 10, pp. 745-752, 2016.
M. Castelluccio, G. Poggi, C. Sansone, and L. Verdoliva, "Land use classification in remote sensing images by convolutional neural networks," arXiv preprint arXiv:1508.00092, 2015.
Y. Liu and C. Huang, "Scene classification via triplet networks," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, pp. 220-237, 2017.
D. Marmanis, M. Datcu, T. Esch, and U. Stilla, "Deep learning earth observation classification using ImageNet pretrained networks," IEEE Geoscience and Remote Sensing Letters, vol. 13, pp. 105-109, 2015.
N. Liu, X. Lu, L. Wan, H. Huo, and T. Fang, "Improving the separability of deep features with discriminative convolution filters for RSI classification," ISPRS International Journal of Geo-Information, vol. 7, p. 95, 2018.
N. Liu, L. Wan, Y. Zhang, T. Zhou, H. Huo, and T. Fang, "Exploiting convolutional neural networks with deeply local description for remote sensing image classification," IEEE access, vol. 6, pp. 11215-11228, 2018.
C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM transactions on intelligent systems and technology (TIST), vol. 2, pp. 1-27, 2011.
K. Qi, C. Yang, Q. Guan, H. Wu, and J. Gong, "A multi-scale deeply described correlatons-based model for land-use scene classification," Remote Sensing, vol. 9, p. 917, 2017.
S. Basu, S. Ganguly, S. Mukhopadhyay, R. DiBiano, M. Karki, and R. Nemani, "Deepsat: a learning framework for satellite imagery," in Proceedings of the 23rd SIGSPATIAL international conference on advances in geographic information systems, 2015, pp. 1-10.
G. Sheng, W. Yang, T. Xu, and H. Sun, "High-resolution satellite scene classification using a sparse coding based multiple feature combination," International journal of remote sensing, vol. 33, pp. 2395-2412, 2012.
H. Li, C. Tao, Z. Wu, J. Chen, J. Gong, and M. Deng, "Rsi-cb: A large scale remote sensing image classification benchmark via crowdsource data," arXiv preprint arXiv:1705.10450, 2017.
Q. Zou, L. Ni, T. Zhang, and Q. Wang, "Deep learning based feature selection for remote sensing scene classification," IEEE Geoscience and Remote Sensing Letters, vol. 12, pp. 2321-2325, 2015.
L. Zhao, P. Tang, and L. Huo, "Feature significance-based multibag-of-visual-words model for remote sensing image scene classification," Journal of Applied Remote Sensing, vol. 10, p. 035004, 2016.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in 2009 IEEE conference on computer vision and pattern recognition, 2009, pp. 248-255.
DOI: http://dx.doi.org/10.18517/ijaseit.11.2.11272
Refbacks
- There are currently no refbacks.
Published by INSIGHT - Indonesian Society for Knowledge and Human Development