Image Retrieval based on the Fusion of Graph Method with Color Moments, GLCM, and Hu Moments

- Akmal, Rinaldi Munir, Judhi Santoso

Abstract


Retrieving images that are similar to the query image in the image database means determining the similarity between the images. This study aims to use a graph method with region adjacency graph representation in conjunction with a non-graph method in image retrieval. We represented an image as a graph and used the Graph Edit Distance (GED) method to calculate the similarity between two graphs. The feature extraction of the image graph, which exposes the content and the relationships between existing content, is a key step in image retrieval based on the graph method. The extraction of graph features is accomplished by the image segmentation method, which divides the image into regions and represents them as a region-adjacency graph (RAG), in which vertices represent regions and edges indicate two neighboring regions. Image retrieval based on the graph method is combined with low-level approaches like Color Moments, Gray Level Co-occurrence Matrix (GLCM), and Hu Moments to boost accuracy. All obtained features are normalized, weighted, and then compared between images to get the similarity value using Euclidean Distance. An image retrieval prototype based on the combined graph method and non-graph method was successfully created in this work, using four datasets: synthetic, batik, COIL-100, and Wang. The MAP of the four datasets is 67.84 percent, but when combined with the low-level feature approach, it rises to between 79.73 and 89.71 percent. The combination of graph and non-graph algorithms improves image retrieval outcomes.

Keywords


Region adjacency graph; graph edit distance; color moment; gray level co-occurrence matrix; hu moment

Full Text:

PDF

References


A. Latif et al., "Content-based image retrieval and feature extraction: A comprehensive review," Math. Probl. Eng., vol. 2019, 2019, doi: 10.1155/2019/9658350.

A. Holzinger, B. Malle, and N. Giuliani, "On Graph Extraction from Image Data," in International Conference on Brain Informatics and Health, 2014, no. 2003, pp. 552–563.

L. Brun, P. Foggia, and M. Vento, "Trends in Graph-based Representations for Pattern Recognition," Pattern Recognit. Lett. Elsevier, vol. 134, pp. 3–9, 2020, doi: 10.1016/j.patrec.2018.03.016.

H. Sun, W. Zhou, and M. Fei, "A Survey On Graph Matching In Computer Vision," in 13th International Congress on Image and Signal Processing , BioMedical Engineering and Informatics (CISP-BMEI), 2020, pp. 225–230.

I. Belahyane, M. Mammass, H. Abioui, and A. Idarrou, "Graph-based image retrieval: State of the art," in 9th International Conference on Image and Signal Processing, 2020, vol. 12119 LNCS, pp. 299–307, doi: 10.1007/978-3-030-51935-3_32.

S. Chavda and G. Mahesh, "Content-Based Image Retrieval: The State of the Art," Int. J. Next-Generation Comput., vol. 10, no. December, pp. 193–212, 2019.

I. M. Hameed, S. H. Abdulhussain, and B. M. Mahmmod, "Content-based image retrieval: A review of recent trends," Cogent Eng., vol. 8, no. 1, 2021, doi: 10.1080/23311916.2021.1927469.

A. K. Nath and A. Wang, "A Survey on Personal Image Retrieval Systems," pp. 1–30, 2020.

S. Tena, R. Hartanto, and I. Ardiyanto, "Content-based image retrieval for fabric images: A survey," Indones. J. Electr. Eng. Comput. Sci., vol. 23, no. 3, pp. 1861–1872, 2021, doi: 10.11591/ijeecs.v23.i3.pp1861-1872.

F. D. Wang, N. Xue, Y. Zhang, G. S. Xia, and M. Pelillo, "A functional representation for graph matching," IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 11, pp. 2737–2754, 2020, doi: 10.1109/TPAMI.2019.2919308.

H. Sharma et al., "Determining similarity in histological images using graph-theoretic description and matching methods for content-based image retrieval in medical diagnostic," Diagn. Pathol., no. October 2012, 2012, doi: 10.1186/1746-1596-7-134.

A. Kumar, J. Kim, D. Feng, and M. Fulham, "Graph-based retrieval of PET-CT images using vector space embedding," Proc. CBMS 2013 - 26th IEEE Int. Symp. Comput. Med. Syst., pp. 413–416, 2013, doi: 10.1109/CBMS.2013.6627829.

S. Luo and H. Z. J. Xu, "Matching images based on consistency graph and region adjacency graphs," Signal, Image Video Process., 2016, doi: 10.1007/s11760-016-0987-1.

D. Valdes-amaro and E. Lopez-prieto, "Image Retrieval using Graphs," in Future Technologies Conference (FTC) 2017, 2017, no. November, pp. 1022–1025.

M. Kashif, G. Raja, and F. Shaukat, "An Efficient Content-Based Image Retrieval System for the Diagnosis of Lung Diseases," J. Digit. Imaging, vol. 33, no. 4, pp. 971–987, 2020, doi: 10.1007/s10278-020-00338-w.

L. P. Valem and D. C. G. Pedronette, "Graph-based selective rank fusion for unsupervised image retrieval," Pattern Recognit. Lett., vol. 135, pp. 82–89, 2020, doi: 10.1016/j.patrec.2020.03.032.

X. Cortés, D. Conte, and H. Cardot, "Learning edit cost estimation models for graph edit distance ✩," Pattern Recognit. Lett. Elsevier, vol. 125, pp. 256–263, 2019, doi: 10.1016/j.patrec.2019.05.001.

H. Ji-Zhao, G.-H. Liu, and S.-X. Song, "Content-based image retrieval using color volume histograms," Int. J. Pattern Recognit. Artif. Intell., vol. 33, no. 9, p. Article ID 6283987, 2019.

B. Patel, K. Yadav, and D. Ghosh, "Current Trend and Methodologies of Content-Based Image Retrieval: Survey," 2021, doi: https://doi.org/10.1007/978-981-15-6707-0_64.

M. K. Alsmadi, "Content-Based Image Retrieval Using Color, Shape and Texture Descriptors and Features," Arab. J. Sci. Eng., vol. 45, no. 4, pp. 3317–3330, 2020, doi: 10.1007/s13369-020-04384-y.

M. Subramanian, V. Lingamuthu, C. Venkatesan, and S. Perumal, "Content-Based Image Retrieval Using Colour, Gray, Advanced Texture, Shape Features, and Random Forest Classifier with Optimized Particle Swarm Optimization," Int. J. Biomed. Imaging, vol. 2022, 2022, doi: 10.1155/2022/3211793.

S. Singh and S. Batra, "An efficient bi-layer content based image retrieval system," Multimed. Tools Appl., vol. 79, no. 25–26, pp. 17731–17759, 2020, doi: 10.1007/s11042-019-08401-7.

E. R. Vimina and M. O. Divya, "Maximal multi-channel local binary pattern with colour information for CBIR," Multimed. Tools Appl., vol. 79, no. 35–36, pp. 25357–25377, 2020, doi: 10.1007/s11042-020-09207-8.

U. A. Khan, A. Javed, and R. Ashraf, "An effective hybrid framework for content based image retrieval (CBIR)," Multimed. Tools Appl., vol. 80, no. 17, pp. 26911–26937, 2021.

M. Garg and G. Dhiman, "A novel content-based image retrieval approach for classification using GLCM features and texture fused LBP variants," Neural Comput. Appl., vol. 33, no. 4, pp. 1311–1328, 2021.

D. Niu, X. Zhao, X. Lin, and C. Zhang, "A novel image retrieval method based on multi-features fusion," Signal Process. Image Commun., vol. 87, p. 115911, 2020, doi: 10.1016/j.image.2020.115911.

S. Chavda and M. Goyani, "2020 Hybrid Approach to Content‑Based Image Retrieval Using Modifed multi scale LBP and color features.pdf," SN Comput. Sci. A Springer Nat. J., vol. 2020, no. 1, p. 305, 2020, doi: https://doi.org/10.1007/s42979-020-00321-w.

N. Kayhan and S. Fekri-Ershad, "Content based image retrieval based on weighted fusion of texture and color features derived from modified local binary patterns and local neighborhood difference patterns," Multimed. Tools Appl., vol. 80, no. 21, pp. 32763–32790, 2021.

K. Chu and G. H. Liu, "Image Retrieval Based on a Multi-Integration Features Model," Math. Probl. Eng., vol. 2020, 2020, doi: 10.1155/2020/1461459.

S. Kugunavar and C. J. Prabhakar, "Content-based medical image retrieval using delaunay triangulation segmentation technique," J. Inf. Technol. Res., vol. 14, no. 2, pp. 48–66, 2021.

X. Zenggang, T. Zhiwen, and C. Xiaowen, "Research on Image Retrieval Algorithm Based on Combination of Color and Shape Features," J. Signal Process. Syst., vol. 93, no. 2, pp. 139-146., 2019.

D. Srivastava, B. Rajitha, S. Agarwal, and S. Singh, "Pattern-based image retrieval using GLCM," Neural Comput. Appl., vol. 32, no. 15, pp. 10819–10832, 2020.

N. Varish, A modified similarity measurement for image retrieval scheme using fusion of color, texture and shape moments, vol. 81, no. 15. Multimedia Tools and Applications, 2022.

J. Pradhan, A. K. Pal, H. Banka, and P. Dansena, "Fusion of region based extracted features for instance-and class-based CBIR applications," Appl. Soft Comput., vol. 102, p. 107063, 2021.

K. T. Ahmed, S. Ummesafi, and A. Iqbal, "Content based image retrieval using image features information fusion," Inf. Fusion, vol. 51, pp. 76–99, 2019.

E. M. Martey, H. Lei, X. Li, and O. Appiah, "Effective Image Representation using Double Colour Histogram for Content-Based Image Retrieval," Inform., vol. 45, no. 7, pp. 97–105, 2021, doi: 10.31449/inf.v45i7.3715.

S. Fadaei, "New Dominant Color Descriptor Features Based on Weighting of More Informative Pixels using Suitable Masks for Content-Based Image Retrieval," Int. J. Eng. Trans. B Appl., vol. 35, no. 8, pp. 1457–1467, 2022, doi: 10.5829/IJE.2022.35.08B.01.

H. Wang, Z. Li, Y. Li, B. Gupta, and C. Choi, "Visual saliency guided complex image retrieval," Pattern Recognit. Lett. Elsevier, vol. 130, pp. 64–72, 2020.

G. U. Nneji, J. Cai, J. Deng, H. N. Monday, E. C. James, and C. C. Ukwuoma, "Multi-Channel Based Image Processing Scheme for Pneumonia Identification," Diagnostics, vol. 12, no. 2, pp. 1–26, 2022, doi: 10.3390/diagnostics12020325.

M. N. Vharkate and V. B. Musande, "Fusion Based Feature Extraction and Optimal Feature Selection in Remote Sensing Image Retrieval," Multimed. Tools Appl., vol. 81, no. 22, pp. 31787–31814, 2022, doi: 10.1007/s11042-022-11997-y.

S. Jardim, J. António, C. Mora, and A. Almeida, "A Novel Trademark Image Retrieval System Based on Multi-Feature Extraction and Deep Networks," J. Imaging, vol. 8, no. 9, 2022, doi: 10.3390/jimaging8090238.

C. Oyarzun Laura, S. Wesarg, and G. Sakas, "Graph matching survey for medical imaging: On the way to deep learning," Methods, no. July 2021, 2021, doi: 10.1016/j.ymeth.2021.06.008.

K. S. Camilus and V. K. Govindan, "A Review on Graph Based Segmentation," Int. J. Image, Graph. Signal Process., vol. 4, no. June, pp. 1–13, 2012, doi: 10.5815/ijigsp.2012.05.01.

P. F. Felzenszwalb and D. P. Huttenlocher, "Efficient graph-based image segmentation," Int. J. Comput. Vis., vol. 59, no. 2, pp. 167–181, 2004, doi: 10.1023/B:VISI.0000022288.19776.77.

E. Rica, S. Ãlvarez, and F. Serratosa, "On-line learning the graph edit distance costs," Pattern Recognit. Lett., vol. 146, pp. 55–62, 2021, doi: 10.1016/j.patrec.2021.02.019.

F. Serratosa, "A general model to define the substitution, insertion and deletion graph edit costs based on an embedded space," Pattern Recognit. Lett., vol. 138, pp. 115–122, 2020, doi: 10.1016/j.patrec.2020.07.010.

R. M. Bommisetty, O. Prakash, and A. Khare, Keyframe extraction using Pearson correlation coefficient and color moments, vol. 26, no. 3. Springer Berlin Heidelberg, 2020.

G. Xie, B. Guo, Z. Huang, Y. Zheng, and Y. Yan, "Combination of Dominant Color Descriptor and Hu Moments in Consistent Zone for Content Based Image Retrieval," IEEE Access, vol. 8, pp. 146284–146299, 2020, doi: 10.1109/ACCESS.2020.3015285.

Z. Huang and J. Leng, "Analysis of Hu's moment invariants on image scaling and rotation," ICCET 2010 - 2010 Int. Conf. Comput. Eng. Technol. Proc., vol. 7, no. May 2010, 2010, doi: 10.1109/ICCET.2010.5485542.

A. Humeau-Heurtier, "Texture feature extraction methods: A survey," IEEE Access, vol. 7, no. c, pp. 8975–9000, 2019, doi: 10.1109/ACCESS.2018.2890743.

A. Eleyan and H. Demirel, "Co-occurrence matrix and its statistical features as a new approach for face Recognition," Turkish J. Electr. Eng. Comput. Sci., vol. 19, no. 1, 2011, doi: 10.3906/elk-0906-27.

Haralick, R.M and K. Shanmugam, "Texture features for image classification," EEETrans. Syst. Man Cybern. SMC, vol. 3, no. 6, pp. 610–621, 1973.

M. Huang, H. Shu, Y. Ma, and Q. Gong, "Content-based image retrieval technology using multi-feature fusion," Opt. - Int. J. Light Electron Opt., vol. 126, no. 19, pp. 2144–2148, 2015, doi: 10.1016/j.ijleo.2015.05.095.




DOI: http://dx.doi.org/10.18517/ijaseit.13.3.17521

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development