Developing an Artificial Neural Network Algorithm for Generalized Singular Value Decomposition-based Linear Discriminant Analysis

Rolysent K Paredes, Ariel M Sison, Ruji P Medina


Artificial Neural Networks (ANN) form a dynamic architecture for machine learning and have attained significant capabilities in various fields. It is a combination of interrelated calculation elements and derives outputs for new inputs after being trained. This study introduced a new mechanism utilizing ANN which was trained using Bayesian Regularization Back Propagation (BRBP) to improve the computational cost problem of the existing algorithm of the Generalized Singular Value Decomposition-based Linear Discriminant Analysis (LDA/GSVD). The proposed approach can minimize the number of iterations and mathematical processes of the existing LDA/GSVD algorithm which suffers time complexity. Through simulation using BLE RSSI Dataset from UCI which has 105 classes and 13 dimensions with 1420 instances, it was found out that ANN improved the computational cost during the classification of the data up to 57.14% while maintaining its accuracy. This new technique is recommended when classifying big data, and for pattern analysis as well.


artificial neural network; bayesian regularization back propagation; generalized singular value decomposition; ANN for LDA/GSVD; linear discriminant analysis.

Full Text:



W. Hadi, F. Aburub, and S. Alhawari, “A new fast associative classification algorithm for detecting phishing websites,†Appl. Soft Comput., vol. 48, pp. 729–734, Nov. 2016.

S. S. Nikam, “A comparative study of classification techniques in data mining algorithms,†Oriental Journal of Computer Science and Technology, vol. 8, no. 1, pp. 13-19, 2015.

N. B. M. Zainee and K. Chellappan, “A preliminary dengue fever prediction model based on vital signs and blood profile,†2016 IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES), pp. 652-656, 2016.

P. P. Markopoulos, “Linear Discriminant Analysis with few training data,†2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4626-4630, March 2017.

X. Gao, X. Wang, X. Li, and D. Tao, “Transfer latent variable model based on divergence analysis,†Pattern Recognition, vol. 44, no. 10-11, pp. 2358–2366, 2011.

X. Gao, X. Wang, D. Tao, and X. Li, “Supervised Gaussian Process Latent Variable Model for Dimensionality Reduction,†IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 41, no. 2, pp. 425–434, 2011.

J. J. D. M. S. Junior and A. R. Backes, “Shape classification using line segment statistics,†Information Sciences, vol. 305, pp. 349–356, 2015.

J. Shao, Y. Wang, X. Deng, and S. Wang, “Sparse linear discriminant analysis by thresholding for high dimensional data,†The Annals of Statistics, vol. 39, no. 2, pp. 1241–1265, 2011.

D. Tao, J. Cheng, X. Lin, and J. Yu, “Local structure preserving discriminative projections for RGB-D sensor-based scene classification,†Information Sciences, vol. 320, pp. 383–394, 2015.

D. Wang, X. Gao, and X. Wang, “Semi-Supervised Nonnegative Matrix Factorization via Constraint Propagation,†IEEE Transactions on Cybernetics, vol. 46, no. 1, pp. 233–244, 2016.

L. Zhang, L. Wang, and W. Lin, “Generalized Biased Discriminant Analysis for Content-Based Image Retrieval,†IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 42, no. 1, pp. 282–290, 2012.

H. Zhao and P. C. Yuen, “Incremental Linear Discriminant Analysis for Face Recognition,†IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 38, no. 1, pp. 210–221, 2008.

C. L. Liu, W. H. Hsaio, C. H. Lee, and F. S. Gou, “Semi-Supervised Linear Discriminant Clustering,†IEEE Transactions on Cybernetics, vol. 44, no. 7, pp. 989–1000, 2014.

J. Zhao, L. Shi, and J. Zhu, “Two-Stage Regularized Linear Discriminant Analysis for 2-D Data,†IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 8, pp. 1669–1681, 2015.

G. Baudat and F. Anouar, “Generalized Discriminant Analysis Using a Kernel Approach,†Neural Computation, vol. 12, no. 10, pp. 2385–2404, 2000.

S. Mika, G. Ratsch, J. Weston, B. Scholkopf, and K. Mullers, “Fisher discriminant analysis with kernels,†Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop, pp. 41-48, 1999.

A. Sharma and K. K. Paliwal, “Linear discriminant analysis for the small sample size problem: an overview,†International Journal of Machine Learning and Cybernetics, vol. 6, no. 3, pp. 443–454, Jul. 2014.

S. Yu, Z. Cao, and X. Jiang, “Robust linear discriminant analysis with a Laplacian assumption on projection distribution,†2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2567-2571, 2017.

W. Deng, J. Hu, and J. Guo, “Extended SRC: Undersampled Face Recognition via Intraclass Variant Dictionary,†IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 9, pp. 1864–1870, 2012.

Z. Wang, Y.-H. Shao, L. Bai, C.-N. Li, L.-M. Liu, and N.-Y. Deng, “MBLDA: A novel multiple between-class linear discriminant analysis,†Information Sciences, vol. 369, pp. 199–220, 2016.

X. Jing, Y. Dong, and Y. Yao, “Uncorrelated optimal discriminant vectors based on generalized singular value decomposition,†International Conference on Automatic Control and Artificial Intelligence (ACAI 2012), 2012.

T. Zhang, B. Fang, Y. Y. Tang, Z. Shang, and B. Xu, “Generalized Discriminant Analysis: A Matrix Exponential Approach,†IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 40, no. 1, pp. 186–197, 2010.

D. Cai, X. He, and J. Han, “Training Linear Discriminant Analysis in Linear Time,†2008 IEEE 24th International Conference on Data Engineering, pp. 209-217, Apr. 2008.

Z. Zhang, G. Dai, C. Xu, and M. I. Jordan, “Regularized discriminant analysis, ridge regression and beyond,†Journal of Machine Learning Research, pp. 2199-2228, Aug. 11, 2010.

H. Yu and J. Yang, “A direct LDA algorithm for high-dimensional data — with application to face recognition,†Pattern Recognition, vol. 34, no. 10, pp. 2067–2070, 2001.

J. Ye and Q. Li, “A two-stage linear discriminant analysis via QR-decomposition,†IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 929–941, 2005.

J. K. P. Seng and K. L.-M. Ang, “Big Feature Data Analytics: Split and Combine Linear Discriminant Analysis (SC-LDA) for Integration Towards Decision Making Analytics,†IEEE Access, vol. 5, pp. 14056–14065, 2017.

P. Howland and H. Park, “Generalizing discriminant analysis using the generalized singular value decomposition,†IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp. 995–1006, 2004.

C. H. Park and H. Park, “A Relationship between Linear Discriminant Analysis and the Generalized Minimum Squared Error Solution,†SIAM Journal on Matrix Analysis and Applications, vol. 27, no. 2, pp. 474–492, 2005.

Z. Chen and T. H. Chan, “A truncated generalized singular value decomposition algorithm for moving force identification with ill-posed problems,†Journal of Sound and Vibration, vol. 401, pp. 297–310, 2017.

W. Wu and M. O. Ahmad, “Orthogonalized linear discriminant analysis based on modified generalized singular value decomposition,†2009 IEEE International Symposium on Circuits and Systems, pp. 1629-1632, 2009.

S. Bahrami, “Three-dimensional inverse scattering approach using analytical singular value decomposition method,†2017 18th International Radar Symposium (IRS), pp. 1-10, June 2017.

M. Berry, D. Mezher, B. Philippe, and A. Sameh, “Parallel computation of the singular value decomposition,†INRIA, 2003.

Y. Dash, and S. K. Dubey, “Quality prediction in object oriented system by using ANN: a brief survey.†International Journal of Advanced Research in Computer Science and Software Engineering, vol. 2, no. 2, 2012.

S. S. Ranhotra, A. Kumar, M. Magarini, and A. Mishra, “Performance comparison of blind and non-blind channel equalizers using artificial neural networks,†2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN), pp. 243-248, July 2017.

H. Tana, G. Yang, B. Yu, X. Liang, and Y. Tang, “Neural Network Based Algorithm for Generalized Eigenvalue Problem,†2013 International Conference on Information Science and Cloud Computing Companion, pp. 446-451, 2013.

K. Jazayeri, M. Jazayeri, and S. Uysal, “Comparative Analysis of Levenberg-Marquardt and Bayesian Regularization Backpropagation Algorithms in Photovoltaic Power Estimation Using Artificial Neural Network,†Advances in Data Mining. Applications and Theoretical Aspects Lecture Notes in Computer Science, pp. 80–95, July 2016.

F. Dalipi and S. Y. Yayilgan, “The impact of environmental factors to skiing injuries: Bayesian regularization neural network model for predicting skiing injuries,†2015 6th International Conference on Computing, Communication and Networking Technologies (ICCCNT), pp. 1-6, July 2015.

M. Mohammadi, A. Al-Fuqaha, M. Guizani, and J. S. Oh, “Semisupervised Deep Reinforcement Learning in Support of IoT and Smart City Services,†IEEE Internet of Things Journal, vol. 5, no. 2, pp. 624–635, 2018.



  • There are currently no refbacks.

Published by INSIGHT - Indonesian Society for Knowledge and Human Development