Question Classification Based on Bloom’s Taxonomy Using Enhanced TF-IDF

Manal Mohammed, Nazlia Omar

Abstract


Bloom’s Taxonomy has been used widely in the educational environment to measure, evaluate and write high-quality exams. Therefore, many researchers have worked on the automation for classification of exam questions based on Bloom’s Taxonomy. The aim of this study is to make an enhancement for one of the most popular statistical feature, which is TF-IDF, to improve the performance of exam question classification in accordance to Bloom’s Taxonomy cognitive domain. Verbs play an important role in determining the level of a question in Bloom’s Taxonomy. Thus, the improved method assigns the impact factor for the words by taking the advantage of the part-of-speech tagger. The higher impact factor assigns to the verbs, then to the noun and adjective, after that, the lower impact factor assigns to the other part-of-speech. The dataset that has been used in this study is consist of 600 questions, divided evenly into each Bloom level. The questions first pass into the preprocessing phase in which they are prepared to be suitable for applying the proposed enhanced feature. For classification purpose, three machine learning classifiers are used Support Vector Machine, Naïve Bayes, and K-Nearest Neighbour. The enhanced feature shows satisfactory result by outperforming the classical feature TF-IDF via all classifiers in terms of weighted recall, precision, and F1-measure. On the other hand, Support Vector Machine has superior performance over other classifiers Naïve Bayes, and K-Nearest Neighbour by achieving an average of 86%, 85%, and 81.6% weighted F1-measure respectively. However, these results are promising and encouraging for further investigations.

Keywords


question classification; bloom’s taxonomy; TF-IDF; support vector machine; naïve bayes; K-Nearest Neighbour.

Full Text:

PDF

References


B. S. Bloom, Taxonomy of Educational Objectives – Handbook 1 Cognitive Domain. London: Longman, 1956.

A. Osman and A. A. Yahya, “Classification of exam questions using linguistically-motivated features: A case study based on bloom’s taxonomy,†in The Sixth International Arab Conference on Quality Assurance in Higher Education (IACQA’2016 ), 2016, vol. 467–474.

N. Omar et al., “Automated Analysis of Exam Questions According to Bloom’s Taxonomy,†Procedia - Soc. Behav. Sci., vol. 59, no. 1956, pp. 297–303, 2012.

K. Jayakodi, M. Bandara, and D. Meedeniya, “An Automatic Classifier for Exam Questions with WordNet and Cosine Similarity,†in Moratuwa Engineering Research Conference (MERCon), 2016. IEEE, 2016, pp. 12–17.

D. Kennedy, Writing and using learning outcomes: a practical guide. Cork, 2006.

D. A. Abduljabbar and N. Omar, “Exam Questions Classification Based on Bloom ’ S Taxonomy Cognitive Level Using Classifiers Combination,†J. Theor. Appl. Inf. Technol., vol. 78, no. 3, pp. 447–455, 2015.

W. C. Chang and M. S. Chung, “Automatic applying Bloom’s taxonomy to classify and analysis the cognition level of english question items,†2009 Jt. Conf. Pervasive Comput. JCPC 2009, pp. 727–733, 2009.

S. S. Haris and N. Omar, “Bloom’s taxonomy question categorization using rules and N-gram approach,†J. Theor. Appl. Inf. Technol., vol. 76, no. 3, pp. 401–407, 2015.

A. Lajis, H. M. Nasir, and N. A. Aziz, “Proposed Assessment Framework Based on Bloom Taxonomy Cognitive Competency: Introduction to Programming,†ICSCA ’18, Kuantan, Pahang, Malaysia, 2018, pp. 1–5, 2018.

A. Shaikh, A. Handore, and P. P. Kudal, “Quality Question Paper Generator on Role Based Hierarchy model,†Int. J. Res. Advent Technol., vol. 5, no. 2, pp. 6–9, 2017.

Rasim, A. Z.R. Langi, Y. Rosmansyah, and Munir, “Generation Quiz with Genetic Algorithm Based on Bloom ’ s Taxonomy Classification in Serious Game Based Virtual Environments,†2016 Int. Conf. ICT Smart Soc. Surabaya, 20-21 July 2016 ISBN 978-1-5090-1620-4, no. July, pp. 20–21, 2016.

A. A. Yahya, Z. Toukal, and A. Osman, “Bloom ’ s Taxonomy – Based Classification for Item Bank Questions Using Support Vector Machines Bloom ’ s Taxonomy – Based Classification for Item Bank,†in Modern Advances in Intelligent Systems and Tools, SCI 431, no. April 2012, Springer-Verlag Berlin Heidelberg 2012, 2012, pp. 135–140.

S. S. Haris and N. Omar, “A rule-based approach in Bloom’s Taxonomy question classification through natural language processing,†in International Conference on Computing and Convergence Technology (ICCCT),Seoul, 2012, pp. 410–414.

A. A. Yahya, A. Osman, A. Taleb, and A. A. Alattab, “Analyzing the Cognitive Level of Classroom Questions Using Machine Learning Techniques,†Procedia - Soc. Behav. Sci., vol. 97, pp. 587–595, 2013.

A. A. Yahya and A. Osama, “Automatic Classification of Questions into Bloom’s Cognitive Levels Using Support Vector Machines,†in The International Arab Conference on Information Technology, Naif Arab University for Security Science (NAUSS), 2011, pp. 1–6.

A. A. Yahya and A. Osman, “Classification of high dimensional Educational Data using Particle Swarm Classification,†2014 11th IEEE/ACS Int. Conf. Comput. Syst. Appl. AICCSA 2014, pp. 34–41, 2015.

A. A. Yahya, “Swarm intelligence-based Approach for Educational Data Classification,†J. King Saud Univ. - Comput. Inf. Sci., no. August, 2017.

A. Sangodiah, M. Muniandy, and L. E. Heng, “Question classification using statistical approach: A complete review,†J. Theor. Appl. Inf. Technol., vol. 71, no. 3, pp. 386–395, 2015.

L. Zhu, G. Wang, and X. Zou, “A Study of Chinese Document Representation and Classification with Word2vec,†in Computational Intelligence and Design (ISCID), 2016 9th International Symposium, 2016, pp. 298–302.

K. Chen, Z. Zhang, J. Long, and H. Zhang, “Turning from TF-IDF to TF-IGM for term weighting in text classification,†Expert Syst. Appl., vol. 66, pp. 1339–1351, 2016.

M. M. Altawaier and S. Tiun, “Comparison of Machine Learning Approaches on Arabic Twitter Sentiment Analysis,†Int. J. Adv. Sci. Eng. Inf. Technol., vol. 6, no. 6, p. 1067, 2016.

M. Pota, M. Esposito, and G. De Pietro, A forward-selection algorithm for SVM-based question classification in cognitive systems, vol. 6. Cham: Springer, 2016.

F. Wang, Z. Zhen, B. Wang, and Z. Mi, “Comparative Study on KNN and SVM Based Weather Classification Models for Day Ahead Short Term Solar PV Power Forecasting,†Appl. Sci., vol. 8, no. 1, p. 28, 2017.

S. N. Das, M. Mathew, and P. K. Vijayaraghavan, “An Approach for Optimal Feature Subset Selection using a New Term Weighting Scheme and Mutual Information,†Int. J. Adv. Sci. Eng. Inf. Technol., 2011.

C. D. Manning, P. Raghavan, and H. Schütze, “Text classification and Naive Bayes,†Introd. to Inf. Retr., no. c, p. 260, 2008.

H. Alshalabi, S. Tiun, N. Omar, and M. Albared, “Experiments on the Use of Feature Selection and Machine Learning Methods in Automatic Malay Text Categorization,†Procedia Technol., vol. 11, no. Iceei, pp. 748–754, 2013.




DOI: http://dx.doi.org/10.18517/ijaseit.8.4-2.6835

Refbacks

  • There are currently no refbacks.



Published by INSIGHT - Indonesian Society for Knowledge and Human Development