Design and Implementation of a Programming Automatic Assessment System in Jupyter Notebook

HakNeung Go, Seong-Won Kim, Youngjun Lee

Abstract


Learning programming is challenging. So, computer educators have developed various tools to help students. In this paper, we have developed a tool that combines the advantages of a Programming Automatic Assessment (PAA) system and Jupyter Notebook (JN) to support learning programming. The design direction of this system is free to use, easy to set up, and supports interactive computing. The Programming Automatic Assessment in Jupyter Notebook (PAAinJN) is available free of charge using the assessment module released on Git and the personal JN. The initialization is completed by executing in a code cell with two lines of code that downloads and executes the assessment module. In an interactive computing environment, presenting problems, writing code to be evaluated, and evaluating code can be executed in the code cells, and the problems and the results of the assessment are presented in the code cell outputs. The performance was verified by the examples presented in a high school informatics textbook using the programming automatic assessment system as teaching learning material. In addition, we propose a way to develop teaching-learning materials using PAAinJN in consideration of teachers and students and a way of distributing and collecting teaching-learning materials using the free Learning Management System. PAAinJN is expected to help students learn programming by eliminating assessment and feedback delays through PAA while learning to program in an interactive computing environment.

Keywords


Automatic assessment system; Jupyter notebook; interactive computing; Python

Full Text:

PDF

References


Lee, D., Hwang, J. Y., Lee, Y., and Kim, S. W, “Informatics and Artificial Intelligence (AI) Education in Korea: Situation Analysis Using the Darmstadt Modelâ€. JOIV: International Journal on Informatics Visualization., vol. 6(2), pp. 427-444, 2022, DOI: 10.30630/joiv.6.2.1000.

Shin, S., and Bae, Y, “A Study on the Hierarchical Instructional System Design of Software Education by School System,†Journal of The Korean Association of Information Education, vol. 19(4), pp. 533-544, 2015, DOI: 10.14352/jkaie.2015.19.4.533.

Wing, J. M, “Computational thinking,†Communications of the ACM, vol. 49(3), pp. 33-35, 2006, DOI:10.1145/1118178.1118215.

Lee, Y. J., Paik, S. H., Shin, J. H., You, H. C., Jeong, I. G., Ahn, S. J., and Jeon, S. G, “Research for introducing computational thinking into primary and secondary education,†Korea Foundation for the Advancement of Science and Creativity, BD14060010, 2014.

C. Cachero, P. Barra, S. Meliá and O. López, “Impact of Programming Exposure on the Development of Computational Thinking Capabilities: An Empirical Study,†IEEE, vol. 8, pp. 72316-72325, 2020, DOI: 10.1109/ACCESS.2020.2987254.

Kim, J. M, “Problem-solving skills based on computing thinking,†Korea Information Processing Society Review, vol. 24(2), pp. 13-21, 2017.

Durak, H. Y., Yilmaz, F. G. K., and Yilmaz, R, “Computational thinking, programming self-efficacy, problem solving and experiences in the programming process conducted with robotic activities,†Contemporary Educational Technology, vol. 10(2), pp. 173-197, 2019, DOI:10.30935/cet.554493.

Malmi, L., Utting, I., and Ko, A, “Tools and Environments,†In S. Fincher & A. Robins (Eds.), The Cambridge Handbook of Computing Education Research. Cambridge, United Kingdom, pp. 173-197, 2019, doi:10.1017/9781108654555.022.

Ramírez-Echeverry, J. J., Restrepo-Calle, F., and González, F. A., “A case study in technologyenhanced learning in an introductory computer programming course,†Global Journal of Engineering Education, vol. 24(1), pp. 65-71, 2022.

Ullah, Z., Lajis, A., Jamjoom, M., Altalhi, A., Alâ€Ghamdi, A., and Saleem, F, “The effect of automatic assessment on novice programming: Strengths and limitations of existing systems,†Computer Applications in Engineering Education, vol. 26(6), pp. 2328-2341, 2018, DOI: 10.1002/cae.21974.

Kurnia, A., Lim, A., & Cheang, B, “Online judge,†Computers & Education, vol 36(4), pp. 299-315, 2001, DOI: 10.1016/S0360-1315(01)00018-5.

Rahmani, A., and Min, J. L, “Automatic Grading System to Supporting Blended Learning in Basic Programming Practice–an Experience Report,†In International Conference on Online and Blended Learning 2019 (ICOBL 2019), pp. 96-101, Atlantis Press, DOI: 10.2991/assehr.k.200521.020.

Cardoso, A., Leitão, J. and Teixeira, C, “The Challenges of the Digital Transformation in Education,†Proceedings of the 21st International Conference on Interactive Collaborative Learning (ICL2018), vol. 2, pp. 227-236, DOI: 10.1007/978-3-030-11935-5_22.

Wang, J., Kuo, T. Y., Li, L., and Zeller, A, “Assessing and restoring reproducibility of Jupyter notebooks,†In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, pp. 138-149, 2020, DOI: 10.1145/3324884.3416585.

Ochkov, V. F., Stevens, A., and Tikhonov, A. I, "Jupyter Notebook, JupyterLab–Integrated Environment for STEM Education," In 2022 VI International Conference on Information Technologies in Engineering Education (Inforino), pp, 1-5, 2022, DOI: 10.1109/Inforino53888.2022.9782924.

Cardoso, A., Leitão, J., and Teixeira, C, "Using the Jupyter Notebook as a Tool to Support the Teaching and Learning Processes in Engineering Courses," In: Auer, M., Tsiatsos, T. (eds) The Challenges of the Digital Transformation in Education. ICL 2018. Advances in Intelligent Systems and Computing, Springer, Cham, vol 917, 2019 DOI: 10.1007/978-3-030-11935-5_22.

González-Carrillo, C. D., Restrepo-Calle, F., Ramírez-Echeverry, J. J., and González, F. A, “Automatic grading tool for jupyter notebooks in artificial intelligence courses,†Sustainability, vol. 13(21), pp. 12050, 2021, DOI: 10.3390/su132112050.

Zamfir, F. S. and Pricop, E, “On the design of an interactive automatic Python programming skills assessment system,†2022 14th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), pp. 1-5, 2022, DOI: 10.1109/ecai54874.2022.9847414.

Chang, W. Y. and Kim, S. S, “A review on trends of programming(algorithm) automated assessment system and it`s applicationâ€, The Journal of Korean Association of Computer Education, vol. 20(1), pp. 13-26, 2017.

Han, S., Liu, X., and Woo G., "Evaluation Methods and Classification of Online Judge Systems Based on Education Level and Purpose," KIISE Transactions on Computing Practices, vol.28(10), vol.487-492, 2022, DOI: 10.5626/KTCP.2022.28.10.487.

Ministry of Education, “Informatics Science Guidelinesâ€, 2015.

Chang, W. Y., and Kim, S. S, “Development and application of algorithm judging system: analysis of effects on programming learning.†The Journal of Korean Association of Computer Education, vol. 17(4), pp. 45-57, 2014.

Jeong, S. S, “The effects of programming education using an automatic programming assessment system on learning flow of general high school students†M.S. thesis, Dept. Computer Education, Graduate School of Korea National University of Education., Cheongju, Korea, 2019.

Johnson, J. W, “Benefits and pitfalls of jupyter notebooks in the classroom,†In Proceedings of the 21st Annual Conference on Information Technology Education, pp. 32-37, 2020, DOI: 10.1145/3368308.3415397.

Zhao, P., and Xia, J, “Use JupyterHub to Enhance the Teaching and Learning Efficiency of Programming Related Courses,†23rd International Conference on ISO & TQM, Florida, USA, pp.1-6, 2019.

Klever, N., “Jupyter Notebook, JupyterHub and Nbgrader. Becoming Greener—Digitalization in My Workâ€, The Publication Series of LAB University of Applied Sciences, pp. 37-43, 2020.

Blank, D. S., Bourgin, D., Brown, A., Bussonnier, M., Frederic, J., G ranger, B., and Willing, C, “nbgrader: A tool for creating and grading assignments in the Jupyter Notebookâ€, The Journal of Open Source Education, vol. 2(11), 2019, DOI: 10.21105/jose.00032.

Barba, L. A., Barker, L. J., Blank, D. S., Brown, J., Downey, A. B., George, T., ... and Zingale, M., “Teaching and learning with Jupyter,†Recuperado: https://jupyter4edu. github. io/jupyter-edu-book, 2019.

Go, H., and Lee, Y, “Exploring how to present the problem of Automatic Assessment system in Jupyter Notebook,†Proceedings of the Korean Society of Computer Information Conference, vol. 31(1), pp. 221-222, 2022.

Gyeonggi Science High School Informatics Teachers, Problems. [Online]. available: http://koistudy.net/?mid=viewProblems (accessed: October 1st, 2022).

Han. G. W, Go. G. S, Kim. E. K, Lim. S. K, Jeon. H. S, Jeong. S. S, Jeong. W.Y, Jeong. J. K, Jeong. J. W, and Jo. H. J, “[2015 Revised–- High School] Informatics Guide,†Seoul, Korea: SamYangMedia, 2019.

Gupta, A., and Pathania, P., “To study the impact of Google Classroom as a platform of learning and collaboration at the teacher education level,†Education and Information Technologies, vol. 26(1), pp. 843-857, 2021.




DOI: http://dx.doi.org/10.18517/ijaseit.13.3.18457

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development