Open Access Peer-reviewed Research Article

Exploring the impact of using Chat-GPT on student learning outcomes in technology learning: The comprehensive experiment

Main Article Content

Muhammad Hakiki corresponding author
Radinal Fadli
Agariadne Dwinggo Samala
Ade Fricticarani
Popi Dayurni
Kurniati Rahmadani
Ayu Dewi Astiti
Arisman Sabir


This experimental study investigates the impact of integrating Chat GPT (Generative Pre-trained Transformer) on student learning outcomes in technology education at Universitas Muhammadiyah Muara Bungo. The research involves an experimental group using Chat GPT and a control group with conventional methods. Data from 31 participants in each group were collected, assessing learning outcomes through final test scores. Analyzing the results with a t-test, the experimental group displayed significantly higher achievements than the control group, highlighting the positive effect of incorporating GPT Chat into educational technology. The study illuminates the potential of AI-powered chatbots like Chat GPT to enhance student learning outcomes. Further exploration is required to gauge its adaptability across diverse educational contexts for more enhanced learning results. T-test results, conducted at a 95% confidence level with α 0.05, and degrees of freedom dk = n1 + n2 - 2 = 60, showed tcount of 5.424 against ttable of 2.000, firmly establishing tcount > ttable (5.424 > 2.000). Consequently, the null hypothesis (H0) proposing no significant impact of Chat GPT utilization is refuted. Conversely, the alternative hypothesis (H1), signifying a significant influence from Chat GPT usage, is upheld, affirming its substantial role in students' technological education.

Chat-GPT, artificial intelligence, learning outcomes, technology learning

Article Details

How to Cite
Hakiki, M., Fadli, R., Samala, A. D., Fricticarani, A., Dayurni, P., Rahmadani, K., Astiti, A. D., & Sabir, A. (2023). Exploring the impact of using Chat-GPT on student learning outcomes in technology learning: The comprehensive experiment. Advances in Mobile Learning Educational Research, 3(2), 859-872.


  1. Ali, M. A., Sahari Ashaari, N., Mat Noor, S. F., & Zainudin, S. (2022). Identifying Students’ Learning Patterns in Online Learning Environments: A Literature Review. International Journal of Emerging Technologies in Learning (IJET), 17(08), 189–205.
  2. Athanassopoulos, S., Manoli, P., Gouvi, M., Lavidas, K., & Komis, V. (2023). The use of ChatGPT as a learning tool to improve foreign language writing in a multilingual and multicultural classroom. Advances in Mobile Learning Educational Research, 3(2), 818–824.
  3. Baharom, M. M., Atan, N. A., Rosli, M. S., Yusof, S., & Abd Hamid, M. Z. (2020). Integration of Science learning Apps based on Inquiry Based Science Education (IBSE) in enhancing Students Science Process Skills (SPS). International Journal of Interactive Mobile Technologies (IJIM), 14(09), 95.
  4. Castillo, A. G. R., Silva, G. J. S., Arocutipa, J. P. F., Berrios, H. Q., Rodriguez, M. A. M., Reyes, G. Y., ... & Arias-Gonzáles, J. L. (2023). Effect of Chat GPT on the digitized learning process of university students. Journal of Namibian Studies: History Politics Culture, 33, 1-15.
  5. Chiesa-Estomba, C. M., Lechien, J. R., Vaira, L. A., Brunet, A., Cammaroto, G., Mayo-Yanez, M., Sanchez-Barrueco, A., & Saga-Gutierrez, C. (2023). Exploring the potential of Chat-GPT as a supportive tool for sialendoscopy clinical decision making and patient information support. European Archives of Oto-Rhino-Laryngology.
  6. Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: the state of the field. International Journal of Educational Technology in Higher Education, 20(1).
  7. Dayawansa, S., Mantziaris, G., & Sheehan, J. (2023). Chat GPT versus human touch in stereotactic radiosurgery. Journal of Neuro-Oncology, 163(2), 481–483.
  8. Elzayady, H., S. Mohamed, M., M. Badran, K., & I. Salama, G. (2022). Detecting Arabic textual threats in social media using artificial intelligence: An overview. Indonesian Journal of Electrical Engineering and Computer Science, 25(3), 1712.
  9. Esplugas, M. (2023). The use of artificial intelligence (AI) to enhance academic communication, education and research: a balanced approach. Journal of Hand Surgery (European Volume), 48(8), 819–822.
  10. Fauzan, A., Triyono, M. B., Hardiyanta, R. A. P., Daryono, R. W., & Arifah, S. (2023). The Effect of Internship and Work Motivation on Students’ Work Readiness in Vocational Education: PLS-SEM Approach. Journal of Innovation in Educational and Cultural Research, 4(1), 26–34.
  11. Hakiki, M., Surjono, H. D., Wagiran, Fadli, R., Budiman, R. D. A., Ramadhani, W., Habibie, Z. R., Suhardiman, S., & Hidayah, Y. (2023). Enhancing Practicality of Web-Based Mobile Learning in Operating System Course: A Developmental Study. International Journal of Interactive Mobile Technologies (IJIM), 17(19), 4–19.
  12. Hassan, A. N., Al-Chlaihawi, S., & Khekan, A. R. (2021). Artificial intelligence techniques over the fifth generation mobile networks: a review. Indonesian Journal of Electrical Engineering and Computer Science, 24(1), 317.
  13. İpek, Z. H., Gözüm, A. İ. C., Papadakis, S., & Kallogiannakis, M. (2023). Educational Applications of the ChatGPT AI System: A Systematic Review Research. Educational Process International Journal, 12(3).
  14. Judd, B. C., & Graves, C. A. (2012). Cellular STEM: Promoting Interest in Science, Technology, Engineering, and Math Education Using Cellular Messaging, Cloud Computing, and Web-Based Social Networks. 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (Ccgrid 2012).
  15. Kaiss, W., Mansouri, K., & Poirier, F. (2023). Effectiveness of an Adaptive Learning Chatbot on Students’ Learning Outcomes Based on Learning Styles. International Journal of Emerging Technologies in Learning (IJET), 18(13), 250–261.
  16. Karakose, T., Demirkol, M., Aslan, N., Köse, H., & Yirci, R. (2023). A Conversation with ChatGPT about the Impact of the COVID-19 Pandemic on Education: Comparative Review Based on Human–AI Collaboration. Educational Process International Journal, 12(3).
  17. Karakose, T., Tülübaş, T., & Papadakis, S. (2023). The scientific evolution of social justice leadership in education: structural and longitudinal analysis of the existing knowledge base, 2003–2022. Frontiers in Education, 8.
  18. Kleebayoon, A., Mungmunpuntipanitp, R., & Wiwanitkit, V. (2023). Chat GPT in stereotactic radiosurgery: correspondence. Journal of Neuro-Oncology, 163(3), 727–728.
  19. Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: how may AI and GPT impact academia and libraries? Library Hi Tech News, 40(3), 26–29.
  20. Mahfud, T., Triyono, M. B., Sudira, P., & Mulyani, Y. (2020). The influence of social capital and entrepreneurial attitude orientation on entrepreneurial intentions: the mediating role of psychological capital. European Research on Management and Business Economics, 26(1), 33–39.
  21. Omar, S. F., Nawi, H. S. A., Mee, R. W. M., Pek, L. S., Shahdan, T. S. T., & Woo, A. (2021). Readiness in using online interactive platforms for remote teaching. Indonesian Journal of Electrical Engineering and Computer Science, 24(2), 1047.
  22. Papadakis, S. (2023). MOOCs 2012-2022: An overview. Advances in Mobile Learning Educational Research, 3(1), 682–693.
  23. Papadakis, S., Kiv, A. E., Kravtsov, H. M., Osadchyi, V. V., Marienko, M. V., Pinchuk, O. P., ... & Semerikov, S. O. (2023). Revolutionizing education: using computer simulation and cloud-based smart technology to facilitate successful open learning. In CEUR Workshop Proceedings (Vol. 3358, pp. 1-18).
  24. Park, K., Li, H., & Luo, N. (2021). Key Issues on Informal Learning in the 21st Century: A Text Mining-based Literature Review. International Journal of Emerging Technologies in Learning (IJET), 16(17), 4.
  25. Porter, E., Murphy, M., & O’Connor, C. (2023). Chat GPT in dermatology: Progressive or problematic? Journal of the European Academy of Dermatology and Venereology, 37(7). Portico.
  26. Putra, K. A. J., Triyono, M. B., & Daryono, R. W. (2022). The Influence of Entrepreneurship Competency and Leadership Challenge to Principals' Leadership Solutions. Jurnal Pendidikan Dan Pengajaran, 55(2), 385-397.
  27. Rudolph, J., Shannon Tan, & Samson Tan. (2023). War of the Chatbots: Bard, Bing Chat, ChatGPT, Ernie and Beyond. The New AI Gold Rush and Its Impact on Higher Education. Journal of Applied Learning and Teaching, 6(1): 364–389.
  28. Sefriani, R., Sepriana, R., Radyuli, P., & Hakiki, M. (2022). Android-Based Blended Learning Media for Computer Maintenance Lectures. Journal of Education Technology, 6(1), 119–125.
  29. Soboleva, E. V. (2019). Quest in a Digital School: The Potential and Peculiarities of Mobile Technology Implementation. European Journal of Contemporary Education, 8(3), 613-626.
  30. Spreafico, C., & Sutrisno, A. (2023). Artificial Intelligence Assisted Social Failure Mode and Effect Analysis (FMEA) for Sustainable Product Design. Sustainability, 15(11), 8678.
  31. Sriadhi, S., Hamid, A., Sitompul, H., & Restu, R. (2022). Effectiveness of Augmented Reality-Based Learning Media for Engineering-Physics Teaching. International Journal of Emerging Technologies in Learning (IJET), 17(05), 281–293.
  32. Sutrisni, D. M., Utaminingsih, S., Murtono, M., Mariam, I. O., & Pratama, H. (2022). The effectiveness of android-based budiran game assisted by smart apps creator 3 to improve science learning outcomes of fourth graders in theme 1. Advances in Mobile Learning Educational Research, 2(2), 483–492.
  33. Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1).
  34. Tsai, M. L., Ong, C. W., & Chen, C.-L. (2023). Exploring the use of large language models (LLMs) in chemical engineering education: Building core course problem models with Chat-GPT. Education for Chemical Engineers, 44, 71–95.
  35. Tülübaş, T., Karakose, T., & Papadakis, S. (2023). A Holistic Investigation of the Relationship between Digital Addiction and Academic Achievement among Students. European Journal of Investigation in Health, Psychology and Education, 13(10), 2006–2034.
  36. Viktor, M., Anna, K., & Olga, M. (2021). Development of a model for evaluating the effectiveness of innovative startups based on information cycles and using neural networks. Indonesian Journal of Electrical Engineering and Computer Science, 23(1), 396.
  37. Wang, N., Rebolledo-Mendez, G., Matsuda, N., Santos, O. C., & Dimitrova, V. (2023). Artificial Intelligence in Education. 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023. url{
  38. Widodo, P., Subandowo, M., Musyarofah, L., Slamet, J., & Ming, D. (2023). Interactive gamification-flip-book for developing students’ outcomes. Advances in Mobile Learning Educational Research, 3(2), 754–762.