Open Access Peer-reviewed Research Article

Exploring the impact of using Chat-GPT on student learning outcomes in technology learning: The comprehensive experiment

Article Sidebar

Frequency Distribution
Download PDF
Submitted   Aug 6, 2023
Published   Oct 18, 2023
Issue    Vol 3 No 2 (2023)
DOI   10.25082/AMLER.2023.02.013

Views

4871

Downloads

1869

Citations

PlumX Metrics

Main Article Content

Muhammad Hakiki corresponding author
Universitas Muhammadiyah Muara Bungo, Bungo, Indonesia
Radinal Fadli
Universitas Muhammadiyah Muara Bungo, Bungo, Indonesia
Agariadne Dwinggo Samala
Universitas Negeri Padang, Padang, Indonesia
Ade Fricticarani
Universitas Bina Bangsa Banten, Serang, Indonesia
Popi Dayurni
Universitas Bina Bangsa Banten, Serang, Indonesia
Kurniati Rahmadani
Universitas Bina Bangsa Banten, Serang, Indonesia
Ayu Dewi Astiti
SD Negeri Sindang III, Majalengka, Indonesia
Arisman Sabir
Universitas Muhammadiyah Muara Bungo, Bungo, Indonesia

Abstract

This experimental study investigates the impact of integrating Chat GPT (Generative Pre-trained Transformer) on student learning outcomes in technology education at Universitas Muhammadiyah Muara Bungo. The research involves an experimental group using Chat GPT and a control group with conventional methods. Data from 31 participants in each group were collected, assessing learning outcomes through final test scores. Analyzing the results with a t-test, the experimental group displayed significantly higher achievements than the control group, highlighting the positive effect of incorporating GPT Chat into educational technology. The study illuminates the potential of AI-powered chatbots like Chat GPT to enhance student learning outcomes. Further exploration is required to gauge its adaptability across diverse educational contexts for more enhanced learning results. T-test results, conducted at a 95% confidence level with α 0.05, and degrees of freedom dk = n1 + n2 - 2 = 60, showed tcount of 5.424 against ttable of 2.000, firmly establishing tcount > ttable (5.424 > 2.000). Consequently, the null hypothesis (H0) proposing no significant impact of Chat GPT utilization is refuted. Conversely, the alternative hypothesis (H1), signifying a significant influence from Chat GPT usage, is upheld, affirming its substantial role in students' technological education.

Keywords
Chat-GPT, artificial intelligence, learning outcomes, technology learning

Article Details

How to Cite
Hakiki, M., Fadli, R., Samala, A. D., Fricticarani, A., Dayurni, P., Rahmadani, K., Astiti, A. D., & Sabir, A. (2023). Exploring the impact of using Chat-GPT on student learning outcomes in technology learning: The comprehensive experiment. Advances in Mobile Learning Educational Research, 3(2), 859-872. https://doi.org/10.25082/AMLER.2023.02.013
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

References

  1. Ali, M. A., Sahari Ashaari, N., Mat Noor, S. F., & Zainudin, S. (2022). Identifying Students’ Learning Patterns in Online Learning Environments: A Literature Review. International Journal of Emerging Technologies in Learning (IJET), 17(08), 189–205. https://doi.org/10.3991/ijet.v17i08.29811
  2. Athanassopoulos, S., Manoli, P., Gouvi, M., Lavidas, K., & Komis, V. (2023). The use of ChatGPT as a learning tool to improve foreign language writing in a multilingual and multicultural classroom. Advances in Mobile Learning Educational Research, 3(2), 818–824. https://doi.org/10.25082/amler.2023.02.009
  3. Baharom, M. M., Atan, N. A., Rosli, M. S., Yusof, S., & Abd Hamid, M. Z. (2020). Integration of Science learning Apps based on Inquiry Based Science Education (IBSE) in enhancing Students Science Process Skills (SPS). International Journal of Interactive Mobile Technologies (IJIM), 14(09), 95. https://doi.org/10.3991/ijim.v14i09.11706
  4. Castillo, A. G. R., Silva, G. J. S., Arocutipa, J. P. F., Berrios, H. Q., Rodriguez, M. A. M., Reyes, G. Y., ... & Arias-Gonzáles, J. L. (2023). Effect of Chat GPT on the digitized learning process of university students. Journal of Namibian Studies: History Politics Culture, 33, 1-15. https://doi.org/10.59670/jns.v33i.411
  5. Chiesa-Estomba, C. M., Lechien, J. R., Vaira, L. A., Brunet, A., Cammaroto, G., Mayo-Yanez, M., Sanchez-Barrueco, A., & Saga-Gutierrez, C. (2023). Exploring the potential of Chat-GPT as a supportive tool for sialendoscopy clinical decision making and patient information support. European Archives of Oto-Rhino-Laryngology. https://doi.org/10.1007/s00405-023-08104-8
  6. Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: the state of the field. International Journal of Educational Technology in Higher Education, 20(1). https://doi.org/10.1186/s41239-023-00392-8
  7. Dayawansa, S., Mantziaris, G., & Sheehan, J. (2023). Chat GPT versus human touch in stereotactic radiosurgery. Journal of Neuro-Oncology, 163(2), 481–483. https://doi.org/10.1007/s11060-023-04353-z
  8. Elzayady, H., S. Mohamed, M., M. Badran, K., & I. Salama, G. (2022). Detecting Arabic textual threats in social media using artificial intelligence: An overview. Indonesian Journal of Electrical Engineering and Computer Science, 25(3), 1712. https://doi.org/10.11591/ijeecs.v25.i3.pp1712-1722
  9. Esplugas, M. (2023). The use of artificial intelligence (AI) to enhance academic communication, education and research: a balanced approach. Journal of Hand Surgery (European Volume), 48(8), 819–822. https://doi.org/10.1177/17531934231185746
  10. Fauzan, A., Triyono, M. B., Hardiyanta, R. A. P., Daryono, R. W., & Arifah, S. (2023). The Effect of Internship and Work Motivation on Students’ Work Readiness in Vocational Education: PLS-SEM Approach. Journal of Innovation in Educational and Cultural Research, 4(1), 26–34. https://doi.org/10.46843/jiecr.v4i1.413
  11. Hakiki, M., Surjono, H. D., Wagiran, Fadli, R., Budiman, R. D. A., Ramadhani, W., Habibie, Z. R., Suhardiman, S., & Hidayah, Y. (2023). Enhancing Practicality of Web-Based Mobile Learning in Operating System Course: A Developmental Study. International Journal of Interactive Mobile Technologies (IJIM), 17(19), 4–19. https://doi.org/10.3991/ijim.v17i19.42389
  12. Hassan, A. N., Al-Chlaihawi, S., & Khekan, A. R. (2021). Artificial intelligence techniques over the fifth generation mobile networks: a review. Indonesian Journal of Electrical Engineering and Computer Science, 24(1), 317. https://doi.org/10.11591/ijeecs.v24.i1.pp317-328
  13. İpek, Z. H., Gözüm, A. İ. C., Papadakis, S., & Kallogiannakis, M. (2023). Educational Applications of the ChatGPT AI System: A Systematic Review Research. Educational Process International Journal, 12(3). https://doi.org/10.22521/edupij.2023.123.2
  14. Judd, B. C., & Graves, C. A. (2012). Cellular STEM: Promoting Interest in Science, Technology, Engineering, and Math Education Using Cellular Messaging, Cloud Computing, and Web-Based Social Networks. 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (Ccgrid 2012). https://doi.org/10.1109/ccgrid.2012.111
  15. Kaiss, W., Mansouri, K., & Poirier, F. (2023). Effectiveness of an Adaptive Learning Chatbot on Students’ Learning Outcomes Based on Learning Styles. International Journal of Emerging Technologies in Learning (IJET), 18(13), 250–261. https://doi.org/10.3991/ijet.v18i13.39329
  16. Karakose, T., Demirkol, M., Aslan, N., Köse, H., & Yirci, R. (2023). A Conversation with ChatGPT about the Impact of the COVID-19 Pandemic on Education: Comparative Review Based on Human–AI Collaboration. Educational Process International Journal, 12(3). https://doi.org/10.22521/edupij.2023.123.1
  17. Karakose, T., Tülübaş, T., & Papadakis, S. (2023). The scientific evolution of social justice leadership in education: structural and longitudinal analysis of the existing knowledge base, 2003–2022. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1139648
  18. Kleebayoon, A., Mungmunpuntipanitp, R., & Wiwanitkit, V. (2023). Chat GPT in stereotactic radiosurgery: correspondence. Journal of Neuro-Oncology, 163(3), 727–728. https://doi.org/10.1007/s11060-023-04375-7
  19. Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: how may AI and GPT impact academia and libraries? Library Hi Tech News, 40(3), 26–29. https://doi.org/10.1108/lhtn-01-2023-0009
  20. Mahfud, T., Triyono, M. B., Sudira, P., & Mulyani, Y. (2020). The influence of social capital and entrepreneurial attitude orientation on entrepreneurial intentions: the mediating role of psychological capital. European Research on Management and Business Economics, 26(1), 33–39. https://doi.org/10.1016/j.iedeen.2019.12.005
  21. Omar, S. F., Nawi, H. S. A., Mee, R. W. M., Pek, L. S., Shahdan, T. S. T., & Woo, A. (2021). Readiness in using online interactive platforms for remote teaching. Indonesian Journal of Electrical Engineering and Computer Science, 24(2), 1047. https://doi.org/10.11591/ijeecs.v24.i2.pp1047-1053
  22. Papadakis, S. (2023). MOOCs 2012-2022: An overview. Advances in Mobile Learning Educational Research, 3(1), 682–693. https://doi.org/10.25082/amler.2023.01.017
  23. Papadakis, S., Kiv, A. E., Kravtsov, H. M., Osadchyi, V. V., Marienko, M. V., Pinchuk, O. P., ... & Semerikov, S. O. (2023). Revolutionizing education: using computer simulation and cloud-based smart technology to facilitate successful open learning. In CEUR Workshop Proceedings (Vol. 3358, pp. 1-18). https://doi.org/10.31812/123456789/7375
  24. Park, K., Li, H., & Luo, N. (2021). Key Issues on Informal Learning in the 21st Century: A Text Mining-based Literature Review. International Journal of Emerging Technologies in Learning (IJET), 16(17), 4. https://doi.org/10.3991/ijet.v16i17.23663
  25. Porter, E., Murphy, M., & O’Connor, C. (2023). Chat GPT in dermatology: Progressive or problematic? Journal of the European Academy of Dermatology and Venereology, 37(7). Portico. https://doi.org/10.1111/jdv.19174
  26. Putra, K. A. J., Triyono, M. B., & Daryono, R. W. (2022). The Influence of Entrepreneurship Competency and Leadership Challenge to Principals' Leadership Solutions. Jurnal Pendidikan Dan Pengajaran, 55(2), 385-397. https://doi.org/10.23887/jpp.v55i2.43711
  27. Rudolph, J., Shannon Tan, & Samson Tan. (2023). War of the Chatbots: Bard, Bing Chat, ChatGPT, Ernie and Beyond. The New AI Gold Rush and Its Impact on Higher Education. Journal of Applied Learning and Teaching, 6(1): 364–389. https://doi.org/10.37074/jalt.2023.6.1.23
  28. Sefriani, R., Sepriana, R., Radyuli, P., & Hakiki, M. (2022). Android-Based Blended Learning Media for Computer Maintenance Lectures. Journal of Education Technology, 6(1), 119–125. https://doi.org/10.23887/jet.v6i1.42514
  29. Soboleva, E. V. (2019). Quest in a Digital School: The Potential and Peculiarities of Mobile Technology Implementation. European Journal of Contemporary Education, 8(3), 613-626. https://doi.org/10.13187/ejced.2019.3.613
  30. Spreafico, C., & Sutrisno, A. (2023). Artificial Intelligence Assisted Social Failure Mode and Effect Analysis (FMEA) for Sustainable Product Design. Sustainability, 15(11), 8678. https://doi.org/10.3390/su15118678
  31. Sriadhi, S., Hamid, A., Sitompul, H., & Restu, R. (2022). Effectiveness of Augmented Reality-Based Learning Media for Engineering-Physics Teaching. International Journal of Emerging Technologies in Learning (IJET), 17(05), 281–293. https://doi.org/10.3991/ijet.v17i05.28613
  32. Sutrisni, D. M., Utaminingsih, S., Murtono, M., Mariam, I. O., & Pratama, H. (2022). The effectiveness of android-based budiran game assisted by smart apps creator 3 to improve science learning outcomes of fourth graders in theme 1. Advances in Mobile Learning Educational Research, 2(2), 483–492. https://doi.org/10.25082/amler.2022.02.017
  33. Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1). https://doi.org/10.1186/s40561-023-00237-x
  34. Tsai, M. L., Ong, C. W., & Chen, C.-L. (2023). Exploring the use of large language models (LLMs) in chemical engineering education: Building core course problem models with Chat-GPT. Education for Chemical Engineers, 44, 71–95. https://doi.org/10.1016/j.ece.2023.05.001
  35. Tülübaş, T., Karakose, T., & Papadakis, S. (2023). A Holistic Investigation of the Relationship between Digital Addiction and Academic Achievement among Students. European Journal of Investigation in Health, Psychology and Education, 13(10), 2006–2034. https://doi.org/10.3390/ejihpe13100143
  36. Viktor, M., Anna, K., & Olga, M. (2021). Development of a model for evaluating the effectiveness of innovative startups based on information cycles and using neural networks. Indonesian Journal of Electrical Engineering and Computer Science, 23(1), 396. https://doi.org/10.11591/ijeecs.v23.i1.pp396-404
  37. Wang, N., Rebolledo-Mendez, G., Matsuda, N., Santos, O. C., & Dimitrova, V. (2023). Artificial Intelligence in Education. 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023. url{https://doi.org/10.1007/978-3-031-36272-9
  38. Widodo, P., Subandowo, M., Musyarofah, L., Slamet, J., & Ming, D. (2023). Interactive gamification-flip-book for developing students’ outcomes. Advances in Mobile Learning Educational Research, 3(2), 754–762. https://doi.org/10.25082/amler.2023.02.002