Questionnaires in Educational Research: Instrument Selection, Design, Validation, and Responsible Use
Main Article Content
Abstract
Questionnaires are among the most widely used data collection instruments in educational research because of their efficiency, scalability, standardisation, and capacity to generate data from large and diverse populations. Despite their widespread use, methodological guidance often focuses primarily on questionnaire construction while giving comparatively limited attention to the more fundamental question of whether questionnaires are the most appropriate instrument for addressing a particular research problem. Consequently, researchers may adopt questionnaires without adequately considering the nature of the research question, the characteristics of the construct being investigated, or the type of evidence required. This paper provides a comprehensive examination of questionnaires as data-collection instruments in educational research, addressing both the methodological question of when to use questionnaires and the practical question of how to design, validate, and implement them. Drawing on contemporary literature in educational research, survey methodology, and measurement theory, this paper examines instrument selection, construct-method alignment, self-report measurement, and the assumptions underlying questionnaire data. Particular attention is given to the conditions under which questionnaires are appropriate and the circumstances in which alternative or complementary methods may provide more valid evidence. This paper further discusses the questionnaire structure, item development, validity and reliability, ethical considerations, methodological challenges, and applications in educational research. It argues that questionnaire development should be viewed as a secondary methodological decision, following the prior determination that questionnaires are appropriate for the research objectives and constructs under investigation. The paper further emphasises that methodological rigour depends not only on questionnaire design but also on informed instrument selection, systematic validation, ethical practice, and responsible interpretation of the findings. By integrating theoretical perspectives with practical guidance, this paper provides a framework for determining when questionnaires should be used, how they should be developed, and how their findings should be interpreted in educational research.
Article Details

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
References
- Cohen L, Manion L, Morrison K. Research Methods in Education. Routledge, 2017. https://doi.org/10.4324/9781315456539
- Creswell JW, Creswell JD. Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications, 2017.
- Fraenkel JR, Wallen NE. How to design and evaluate research in education. Order Department, McGraw Hill Publishing Co., Princeton Rd., Hightstown, NJ 08520, 1990.
- Bryman A. Social research methods[M]. Oxford university press, 2016.
- Johnson RB, Christensen LB. Educational research: Quantitative, qualitative, and mixed approaches. Sage publications, 2024.
- Dillman DA, Smyth JD, Christian LM. Internet, Phone, Mail, and Mixed‐Mode Surveys. Published online August 18, 2014. https://doi.org/10.1002/9781394260645
- Evans JR, Mathur A. The value of online surveys: a look back and a look ahead. Internet Research. 2018, 28(4): 854-887. https://doi.org/10.1108/intr-03-2018-0089
- Alcott B, Bhattacharjea S, Sabates R, et al. Reflexive approaches to sampling, survey design and implementation: some practical examples from rural India. International Journal of Research & Method in Education. 2024, 48(4): 339-352. https://doi.org/10.1080/1743727x.2024.2432282
- Sharma H, Ruikar M. Crafting an effective questionnaire: An essential prerequisite of engaging surveys. Perspectives in Clinical Research. 2025, 16(3): 118-126. https://doi.org/10.4103/picr.picr_157_24
- Chand SP. Methods of Data Collection in Qualitative Research: Interviews, Focus Groups, Observations, and Document Analysis. Advances in Educational Research and Evaluation. 2025, 6(1): 303-317. https://doi.org/10.25082/aere.2025.01.001
- Saini SK. Data Collection Methods and Tools for Research; Technique for Academic and Business Research Projects. SKCON-2023, 241.
- Denzin NK. The Research Act. Routledge, 2017. https://doi.org/10.4324/9781315134543
- Leedy RJ, Ormrod JE. Practical research. Planning and design . Boston, MA: Pearson. 2018.
- Tashakkori A, Teddlie C. Mixed methodology: Combining qualitative and quantitative approaches. sage, 1998.
- Kvale S. Interviews: Learning the craft of qualitative research interviewing. Sage, 2009.
- Tourangeau R, Rips LJ, Rasinski K. The Psychology of Survey Response. Published online March 13, 2000. https://doi.org/10.1017/cbo9780511819322
- Tourangeau R, Conrad FG, Couper MP. The Science of Web Surveys. Published online May 1, 2013. https://doi.org/10.1093/acprof: oso/9780199747047.001.0001
- DeVellis RF, Thorpe CT. Scale development: Theory and applications. Sage publications, 2021.
- Repke L, Birkenmaier L, Lechner C. Validity in survey research-from research design to measurement instruments. 2024.
- Koo M, Yang S-W. Questionnaire Use and Development in Health Research. Encyclopedia. 2025, 5(2): 65. https://doi.org/10.3390/encyclopedia5020065
- Presser S, Couper MP, Lessler JT, et al. Methods for Testing and Evaluating Survey Questions. Public Opinion Quarterly. 2004, 68(1): 109-130. https://doi.org/10.1093/poq/nfh008
- Johns R. One Size Doesn’t Fit All: Selecting Response Scales For Attitude Items. Journal of Elections, Public Opinion & Parties. 2005, 15(2): 237-264. https://doi.org/10.1080/13689880500178849
- Preston CC, Colman AM. Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychologica. 2000, 104(1): 1-15. https://doi.org/10.1016/s0001-6918(99)00050-5
- Fowler FJ. Survey research methods, 5th ed. London: SAGE Publications, 2014.
- Galesic M, Bosnjak M. Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey. Public Opinion Quarterly. 2009, 73(2): 349-360. https://doi.org/10.1093/poq/nfp031
- Rolstad S, Adler J, Rydén A. Response Burden and Questionnaire Length: Is Shorter Better? A Review and Meta-analysis. Value in Health. 2011, 14(8): 1101-1108. https://doi.org/10.1016/j.jval.2011.06.003
- Baker MJ. Data Collection – Questionnaire Design. The Marketing Review. 2003, 3(3): 343-370. https://doi.org/10.1362/146934703322383507
- Forza C, Sandrin E. Surveys. Research Methods for Operations and Supply Chain Management. Published online September 26, 2023: 76-158. https://doi.org/10.4324/9781003315001-4
- Connell J, Carlton J, Grundy A, et al. The importance of content and face validity in instrument development: lessons learnt from service users when developing the Recovering Quality of Life measure (ReQoL). Quality of Life Research. 2018, 27(7): 1893-1902. https://doi.org/10.1007/s11136-018-1847-y
- Messick S. Validity, 3rd ed. American Council on Education/Macmillan, 1989: 13–103.
- Chen X, Jin J, Ke W, et al. Exploring cognitive behavioral aspects in educational psychology: A rigorous analysis of reliability and validity measures. SLAS Technology. 2024, 29(4): 100144. https://doi.org/10.1016/j.slast.2024.100144
- Clark LA, Watson D. Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment. 2019, 31(12): 1412-1427. https://doi.org/10.1037/pas0000626
- Van de Vijver F, Leung K. Methods and data analysis of comparative research. Handbook of cross-cultural psychology. 1997, 1: 257-300.
- He J, van de Vijver F. Bias and Equivalence in Cross-Cultural Research. Online Readings in Psychology and Culture. 2012, 2(2). https://doi.org/10.9707/2307-0919.1111
- Cronbach LJ. Coefficient Alpha and the Internal Structure of Tests. Psychometrika. 1951, 16(3): 297-334. https://doi.org/10.1007/bf02310555
- Yusoff N, Abd Razak A, Mohd Abbas N. Assessing the Reliability and Validity of a Survey Questionnaire for Online Laboratory Courses in Mechanical Engineering Programs. Asean Journal of Engineering Education. 2024, 8(1): 70-75. https://doi.org/10.11113/ajee2024.8n1.148
- Hossan D, Wolfs B, Petkovic M. Questionnaire validity and reliability: A review with practical guidelines. Journal of Entrepreneurship, Business and Economics. 2025, 13(1): 135-186.
- Kline P. The handbook of psychological testing, 2nd ed. London; New York: Routledge, 2000.
- Le TTT, Tran MT, Le HBH. Factors affecting students’ decision to choose regional public universities: An empirical study from Vietnam. The Journal of Asian Finance, Economics and Business (JAFEB). 2022, 9(4): 143-152.
- Goretzko D, Siemund K, Sterner P. Evaluating Model Fit of Measurement Models in Confirmatory Factor Analysis. Educational and Psychological Measurement. 2023, 84(1): 123-144. https://doi.org/10.1177/00131644231163813
- Seyedhassan S, Mahsa M, Jeyran O, et al. Scan of the postgraduate educational environment domains questionnaire: a reliable and valid tool for the evaluation of educational environment in postgraduate medical education. BMC Medical Education. 2024, 24(1). https://doi.org/10.1186/s12909-024-06125-3
- Boateng GO, Neilands TB, Frongillo EA, et al. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Frontiers in Public Health. 2018, 6. https://doi.org/10.3389/fpubh.2018.00149
- Embretson SE, Reise SP. Item response theory for psychologists. Mahwah, N.J.: L. Erlbaum Associates, 2000.
- Zumbo BD. Three Generations of DIF Analyses: Considering Where It Has Been, Where It Is Now, and Where It Is Going. Language Assessment Quarterly. 2007, 4(2): 223-233. https://doi.org/10.1080/15434300701375832
- Regmi PR, Waithaka E, Paudyal A, et al. Guide to the design and application of online questionnaire surveys. Nepal Journal of Epidemiology. 2017, 6(4): 640-644. https://doi.org/10.3126/nje.v6i4.17258
- Schwarz N. Self-reports: How the questions shape the answers. American Psychologist. 1999, 54(2): 93-105. https://doi.org/10.1037/0003-066x.54.2.93
- Kemmelmeier M. Cultural differences in survey responding: Issues and insights in the study of response biases. International Journal of Psychology. 2016, 51(6): 439-444. https://doi.org/10.1002/ijop.12386
- Bastos JL, Duquia RP, González-Chica DA, et al. Field work I: selecting the instrument for data collection. Anais Brasileiros de Dermatologia. 2014, 89(6): 918-923. https://doi.org/10.1590/abd1806-4841.20143884
- Fredricks JA, Blumenfeld PC, Paris AH. School Engagement: Potential of the Concept, State of the Evidence. Review of Educational Research. 2004, 74(1): 59-109. https://doi.org/10.3102/00346543074001059
- Ornstein AC, Hunkins FP. Curriculum: Foundations, principles and issues, 7th ed. Harlow Etc.: Pearson, 2018.
- Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health. 2007, 30(4): 459-467. https://doi.org/10.1002/nur.20199
- Lynn MR. Determination and Quantification Of Content Validity. Nursing Research. 1986, 35(6): 382???386. https://doi.org/10.1097/00006199-198611000-00017
- Willis GB. Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks: Sage Publications, 2005.
- Kane MT. Validating the Interpretations and Uses of Test Scores. Journal of Educational Measurement. 2013, 50(1): 1-73. https://doi.org/10.1111/jedm.12000
- Vandenberg RJ, Lance CE. A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research. Organizational Research Methods. 2000, 3(1): 4-70. https://doi.org/10.1177/109442810031002
- Putnick DL, Bornstein MH. Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review. 2016, 41: 71-90. https://doi.org/10.1016/j.dr.2016.06.004
- McDonald RP. Test theory: A unified treatment. New York; London: Routledge, 2011.
- Dunn TJ, Baguley T, Brunsden V. From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology. 2013, 105(3): 399-412. https://doi.org/10.1111/bjop.12046


