Psychometric Properties of the Persian Version of the Online Student Engagement Questionnaire: A Transcultural Adaptation and Psychometric Study

Document Type : Original Article

Authors

Department of Educational Sciences, Farhangian University, Tehran, Iran

Abstract

Background: Given the significance of learner engagement in distance education, particularly in online learning settings, scholars are continuously seeking appropriate tools to assess it with greater precision. Therefore, this research aimed to investigate the psychometric properties of the Online Student Engagement Questionnaire (OSEQ) within the Iranian context.
Methods: This transcultural adaptation and psychometric study was carried out on 330 students who were engaged in online studies using a convenience sampling across five Iranian universities (Farhangian University, Shahid Rajaee Teacher Training University, Tehran University, Mehr Alborz University, and Iran University of Science and Technology) from October 2022 to December 2023. The OSEQ, comprising 16 self-report items across four engagement dimensions (cognitive, behavioral, social, and affective engagements), was employed. The questionnaire underwent initial translation using a standard forward-backward technique. The psychometric characteristics including face, content, and construct validities, along with reliability, were appraised using both Cronbach’s alpha and McDonald’s omega coefficients.
Results: The linguistic and conceptual equivalence of the translated questionnaire exceeded 1.5, while the content validity values (CVR= 0.81; CVI= 0.85) were determined based on the viewpoints of nine experts. No items out of 16 were excluded, considering the face and content validity coefficients. Through the execution of an Exploratory Factor Analysis (EFA), four factors were extracted, accounting for 56.74% of the overall variance. The structure of the factors, supported by suitable fit indices (Χ2/df=2.33, RMSEA=0.064, GFI=0.92, NFI=0.90, CFI=0.93, RMR=0.044, SRMR=0.046) derived from four first-order factors, was validated. The questionnaire demonstrated satisfactory reliability, as indicated by McDonald’s omega coefficient ranging from 0.70 to 0.79 and Cronbach alpha coefficient ranging from 0.70 to 0.79.
Conclusion: The findings demonstrated that the OSEQ has strong psychometric properties, making it an appropriate instrument for assessing online student engagement within the Iranian setting.

Highlights

Abbas Taghizade (Google Scholar)

Keywords

References

  1. Ferrer J, Ringer A, Saville K, A Parris M, Kashi K. Students’ motivation and engagement in higher education: the importance of attitude to online learning. High Educ. 2022;19;83(2):317–38. doi:10.1007/s10734-020-00657-5.
  2. Henrie CR, Halverson LR, Graham CR. Measuring student engagement in technology-mediated learning: A review. Comput Educ. 2015;90:36–53. doi:10.1016/j.compedu.2015.09.005.
  3. Sanders LD, Daly AP, Fitzgerald K. Predicting retention, understanding attrition: A prospective study of foundation year students. Widening Particip Lifelong Learn. 2016;18(2):50–83. doi:10.5456/WPLL.18.2.50.
  4. Miller CJ, Perera HN, Maghsoudlou A. Students multidimensional profiles of math engagement: Predictors and outcomes from a self-system motivational perspective. Br J Educ Psychol. 2021;91(1):261–85. doi:10.1111/bjep.12358. PubMed PMID: 32567681.
  5. Wang M-T, Fredricks JA, Ye F, Hofkens TL, Linn JS. The math and science engagement scales: Scale development, validation, and psychometric properties. Learn Instr. 2016;43:16–26. doi:10.1016/j.learninstruc.2016.01.008.
  6. Dixson MD. Measuring student engagement in the online course: The Online Student Engagement scale (OSE). Online Learn. 2015;19(4):n4. doi:10.24059/olj.v19i4.561.
  7. Wong L. Student engagement with online resources and its impact on learning outcomes. Proceedings of the informing science and information technology education conference; 2013. Journal of Information Technology Education: Innovations in Practice, 12(1),129-146. Available from: https://www.learntechlib.org/p/174797.
  8. Lin L-C, Hung I-C, Kinshuk, Chen N-S. The impact of student engagement on learning outcomes in a cyber-flipped course. Educ Technol Res Dev. 2019;67:1573–91. doi:10.1007/s11423-019-09698-9.
  9. Wang C, Mirzaei T, Xu T, Lin H. How learner engagement impacts non-formal online learning outcomes through value co-creation: An empirical analysis. Int J Educ Technol High Educ. 2022;19(1):32. doi:10.1186/s41239-022-00341-x. PubMed PMID: 35821974; PubMed Central PMCID: PMC9263066.
  10. Bergdahl N, Nouri J, Fors U, Knutsson O. Engagement, disengagement and performance when learning with technologies in upper secondary school. Comput Educ. 2020;149:103783. doi:10.1016/j.compedu.2019.103783.
  11. Wang M-T, Peck SC. Adolescent educational success and mental health vary across school engagement profiles. Dev Psychol. 2013;49(7):1266. doi:10.1037/a0030028. PubMed PMID: 23066673.
  12. Bae CL, Les DeBusk-Lane M, Lester AM. Engagement profiles of elementary students in urban schools. Contemp Educ Psychol. 2020;62:101880. doi:10.1016/j.cedpsych.2020.101880.
  13. Rimm-Kaufman SE, Baroody AE, Larsen RAA, Curby TW, Abry T. To what extent do teacher-student interaction quality and student gender contribute to fifth graders engagement in mathematics learning? J Educ Psychol. 2015;107(1):170-185. https://psycnet.apa.org/doi/10.1037/a0037252.
  14. Fredricks JA, Blumenfeld PC, Paris AH. School engagement: Potential of the concept, state of the evidence. Rev Educ Res. 2004;74(1):59–109. doi:10.3102/00346543074001059.
  15. Fredricks JA, McColskey W. The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In: Handbook of research on student engagement. Springer; 2012. P. 763–82. https://awspntest.apa.org/doi/10.1007/978-1-4614-2018-7_37.
  16. Lamborn S, Newmann F, Wehlage G. The Significance and Sources of Student Engagement. In: Newman, F.M., editor. Student Engagement and Achievement in American Secondary Schools. Teachers College Press: New York; 1992. P. 11-39.
  17. Deng R, Benckendorff P, Gannaway D. Learner engagement in MOOCs: Scale development and validation. Br J Educ Technol. 2020;51(1):245–62. doi:10.1111/bjet.12810.
  18. Skinner EA, Pitzer JR. Developmental dynamics of student engagement, coping, and everyday resilience. In: Handbook of research on student engagement. Springer; 2012. p. 21–44. doi:10.1007/978-1-4614-2018-7_2.
  19. Fredricks JA, Hofkens TL, Wang M-T. 27 Addressing the Challenge of Measuring Student Engagement. Cambridge University Press; 2019. P. 689-712. doi:10.1017/9781316823279.029.
  20. Bebell D, Kay R. One to one computing: A summary of the quantitative results from the Berkshire wireless learning initiative. J Technol Learn Assess. 2010;9(2):n2. Available from: https://ejournals.bc.edu/index.php/jtla/article/view/1607.
  21. Wang M. Online collaboration and offline interaction between students using asynchronous tools in blended learning. Australas J Educ Technol. 2010;26(6):830-46. doi:10.14742/ajet.1045.
  22. Appleton JJ, Christenson SL, Kim D, Reschly AL. Measuring cognitive and psychological engagement: Validation of the Student Engagement Instrument. J Sch Psychol. 2006;44(5):427–45. doi:10.1016/j.jsp.2006.04.002.
  23. Thai M, Sheeran N, Cummings DJ. We’re all in this together: The impact of Facebook groups on social connectedness and other outcomes in higher education. Internet High Educ. 2019;40:44–9. doi:10.1016/J.IHEDUC.2018.10.001.
  24. Sun Y, Guo Y, Zhao Y. Understanding the determinants of learner engagement in MOOCs: An adaptive structuration perspective. Comput Educ. 2020;157:103963. doi:10.1016/j.compedu.2020.103963.
  25. Hoi VN, Le Hang H. The structure of student engagement in online learning: A bi-factor exploratory structural equation modeling approach. J Comput Assist Learn. 2021;37(4):1141–53. doi:10.1111/jcal.12551.
  26. Ben-Eliyahu A, Moore D, Dorph R, Schunn CD. Investigating the multidimensionality of engagement: Affective, behavioral, and cognitive engagement across science activities and contexts. Contemp Educ Psychol. 2018;53:87–105. doi:10.1016/j.cedpsych.2018.01.002.
  27. Garavand Y, Koohi M, Seyed kaboli E, Alizadeh A. Construction and Investigating of Psychometric Characteristics of Research Engagement Questionnaire in Postgraduate Students. Q Educ Meas. 2020;10(39):83–104. [In Persian] Available from: https://jem.atu.ac.ir/article_11749.html.
  28. Amirian SK, Hajhosseini M, Nezami M, Ebrahimi S. Psychometric properties of the student discussion engagement scale. Q Educ Meas. 2023;14(53). [In Persian] Available from: https://jem.atu.ac.ir/article_15881.html.
  29. Ramazani M, khamesan,  A. Psychometric characteristics of Reeve’s academic engagement questionnaire 2013: with the introduction of the Agentic Engagement. Q Educ Meas. 2017;8(29):185–204. [In Persian] Available from: https://jem.atu.ac.ir/article_8380.html.
  30. Hajializadeh K. Evaluation of the Psychometric Properties of the Academic Engagement Questionnaire among High School Students. Q Educ Meas [In Persian]. 2016;6(24):83–102. Available from: https://jem.atu.ac.ir/article_7541.html.
  31. Kline RB. Principles and practice of structural equation modeling. Guilford publications. 5th ed. Routledge; 2023. https://www.routledge.com/Principles-and-Practice-of-Structural-Equation-Modeling-Fifth-Edition/Kline/p/book/9781462551910.
  32. Broder HL, McGrath C, Cisneros GJ. Questionnaire development: face validity and item impact testing of the Child Oral Health Impact Profile. Community Dent Oral Epidemiol. 2007;35:8–19. doi:10.1111/j.1600-0528.2007.00401.x. PubMed PMID: 17615046.
  33. Lawshe CH, others. A quantitative approach to content validity. Pers Psychol. 1975;28(4):563–75. doi:10.1111/j.1744-6570.1975.tb01393.x.
  34. Waltz CF, Bausell BR. Nursing Research: Design Statistics and Computer Analysis. F.A. Davis Company; 1981.
  35. Watted A, Barak M. Motivating factors of MOOC completers: Comparing between university-affiliated students and general participants. Internet High Educ. 2018;37:11–20. doi:10.1016/j.iheduc.2017.12.001.
  36. Bond M, Buntins K, Bedenlier S, Zawacki-Richter O, Kerres M. Mapping research in student engagement and educational technology in higher education: A systematic evidence map. Int J Educ Technol High Educ. 2020;17:1–30. doi:10.1186/s41239-019-0176-8.
  37. Gray JA, DiLoreto M. The effects of student engagement, student satisfaction, and perceived learning in online learning environments. Int J Educ Leadersh Prep. 2016;11(1). https://eric.ed.gov/?id=EJ1103654.
  38. Zen Z, Ariani F, others. Academic achievement: the effect of project-based online learning method and student engagement. Heliyon. 2022;8(11). doi:10.1016/j.heliyon.2022.e11509. PubMed PMID: 36411883; PubMed Central PMCID: PMC9674908.
  39. DeVellis RF, Thorpe CT. Scale development: Theory and applications. Sage Publications; 2021.
  40. Abbasi M, Ghamoushi M, Mohammadi Zenouzagh Z. EFL learners engagement in online learning context: development and validation of potential measurement inventory. Univers Access Inf Soc. 2023;1–15. doi:10.1007/s10209-023-00993-0. PubMed PMID: 37361676; PubMed Central PMCID: PMC10106874.
  41. Roblyer MD, Wiencke WR. Exploring the interaction equation: Validating a rubric to assess and encourage interaction in distance courses. J Asynchronous Learn Networks. 2004;8(4):24–37. doi:10.24059/olj.v8i4.1808.
  42. Devaux M, Sassi F. Social disparities in hazardous alcohol use: self-report bias may lead to incorrect estimates. Eur J Public Health. 2016;26(1):129–34. doi:10.1093/eurpub/ckv190. PubMed PMID: 26585784; PubMed Central PMCID: PMC4851717.
Interdiscip J Virtual Learn Med Sci
Volume 15, Issue 3 - Serial Number 58
September 2024
Pages 226-240

Files

Share

How to cite

Statistics