Psychometric Validation and Preliminary Cross Cultural Comparison of an Artificial Intelligence Chatbot Perception Scale(G-CAVS)Across German and Turkish Samples

Seyat Polat; Günter Renner

doi:10.66815/ejecs/2984

Authors

Seyat Polat Otto-Friedrich-University Bamberg https://orcid.org/0000-0002-8570-2161
Günter Renner Friedrich-Alexander-Universität Erlangen-Nürnberg https://orcid.org/0009-0008-8290-9854

DOI:

https://doi.org/10.66815/ejecs/2984

Keywords:

Artificial Intelligence, Chatbots, Teacher Perceptions, Cross-Cultural Comparison

Abstract

This study was conducted to adapt the “General Chatbot Acceptance, Enjoyment, Perceived Risk and Value Scale (G-CAVS)” to Turkish, which was originally developed with teacher candidates, and to test measurement invariance of the scale between samples consisting of German pre-service teachers and Turkish in-service teachers. The research was designed as a scale adaptation and cross-cultural comparison study within the quantitative research paradigm. The data of the study were collected from two different groups: German teacher candidates (n = 184) and Turkish teachers (n = 263). Confirmatory factor analysis (CFA) confirmed the four-factor structure (Acceptance –Enjoyment of Use –Perceived Value –Perceived Risk) as in the original form of the scale and demonstrated that the Turkish version of the scale is valid and reliable. The results of the multi-group confirmatory factor analysis revealed that the scale demonstrates preliminary cross-cultural equivalence at the configural, metric, and partial scalar invariance levels. These findings indicate that the G-CAVS scale is a valid and reliable measurement tool for the Turkish sample and enables tentative comparison of teachers' perceptions of chatbots between the Turkish and German samples.

Downloads

Download data is not yet available.

👁 Abstract Views: 432📥 PDF Downloads: 44

Author Biographies

Seyat Polat, Otto-Friedrich-University Bamberg

is a scholar in educational sciences at the University of Bamberg. His research focuses on multilingual education, migration and language pedagogy, as well as the pedagogical integration of artificial intelligence in teaching and teacher education. He has conducted scale development and cross-cultural adaptation studies in the field of educational psychology and technology-enhanced learning. In addition, he serves as Editor-in-Chief of the Journal of Research in Social Science and Language (JSSAL). His scholarly work centers on culturally responsive pedagogy, teacher education, and evidence-based measurement in educational research.

Günter Renner, Friedrich-Alexander-Universität Erlangen-Nürnberg

is a senior lecturer (Akademischer Oberrat) at the Institute of Primary Education Research, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Germany. He studied Protestant theology and primary education (Lehramt Grundschule) and completed both the First and Second State Examinations for primary school teaching, graduating among the top five percent of his cohort. Following teaching experience in Bavaria and at Goroka International School in Papua New Guinea, he joined the Institute of Primary Education Research, where he has served in various academic positions since 2012. His research focuses on primary education, didactics, higher education, and individual and cooperative learning. His scholarly work contributes to research on teaching and learning processes in primary education.

References

Andriessen, J., & Sandberg, J. (1999). Where is education heading and how about AI. International Journal of Artificial Intelligence in Education, 10(2), 130–150.

APA. (2017). Ethical principles of Psychologist and code of conduct. Published online. http://www.apa.org/ethics/code/index.aspx

Beck, J., Stern, M., & Haugsjaa, E. (1996). Applications of AI in education. Crossroads, 3 (1), 11–15. https://dl.acm.org/doi/fullHtml/10.1145/332148.332153 DOI: https://doi.org/10.1145/332148.332153

Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186-3191. DOI: https://doi.org/10.1097/00007632-200012150-00014

Bilgiç, E. T. (2024). Genel veri koruma ilkelerinin yapay zekâ karşisinda uygulanabilirliği sorunu. Türkiye Adalet Akademisi Dergisi, (57), 247-282. https://doi.org/10.54049/taad.1418236 DOI: https://doi.org/10.54049/taad.1418236

Blair, F., & Walsh, T. (2026). Mother/nature: An intersectional analysis of the patriarchal nature of anthropocentric practices and discourses surrounding nuclear testing and industrial pollution. Feminist Encounters: A Journal of Critical Studies in Culture and Politics, 10(2), 7. https://doi.org/10.20897/femenc/18754 DOI: https://doi.org/10.20897/femenc/18754

Boonmoh, A., Jumpakate, T., & Karpklon, S. (2021). Teachers’ perceptions and experience in using technology for the classroom, Computer-Assisted Language Learning Electronic Journal, 22(1), 1-24. https://callej.org/index.php/journal/article/view/320

Borsci, S., Malizia, A., Schmettow, M., Van Der Velde, F., Tariverdiyeva, G., Balaji, D., & Chamberlain, A. (2022). The chatbot usability scale: The design and pilot of a usability scale for interaction with AI-based conversational agents. Personal and Ubiquitous Computing, 26(1), 95-119. https://doi.org/10.1007/ s00779-021-01582-9 DOI: https://doi.org/10.1007/s00779-021-01582-9

Butler, L., & Starkey, L. (2024). OK Google, help me learn: An exploratory study of voice-activated artificial intelligence in the classroom. Technology, Pedagogy and Education, 33(2), 135–148. https://doi.org/10.1080/1475939X.2024.2311779 DOI: https://doi.org/10.1080/1475939X.2024.2311779

Chambers, C. (2004). Technological advancement, learning, and the adoption of new technology. European Journal of Operational Research, 152(1), 226-247. https://doi.org/10.1016/S0377-2217(02)00651-3 DOI: https://doi.org/10.1016/S0377-2217(02)00651-3

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural equation modeling, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5 DOI: https://doi.org/10.1207/S15328007SEM0902_5

Dai, Y., Lin, Z., Liu, A., Dai, D., & Wang, W. (2023). Effect of an Analogy-Based Approach of Artificial Intelligence Pedagogy in Upper Primary Schools. Journal of Educational Computing Research, 61(8), 1695–1722. https://doi.org/10.1177/07356331231201342 DOI: https://doi.org/10.1177/07356331231201342

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 319-340. https://doi.org/10.2307/249008 DOI: https://doi.org/10.2307/249008

Elmaadaway, M. A. N., El‐Naggar, M. E., & Abouhashesh, M. R. I. (2025). Improving Primary School Students’ Oral Reading Fluency Through Voice Chatbot‐Based AI. Journal of Computer Assisted Learning, 41(2), e70019. https://doi.org/10.1111/jcal.70019 DOI: https://doi.org/10.1111/jcal.70019

Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE access, 8, 75264-75278. https://ieeexplore.ieee.org/abstract/document/9069875 DOI: https://doi.org/10.1109/ACCESS.2020.2988510

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural equation modeling: a multidisciplinary journal, 14(3), 464-504. https://doi.org/10.1080/10705510701301834 DOI: https://doi.org/10.1080/10705510701301834

Dalle, J., Raisinghani, M. S., Putra, A. P., Suriansyah, A., Hadi, S., & Sahara, B. (2021). A technology acceptance case of Indonesian senior school teachers: Effect of facilitating learning environment and learning through experimentation. International Journal of Online Pedagogy and Course Design, 11(4), 45–60. https://doi.org/10.4018/IJOPCD.2021100104 DOI: https://doi.org/10.4018/IJOPCD.2021100104

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50. DOI: https://doi.org/10.1177/002224378101800104

https://doi.org/10.1177/002224378101800104

Herwin, H., Che Dahalan, S., Olapane, E., Prananto, I. W., & Wirawan, P. R. (2026). Assessing STEM thinking in elementary classrooms: Evidence from teachers' perspectives. European Journal of STEM Education, 11(1), Article 31. https://doi.org/10.20897/ejsteme/18597 DOI: https://doi.org/10.20897/ejsteme/18597

Hoofnagle, C. J., Van Der Sloot, B., & Borgesius, F. Z. (2019). The European Union general data protection regulation: what it is and what it means. Information & Communications Technology Law, 28(1), 65-98. https://doi.org/10.1080/13600834.2019.1573501 DOI: https://doi.org/10.1080/13600834.2019.1573501

Huang, J., Saleh, S., & Liu, Y. (2021). A review on artificial intelligence in education. Academic Journal of Interdisciplinary Studies, 10(3). https://doi.org/10.36941/ajis-2021-0077 DOI: https://doi.org/10.36941/ajis-2021-0077

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118 DOI: https://doi.org/10.1080/10705519909540118

Ilhan, A., & Fietkiewicz, K. J. (2021). Data privacy-related behavior and concerns of activity tracking technology users from Germany and the USA. Aslib Journal of Information Management, 73(2), 180-200. https://doi.org/10.1108/AJIM-03-2020-0067 DOI: https://doi.org/10.1108/AJIM-03-2020-0067

Jiang, Y., Li, X., Luo, H., Yin, S., & Kaynak, O. (2022). Quo vadis artificial intelligence?. Discover Artificial Intelligence, 2(1), 4. https://doi.org/10.1007/s44163-022-00022-8 DOI: https://doi.org/10.1007/s44163-022-00022-8

Katz, N. (2026). Digital inequality in context: A socio-technical analysis of Arab students' remote learning in Israel. Asia Pacific Journal of Education and Society, 14(1), 6. https://doi.org/10.20897/apjes/17975 DOI: https://doi.org/10.20897/apjes/17975

Kim, H. W., Chan, H. C., & Gupta, S. (2007). Value-based adoption of mobile internet: an empirical investigation. Decision support systems, 43(1), 111-126. https://doi.org/10.1016/j.dss.2005.05.009 DOI: https://doi.org/10.1016/j.dss.2005.05.009

Köhler, C., & Hartig, J. (2024). ChatGPT in higher education: Measurement instruments to assess student knowledge, usage, and attitude. Contemporary educational technology, 16(4), ep528. https://doi.org/10.30935/cedtech/15144 DOI: https://doi.org/10.30935/cedtech/15144

Li, Y., & Hilliges, O. (Eds.). (2021). Artificial intelligence for human computer interaction: a modern approach (pp. 463-493). Cham: Springer. https://link.springer.com/content/pdf/10.1007/978-3-030-82681-9.pdf DOI: https://doi.org/10.1007/978-3-030-82681-9

Mundy, M.-A., Kupczynski, L., & Kee, R. (2012). Teacher’s Perceptions of Technology Use in the Schools. Sage Open, 2(1). https://doi.org/10.1177/2158244012440813 DOI: https://doi.org/10.1177/2158244012440813

Nemt-Allah, M., Khalifa, W., Badawy, M., Elbably, Y., & Ibrahim, A. (2024). Validating the ChatGPT Usage Scale: psychometric properties and factor structures among postgraduate students. BMC psychology, 12(1), 497. https://doi.org/10.1186/s40359-024-01983-4 DOI: https://doi.org/10.1186/s40359-024-01983-4

Özcan, A., & Polat, S. (2023). Artificial Intelligence and Chat Bots in Academic Research. Journal of Research in Social Sciences and Language, 3(2), 81–90. https://doi.org/10.71514/jssal/2023.111

Polat, S., & Renner, G. (2026). General Chatbot Acceptance, Enjoyment, Perceived Risk and Value (G-CAVS): Scale Development and Validation, Contemporary educational technology. https://doi.org/10.30935/cedtech/17878 DOI: https://doi.org/10.30935/cedtech/17878

Pedro, F., Subosa, M., Rivas, A., & Valverde, P. (2019). Artificial intelligence in education: Challenges and opportunities for sustainable development. https://unesdoc.unesco.org/ark:/48223/pf0000366994

Roll, I., & Wylie, R. (2016). Evolution and revolution in artificial intelligence in education. International journal of artificial intelligence in education, 26(2), 582-599. https://doi.org/10.1007/s40593-016-0110-3 DOI: https://doi.org/10.1007/s40593-016-0110-3

Rudra, S., Weber, P., Tillmanns, T., Filho, A., Wiersma, E., Dawitz, J., Dudenaite, D., & Reynolds, S. (2025). Use and perception of generative AI in higher education: Insights from the ERASMUS+ project 'Teaching and Learning with Artificial Intelligence' (TaLAI). Proceedings of the 17th International Conference on Computer Supported Education (CSEDU 2025). DOI: https://doi.org/10.5220/0013360200003932

Sharma, P. (2010). Measuring personal cultural orientations: Scale development and validation. Journal of the Academy of Marketing Science, 38(6), 787-806. https://doi.org/10.1007/s11747-009-0184-7 DOI: https://doi.org/10.1007/s11747-009-0184-7

Taylor, Z. (2024). Using Chat GPT to Clean Qualitative Interview Transcriptions: A Usability and Feasibility Analysis. American Journal of Qualitative Research, 8(2), 153-160. https://doi.org/10.29333/ajqr/14487 DOI: https://doi.org/10.29333/ajqr/14487

Tobbi, S. (2026). Companion AI as a Practice Ecology: Identity Rehearsal, Affective Affordances and Emerging AI Literacies among Algerian Students of English as a Foreign Language. Journal of Research in Social Sciences and Language. https://doi.org/10.71514/jssal/2026.247

Uğraş, H., Uğraş, M., Papadakis, S., & Kalogiannakis, M. (2024a). ChatGPT-supported education in primary schools: The potential of ChatGPT for sustainable practices. Sustainability, 16(22), 9855. https://doi.org/10.3390/su16229855 DOI: https://doi.org/10.3390/su16229855

Uğraş, H., Uğraş, M., Papadakis, S., & Kalogiannakis, M. (2024b). Innovative early childhood STEM education with ChatGPT: Teacher perspectives. Technology, Knowledge and Learning, 30(2), 809-831. https://doi.org/10.1007/s10758-024-09804-8 DOI: https://doi.org/10.1007/s10758-024-09804-8

Valdez, G., McNabb, M., Foertsch, M., Anderson, M., Hawkes, M., & Raack, L. (1999). Computer-based technology and learning: Evolving uses and expectations. NCREL, 1900 Spring Rd., Suite 300, Oak Brook, IL 60523-1480.

Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decision sciences, 39(2), 273-315. https://doi.org/10.1111/j.1540-5915.2008.00192.x DOI: https://doi.org/10.1111/j.1540-5915.2008.00192.x

Veith, J., Bauer, A., Becker, E., & Bitzenbauer, P. (2026). Exploring physics students’ attitudes toward ChatGPT using the ABC model. EURASIA Journal of Mathematics, Science and Technology Education, 22(2), em2770. https://doi.org/10.29333/ejmste/17822 DOI: https://doi.org/10.29333/ejmste/17822

Yurt, E. (2025). The self-regulation for AI-based learning scale: Psychometric properties and validation. International Journal of Current Educational Studies, 4(1), 95-118. https://doi.org/10.46328/ijces.176 DOI: https://doi.org/10.46328/ijces.176