Between Interest and Skill: How Students Perceive and Use AI
https://doi.org/10.31992/0869-3617-2025-34-8-9-9-32
Abstract
As AI becomes an integral part of education and the future labor market, it is important to understand how students perceive and use it. This study explores Russian university students’ attitudes toward AI and examines the relationship between their beliefs and actual skills in effectively using AI. A specially developed questionnaire was used to assess students’ conceptions of AI, covering four dimensions: interest in AI, subjective experience with AI, perceived future value of AI, and perceived risks associated with AI. Skills of using AI were measured through a practical task involving the creation of an effective prompt for a large language model to solve an authentic work-related problem. Results showed that many students struggled to create effective prompts. Those who considered themselves more experienced with using AI performed only slightly better (r = 0,20), as did students with a higher level of interest in AI (r = 0,12). Overall, the connection between attitudes and actual skills was weak. Students who perceived AI as risky tended to assign it less value for the future (r = –0,09), but this perception did not affect their interest in AI or their sense of personal experience. Ultimately, despite students’ strong interest in AI, their ability to use it effectively remains limited.
Supporting agencies: The research was carried out at the expense of the grant of the Russian Science Foundation 25-28-01485.
About the Authors
A. E. Ivanova
National Research University Higher School of Economics
Russian Federation
Russian Federation
Alina E. Ivanova – Ph.D. (Education), Senior Research Fellow, Center for Psychometrics and Measurement in Education, Institute of Education,
16, bldg. 10, Potapovsky lane, Moscow, 101000.
Researcher ID: I-4300-2015.
K. V. Tarasova
National Research University Higher School of Economics
Russian Federation
Russian Federation
Ksenia V. Tarasova – Ph.D. (Education), Director of the Center for Psychometrics and Measurement in Education, Institute of Education,
16, bldg. 10, Potapovsky lane, Moscow, 101000.
Researcher ID: ABD-33272020.
D. P. Talov
National Research University Higher School of Economics
Russian Federation
Russian Federation
Daniil P. Talov – Ph.D. student, Junior Research Fellow, Center for Psychometrics and Measurement in Education, Institute of Education,
16, bldg. 10, Potapovsky lane, Moscow, 101000.
Researcher ID: MDS-9225-2025.
References
1. Almatrafi, O., Johri, A., Lee, H. (2024). A Systematic Review of AI Literacy Conceptualization, Constructs, and Implementation and Assessment Efforts (2019–2023). Computers and Education Open. Article no. 100173, doi: 10.1016/j.caeo.2024.100173
2. Mills, K., Ruiz, P., Lee, K.W., Coenraad, M., Fusco, J., Roschelle, J., Weisgrau, J. (2024). AI Literacy: A Framework to Understand, Evaluate, and Use Emerging Technology. Digital Promise. Doi: 10.51388/20.500.12265/218
3. Sindermann, C., Sha, P., Zhou, M., Wernicke, J., Schmmitt, H.S. et al. (2021). Assessing the Attitude Towards Artificial Intelligence: Introduction of a Short Measure in German, Chinese, and English Language. Künstliche Intelligenz. Vol. 35, pp. 109-118, doi: 10.1007/s13218020-00689-0
4. Ibrahim, F., Münscher, J.C., Daseking, M., Telle, N.T. (2025). The Technology Acceptance Model and Adopter Type Analysis in the Context of Artificial Intelligence. Frontiers in Artificial Intelligence. Vol. 7, article no. 1496518, doi: 10.3389/frai.2024.1496518
5. Wang, J., Fan, W. (2025). The Effect of ChatGPT on Students’ Learning Performance, Learning Perception, and Higher-Order Thinking: Insights from a Meta-Analysis. Humanities and Social Sciences Communications. No. 12, article no. 621, doi: 10.1057/s41599-025-04787-y
6. Ali, D., Fatemi, Y., Boskabadi, E., Nikfar, M., Ugwuoke, J., Ali, H. (2024). ChatGPT in Teaching and Learning: A Systematic Review. Education Sciences. Vol. 14, no. 6, article no. 643, doi: 10.3390/educsci14060643
7. Laupichler, M.C., Aster, A., Schirch, J., Raupach, T. (2022). Artificial Intelligence Literacy in Higher and Adult Education: A Scoping Literature Review. Computers and Education. Vol. 3, article no. 100101, doi: 10.1016/j.caeai.2022.100101
8. Zhou, X., Schofield, L., Zhang, J.J., Abuelmaatti, A., Howell, L. (2024). Building Bridges in AI: Enhancing AI Literacy for Students and Staff Across Disciplines. PGR Student Partners – Empowering Doctoral Students Through Partnership And Co-Creation in Institutional Equality, Diversity and Inclusion Change Projects, 20. Available at: https://www.researchgate.net/publication/384291235_Building_bridges_in_AI_Enhancing_AI_literacy_for_students_and_staff_across_disciplines (accessed: 23.06.2025).
9. Lintner, T. (2024). A Systematic Review of AI Literacy Scales. NPJ Science of Learning. Vol. 9, no. 1, article no. 50, doi: 10.1038/s41539-024-00264-4
10. Mansoor, H.M., Bawazir, A., Alsabri, M.A., Alharbi, A., Okela, A.H. (2024). Artificial Intelligence Literacy among University Students – A Comparative Transnational Survey. Frontiers in Communication. Vol. 9, article no. 1478476, doi: 10.3389/fcomm.2024.1478476
11. Fernandes, D., Villa, S., Nicholls, S., Haavisto, O., Buschek, D. et al. (2024). Performance and Metacognition Disconnect When Reasoning in Human-AI Interaction. arXiv preprint. Article no. 2409.16708, doi: 10.48550/arXiv.2409.16708
12. Mousavi Baigi, S.F., Sarbaz, M., Ghaddaripouri, K., Ghaddaripouri, M., Mousavi, A.S. et al. (2023). Attitudes, Knowledge, and Skills Towards Artificial Intelligence among Healthcare Students: A Systematic Review. Health Science Reports. Vol. 6, no. 3, article no. e1138, doi: 10.1002/hsr2.1138
13. Klar, M. (2025). Using ChatGPT Is Easy, Using It Effectively Is Tough? A Mixed-Methods Study on K-12 Students’ Perceptions, Interaction Patterns, and Support for Learning with Generative AI Chatbots. Smart Learning Environments. Vol. 12, no. 1, pp. 1-19, doi: 10.1186/s40561-02500385-2
14. Mahmood, K. (2016). Do People Overestimate Their Information Literacy Skills? A Systematic Review of the Dunning-Kruger Effect. Communications in Information Literacy. Vol. 10, no. 2, pp. 3-24, doi: 10.15760/comminfolit.2016.10.2.2
15. Giray, L. (2023). Prompt Engineering with ChatGPT: A Guide for Academic Writers. Annals of Biomedical Engineering. Vol. 51, no. 6, pp. 2692-2633, doi: 10.1007/s10439-023-03272-4
16. Yang, M., Jiang, Sh., Li, B., Herman, K. (2025). Analysing Nontraditional Students’ ChatGPT Interaction, Engagement, Self-Efficacy and Performance: A Mixed-Methods Approach. British Journal of Educational Technology. Vol. 56, no. 5, pp. 1973-2000, doi: 10.1111/bjet.13588
17. Ekin, S. (2023). Prompt Engineering for ChatGPT: A Quick Guide to Techniques, Tips, and Best Practices. TechRxiv. Doi: 10.36227/techrxiv.22683919.v2
18. Nori, H., King, N., McKinney, S.M., Carignan, D., Horvitz, E. (2023). Capabilities of GPT-4 on Medical Challenge Problems. arXiv preprint. Article no. 2303.13375, doi: 10.48550/arXiv.2303.13375
19. Lee, D., Palmer, E. (2025). Prompt Engineering in Higher Education: A Systematic Review to Help Inform Curricula. International Journal of Educational Technology in Higher Education. Vol. 22, no. 1, article no. 7, doi: 10.1186/s41239-025-00503-7
20. Krupp, S., Rinas, R., Klieme, E., Nagel, M. (2023). Large Language Models in the Classroom: Opportunities and Challenges of ChatGPT for Higher Education. Zeitschrift für Erziehungswissenschaft. Vol. 26, no. 3, pp. 573-594, doi: 10.1007/s11618-023-01128-6
21. Federiakin, D., Molerov, D., Zlatkin-Troitschanskaia, O., Maur, A. (2024). Prompt Engineering as a New 21st Century Skill. Frontiers in Education. No. 9, article 1366434, doi: 10.3389/feduc.2024.1366434
22. Maharjan, J., Garikipati, A., Singh, N.P., Cyrus, L., Sharmma, M. et al. (2024). OpenMedLM: Prompt Engineering Can Out-Perform Fine-Tuning in Medical Question-Answering with Open-Source Large Language Models. Scientific Reports. Vol. 14, no. 2, doi: 10.1038/s41598-024-64827-6
23. Hwang, G.J., Chiu, L.Y., Tseng, J.C.R. (2023). A Framework of Prompt Literacy for Supporting AI Learning: Conceptualization and Application. Computers and Education: Artificial Intelligence. Vol. 4, article no. 100160, doi: 10.1016/j.caeai.2023.100160
24. Busch, K., Rochlitzer, A., Sola, D., Leopold, H. (2023). Just Tell Me: Prompt Engineering in Business Process Management. In: Enterprise, Business-Process and Information Systems Modeling. Pp. 3-11, doi: 10.1007/978-3-031-34241-7_1
25. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. et al. (2020). Language Models Are FewShot Learners. Advances in Neural Information Processing Systems. Vol. 33, pp. 1877-1901, doi: 10.48550/arXiv.2005.14165
26. Kojima, T., Gu, S.S., Reid, M., Matsuo, Yu., Iwasawa, Yu. (2022). Large Language Models Are Zero-Shot Reasoners. Advances in Neural Information Processing Systems. Vol. 35, doi: 10.48550/arXiv.2205.11916
27. Wang, X., Wei, J., Schuurmans, D., Lee, Q., Chi, E. et al. (2022). Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv preprint. Article no. 2203.11171, doi: 10.48550/arXiv.2203.11171
28. Walter, S. (2024). Integrating LLMs in Higher Education: Implications for Assessment and Academic Integrity. Journal of Learning Analytics and AI in Education. Vol. 5, no. 1, pp. 45-61, doi: 10.61669/001c.131915
29. Semnani, S.H., Li, X., Ye, Q., Callison-Burch, C. (2023). SelfCheckGPT: Zero-Resource BlackBox Hallucination Detection for Generative Large Language Models. arXiv preprint. Article no. 2303.08896, doi: 10.48550/arXiv.2303.08896
30. Gattupalli, S., Maloy, R.W., Edwards, S.A. (2023). Prompt Literacy: A Pivotal Educational Skill in the Age of AI. College of Education Working Papers and Reports Series. Doi: 10.7275/3498wx48
31. Carolus, A., Koch, M.J., Straka, S., Latoschik, M.E., Wienrich, C. (2023). MAILS-Meta AI Literacy Scale: Development and Testing of an AI literacy Questionnaire Based on Well-Founded Competency Models and Psychological Change-And Meta-Competencies. Computers in Human Behavior: Artificial Humans. Vol. 1, no. 2, article no. 100014, doi: 10.1016/j.chbah.2023.100014
32. Katsantonis, A., Katsantonis, I.G. (2024). University Students’ Attitudes Toward Artificial Intelligence: An Exploratory Study of the Cognitive, Emotional, and Behavioural Dimensions of AI Attitudes. Education Sciences. Vol. 14, no. 9, article no. 988, doi: 10.3390/educsci14090988
33. Roberts-Tyler, E.J., Roberts, S., Watkins, R., Gillespie, D. (2023). Effects of Implementation Support on Children’s Reading Outcomes Following an Online Early Reading Programme: A Cluster-Randomised Controlled Trial. British Journal of Educational Technology. Vol. 54, no. 5, pp. 1373-1396, doi: 10.1111/bjet.13312
34. Long, D., Magerko, B. (2020). What Is AI Literacy? Competencies and Design Considerations. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Pp. 1-16, doi: 10.1145/3313831.3376727
35. Yang, W. (2022). Artificial Intelligence Education for Young Children: Why, What, and How in Curriculum Design and Implementation. Computers and Education. Vol. 3, article no. 100061, doi: 10.1016/j.caeai.2022.100061
36. Pedro, F., Subosa, M., Rivas, A., Valverde, P. (2019). Artificial Intelligence in Education: Challenges and Opportunities for Sustainable Development. UNESCO. 46 p. Available at: https:// unesdoc.unesco.org/ark:/48223/pf0000366994 (accessed: 23.06.2025).
37. Miao, F., Holmes, W., Huang, R., Zhang, H. (2021). AI and Education: A Guidance for Policymakers. UNESCO Publishing. Available at: https://unesdoc.unesco.org/ark:/48223/pf0000376709 (accessed: 23.06.2025).
38. Casal-Otero, L., Catala, A., Fernández-Morante, C. Taboada, M., Cebreiro, B. et al. (2023). AI Literacy in K-12: A Systematic Literature Review. International Journal of STEM Education. Vol. 10, article no. 29, doi: 10.1186/s40594-023-00418-7
39. Hwang, Y., Lee, J.H., Shin, D. (2023). What Is Prompt Literacy? An Exploratory Study of Language Learners’ Development of New Literacy Skill Using Generative AI. arXiv preprint. Article no. 2311.05373, doi:10.48550/arXiv.2311.05373
40. Li, X., Jiang, M.Y., Jong, M.S., Zhang, X., Chai, Ch. (2022). Understanding Medical Students’ Perceptions of and Behavioral Intentions toward Learning Artificial Intelligence. International Journal of Environmental Research and Public Health. Vol. 19, article no. 8733, doi: 10.3390/ijerph19018733
41. Morales-García, W.C., Sairitupa-Sanchez, L.Z., Morales-García, S.B., Morales-García, M. (2024). Adaptation and Psychometric Properties of a Brief Version of the General Self-Efficacy Scale for Use with Artificial Intelligence (GSE-6AI) among University Students. Frontiers in Education. Vol. 9, article no. 1293437, doi: 10.3389/feduc.2024.1293437
42. Wang, Y.Y., Chuang, Y.W. (2023). Artificial Intelligence Self-Efficacy: Scale Development and Validation. Education and Information Technologies. Vol. 28, pp. 1-24, doi: 10.1007/s10639023-12001-y
43. Soto-Sanfiel, M.T., Angulo-Brunet, A., Lutz, C. (2024). The Scale of Artificial Intelligence Literacy for All (SAIL4ALL): A Tool for Assessing Knowledge on Artificial Intelligence in All Adult Populations and Settings. OSF preprint. Doi: 10.31235/osf.io/bvyku
44. Ng, D.T.K., Wu W., Leung J.K.L., Chiu T.K.F., Chu S.K.W. (2023). Design and Validation of the AI Literacy Questionnaire: The Affective, Behavioural, Cognitive and Ethical Approach. British Journal of Educational Technology. Vol. 54, pp. 1-23, doi: 10.1111/bjet.13411
45. Hwang, H.S., Zhu, L.C., Cui, Q. (2023). Development and Validation of a Digital Literacy Scale in the AI Era for College Students. KSII Transactions on Internet and Information Systems. Vol. 17, no. 8, pp. 2241-2258, doi: 10.3837/tiis.2023.08.016
46. Eccles, J.S., Wigfield, A. (2020). From Expectancy-Value Theory to Situated ExpectancyValue Theory: A Developmental, Social Cognitive, and Sociocultural Perspective on Motivation. Contemporary Educational Psychology. Vol. 61, article no. 101859, doi: 10.1016/j.cedpsych.2020.101859
47. Zhai, X., Chu, X., Chai, C.S., Jong, M.S.Y., Istenic, A. et al. (2021). A Review of Artificial Intelligence in Education from 2010 to 2020. Complexity. Vol. 2021, doi: 10.1155/2021/8812542
48. Chan, C.K.Y., Hu, W. (2023). Students’ Voices on Generative AI: Perceptions, Benefits, and Challenges in Higher Education. International Journal of Educational Technology in Higher Education. Vol. 20, no. 1, article no. 43, doi: 10.1186/s41239-023-00411-8
49. Venkatesh, V., Davis, F.D. (2000). A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science. Vol. 46, no. 2, pp. 186-204, doi: 10.1287/mnsc.46.2.186.11926
50. Zhang, B., Dafoe, A. (2019). Artificial Intelligence: American Attitudes and Trends. Oxford, UK: Centre for the Governance of AI; Future of Humanity Institute, University of Oxford. Doi: 10.2139/ssrn.3312874
51. de Ayala, R.J. (2022). The Theory and Practice of Item Response Theory (2nd ed.). Methodology in the Social Sciences. New York: The Guilford Press. 643 p. Available at: https://www.guilford.com/books/The-Theory-and-Practice-of-Item-Response-Theory/R-de-Ayala/9781462547753 (accessed: 23.06.2025).
52. Robitzsch, A., Kiefer, T., Wu, M., Robitzsch, A.C. (2022). TAM: Test Analysis Modules (R package, ver. 4.1-0). Doi: 10.32614/CRAN.package.TAM
53. Revelle, W. (2023). Psych: Procedures for Psychological, Psychometric, and Personality Research (R package, ver. 2.3-6). Doi: 10.32614/CRAN.package.psych
54. Smith, E.V., Conrad, K.M., Chang, K., Piazza, J. (2002). An Introduction to Rasch Measurement for Scale Development and Person Assessment. Journal of Nursing Measurement. Vol. 10, no. 3, pp. 189-206, doi: 10.1891/jnum.10.3.189.52562
55. Reckase, M.D. (2009). Unidimensional IRT Models. In: Multidimensional Item Response Theory. New York: Springer. Pp. 17-30, doi: 10.1007/978-0-387-89976-3
56. Dunning, D., Heath, C., Suls, J.M. (2004). Flawed Self-Assessment: Implications for Health, Education, and the Workplace. Psychological Science in the Public Interest. Vol. 5, no. 3, pp. 69106, doi: 10.1111/j.1529-1006.2004.00018.x
57. Brantmeier, C., Vanderplank, R., Strube, M. (2012). What About Me?: Individual Self-Assessment by Skill and Level of Language Instruction. System. Vol. 40, no. 1, pp. 144-160, doi: 10.1016/j.system.2012.01.003
58. Fošner, A. (2024). University Students’ Attitudes and Perceptions Towards AI Tools: Implications for Sustainable Educational Practices. Sustainability. Vol. 16, no. 19, article no. 8668, doi: 10.3390/su16198668
59. Ananin, D.P., Komarov, R.V., Remorenko, I.M. (2025). When Honesty is Good, for Imitation is Bad”: Strategies for Using Generative Artificial Intelligence in Russian Higher Education Institutions.. Vysshee obrazovanie v Rossii = Higher Education in Russia. Vol. 34, no. 2, pp. 31-50, doi: 10.31992/0869-3617-2025-34-2-31-50 (In Russ., abstract in Eng.).
60. Wu, W., Zhang, B., Li, S., Liu, H. (2022). Exploring Factors of the Willingness to Accept AIAssisted Learning Environments: An Empirical Investigation Based on the UTAUT Model and Perceived Risk Theory. Frontiers in Psychology. Vol. 13, article no. 870777, doi: 10.3389/fpsyg.2022.870777
61. Buyakova, K.I., Dmitriev, Y.A., Ivanova, A.S., Feschenko, A.V., Yakovleva, K.I. (2024). Attitudes of Students and Instructors Towards the Use of Generative AI Tools in Higher Education. Obrazovanie i Nauka. Vol. 26, no. 7, pp. 160-193, doi: 10.17853/1994-5639-2024-7-160-193 (In Russ., abstract in Eng.).
62. Kuzminov, Ya.I., Kiryushina, M.A., Vorochkov, A.P., Kruchinskaya, E.V., Terentyev, E.A., Frumin, I.D. (2024). Nachalo kontsa ili novoy epokhi? Effekty generativnogo iskusstvennogo intellekta v vysshem obrazovanii [The Beginning of the End or a New Era? Effects of Generative AI in Higher Education]. Moscow: NIU HSE, Institute of Education Publ., 64 p. Available at: https://ioe.hse.ru/pubs/share/direct/991726366.pdf (accessed: 23.06.2025). (In Russ.).
Review
Views:
29
Email this article 