The Impact of AI Tools on the Development of Foreign Language Skills and Digital Competence: A Longitudinal Study and Assessment Matrix

The article presents the results of a two‑year longitudinal study (2023–2025) conducted with a sample of 50 first- and second‑year students of Moscow State Institute of International Relations (17 German‑track students and 33 English‑track students). Drawing on a triangulation of methods (standardized CEFR‑based language proficiency tests, digital competence questionnaires, analysis of student portfolios, semi‑structured interviews with 10 teachers and 20 students, and reflective narratives), the authors examined the impact of the purposeful integration of artificial intelligence (AI) tools on the development of foreign language skills and digital competence. All participants had achieved at least a B2 level according to their Unified State Exam (≥75 points), which reflects the selective nature of the institution. The results show a significant improvement in all four language skills (speaking: Cohen’s d = 0.97, writing: d = 0.70, reading: d = 0.54, listening: d = 0.48, all p < 0.01). Qualitative analysis revealed a three‑stage evolution of students’ value‑laden attitudes towards AI: from techno‑scepticism through pragmatism to an integrative understanding. The authors developed a matrix of digital AI competence comprising four components (cognitive, operational, affective and axiological) and three developmental levels (novice, practitioner, expert). The study found that students with higher digital competence demonstrate significantly greater improvement in language skills (correlation with speaking r = 0.42, p = 0.002). Four patterns of AI use were identified and classified; 71% of students use AI responsibly. The authors argue that value‑laden attitudes towards AI are a critical precondition for the effective use of technology in the training of future international affairs professionals. In practical terms, the competence matrix can serve as a tool for assessment, course design and teacher training in the context of Education 4.0.

1. Titova, S.V. (2024). Technological Solutions Based on Artificial Intelligence in Teaching Foreign Languages: An Analytical Review. Vestnik Moskovskogo universiteta. Seriia 19. Lingvistika i mezhkul’turnaia kommunikatsiia = Moscow University Bulletin. Series 19. Linguistics and Intercultural Communication. Vol. 27, no. 2, pp. 18-37, doi: 10.55959/MSU-2074-1588-19-27-2-2. (In Russ., abstract in Eng.).

2. Sysoev, P.V., Filatov, E.M. (2024). Artificial Intelligence in Teaching Russian as a Foreign Language. Rusistika = Russian Language Studies. Vol. 22, no. 2, pp. 300-317, doi: 10.22363/2618- 8163-2024-22-2-300-317. (In Russ., abstract in Eng.).

3. Ivakhnenko, E.N., Nikol’skii, V.S. (2023). ChatGPT in Higher Education and Science: a Threat or a Valuable Resource? Vysshee obrazovanie v Rossii = Higher Education in Russia. Vol. 32, no. 4, pp. 9-22, doi: 10.31992/0869-3617-2023-32-4-9-22. (In Russ., abstract in Eng.).

4. Shubina, Iu.V. (2024). Teaching Foreign Language Vocabulary in Universities: Artificial Intelligence as a Teaching Assistant in Russian Universities. Upravlenie obrazovaniem: teoriia i praktika = Education Management Review. Vol. 14, no. 2-2, pp. 87-95, doi: 10.25726/z9377-6937-4106-o. (In Russ., abstract in Eng.).

5. Evstigneev, M.N. (2024). Principles of Foreign Language Teaching Based on Artificial Intelligence Technologies. Vestnik Tambovskogo universiteta. Seriia: Gumanitarnye nauki = Tambov University Review. Series: Humanities. Vol. 29, no. 2, pp. 309-323, doi: 10.20310/1810-0201-2024-29-2-309-323. (In Russ., abstract in Eng.).

6. Ferrari, A. (2013). DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe. JRC Scientific and Technical Reports EUR 25351 EN. Luxembourg : Publications Office of the European Union, 45 p., doi: 10.2788/52966.

7. Carretero, S., Vuorikari, R., Punie, Y. (2017). DigComp 2.1: The Digital Competence Framework for Citizens with Eight Proficiency Levels and Examples of Use. EUR 28558 EN. Luxembourg : Publications Office of the European Union, 48 p., doi: 10.2760/38842.

8. Raitskaya, L., Tikhonova, E. (2024). Appliances of Generative AI-powered Language Tools in Academic Writing: A Scoping Review. Journal of Language and Education. Vol. 10, no. 4, pp. 5-30, doi: 10.17323/jle.2024.24181.

9. Kizrina, N.G., Levina, E.A., Vishlenkova, S.G. (2022). Formation of the Subject Digital Competence of Future Foreign Language Teacher. Perspektivy nauki i obrazovaniia = Perspectives of Science and Education. No. 2 (56), pp. 183-199, doi: 10.32744/pse.2022.2.11. (In Russ., abstract in Eng.).

10. Kuvshinova, E.E. (2024). Application of Artificial Intelligence in Teaching a Foreign Language. Gumanitarii Iuga Rossii = Humanities of the South of Russia. Vol. 13, no. 2, pp. 75-84, doi: 10.18522/2227-8656.2024.2.7. (In Russ., abstract in Eng.).

11. Voronina, D.K., Shamov, A.N. (2025). Generative Artificial Intelligence in Language Education: Ways to Solve the Plagiarism Problem. Vestnik Mininskogo universiteta = Vestnik of Minin University. Vol. 13, no. 3, p. 5, doi: 10.26795/2307-1281-2025-13-3-5. (In Russ., abstract in Eng.).

12. Warschauer, M. (2003). Technology and Social Inclusion: Rethinking the Digital Divide. Cambridge, Massachusetts : The MIT Press, 275 p. ISBN: 9780262731737.

13. North B., Piccardo E., Goodier T. (2020). Council of Europe (2020). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Companion volume. Strasbourg : Council of Europe Publishing, 278 p. ISBN: 978-92-871-8621-8.

14. Kern, R.G., Warschauer, M. (2000). Theory and Practice of Network-Based Language Teaching. In: Warschauer M., Kern R. (Eds.). Network-Based Language Teaching: Concepts and Practice. Cambridge : Cambridge University Press, pp. 1-19. ISBN: 0-521-66136-6.

15. Braun, V., Clarke, V. (2006). Using Thematic Analysis in Psychology. Qualitative Research in Psychology. Vol. 3, no. 2, pp. 77-101, doi: 10.1191/1478088706qp063oa.

16. Huang, X., Zou, D., Cheng, G., Chen, X., Xie, H. (2023). Trends, Research Issues and Applications of Artificial Intelligence in Language Education. Educational Technology & Society. Vol. 26, no. 1, pp. 112-131, doi: 10.30191/ETS.202301_26(1).0009.

17. Cotton, D.R.E., Cotton, P.A., Shipway, J.R. (2024). Chatting and Cheating: Ensuring Academic Integrity in the Era of ChatGPT. Innovations in Education and Teaching International. Vol. 61, no. 2, pp. 228-239, doi: 10.1080/14703297.2023.2190148.

18. Sullivan, M., Kelly, A., McLaughlan, P. (2023). ChatGPT in Higher Education: Considerations for Academic Integrity and Student Learning. Journal of Applied Learning & Teaching. Vol. 6, no. 1, pp. 1-10, doi: 10.37074/jalt.2023.6.1.17.