How to Design a National Aggregated Ranking?

The article presents a new method of designing a national ranking of higher education institutions on the basis of aggregating all existing public national rankings. The new aggregating method MetALeague (Method of Analysis of Leagues) can help to do away with monopoly in evaluation, thus, providing room for independent evaluation of HEIs. The analysis of the current state of procedures and methods of independent evaluation of Russian HEIs in the form of rankings and databases was carried out. The suggested MetALeague method is based on the mathematical tool of the voting theory in small groups and helps to combine different independent evaluations in a single format. A possible approach to creating such a unified national ranking in the form of 10 leagues is considered and the calculation results are presented. The developed ranking methods and the obtained results can be useful for self-diagnosis and promotion of research, organizational and methodological activities of HEIs, and also when introducing a risk-oriented approach in oversight activities.

1. Navodnov V.G., Motova, G.N., Ryzhakova O.E. (2019). The Method of League Analysis and Its Application in Comparing Global University Rankings and Russia’s University Performance Monitoring. Voprosy obrazovaniya =Educational Studies Moscow. No. 3, pp. 130-151. DOI:10.17323/1814-9545-2019-3-130-151 (In Russ., abstract in Eng.)

2. Bolotov, V.A., Valdman, I.A., Gorbovsky, R.V., Zakhir, Yu.S., Mertsalova, T.A. (2016). Kluchevye voprosy rasvitia natsionalnykh i regionalnykh system otsenki katchestva obrazova-nia (expertny obzor) [Key Issues of Development of National and Regional Systems of Education Quality Assessment (Expert Review)]. Moscow: Higher School of Economics Publ., 232 p. (In Russ.)

3. Bolotov, V.A. (2018). The Past, Present, and Possible Future of the Russian Education Assessment System. Voprosy obrazovaniya =Educational Studies Moscow. No. 3, pp. 287-297. DOI: https://doi.org/10.17323/1814-9545-2018-3-287-297

4. Larichev, O.I. (2000). Teoriya i metody prinyatiya resheniy, a takzhe Khronika sobytiy v Volshebnykh Stranakh: uchebnik [Theory and Methods of Decision Making, and The Chronicles of Fairy Lands: A Textbook]. Moscow: Logos Publ., 296 p. (In Russ.)

5. Petrovsky, A.B. (2009). Teoriya prinyatiya resheniy [Decision Theory]. Moscow: Akademiya Publ., 396 p. (In Russ.)

6. Balatskii, E.V. (2012). [Russian Practices in Rating the Effectiveness of University Programs]. Obshestvo i ekonomika = Society and Economy. No. 11, pp. 68-84. (In Russ.)

7. Navodnov, V.G., Ryzhakova, O.E. (2018). [The «First Mission» Ranking Was Created on the Basis of the Project «The Best Educational Programs of Innovative Russia»]. Akkreditatsiya v obmzovanii [Accreditation in Education]. No. 7 (107), pp. 50-64. (In Russ.)

8. Navodnov, V.G., Ryzhakova, O.E. (2018). [Russian Universities Were Divided into 14 Leagues of the «First Mission» Aggregated Ranking]. Akkreditatsiya v obrazovanii [Accreditation in Education]. No. 8 (108), pp. 50-57. (In Russ.)

9. Navodnov, V.G., Ryzhakova, O.E. (2018). [Generalization of the McKinsey Model to Rank Higher Educational Institutions with Equal Criteria]. Vestnik Povolzhskogo gosudarstvennogo tekh-nologicheskogo universiteta. Ser. Ekonomika i upravlenie = Vestnik of Volga State University of Technology. Series “Economics and Management”, No. 2 (38), pp. 5-18. DOI: https://doi.org/10.15350/2306-2800.2018.2.5 (In Russ.)

10. Matveeva, O.A. (2019). Development of the Voluntary Accreditation for Study Programmes in Russia. Vysshee obrazovanie v Rossii = Higher Education in Russia.. Vol. 28, no. 7, pp. 19-28. DOI: https://doi.org/10.31992/0869-3617-2019-28-7-19-28 (In Russ., abstract in Eng.)

11. Gaisenok, V.A., Naumovich, O.A., Samokhval, V.V. (2018). Correlational Relationships between International Rankings for Higher Education Institutions. Vysshee obmzovanie v Rossii = Higher Education in Russia. Vol. 27, no. 12, pp. 20-28. DOI: https://doi.org/10.31992/0869-3617-2018-27-12-20-28 (In Russ., abstract in Eng.)

12. Zadorozhnyuk, I.E., Korosteleva, L.Yu., Tebiyev, B.K. (2019). TOP-200 Higher Education Institutions in Four International Ratings: Comparative Analysis. Vysshee obrazovanie v Rossii = Higher Education in Russia. Vol. 28, no. 3, pp. 85-95. DOI: https://doi.org/10.31992/0869-3617-2019-28-3-85-95 (In Russ., abstract in Eng.)