International scholarship schemes for higher education are widely supported by governments, supranational bodies, and charitable organisations as part of both public diplomacy and developmental assistance commitments. Although the objectives of many scholarship schemes have evolved over the period of their administration, commitment to scholarships remains strong. Increasingly, scholarship providers have invested time and resources into the evaluation of scheme outcomes, tracing alumni and examining how their experiences post-scholarship reflect progress toward the policy objectives of scholarship programmes. Despite the elevated importance of evaluation practices to both understanding outcomes and securing renewed funding, surprisingly little analysis has been conducted of evaluation practices currently employed across the scholarships ‘sector’. This scoping study was undertaken by the CSC in an attempt to identify the main trends and ambiguities in evaluation approaches and to analyse where different, or more detailed, methodology might lead to more robust evaluation. Four aspects of the available documentation were analysed: methodology, methods, variables, and data analysis. Three thematic issues – counterfactuals, value for money, and aid harmonisation – were also considered. In addition to descriptive findings, the scoping study offers more speculative conclusions regarding the ambiguities in the sector that might be addressed in order to strengthen evaluation methodology. Although some reference is made to research literature on evaluation methodology, this study has been designed to address current – not ‘best’ – practice. The results of the scoping study are thus intended to offer both an insight into the general trends in evaluation methodology within the sector and provide an opportunity for dialogue and cooperation going forward.
A study of research methodology used in evaluations of international scholarship schemes for higher education
DIS, Research & Working Papers
