Strategic Studies of public policy

Strategic Studies of public policy

Synthesizing the integrated performance evaluation framework of the Ministry of Science, Research and Technology with a functional approach

Document Type : Research Paper

Authors
1 Department of Science, Technology and Innovation Policy Evaluation, National Research Institute of Science Policy
2 Department of Technology and Innovation Policy, National Research Institute of Science Policy
3 Assistant Professor, Faculty of Management and Accounting, Farabi Faculties, University of Tehran
Abstract
Obtaining feedback from the results and achievements of the science, technology, and innovation systems, especially at the national level, is a requirement for any policy system's long-term success. The Ministry of Science, Research and Technology (MSRT), the most vital player in Iran's science, technology, and innovation system, needs to have such a feedback mechanism based on its priority functions of the science, technology, and innovation system, obtaining the best performance.

For this purpose, in this research, using the methodology of framework synthesis, an integrated and comprehensive framework for evaluating the performance of the Ministry of Science, Research and Technology based on the selection of 40 key indicators has been presented and validated using the expert panel method. The results of applying the proposed evaluation framework should be adapted to the goals and policies of the national STI system. Based on the latest statistics and information available (from September to March 2022), the performance of the MSRT in the evaluation quadrants is lower than average. Also, we offered some policy recommendations focused on improving key indicators' performance.
Keywords

Subjects


References
Aitsi-Selmi, A., Murray, V., Wannous, C., Dickinson, C., Johnston, D., Kawasaki, A., ... & Yeung, T. 2016. Reflections on a science and technology agenda for 21st century disaster risk reduction. International Journal of Disaster Risk Science, 7(1), 1-29.
Barker, K. 2007. The UK Research Assessment Exercise: the evolution of a national research evaluation system. Research evaluation, 16(1), 3-12. 
Brunton, G., Oliver, S., & Thomas, J. (2020). Innovations in framework synthesis as a systematic review method. Research synthesis methods, 11(3), 316-330.‏ 
Canberra Manual. 1995. The measurement of scientific and technological activities. Manual on the measurement of human resources devoted to S&T. Brussels, Luxembourg.
Carroll, C., Booth, A., & Cooper, K. (2011). A worked example of” best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents. BMC medical research methodology, 11(1), 1-9. doi:https://doi.org/10.1186/1471-2288-11-29
Chelimsky, E. (2006). The purposes of evaluation in a democratic society. The Sage handbook of evaluation, 33-55. 
Chen, J., Heng, C. S., Tan, B. C., & Lin, Z. (2018). The distinct signaling effects of R&D subsidy and non-R&D subsidy on IPO performance of IT entrepreneurial firms in China. Research Policy, 47(1), 108-120.
Coryn, C. L., Hattie, J. A., Scriven, M., & Hartmann, D. J. (٢٠٠٧). Models and Mechanisms for Evaluating Government-Funded Research An International Comparison. American Journal of Evaluation, ٢٨(٤), ٤٣٧-٤٥٧.
Cruz-Castro, L., & Sanz-Menéndez, L. (2007). Research Evaluation in transition: individual versus organisational assessment in Spain. The Changing Governance of the Sciences: The Advent of Research Evaluation Systems, 205-223. 
European Environment Agency (2017). EEA guidance document-policy evaluation.
Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development, Organization for Economic Co-operation and Development (OECD).
Frederiksen, L. F., Hansson, F., & Wenneberg, S. B. 2003. The Agora and the role of research evaluation. Evaluation, 9(2), 149-172.‏ 
Gasper, D. (2018), Policy Evaluation: From Managerialism and Econocracy to a Governance Perspective, in International Development Governance, Routledge.
Gibbons, M., & Georghiou, L. (1987). Evaluation of Research. A Selection of Current Practices. OECD Publications Service, Sales and Distribution Division, 2, rue Andre-Pascal, 75775 Paris, France.
GII 2022 theme: Energizing the World with Innovation. 2022. World Intellectual Property Organization (WIPO).
Hackett, P. (1997). Introduction to training. Universities Press. 
Jordan, B. (2008). Welfare and well-being: Social value in public policy. Policy Press. 
Molas-Gallart, J. 2012. Research governance and the role of evaluation: A comparative study. American Journal of Evaluation, 33(4), 583-598.‏ 
OECD-DAC (2002). Glossary of Key Terms in Evaluation and Results Based Management, https://www.oecd.org/dac/evaluation/18074294.pdf (accessed on 18 June 2023).
OECD (2018). Draft Policy Framework on Sound Public Governance, http://www.oecd.org/gov/draft-policy-framework-on-sound-public-governance.pdf (accessed on 8 July 2023).
OECD Patent Statistics Manual. 2009. Organization for Economic Co-operation and Development (OECD).
Oh, D. S., & Phillips, F. (2014). Technopolis. Springer.
Oslo Manual 2005: Guidelines for Collecting and Interpreting Innovation Data, Organization for Economic Co-operation and Development (OECD).
Smismans, S. (2015). Policy Evaluation in the EU: The Challenges of Linking Ex Ante and Ex Post Appraisal, Symposium on Policy Evaluation in the EU.  http://dx.doi.org/10.1017/S1867299X00004244.
Technology Balance of Payments (TBP) Manual. 1999. Organization for Economic Co-operation and Development (OECD).
UNESCO Science Report 2010. The Current Status of Science around the World. Paris: UNESCO, 2010.