Strategic Studies of public policy

Strategic Studies of public policy

Developing Evaluation Indicators for Science and Technology Policies: An Application of the Logical Framework Approach

Document Type : Research Paper

Author
National Research Institute for Science Policy of Iran
Abstract
In the present study, an evaluation framework for science and technology policies was developed and validated using a logical approach tailored to the policy context of Iran in the field of science and technology. The framework was weighted based on expert opinions. This research is applied in purpose and employed both qualitative and quantitative methods for data collection, thus falling under the category of mixed-methods research. Specifically, the evaluation framework was developed, validated, and prioritized through a review of existing models, analysis of upstream policy documents, and expert surveys. The findings revealed that the main dimensions of evaluation include: Input Stage: Financial resources, human resources, infrastructure, access to technology, laws and regulations, and institutional capacities. Process Stage: Coordination and participation, resource allocation, monitoring and evaluation, planning and policymaking, commercialization and technology transfer, capacity building and training, science diplomacy, and licensing. Output Stage: Financial resources, human resources, infrastructure, access to technology, laws and regulations, and institutional capacities. Outcome Stage: Economic, scientific and technological, educational and academic, institutional and infrastructural, environmental, self-sufficiency and competitiveness, and international rankings. Impact Stage: Economic, social, scientific and technological, cultural, environmental, and international. Finally, a Key Connections Table (origin, destination, causal relationship, and type of influence) was presented based on the logical framework. This table not only serves as an evaluation framework but also as a strategic management tool, helping policymakers make evidence-based decisions. It also enables institutions to demonstrate accountability for national science and technology policies and programs
Keywords
Subjects

منابع فارسی
آزادی احمدآبادی، قاسم. (1404). ابعاد و شاخص‌های ارزیابی سیاست‌های علم و فناوری براساس چرخه سیاستگذاری. سیاستگذاری عمومی، 11(4)، 150-169.
 doi: 10.22059/jppolicy.2025.105205 
اسفندیاری، نیما، مرادی، محمود، رمضانیان، محمدرحیم و ابراهیم‌پور ازبری، مصطفی. (1403). تحلیل زیست‌بوم نوآوری ایران از سیاست‌گذاری تا عمل: رویکرد مبتنی بر تحلیل تم. سیاست نامه علم و فناوری، 14(2)، 59-78.
انتظاری، یعقوب. (1401). الزامات توسعه زیست‌بوم کارآفرینی دانشگاه بنیان در ایران. فصلنامه پژوهش و برنامه‌ریزی در آموزش عالی، 25(1)، 1-25.
بوشهری، علیرضا؛ باقری، ابوالفضل (1395). ارزیابی سیاست‌های علم و فناوری: موردکاوی به‌کارگیری نخبگان وظیفه در پروژههای پژوهشی. بهبود مدیریت، 3(33): 107-129.
جاهد، حسینعلی، آراسته، حمیدرضا و جعفری، پریوش. (1390). تعیین و تبیین عوامل فردی مؤثر در تجاری‌سازی نتایج پژوهشی؛ مطالعه موردی دانشگاه آزاد اسلامی واحد علوم و تحقیقات. سیاست علم و فناوری، 4(3)، 1-17.
حیدری، حسن. (1386). نظام ملی نوآوری به‌عنوان چارچوبی برای تحلیل نوآوری: رویکردی نظری. پژوهش‌های اقتصادی ایران، 9(33)، 129-163.
خمسه، عباس و علیمرادیان، محیا. (1389). ویژگی و اهمیت مراکز توسعه و نوآوری و مدیریت آنها. فصلنامه توسعه تکنولوژی صنعتی، 8(15)، 25-32.
ریاحی، پریسا و دانایی‌فرد، حسن. (1398). سیاست‌های علم و فناوری برای تقویت نظام‌های منطقه‌ای نوآوری. سیاست علم و فناوری، 12(2)، 193-208.
سلطانی، علیمحمد؛ طباطبائیان، سیدحبیبالله. (1398). ارزیابی سیاست‌های علم، فناوری و نوآوری. سیاست علم و فناوری، 11(2)،560-578.
شجاعی، محمدحسین؛ فاتحراد، مهدی؛ طباطبائیان، سیدحبیبالله. (1393). طرحی برای نظام ارزیابی سیاست‌های علم، فناوری و نوآوری. رهیافت، 24(57)، 1-15.
عباسی، محمد. (۱۳۹۳). پ‍ی‍ش‍ن‍ه‍ادی‍ه‌ ان‍ج‍ام‌ پ‍روژه‌ طرح‌ ک‍لان‌ رص‍د شاخص‌ها و اول‍وی‍ت‌ه‍ای‌ ع‍ل‍م‌ و ف‍ن‍اوری‌ در ای‍ران. تهران: پ‍ژوه‍ش‍گ‍اه‌ ع‍ل‍وم‌ و ف‍ن‍اوری‌ اطلاع‍ات‌ ای‍ران.‌
کلانتری، اسماعیل؛ منتظر، غلامعلی؛ قاضی نوری، سیدسپهر. (1400). ارزیابی حسن اجرای سیاست‌های راهبردی و کلان علم و فناوری در ایران مبتنی بر رویکرد شبکه سیاستی، سیاست‌های راهبردی و کلان، 9(36)، 712-746.
 magiran.com/p2397637
مرادی پور، حجت اله، حاجیانی، ابراهیم و خلیفه سلطانی، حشمت. (1396). رهیافتی به پیامدهای سیاست گذاری علم و فناوری در ایران براساس تحلیل اسناد بالادستی. پژوهش در نظام‌های آموزشی، 11(37)، 151-178. 
doi: 10.22034/jiera.2017.57770
ملک زاده، رضا. (1401). نقش همکاری‌های علمی بین‌المللی در توسعه علمی کشور. فصلنامه پژوهش و برنامه‌ریزی در آموزش عالی، 8(3)، 63-78. 
وزیری، اسماعیل و صبوری، علی اکبر. (1400). تعاملات علمی بین‌المللی، پیشران دیپلماسی علم و فناوری. نشاء علم، 12(1)، 13-20.
Refrences
Bell, M., & Pavitt, K. (1993). Technological Accumulation and Industrial Growth: Contrasts Between Developed and Developing Countries. Industrial and Corporate Change, 2, 157-210.
Castellacci, F. (2023). Innovation and social welfare: A new research agenda. Journal of Economic Surveys, 37(4), 1156-1191. https://doi.org/10.1111/joes.12537
Chávez, J. C. R., & Chávez, J. C. L. N. (2013). A Science and Technology Policy Model to Support Regional Innovation Systems. In Revista Nicolaita de Estudios Económicos (Vol. 6, Issue 2, pp. 73–86). Instituto de Investigaciones Economicas y Empresariales. https://doi.org/10.33110/rnee.v6i2.137
Cooke, P. (2001). Regional innovation systems, clusters, and the knowledge economy. Industrial and corporate change, 10(4), 945-974.
Duan, Y., Jia, Z., & Wang, M. (2020, November). Minimax-optimal off-policy evaluation with linear function approximation. In International Conference on Machine Learning (pp. 2701-2709). PMLR.
Etzkowitz, H. (2006). The new visible hand: an assisted linear model of science and innovation policy. Science and public policy, 33(5), 310-320.
Etzkowitz, H., & Leydesdorff, L. (2000). The dynamics of innovation: From National Systems and “Mode 2” to a Triple Helix of university–industry–government relations. Research Policy, 29(2), 109–123.  
Godin, B. (2006). The linear model of innovation: The historical construction of an analytical framework. Science, Technology, & Human Values, 31(6), 639-667.
Hautamäki, A., Turkki, T., Brandenburg, C., & Areblad, C. (2010). Creative economy and culture in the innovation policy. Publications of the Ministry of Education, Finland [online], Available from Internet: http://www. minedu. fi/export/sites/default/OPM/Julkaisut/2010/liitteet/OPM13. pdf.
Jordan, G. (2010). A theory-based logic model for innovation policy and evaluation. Research Evaluation, 19(4), 263–273. https://doi.org/10.3152/095820210X12827366906445
Keršuliene, V., Zavadskas, E. K., & Turskis, Z. (2010). Selection of rational dispute resolution method by applying new step‐wise weight assessment ratio analysis (SWARA). Journal of business economics and management, 11(2), 243-258. http://dx.doi.org/10.3846/jbem.2010.12
Kim, J., & Yoo, J. (2019). Science and technology policy research in the EU: from framework programme to HORIZON 2020. Social Sciences, 8(5), 153.
Kim, Y. K., Lee, K., Park, W. G., & Choo, K. (2012). Appropriate intellectual property protection and economic growth in countries at different levels of development. Research policy, 41(2), 358-375.
Kwiek, M. (2020). What large-scale publication and citation data tell us about international research collaboration in Europe: changing national patterns in global contexts. Studies in Higher Education, 46(12), 2629–2649. https://doi.org/10.1080/03075079.2020.1749254
Ma, N. (2024). Evaluation of the Implementation of Baotou’s Science and Technology Talent Policies within the Context of Comprehensive Model Autonomous Region Development. Occupation and Professional Education, (1)4, 17-23. DOI: https://doi.org/10.62381/O242404
Machado, D., Qu, Y., & Cervantes, M. (2019). Innovation policies for sustainable development: Low-carbon energy and smart-city initiatives. https://www.oecd.org/content/dam/oecd/en/publications/reports/2019/10/innovation-policies-for-sustainable-development_f616c02b/6287ddb2-en.pdf
Molas-Gallart, J., & Davies, A. (2006). Toward theory-led evaluation: The experience of European science, technology, and innovation policies. American Journal of Evaluation, 27(1), 64-82. https://doi.org/10.1177/1098214005281701
Neupane, B. (2015). A more developmental approach to science. Unesco Science Report. Towards, 2030, 6-8. https://unesdoc.unesco.org/ark:/48223/pf0000235442
Newcomer, K. E., Hatry, H. P., & Wholey, J. S. (Eds.). (2015). Handbook of practical program evaluation (pp. 1-864). San Francisco, CA: Jossey-Bass & Pfeiffer Imprints, Wiley.
OECD (2012). OECD Science, Technology and Industry Outlook 2012, OECD Publishing, Paris, https://doi.org/10.1787/sti_outlook-2012-en
OECD. (2019). Science, Technology and Innovation Policies for Inclusive Growth. Organisation for Economic Co-operation and Development.
Olinto, G. (2009). Human resources in science and technology indicators: longitudinal evidence from Brazil. In Proceedings of ISSI (pp. 359-368).
Oliveira, M. B. D. (2014). Technology and basic science: the linear model of innovation. Scientiae Studia, 12, 129-146.
Organisation for Economic Co-operation and Development. (2007). Integrating science & technology into development policies: an international perspective. OECD Publishing.
production. United Nations Conference on Trade and Development.  United Nations Publications. https://unctad.org/system/files/official-document/dtltikd2023d2_en.pdf
Reynolds, J. L., Kennedy, E. B., & Symons, J. (2023). If deliberation is the answer, what is the question? Objectives and evaluation of public participation and engagement in science and technology. Journal of Responsible Innovation, 10(1), 2129543.
Rodríguez, J. C., & Navarro-Chávez, C. L. (2015). A system dynamics model of science, technology and innovation policy to sustain regional innovation systems in emerging economies. International Journal of Innovation and Regional Development, 6(1), 7-30.
Sanderson, I. (2000). Evaluation in Complex Policy Systems. Evaluation, 6, 433-454.
Shahabadi, A., & Heidarian, M. (2024). The relationship between human development indicators and technology spillover with environmental quality: Evidence from panel ARDL model for D8 countries. Social Indicators Research, 174(1), 421-444.
Silvestre, B. S., & Ţîrcă, D. M. (2019). Innovations for sustainable development: Moving toward a sustainable future. Journal of cleaner production, 208, 325-332. 
Suparwata, D. O. (2024). The Role of Social and Technological Innovation in Enhancing Economic Well-being and Empowering Rural Communities. Jurnal Terobosan Peduli Masyarakat (TIRAKAT), 1(3), 182-191. DOI: https://doi.org/10.61100/j.tirakat.v1i3.219
Takalo, T., Tanayama, T., & Toivanen, O. (2008). Evaluating innovation policy: a structural treatment effect model of R&D subsidies. Bank of Finland Research Discussion Paper, (7).
UNCTAD. (2023). Technology and innovation for cleaner and more productive and competitive. Commission on Science and Technology for Development, Twenty-sixth session, Geneva, 27–31 March 2023. https://unctad.org/system/files/official-document/ecn162023d2_en.pdf
UNCTAD. (2024). Global cooperation in science, technology and innovation for development. Commission on Science and Technology for Development Twenty-seventh session. Geneva, 15–19 April 2024.  https://unctad.org/system/files/official-document/ecn162024d3_en.pdf
Xu, S., Lian, G., Song, M., & Xu, A. (2024). Do global innovation networks influence the status of global value chains? Based on a patent cooperation network perspective. Humanities and Social Sciences Communications, 11(1), 1-12. http://dx.doi.org/10.1057/s41599-024-03413-7
Yong’an, Z., Zhe, G., & Jie, T. (2016). China’s regional science and technology innovation policy: A classification based on data from Zhongguancun science park. International Journal of Innovation Science, 8, 71-88.
Zhamankaraev, Zh. (2021). THE PLACE OF SANCTIONS IN THE MODERNIZATION OF NATIONAL ECONOMIC POLICY. BULLETIN Series of Sociological and Political Sciences, 74(2), 51–55. https://doi.org/10.51889/2021-2.1728-8940.08
БАРКОВ, А. В., & ЗАВЬЯЛОВ, М. М. (2022). THE CONCEPT OF MANAGING STRATEGIC ENTERPRISES   UNDER FOREIGN SANCTIONS   WITHIN THE FRAMEWORK OF TECHNOLOGICAL SOVEREIGNTY. The Rule-of-Law State: Theory and Practice, 18(3(69)), 34–40. https://doi.org/10.33184/pravgos-2022.3.5