Methodology

The INNO-Appraisal project was finalised in February 2010. The project final report is presented below:

This exercise applies a meta-evaluation approach, or to be more precise, a secondary analytical approach, building on existing data and qualifying it through expert assessment. The feedback to policy-makers is a crucial element, as this will reflect the perceived usefulness and the actual consequences of evaluation exercises. Data collection is achieved through the use of templates that are transferred to a database. This database allows multi-variate analysis as well as in-depth case studies along various analytical dimensions (countries, types of policies, evaluation methodologies used, intended role of evaluation, budgets, etc.). The whole collection and storage process is described in a data collection manual that is binding for all participants and which allows for transparency and validity of data.

The template is designed to be first prefilled by our own team and then sent to the respective policy-maker responsible for the related policy measure. This process is intended to minimise the effort from the policy-makers while still utilising their valuable insight concerning the measures and appraisals. The structure of the template is as follows:

  • Cover page: basic information on the project and instructions on how to fill in the template (for the policy makers)
  • Part A: Information about the policy maker responsible for the measure
  • Part B: Short information about the corresponding policy measure
  • Part C: Basis characteristics of the Appraisal:
    Who conducted it, tender procedure, timing and purpose, costs, reason for the appraisal, topics covered, impacts analysed, sponsor, data analysis and collection methods used, dissemination policies, main audience
  • Part D: Quality of the Appraisal:
    Assessment of the methods, data and analysis (starting with terms of reference)
  • Part E: Conclusions and Recommendations of the Appraisal
  • Part F: Dissemination and Implementation of the Appraisal Results  
In the second phase of the project, case studies are developed from the basis of the data collected. A more qualitative approach is also utilised by conducting various interviews with policy makers, evaluators and other stakeholders.