OECD Develops Methodology for Evaluating Anti-Fraud Strategies

The document was prepared with financial support from the European Union under the Technical Support Instrument.

The methodology is intended primarily for bodies responsible for co-ordinating the implementation, monitoring and evaluation of anti-fraud strategies, including Anti-Fraud Coordination Services in EU Member States. At the same time, the proposed approach can be applied to strategies adopted at the national, regional, sectoral and organisational levels.

According to the OECD, countries are increasingly adopting dedicated strategic documents in this area. Of the 63 countries examined, seven have comprehensive national anti-fraud strategies in force, 19 have strategies adopted by individual ministries or agencies, and a further ten have sectoral strategies.

However, the adoption of a strategy is not always accompanied by an effective framework for evaluating its implementation. Among the 46 countries with strategic objectives relating to anti-corruption and public integrity, fewer than half use outcome-level indicators and conduct evaluations to determine the actual effects of the measures taken. Only 37% of countries formally provide for an end-of-term evaluation as a separate activity.

Monitoring and Evaluation

The OECD recommends establishing monitoring and evaluation mechanisms at the strategy design stage. This requires defining indicator baselines, formulating measurable objectives and establishing a logical link between planned measures and expected results.

The document identifies four groups of indicators:

inputs, including funding, personnel and time;
outputs, such as the adoption of documents or the delivery of training;
outcomes, such as improved inter-agency co-operation or increased use of reporting channels;
impacts, including the development of an anti-fraud culture, stronger fraud-detection capabilities and increased trust in public authorities.

The authors caution against treating the number of detected or reported violations as an unambiguous measure of effectiveness. An increase in detected cases or reports may indicate not that the situation has deteriorated, but that detection mechanisms have improved and confidence in reporting channels has grown. Conversely, focusing on reducing this indicator may create incentives to conceal violations or restrict the acceptance of reports.

Quantitative data should therefore be supplemented by surveys, expert assessments, case reviews and other qualitative information. Perception-based indicators should not be used in isolation either: high-profile cases may worsen public perceptions, even where the increase reflects more effective detection and investigation of violations or greater transparency.

How Strategies Should Be Evaluated

The OECD proposes using six criteria developed by its Development Assistance Committee:

relevance;
coherence;
effectiveness;
efficiency;
impact;
sustainability.

Evaluations may be conducted before implementation begins, at the midpoint of the strategy cycle and after completion. A mid-term evaluation makes it possible to adjust measures, deadlines and resource allocation in a timely manner, while an end-of-term evaluation helps incorporate lessons learned into the preparation of the next strategic document.

Particular attention is paid to assessing the financial effects of anti-fraud measures. The proposed methods include estimating expected losses, comparing results before and after the introduction of new controls, using control groups and calculating the relationship between costs and benefits. However, financial indicators should be considered alongside non-financial effects, such as stronger control systems, increased trust and changes in organisational culture.

Allocation of Responsibilities

The OECD recommends empowering a central co-ordinating body to collect and verify data, prepare reports and organise strategy reviews. Authorities directly responsible for implementing measures should appoint designated staff and regularly report on implementation progress.

At the same time, implementation data should not be based exclusively on self-assessments by the responsible authorities. To improve objectivity, the methodology recommends independent validation of results by the co-ordinating body, another public authority or an external organisation.

Civil society organisations, academic institutions, the media, business representatives and citizens should also be involved in monitoring and evaluation. Such mechanisms can provide additional information on emerging risks and implementation challenges, while also strengthening transparency and public trust.

The OECD also draws particular attention to the content of awareness-raising campaigns. Messages that overemphasise the prevalence of fraud may create the impression that violations are common practice or that combating them is futile. Communication should therefore be specific, credible and linked to concrete measures and accountability mechanisms.

The report is supplemented by practical tools, including a sample evaluation questionnaire, a catalogue of outcome and impact indicators, and a checklist for updating anti-fraud strategies and action plans.