The review of literature undertaken as part of the inception phase of LASER highlighted the lack of rigorous impact assessment of donor investments in commercial law and justice reforms. There is very little robust evidence that such reforms bring about improved business and economic performance, for example in terms of employment, investment, profitability, etc. Given LASER’s mandate to support evidence based decision making in CLJ reform DFID included the design and delivery of two rigorous impact evaluations (RIE) in the programme. RIE’s would provide data that could enable us to determine the extent to which a (completed) reform programme actually caused observed changes.
LASER together with the World Bank and IFC identified an opportunity to undertake a RIE of the International Finance Corporation (IFC) funded Alternative Dispute Resolution (ADR) scheme for tax disputes in Bangladesh.
It was envisioned that LASER would conduct an 18 month rigorous impact evaluation (RIE) of the ADR scheme, and that the evaluation would generate new evidence about the impact of introducing ADR mechanisms in the resolution of taxation disputes, as well as explore questions on cost-effectiveness and demand for ADR in tax disputes, and contribute to a limited but growing body of RIEs of justice interventions.
During the initial scoping for the RIE, LASER identified two key risks which could potentially impact on the success of the programme. After appropriate mitigation and monitoring activities, it was decided to go ahead with the RIE, and the inception phase was completed. It soon became clear however that three key challenges remained. Nearly three months after the inception report was completed it was evident that further delays were likely, and that LASER would, as a result, struggle to complete the evaluation within the lifetime of the programme, and to the level of rigour required.
At the centre of LASER’s approach is two principles: 1) learning about what works and what doesn’t, and 2) adapting as a result, changing what we do and how we do it to ensure the greatest impact. It became clear that the resources allocated to the RIE could be better used elsewhere in the programme. DFID, LASER and the research implementing partner jointly agreed to halt the RIE. For more information on an adaptive approach, click here
One of the guiding principles of an adaptive approach is to be open about what is working and what is not, and to scale up – or at times close down – those initiatives that doesn’t appear likely to achieve the desired impact. Although it was a difficult decision to withdraw support to the RIE, it provided both LASER and DFID with practical experience of how to operate in a way that was aligned with the spirit of adaptive programming.