How Do We Know If Justice Interventions Are Working? The New OECD Measurement Framework Offers an Answer

This is the question at the heart of the recently published working paper of the Organisation for Economic Co-operation and Development (OECD) on public governance: Towards a measurement framework for people-centred justice systems. This paper is a contribution from the OECD to the Justice Action Coalition’s Workstream on Data and Evidence.

While this framework may sound quite technical, it is in fact a very practical tool to help governments understand whether their justice systems are delivering the outcomes people really need. The framework begins with a shift in the point of view. Instead of looking at justice primarily through the lens of the institutions that deliver it, the OECD—in line with the principles of people-centered justice—invites us to look at justice through the experience of the people who need it.

One of the framework’s most important contributions is what it chooses to measure. Its proposed outcome indicators focus on whether people’s justice problems are resolved, whether people trust their justice system, and whether people can ultimately obtain the information, assistance and dispute-resolution mechanisms they need. The result is a much more nuanced and holistic picture of justice.

This approach is a significant departure from measuring justice primarily through institutional activity. It reinforces the importance of legal needs surveys and user experience data. Such data can reveal problems that administrative statistics never see, including those that never enter the formal justice system, as well as the barriers people encounter along the way.

The result is a much more nuanced and holistic picture of justice. Most justice problems never reach a court. People resolve problems, or fail to resolve them, through a much more diverse set of pathways: arbitration, mediation, conciliation, administrative processes, customary and community-based forums, and other forms of legal support. Courts remain essential, but from the perspective of the people seeking justice, they are only one part of a much larger ecosystem.

This is where the OECD framework offers something transformative. It connects three elements that are too often considered separately: government policy and resources, the justice services that people actually encounter, and the outcomes those services produce.

Governments decide on legislation, budgets, institutions, technology, and staff. This shapes the services available to people. Those services should, in turn, contribute to outcomes: resolving justice problems, preventing problems from escalating, ensuring fair processes, and strengthening trust between people and their government.

The framework’s value is that it lets policymakers use mostly available data to see those connections. For example, at a Ministry of Justice, this creates a more useful basis for deciding where resources should go, taking into account which services are more effective, which approaches offer better value for money, and where people are falling out of the system.

Justice can learn from health

Nowadays, it would be odd to imagine evaluating a health system’s impact by simply counting hospital beds, doctors, or consultations. For decades, the international health community has moved beyond those approaches, focusing on whether people receive the care they need, services are accessible and effective, and the population’s overall health improves. Inherently, this nuanced approach brings with it a greater understanding of health disparities in public services, between regions and socioeconomic contexts, that must be addressed to ensure equal access for all.

Justice needs a similar [r]evolution

In the measurement framework, the OECD notes that the health sector has spent years building common classifications, indicators, and measurement systems that allow countries to better understand the performance in that sector, and compare experiences across populations. The new proposed approach seeks the same kind of shared foundation for the justice system.

This should not mean creating another reporting exercise, but changing the way the available data is processed and analyzed. Indeed, one of the framework’s most useful features is precisely that it does not just call for collecting more data.

Justice systems are already “data-rich but information-poor,” the report states. Large quantities of information exist, but institutions often collect them using different definitions and methodologies, store them in separate systems, and keep them disconnected from the policy questions the public sector is trying to answer. Therefore, the challenge is to make better use of existing data and fill most of the relevant gaps.

This is precisely why the OECD Framework is so important for the Justice Action Coalition.

Since the publication of the 2019 “Justice for All” report, and based on data from the World Justice Project, we know how massive the global justice gap is, but our ability to measure progress remains uneven. Countries, institutions, and organizations often collect valuable data, but not necessarily in ways that allow us to connect justice needs with services or outcomes.

The new framework provides a common structure to be used by the Justice Action Coalition countries, and that can continue to be developed, including through the Coalition’s Workstream 1 on Data and Evidence, co-led by the OECD and Canada, and in collaboration with the Hague Institute for Innovation of Law (HiiL), the World Justice Project, Pathfinders for Peaceful, Just and Inclusive Societies, and many other partners. That gives the Justice Action Coalition an opportunity to move an important conversation forward, from how many justice interventions exist, to what difference the entire justice system is actually making.

When applied to the Justice Action Coalition Justice Financing Framework, the OECD framework also provides the foundation for something the justice community has long lacked: a way to make evidence-based decisions on resource allocation. By linking funding to outcomes, decision-makers can move from funding institutions based on historical budgetary allocations to directing resources where they can have the greatest impact on people’s justice needs.

Finally, it also provides greater comparability without pretending that every country’s justice system is the same. The OECD explicitly envisages this as a dynamic tool that can be improved based on national experiences, peer learning, and technical expertise, not as a prescriptive model.

Finally, the new framework is best understood not as the final word on measuring justice, but as an invitation to begin a new phase of collective work, looking into justice data from a different perspective.

It is in every justice actor’s interest to improve how we all understand the impact of our daily work, so we hope the Justice Action Coalition can contribute to these discussions by strengthening collaboration between governments, judiciaries, civil society, and international and regional organizations.

People-centered justice is not simply about changing how we deliver justice. It is about changing how we understand the process that makes that happen.