Our approach

Measuring the impact of media support has long been a major challenge across the sector. However, recognising how valuable a reliable measurement of our impact would be, we track the performance of each client throughout the length of our relationship, enabling us to assess our impact on their business over time. This measurement helps us better understand the results of our work and allows us to make more informed decisions around the performance of our portfolio. Additionally, we also track the impact of our clients upon society to better convey the societal value of their journalistic work. On a broader level, we simply want to inform the public about why journalism matters and why it has critical civic importance. Overall, by understanding this very real return on investment – the social and financial results of media financing – MDIF, its donors and the development community may better support independent news businesses.

We focus our impact assessment efforts on two areas: first, direct impact of our investment on clients; and, second, our clients’ impacts on their societies. The methodology is designed to assess established media outlets that produce news and information content and are focused on generating revenues to support their operations, rather than early-stage digital startups or businesses that provide platforms to connect users instead of providing news and information to audiences. While the outlets we invest in are diverse in terms of their geographic focus, revenue models and media type, we believe that these metrics are relevant for the largest number of clients in our portfolio.

To assess our impact on the business development of clients, we need to know that our financing and advisory support are making a difference. Are we achieving our purposes as agents of change? What are the business changes experienced by the media we support? To do that, we track various performance metrics that measure growth and resilience. We look at clients’ long-term sustainability and monitor how their reach, revenues, staff count and risk rating evolve over the course of their involvement with MDIF. Numerical performance metrics are supported by illustrative case studies showcasing change at the client level. Although we view our investment as a contributor to, not the sole cause of, our clients’ performance over the years, we believe that evidence of clients’ growth collected over the length of our relationship validates our mission and approach. Still, we think that numbers alone are insufficient to unearth or communicate impact. To add context to performance trends, we listen to those who matter most – the independent media companies we support – and ask them directly to evaluate our support and tell us if they experience changes while working with us.

To assess our effectiveness in achieving our ultimate goal of supporting independent media that make a difference to their societies, we also need to understand the impact clients have on the communities they serve. To answer this question, we align our reporting with the SDGs. First, we monitor the number of people having access to information through our clients in a given year (SDG 16.10). Second, to ensure that information received was of high quality, we look at the awards won by our clients, a recognised measure of excellence. Third, to show that our clients bring about social change across various SDG sectors, we present the number of clients who report having contributed to a real-world change alongside examples of stories of change mapped against the relevant SDGs. When presenting these concrete examples we go beyond the metrics of audience reach and website traffic and try to focus on the ultimate social outcomes that followed, for example investigations conducted, laws changed or protests staged. Still, we are very careful not to attribute causality – in most cases we view our clients’ work as a contributor to, not the sole cause of, changes in communities. Lastly, to monitor system-level developments, we map our portfolio against pertinent indicators – the SDG Index and the Social Progress Index.

MDIF gathers the data directly from our clients through monitoring and questionnaires. At the beginning of each calendar year, we approach MDIF-supported media with a survey that collects information on their reach, as well as corruption, accountability, election and social issues reporting for the preceding year. To the extent possible, we validate the reach results clients report, eliminating or adjusting anomalous figures. We also verify the stories by reviewing media coverage of those events, both by our client and other media outlets in the market. For a small number of clients that are unable to complete the survey, we evaluate a sample of their content from the previous year to identifying instances of high-impact reporting. Revenues data and financial viability, composed of seven company-specific indicators, are collected and updated regularly, with the final assessment for the year used for the annual analysis.

In addition to client records, we ingest data from several external data sources. For instance, to monitor the online reach of our clients, we rely on data gathered by Google Analytics. Across different impact areas, we quantify survey responses against comparable and pertinent indicators, including the World Press Freedom Index published by Reporters Without Borders, Corruption Perceptions Index by Transparency International, Social Progress Index by Social Progress Imperative, as well as the World Bank’s Political Stability and Absence of Violence Index and Ease of Doing Business Index.

Since 2005, we have published our Media Developoment Impact Dashboard to publicly present the findings of our annual analysis. The Dashboard provides a comprehensive description of impact results from the preceding year, as well as a longitudinal analysis of outcomes across our portfolio, including quantitative data on sales, reach and financial viability, in order to assess our clients’ economic sustainability, as well as qualitative data on our clients’ impact on society. You are able to locate links to previous Dashboards conveniently on the right-hand side of the website. Additionally, we present our impact in a form of case studies published on our website.

Overall, our impact measurements follow a core principle of prioritising efficiency and reflect the day-to-day business realities of media companies we support. We acknowledge that the data we collect has its limitations and that the absence of a relevant control group means that we are unable to attribute impact to a particular intervention. Although we grapple with issues in both collecting and standardising data across our diverse portfolio, from complicated causality to unreliability of audience research data in many emerging markets, our objective is to collect data with an appropriate degree of rigour that allows us to provide an accurate and reliable insight into our work. Given the still-evolving status of tracking impact and the sweeping changes in the media sector, we are constantly learning and striving to improve our approach. For a complete description of the challenges and how we try to address them see the full Impact Dashboard Methodology.