©Center for Global Development/Mayra Buvinic
The rhetoric around women’s economic empowerment (WEE) in global development is finally being translated into action. Development organizations are using this objective to guide operations and exploring ways to measure impact by integrating WEE indicators into project results frameworks.
But which WEE indicators should be used? There is no easy answer to this question, since selecting and harmonizing core WEE indicators remains a major challenge for development organizations.
This is partly because women’s economic empowerment is intrinsically difficult to measure. WEE has objective (e.g., increased income) and subjective (e.g., improvements in self-confidence) dimensions that both need to be captured, so a single indicator in a project results framework is often not enough. Further, how WEE manifests can vary significantly across contexts and cultures, increasing the challenge of identifying common (rather than customized) WEE indicators to apply across projects, sectors, regions or (even) organizations. And WEE indicators are needed in projects that usually have more than one objective—such as those focused on infrastructure development, agriculture, or social protection—and, therefore, already have many other indicators in their results frameworks. As a result, development organizations often face difficulty in choosing and integrating WEE indicators, especially those that are both reliable and valid as well as practical and comparable.
How different organizations identify and integrate WEE indicators into project results frameworks was one of the main topics discussed at a meeting of researchers and practitioners convened this past July by CGD and Data2X as part of our joint WEE measurement learning collaborative. Here we summarize five takeaway messages from the meeting:
1. Measuring WEE holistically is doable and brings substantial project benefits
ADPP Angola, a small NGO operating in in rural communities with very high poverty rates, exemplifies that measuring WEE is feasible and worthwhile for agencies big and small, with widely different levels of measurement sophistication. Technology (tablets and statistical software) are convenient, but not essential.
Since 2015, ADPP has been using a simple set of five WEE indicators, put together by the ExxonMobil Foundation and their grantees (including ADPP), to monitor the outcomes of ADPP’s work with Women’s Farmer Clubs. The results of the twice-yearly measurement exercise, collected by local staff and displayed in simple tabulations and graphs for all to see (see photo), are discussed in the communities. ADPP has learned that measuring both objective and subjective WEE outcomes—rather than just project inputs and outputs—provides critical information to assess project benefits. Before 2015, ADPP used to only track data on farm production and income indicators. Now, thanks to the addition of indicators related to women’s satisfaction and self-confidence, they are able to judge project impact independent of the negative effects of recurrent economic and environmental shocks (common in Angola). Women farmers, used to experiencing these shocks, judge what is possible within restricted circumstances, and can still report feelings of self-confidence and satisfaction as result of participating in the project, even when income indicators don’t move.
2. Standardizing a core set of WEE indicators can help reduce variation in indicators currently in use
In her recent review of World Bank project results frameworks, Jennifer Solotaroff found wide variation in the indicators used to measure WEE. She identified 44 World Bank operations under the “more and better jobs” and “asset ownership and control” twin pillars of the bank’s gender strategy, which incorporated a wide range of WEE-related indicators in their results frameworks, especially under “jobs/employment” and “technical skills/training” (less so under entrepreneurship and asset indicators), and only one operation measured the subjective or agency component of WEE.
Thomas de Hoop, leading the Evidence Consortium on Women’s Groups (ECWG), similarly found considerable variation in the outcome indicators used in evaluations of women’s self-help groups in India, Nigeria and Uganda, limiting the ability to synthesize evidence on the impact of these groups. ECWG, supported by the Bill & Melinda Gates Foundation, provides guidance on the outcomes to include and measurement approaches to use in impact evaluations of women’s groups, including standardized measures of WEE outcomes to support comparisons across the Gates Foundation’s funding portfolio.
Standardization should be an aspirational goal, but it should not be imposed. The complexity of WEE as a concept and its variability across contexts and cultures, the differences between operational business lines’ objectives (e.g., road building or active labor market or social protection operations), and client countries’ preferences call for flexibility in indicator selection. Harmonization, or procedures to improve comparability of data, can still happen with non-standardized indicators, if they are grounded in consistent and transparent approaches to measurement.
3. A smart approach is a deliberate approach: Indicators should be grounded in an explicit theory of change
A deliberate approach to measuring WEE requires articulating an explicit theory of change that identifies the causal links between the project’s interventions and its intended final outcomes. This theory of change guides the choice of variables and indicators to use in the project result framework. For example, to design innovative employment solutions in response to the COVID-19 crisis and measure their impact on (hopefully) transforming women’s employment, Maria Fernanda Prada of the IDB presented the outlines of a systematic, gender-sensitive approach the IDB is developing to quantify its employment impacts across the operational portfolio. This work requires changing the mindsets of operational staff and clients on the benefits of measuring employment effects.
More efforts that ground WEE indicators in action frameworks or theories of change, rationalizing the indicators chosen for results frameworks, are welcome and needed. Selecting indicators based on a solid rationale and standardizing a core set of WEE indicators to use across projects, sectors, regions or organizations enables monitoring and assessing the performance of a project portfolio, facilitating accountability; building valuable operational knowledge by comparing outcomes and learning lessons across projects; and increasing the efficiency of measurement.
4. WEE indicators should be simple and transparent and guaranteed to yield high-quality data
Simplicity in the choice of indicator and transparency in communicating about the indicator are critical. Simplicity helps ensure quality data and eases adoption and harmonization across projects or organizations. For instance, the government of Angola, led by the Ministry of Social Action and Empowerment of Women and Family, has expressed interest in using ADPP’s WEE indicators across different ministries. The simplicity of these indicators makes their use across government agencies more feasible, which is promising and may help fill in large existing gender data gaps in the country. Transparency in communicating the basic features of the indicators (definition, data source, metadata) is critical for harmonization. See here (table 5 page 39) for a list of desirable features of a WEE indicator.
Also paramount is the choice of WEE indicators that yield quality data. Jim Knowles’ analysis of a rich dataset of men and women business owners in Indonesia tested 49 different WEE indicators (using our WEE conceptual framework) and shows that simple solutions are often available to strengthen the reliability and validity of WEE economic indicators. These include:
- avoiding indicators based on extended recall periods. For example, enumerators should ask respondents about their current savings balances instead of savings over the past year (the latter provides more reliable information that is not distorted by recall issues)
- suppressing the effects of extreme values, which often happen when asking about income or profits, through statistical tools—e.g., the natural log transformation of reported business profits is both reliable and valid (unlike the untreated profit values)
- using statistical models designed to deal with large concentration of zero values, which are common when asking about issues such as savings and physical assets.
5. Collaboration between researchers and practitioners should foster advances in integrating a core set of WEE indicators in projects results frameworks.
To move towards selecting and applying a core set of WEE indicators into their project results frameworks, development organizations can first select a core set of indicators for operations within a specific program or sector based on a theory of change as, for instance, the Government-to-Person Payment (G2Px) Initiative at the World Bank has done for their operations. The same specific program or sector approach can be tried across development organizations, defining a common theory of change and testing WEE indicators to assess their fit. Collaborations between researchers and practitioners should be strongly encouraged to advance the practical application of quality and harmonized WEE indicators in development projects.
This blog was originally published on the website of the Center for Global Development (CGD) here, as part of the WEE Measurement Learning Collaborative. The website provides an overview of the collaborative and its workstreams. It features several blogs and publications on WEE.