Question #4: Tools for measuring social inclusion

35 views
Skip to first unread message

Laura Courbois, MarketShare Associates (MSA)

unread,
Jun 21, 2023, 10:19:51 AM6/21/23
to Advancing Women's Empowerment Virtual Learning Series
This is a question for everyone to respond to, including experts and participants:

What are some of the measurement tools and approaches that the private sector can use to measure the impact of their inclusive business practices on women’s empowerment and social inclusion?
 
Looking forward to the conversation!

Yaquta Fatehi, William Davidson Institute

unread,
Jun 21, 2023, 5:14:55 PM6/21/23
to Advancing Women's Empowerment Virtual Learning Series, Laura Courbois, MarketShare Associates (MSA)

Hi everyone

 

I love this question's focus on "how to conduct measurement" because we often leave the conversation on the benefits and challenges of social impact measurement! 


I use various methods, tools, and approaches to measure WEE and social inclusion, as determined by the research questions, stakeholders' (often the funders) needs for contribution versus attribution of outcomes, and budget. It includes discussions on whether we should use qualitative or quantitative approaches (ideally, use both if resources permit). One tip to determine your way forward is to work with your key stakeholders to deeply understand the data and insights that will help the company in decision-making (this may mean that you want to sketch out a rough analysis plan in advance as well).


Because measurement is not only for accounting and reporting purposes but also for continuous improvement, here is the range I've used so far:

1. Theory of change (the foundation of my measurement strategies),

2. Value stream mapping and process mapping (key for continuous improvement of gender-inclusive operations); One way I think through measuring sustainability in company supply chains for products and services is the plan, source, make, deliver (and return) SCOR framework (lay your process mapping and indicators along this model).

3. Developmental evaluation (particularly important for learning and/or when you can't hold a business process constant),

4. Quasi-experimental designs and experimental design (can be tricky when you need a business to co-determine a comparison group or hold operations constant for a set duration for study fidelity),

5. Retrospective research (used in G-SEARCh; here's our technical note),

6. Most significant change (a participatory approach that ensures you speak to all key stakeholders of the impacts) and

7. Outcome harvesting (when you dont know how impact occurs).


These approaches can be found on better evaluation - one of my favourite measurement resources. I've heard my colleagues use SROI, case studies, success case method, A/B testing, rapid feedback loops, etc. Additionally, a tool I like but have yet to use is WEP. And speaking of tools, we documented our G-SEARCh investors' tools used with their portfolio companies, such as the gender diagnostic tool (page 46) and the Gender Equality Scorecard in this guide.


Another quick tip: consciously apply a gender lens over the selected approaches, methods, and tools and consider whether you are truly including women's voices; for example, if you are conducting phone-based surveys, consider how you will get the man to give the mobile phone over to his wife because this is often a shared asset but mainly controlled by the man (true story; happened to us in Sierra Leone but a common feature in so many low- and middle-income countries where we work). 


To close, there is so much creativity in measurement approaches and an absolute explosion in tools. Even after a decade, I'm still learning new methods; case in point, contribution analysis on a new project to measure the ROI of gender-inclusive approaches with three companies with my Market Share Associates colleagues. This is also why I love participating in these learning exchanges to hear from other folks about their methods and discuss their pros and cons. 


I hope this provides some new food for thought, and I look forward to learning about other approaches! 


Best

Yaquta

Laura Courbois, MarketShare Associates (MSA)

unread,
Jun 22, 2023, 4:41:50 AM6/22/23
to Advancing Women's Empowerment Virtual Learning Series, Yaquta Fatehi, William Davidson Institute, Laura Courbois, MarketShare Associates (MSA), food...@gmail.com
Hi Yaquta, thanks for sharing some the various methods, tools, and approaches that can be used to measure WEE and social inclusion with the private sector. Sharing a response from John Lamb that some of our experts as well as other participants may want to respond to:

All very interesting and informative, thanks

However, since I am no expert in evaluation, nor M&E, but rather a practitioner of MSD, especially for ag-led development, a more mundane and practical question comes to mind. Some of these approaches sound time and effort intensive, hence costly. Way back when, there used to be a rule of thumb that M&E should take up about 5% of the total project budget.

If impact assessment using this whole array of approaches is carried out for the whole of a project, what share of total project budget does it employ typically? And if applied only to evaluate WEE impacts, what (presumably reduced) share should it utilize? 

Or are those inconvenient questions, impossible to answer?

Yaquta Kanchwala

unread,
Jun 22, 2023, 11:14:53 AM6/22/23
to Advancing Women's Empowerment Virtual Learning Series, Laura Courbois, MarketShare Associates (MSA), Yaquta Fatehi, William Davidson Institute, food...@gmail.com
Hi John 

Resources (human, technical, and financial) are a key component in deciding which approach, method, tool, and analysis plan you will use. You are right that I dont have a straightforward answer -- and often, as the third-party measurement lead, I am not privy to the total budget to be able to accurately say what percentage was the M&E piece of the total budget. I am often given a ceiling to work within; my goal is to make M&E sustainable and integral to operations and success, such that more resources are diverted to this essential activity and towards building capacity for this activity within companies. 

My quick tip is that whether you are working as the measurement lead within the organisation or the third party evaluator, it is so important to co-create measurement plans with the program design team, leadership and possibly key stakeholders (such as the funder) to understand the measurement needs, the resources available, and how the insights will be used. It really is a dance between what approach is needed, what are the tasks within that approach, how much level of effort each task will take and does it fit within the ceiling or can you ask for more resources. And please know that M&E plans are organic just as a theory of change; you can grow the measurement strategy with time as more resources are directed towards it. When you first start out, you are likely going to measure just a few key process and outcome indicators to ensure you are on track - and if measuring WEE outcomes are critical, then those should definitely take front and centre in the measurement strategy. 

Please know there have been times when we have had to drop a particular approach (in a multi-approach measurement strategy) because there were limited resources even after the initial ceiling was given; this happened to many evaluators during COVID-19 when we were given no cost extensions or had to change our methodology in real-time because of travel restrictions, business distress and so many other challenges that the pandemic raged on us. 

If anyone else has thoughts on this, please do share! 

Best
Yaquta
Reply all
Reply to author
Forward
This conversation is locked
You cannot reply and perform actions on locked conversations.
0 new messages