Feedback on the first MCS Monthly Online Gathering
rick davies
I have a transcript of an audio file of a workshop discussion. I would like you to summarise this discussion. Previously I asked you what options there are, in terms of kinds of summaries you could produce. This is what you told me:
"There are a few main options for summarizing a workshop discussion from a transcript:
- Highlights summary - Focus on pulling out the key points, conclusions, action items, and decisions from the discussion. This would provide a brief overview of the main takeaways.
- Thematic summary - Identify the main themes or topics covered in the discussion and summarize the key points made about each one. This provides more detail than a simple highlights summary.
- Chronological summary - Walk through the discussion chronologically, summarizing the main points and flow of the conversation. This retains the narrative structure of the discussion.
- Focused summary - Summarize only certain aspects of the discussion, such as questions posed, solutions proposed, arguments made, etc. Allows focusing in on specific areas of interest.
- Abstractive summary - Use natural language generation techniques to produce a new summary reflecting the main points, rather than simply excerpting from the transcript. Requires more advanced AI capabilities." Please now summarise the attached text, using the first method (highlights summary)
RJ
Here is a highlights summary of the key points from the workshop discussion transcript:
- The group discussed experiences using the Most Significant Change (MSC) participatory evaluation technique in various contexts.
- Person 1 has limited experience using MSC so far, having only done one pilot test of it in a mental health services context. Others like Person 2 and Person 3 have more extensive experience with MSC.
- A key theme was adaptations made to the standard MSC methodology to fit time constraints and objectives. Many have had to streamline the process, dropping some steps like verification.
- Ensuring enough time is a common challenge. Person 1's pilot was done in just 3 weeks, which limited the process. Person 2 is anticipating needing more than the 3 months allotted for an upcoming MSC evaluation.
- Another topic was integrating MSC with other evaluation methods like outcomes harvesting and quantitative measures. Using MSC alongside other tools can provide complementary data.
- Ethical use of stories was discussed - ensuring transparency with participants on how their stories will be used. Marketing wishes to repurpose stories posed an issue in one case.
- Overall the discussion focused on sharing practical experiences with MSC implementations, adaptations made, challenges faced, and best practices.
Copy
Please now summarise the original attached text, using the second method (thematic summary)
RJ
Here is a thematic summary of the key points covered in the workshop discussion transcript:
MSC Experience and Background
- Person 1 has limited experience with MSC so far, having only done one pilot test of it recently in a mental health services context.
- Person 2 has used MSC before with UNFPA and is now bidding on an MSC evaluation project.
- Person 3 has extensive experience with MSC since 2006, having worked as an independent consultant and conducted MSC training.
Adapting the MSC Methodology
- Many have had to streamline or adapt the standard MSC process to fit time constraints and objectives.
- Steps like verification are often dropped or limited due to short timeframes.
- Stakeholder analysis and mapping exercises are important but can be condensed or simplified.
- Focus is usually kept on the core steps of collecting, selecting, and providing feedback on stories.
Time Considerations
- Ensuring adequate time is a common challenge with MSC.
- Person 1's first pilot was done in just 3 weeks, which limited the process.
- Person 2 has been allotted 3 months for an upcoming evaluation using MSC but believes more time will be needed.
Integrating MSC with Other Methods
- MSC can complement quantitative measures and other set evaluative tools.
- It can be integrated with methods like outcomes harvesting, though the logistics require further discussion.
- Techniques like Ten Seed Technique have been used alongside MSC successfully.
Ethical Use of Stories
- A key consideration is transparency with participants on how their stories will be used.
- Marketing wishes to repurpose stories posed an ethical issue in one case.
- Proper permissions must be obtained before sharing stories externally.
Overall the discussion centered on practical experiences with MSC, including adaptations made, challenges faced, and recommendations.
Copy
Please now summarise the original attached text, using the third method (chronological summary)
RJ
Here is a chronological summary of the key points covered in the workshop discussion transcript:
Introduction
- Person 1 provides background that she has limited MSC experience, having only done one pilot test of it recently with a mental health services team in Wales. She found it a rich process but felt rushed.
MSC in Wales
- Person 1 notes MSC seems well adopted in Wales. Her colleague Mike Corcoran uses it a lot. Another organization called ACE has used it for ongoing monitoring and evaluation.
Adapting the MSC Process
- Person 1 did only 6 of the 10 steps outlined in the MSC guide due to time constraints, skipping things like verification of stories.
- Person 3 notes he focuses on the core 3 steps - collection, selection, feedback. Other steps may be done beforehand or not needed.
- Person 4 had issues arise when stakeholders realized they could use stories for marketing purposes, beyond what participants had consented to. More care may be needed around ethics of sharing.
Time Considerations
- Person 1 did her pilot in just 3.5 weeks due to an urgent student project deadline. Felt too rushed.
- Person 2 has been allotted 3 months for an upcoming MSC evaluation but expects to need more time.
- Person 3 notes MSC can be used for one-off evaluations or periodic monitoring, so timeframes vary greatly. Need to avoid squeezing dialogue though.
Stakeholder Analysis
- Person 2 is planning an MSC evaluation across multiple countries. Careful stakeholder analysis will be needed to determine whose stories to collect.
- Person 3 notes stakeholder analysis often happens beforehand. He sometimes uses card sorting techniques to inform MSC design.
Combining MSC with Other Methods
- Person 1 uses MSC alongside mandated quantitative measures in mental health services. Provides a complementary perspective.
- Person 4 hasn't combined MSC with other methods but is open to exploring it more.
- Person 3 has used Ten Seed Technique successfully alongside MSC, as a way to also gather community-level perspectives.
Conclusion
- The group reflected on key learnings, challenges like lack of time, and plans for future MSC applications.
Copy
Please now summarise the original attached text, using the fourth method (focused summary)
RJ
Here is a focused summary of the workshop discussion transcript, concentrating specifically on the adaptations made to the MSC methodology:
- Person 1 only did 6 of the 10 steps laid out in the MSC guide due to having just 3.5 weeks to complete her pilot. She had to skip steps like validation of stories with participants.
- Person 3 focuses on just the core 3 steps - collection, selection, and feedback. Other activities like stakeholder analysis may happen beforehand.
- For some clients Person 3 has only done the basic 3 steps as requested, omitting things like quantification and secondary analysis.
- Person 4 has found ethical issues arising around potential reuse of stories collected. More work may be needed to ensure permissions and transparency with participants upfront.
- Time limitations frequently force adaptations. Person 2 has 3 months allotted for an upcoming MSC evaluation but expects to need more time.
- Domains of change are often not pre-defined, allowing themes to emerge from the stories collected.
- Person 3 takes a flexible approach to verification, generally trusting stories unless they will be made public or need more detail. He suggests "elaboration" may be a better framing than "verification".
- Combining MSC with other methods like outcomes harvesting requires care around how the selection discussions play out. Using MSC as an entry point to then harvest outcomes may be best rather than intermixing the discussions.
In summary, adaptations to the MSC methodology are very common, whether streamlining steps, limiting verification, adjusting analysis, or interfacing with complementary methods. Time constraints in particular force creativity.
Copy
Please now summarise the original attached text, using the fifth method (abstractive summary)
RJ
Here is an abstractive summary of the key points from the workshop discussion:
The group explored experiences and best practices around using Most Significant Change (MSC), a participatory story-based evaluation approach, in their various contexts. A central theme was the need for flexibility and adaptation when applying the standard MSC methodology in real-world settings.
Many shared how they modified the typical 10-step process due to limited timeframes and specific organizational needs. For example, several reported streamlining down to just the core steps of story collection, selection, and feedback. Areas like verification of stories were often abbreviated or skipped. Defining domains of change upfront was also frequently adjusted to allow themes to emerge from the stories organically.
Combining MSC with complementary data like quantitative metrics or outcome harvesting was discussed. This requires care to ensure appropriate synthesis and use of different data streams. Questions arose around handling the selection discussions for both MSC stories and harvested outcomes jointly.
Ethical considerations around sharing personal stories were highlighted. In one case, marketing wished to repurpose stories beyond their intended use, raising consent issues. This exemplified the importance of transparency with participants regarding how their stories will be utilized.
The breadth of experiences shared underscores how MSC as a flexible, dialogic tool can be molded to fit various evaluation settings and needs. However, common challenges like insufficient time persist, requiring creative solutions to adapt the methodology while upholding participatory principles. Further guidance on interfacing MSC with emerging methods and navigating ethical dilemmas would benefit practitioners.
You have now done five different types of summary. Please now summarise all of these summaries into one summary, with minimal loss of detail but also avoiding duplication. You can use up to 300 words.
RJ
Here is a consolidated 300 word summary of the key points covered across the different summaries of the MSC workshop discussion:
The group shared diverse experiences using Most Significant Change (MSC) for participatory evaluation in various contexts. A major theme was adaptations made to the standard 10-step MSC process to accommodate real-world needs and constraints. Many streamlined the approach, focusing on just collecting, selecting, and providing feedback on stories. Steps like verification were often abbreviated or omitted due to short timeframes. Defining domains of change upfront was frequently adjusted to let themes emerge from stories organically.
Combining MSC with other methods like outcomes harvesting or quantitative data was discussed but requires care in synthesizing different data appropriately. Ethical considerations around sharing personal stories were also highlighted, underscoring the need for transparency regarding how stories will be used.
Insufficient time was a common challenge, forcing creative solutions to adapt MSC while upholding participatory principles. But MSC's flexibility was seen as a strength in molding it to diverse settings. Further guidance on tailoring the methodology, integrating it with complementary approaches, and addressing ethical dilemmas would benefit practitioners.
Overall, the breadth of experiences affirmed MSC's versatility but also the need for adaptation and streamlining in practice. Adhering to the core principles of elevating participant perspectives through story gathering and collaborative sensemaking allows MSC to serve as a "quick and dirty" evaluative tool or more in-depth ongoing monitoring approach.
Jess Dart
Hey Rick
Sorry for the radio silence, things have been hectic here, and it hasn’t been so easy to engage in the evenings. I have been on the road and travelling again.
Looks like things are going well, and I love the approach with AI summaries! I think you are onto something. I will be sure to come along to at least one monthly session on MSC. Cant wait to try to platform
Jess
Dr Jess Dart (she/her)
| Chief Evaluator/Founder Clear Horizon | e: je...@clearhorizon.com.au p: +61 3 9425 7777 I m: 0425 735 530 w: www:clearhorizon.com.au |
Build your evaluation skills with the Clear Horizon Academy, and join our free Community for tools, resources and advice from peers.
Keep up to date with the latest news, sign up to our monthly e-news.
Clear Horizon acknowledges and pays respect to the past, present and future Traditional Custodians and Elders of this nation and the continuation of cultural, spiritual and educational practices of Aboriginal and Torres Strait Islander peoples.
Notice: The information contained in this email is confidential. If you are not the intended recipient, you may not disclose or use the information in this email in any way. If you receive it in error, please tell us immediately by return email and delete this document. We do not guarantee the integrity of any emails or attached files are not responsible for any changes made to them by any other person.
--
If you have any concerns about any of the postings on this email list please email me directly at
rick....@gmail.com
---
You received this message because you are subscribed to the Google Groups "MostSignificantChange (MSC) email list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
mostsignificantchange-msc-...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mostsignificantchange-msc-2020-email-list/cb3405cd-16ae-438f-b2cc-dc45d913dcd5n%40googlegroups.com.