Asking about negative change in MSC interviews

24 views
Skip to first unread message

Sasquia Antunez Pineda

unread,
Jan 7, 2022, 4:26:17 AM1/7/22
to MostSignificantChange (MSC) email list
Happy New Year Everyone!

I hope you are all doing well, even with the current challenges of COVID-19 worldwide. I am preparing my first MSC interview protocol for a story-gathering project on juvenile justice in Honduras, where I will be interviewing practitioners at juvenile correctional centres and former inmates. I understand that MSC shares some principles with Appreciative Inquiry interview protocols. On these lines, I am feeling stuck because given the complex prison context of my storytellers – where there are many negative aspects of recidivism, riots within the prison, institutional challenges and so on– I find it difficult to only focus on the strengths of 'what has been' and 'what is' (positive change). Do you have examples of how to ask about negative changes during MSC story-gathering? Or the transition between asking about positive changes to negative changes?

Any guidance you might offer would be truly appreciated.

Wishing you all health and wellness this year,

Sasquia Antúnez Pineda
(Graduate Student of Theology and International Development, University of Toronto)

Fiona Kotvojs

unread,
Jan 7, 2022, 6:28:56 AM1/7/22
to mostsignificantchang...@googlegroups.com

Hi Sasquia,

 

From my perspective, the only common element between appreciative enquiry and MSC is that they are both qualitative methods and generally use semi structured interviews. One of the great strengths of MSC is that it identifies positive and negative change – I specifically use it as an early warning sign for problems.

 

I generally ask a question like:

What good and bad changes have you seen in XYZ over the last year?

 

I use the words good and bad as they are simple and everyone understands. I found positive and negative confused some and I didn’t get as many of the negative.

 

My experience is that people will list good and bad changes. The negatives are my early warning signs and we look at those in the analysis.

 

Then I ask, which of these changes is the most significant to you? What was it like before? What was it like after? What caused this change? Why did you select this change?....

 

I think a structure like that may work for you.

 

Good luck

 

Fiona

 

 

Dr Fiona Kotvojs GAICD

Monitoring and Evaluation Specialist

Kurrajong Hill Pty Ltd

 

fi...@kurrajonghill.com.au

Phone: 0448 453 422

--
If you have any concerns about any of the postings on this email list please email me directly at rick....@gmail.com
---
You received this message because you are subscribed to the Google Groups "MostSignificantChange (MSC) email list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mostsignificantchange-msc-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mostsignificantchange-msc-2020-email-list/3a496c57-3a84-4d00-94de-f7bb13295bc9n%40googlegroups.com.

rick davies

unread,
Jan 7, 2022, 6:36:41 AM1/7/22
to MostSignificantChange (MSC) email list
and a point to add to what Fiona has said...

MSC is not just about collection of MSC stories, it is about their analysis. Particularly the use of a participatory selection process. No selection = no MSC, just story collection

regards, rick davies

Jaqui Goldin

unread,
Jan 7, 2022, 8:42:27 AM1/7/22
to mostsignificantchang...@googlegroups.com
Dear Sasquia


It is helpful also to take your cues from the people you interview - let them 'free flow' in their story - and then ask them at certain points 'how did that make you feel' .. its often helpful to 'stop' and then say if you had to describe how you feel what comes to mind. Its often suprisinng - leading to more 'stories' and opening up avenues of 'information' .. I agree totally with Fiona its not negative or positive - and good and bad are more simple - and direct - ways of asking. But sometimes there is a very grey area inn the middle. You could 'code' words afterwards when you are analysing your 'story' and say that the adjectives used were strongly expressive about a good or a bad experience - but sometimes its more in the middle and you can either colour code (say from black to grey) or bright colours to something that fades - to capture the mood and experience of the person

Do let me know once you start your work in Honduras if you need any guidance on a specific experience.

Also - its helpful to say - 'could I come back and ask you again later on' which gives you that opening and doesn't shut out the conversation. Just make sure to leave that door open. You can say something like 'wow, that is amazing. thank you so much for sharing. I might not fully understand and could I come back if I have forgotten to ask you something or if I have got something 'wrong'

Good luck

Keep me posted

Jaqui

Professor Jacqueline Goldin

Extra-ordinary Associate Professor of Anthropology and Water Sciences

Centre of UNESCO Chair in Groundwater

Earth Sciences

University of the Western Cape

 

Private Bag X 17 Bellville 7537

Tel: 021 9592439

Cell: 27 844380203

 

 


Nick Andrews

unread,
Jan 7, 2022, 8:43:11 AM1/7/22
to MostSignificantChange (MSC) email list
Hi Sasquia, I concur with Fiona, in that we ask people about good or bad changes. This is consistent with another storytelling methodology we use called Experience Based Co-Design which focuses on good and bad experiences rather than changes. You can read more here, so long as you are happy to register with your e-mail:
This toolkit gives a step-by-step guide to improving patient experience of health care using a technique called experience-based co-design (EBCD).
One of the criticisms of Appreciative Inquiry is the danger of not discovering and exploring the bad, although I have a colleague who is expert in AI who says this is a misunderstanding of AI. I can put you in contact with him if you would like that. 

Regards, 

Nick


To: MostSignificantChange (MSC) email list <mostsignificantchang...@googlegroups.com>
Subject: Re: [MostSignificantChange] Asking about negative change in MSC interviews
 

Assels, Rob

unread,
Jan 7, 2022, 10:08:30 AM1/7/22
to mostsignificantchang...@googlegroups.com

Hi Sasquia.

Full disclosure, I have participated in a ParEvo exercise, but I have no expertise in utilizing MSC. That said, I concur with Jacqui’s note about asking to follow up. Interviews are snap shots in time as are the feelings of the interviewee. A follow up interview, after a clear narrative emerges from the coding process, would allow the interviewee an opportunity to clarify (both the good and the bad). Assuming for a moment that some changes have more longevity or momentum than others, if the follow up interviews were conducted at specific intervals they could allow for an interesting analysis.

 

Rob Assels

Evaluation Manager
Research Nova Scotia

Cell 902.759.1878  |  rob.a...@researchns.ca

 

 

 

 

 

 

From: mostsignificantchang...@googlegroups.com <mostsignificantchang...@googlegroups.com> On Behalf Of Jaqui Goldin
Sent: January 7, 2022 6:43 AM
To: mostsignificantchang...@googlegroups.com
Subject: Re: [MostSignificantChange] Asking about negative change in MSC interviews

 

** EXTERNAL EMAIL / COURRIEL EXTERNE **
Exercise caution when opening attachments or clicking on links / Faites preuve de prudence si vous ouvrez une pièce jointe ou cliquez sur un lien

Sasquia Antunez Pineda

unread,
Jan 7, 2022, 11:22:14 AM1/7/22
to mostsignificantchang...@googlegroups.com

Thank you so much Fiona!

 

This is great insight and clarifies things for me. I’ll keep you posted on how the final instrument will look like. I am very excited because my proposal got approved by my academic supervisor – with some revisions, of course – and by the executive director and consultant at ASJ Honduras. ASJ is doing great work in Honduras and I am so excited to begin this project. They were really intrigued by the MSC methodology and its approach.

 

I’ll stay in touch,

 

Happy New Year in Australia!

 

 

SASQUIA ANTÚNEZ PINEDA

Graduate Student Theology & International Development

Justice & Security Intern ASJHonduras | Wycliffe College, University of Toronto

signature_1830095087

647-782-8068

signature_790039695

s.antun...@mail.utoronto.ca

signature_1008740200

www.linkedin.com/in/sasquiaantúnezpineda

twitter

Sasquia Antunez Pineda

unread,
Jan 7, 2022, 11:22:30 AM1/7/22
to mostsignificantchang...@googlegroups.com

Greetings from Canada Jaqui!

 

Thank you so much for this excellent insights. It is my first time applying MSC and because I am going into a vulnerable context (juvenile prisons in Honduras) I want to make sure to develop a solid instrument that make participants feel connected and understand the purpose of what I am doing. I will be testing the MSC interview with local stakeholders in Honduras working in the area of juvenile justice. I will keep you posted on the progress and certainly will reach out in any questions arise.

 

I truly appreciate your support!

 

 

SASQUIA ANTÚNEZ PINEDA

Graduate Student Theology & International Development

Justice & Security Intern ASJHonduras | Wycliffe College, University of Toronto

Subject: Re: [MostSignificantChange] Asking about negative change in MSC interviews

You don't often get email from jaqui...@gmail.com. Learn why this is important

Sasquia Antunez Pineda

unread,
Jan 7, 2022, 11:22:43 AM1/7/22
to mostsignificantchang...@googlegroups.com

Many thanks Andrew and Happy New Year!!

 

Thank you for this two resources, and for sure I will be registering on the mailing list for EBCD! I would also appreciate if you could put me in contact with your colleague who is an expert on AI.

 

I very much appreciate your support!

 

Blessings,

 

 

SASQUIA ANTÚNEZ PINEDA

Graduate Student Theology & International Development

Justice & Security Intern ASJHonduras | Wycliffe College, University of Toronto

Sasquia Antunez Pineda

unread,
Jan 7, 2022, 11:22:51 AM1/7/22
to mostsignificantchang...@googlegroups.com

Thank you Rob for this valuable insight!

 

I am very happy to discover this idea of follow-up interview, making the process more engaging and relational.

 

 

SASQUIA ANTÚNEZ PINEDA

Graduate Student Theology & International Development

Justice & Security Intern ASJHonduras | Wycliffe College, University of Toronto

Subject: RE: [MostSignificantChange] Asking about negative change in MSC interviews

You don't often get email from rob.a...@researchns.ca. Learn why this is important

Sasquia Antunez Pineda

unread,
Jan 7, 2022, 11:23:01 AM1/7/22
to mostsignificantchang...@googlegroups.com

I wanted to thank you all for the valuable insights you gave me today and MSC and AI. This really helps!! I will be sure to share the process of this work in Honduras with you.

 

Your support is truly appreciated.

 

Have a wonderful 2022!

 

 

SASQUIA ANTÚNEZ PINEDA

Graduate Student Theology & International Development

Justice & Security Intern ASJHonduras | Wycliffe College, University of Toronto

Sasquia Antunez Pineda

unread,
Jan 7, 2022, 11:46:23 AM1/7/22
to mostsignificantchang...@googlegroups.com
Thank you Rick,

I am happy to share that we are gathering together a group of stakeholders in Honduras in the area of juvenile justice that will be part of the participatory selection process to assess the stories. This will be done through the collaboration and support of two organizations doing great work in Honduras juvenile detention centres – Orphan Helpers and ASJHonduras. I will be sharing more in the next few weeks. 

To view this discussion on the web visit https://groups.google.com/d/msgid/mostsignificantchange-msc-2020-email-list/a7163d37-1b4e-47dc-bca1-261f912f50a1n%40googlegroups.com.


--

SASQUIA ANTÚNEZ PINEDA

Graduate Student Theology & International Development

Justice & Security Intern ASJHonduras | Wycliffe College, University of Toronto

Bob

unread,
Jan 8, 2022, 3:28:30 AM1/8/22
to mostsignificantchang...@googlegroups.com
Kia ora everyone

I’ve come a bit late to this particular party, but two things occur to me.

Firstly, I was once firmly told off when I made some kind of throw-away comment around ‘appreciative inquiry’ only focusing on the positive.  It got me going back to the original texts and oh my goodness it doesn’t only focus on the positive.  What it does do is generate solutions using an appreciative orientation.  It’s a subtle but important difference between ‘appreciative’ and ‘positive’ that often gets lost as the original ideas of Cooperider and colleagues become coopted and subjected to that whispering game we played as kids.  But the original idea was to generate solutions by identifying and building on what people most value rather than ’solve’ a problem in a more technocratic and instrumental sense.

Secondly, MSC is about the most significant change.   I’ve often meant to ask Rick what happens when the most significant change is a bad one; at least from the perspective of those involved in the process.  Again this is a value based process not a merit based process; what is significant or marginal, and whether it was good or bad will depend on the values, histories and cultures of those involved in the process.  I know that is a terribly obvious statement - but I’ve often said that staying in the same place at least not going significantly backwards is often a significant victory for those who we are working with.

On 7/01/2022, at 1:15 PM, Sasquia Antunez Pineda <s.antun...@mail.utoronto.ca> wrote:

Many thanks Andrew and Happy New Year!!
 
Thank you for this two resources, and for sure I will be registering on the mailing list for EBCD! I would also appreciate if you could put me in contact with your colleague who is an expert on AI.
 
I very much appreciate your support!
 
Blessings,
 
 
SASQUIA ANTÚNEZ PINEDA
Graduate Student Theology & International Development
Justice & Security Intern ASJHonduras | Wycliffe College, University of Toronto

Mobile Argentina +54 9 11 3602 2284
Skype : bobwill
Twitter : @boblwil

"Only Connect ..... "  E. M. Forster

System Diagrams: A Practical Guide

Systemic Evaluation Design 
https://gum.co/evaldesign
Video -https://www.youtube.com/watch?v=VUTqG5j21mU

Wicked Solutions: A systems approach to complex problems.
https://gum.co/wicked



Rituu B Nanda

unread,
Jan 8, 2022, 3:50:48 AM1/8/22
to mostsignificantchang...@googlegroups.com
Hi Sasquia and All,

Thanks to those who shared their experiences. Found them very valuable.

I recently facilitated MSC on the issue of a gender equality initiative. The aim of MSC was that the implementation NGO team understands how the project is progressing and make course corrections if required.
During MSC, I was initially concerned as positive stories of change were emerging.  Finally, came a story of what did not work. A few factors which seem to have made it possible:
  • Build basic M&E  capacity in the NGO staff with a focus on Monitoring, learning, and Evaluation
  • Placed emphasis on improving rather than proving
  • Building ownership of the NGO staff in the process
  • Used strength-based SALT to create a safe space
  • Facilitation skills
Some points:
  1. MSC was accompanied by another tool-  self-assessment by the project team. 
  2. Result-NGO staff decided to make changes in their approach to the project, partially due to the MSC process. 
  3. What we could not do- Due to frequent lockdowns, unfortunately, we could not engage the community in the selection of stories and analysis.
Warm regards,
Rituu


Rituu B Nanda 
Evaluation Facilitator, Institute of Social Studies Trust (ISST)
Gender & Evaluation online community: http://gendereval.ning.com
Core group Member: Evaluation Community of India (ECOI)
Member, EvalGender+: Management Group



To view this discussion on the web visit https://groups.google.com/d/msgid/mostsignificantchange-msc-2020-email-list/YT3PR01MB619523552B6C70FA9EDD754A984D9%40YT3PR01MB6195.CANPRD01.PROD.OUTLOOK.COM.

rick davies

unread,
Jan 8, 2022, 3:50:54 AM1/8/22
to MostSignificantChange (MSC) email list
Hi Bob

1. Thanks for the clarification re appreciative inquiry. Appreciated :-)

2. Re " what happens when the most significant change is a bad one?" I think you sort of answered your own question in your text that follows. I.e. You ask the same questions you would about a positive change: Seek a detailed description, seek a detailed explanation of why the respondent sees it as bad, including existing and anticipated consequences, and perhaps in some circumstances any implication for subsequent actions by any party

3. Re "staying in the same place at least not going significantly backwards is often a significant victory for those who we are working with" This is an interesting question, which I would like to hear other people's answers to. My almost reflex response would be to say/think that there is almost never no change, so focus on identifying the changes that did occur, however "small". But this does not sound appropriate as a blanket prescription. Firstly,  it sounds a bit like an overly intellectual take on peoples lived experience. Secondly, and related, the respondent may genuinely feel that in the face of various minor changes, it is nevertheless the overall feeling of stasis which is most important to them.  

Another angle is to consider whether we include changes in peoples expectations, as part of the realm of inquiry, not just changes in the physical and social world. If so, then some periods of stasis might challenge and change peoples expectations in a way that is significant to them. A question to all: How good are we at seeking changes in expectations? I know a good interviewer will uncover psychological dimensions of reported changes, that is a lot of what significant is all about.  But how good are we at uncovering changes which are primarily psychological?

regards, rick

Bob

unread,
Jan 8, 2022, 6:09:51 AM1/8/22
to mostsignificantchang...@googlegroups.com
Those are great points you make Rick.  

My comment around ‘change’ generally is that much of the way that ‘change’ is spoken of in practice and in the literature is that ‘change’ is inherently a good thing.  I’m often surprised how evaluators take this as a given.  That seems to confuse merit with worth, something evaluators should be very sensitive to.  I find the same attitude in the so-called ‘learning’ literature.  There is always an assumption that learning is good, that it has intrinsic merit.  But my years working on health and safety issues made me realise that people are always learning, the important thing is what they are learning.  Are they learning that to survive in an organisation you do not to report breaches of health and safety rules?

I loved your question about changed expectations, not only inherently but also it challenged me to think about what I mean when I talk about ‘changes’.  As you point out in your second paragraph nothing is static - even ’no change’ is the result of smaller changes that balance each other out.  So it is again a question of what is changing rather than whether things are changing.

--
If you have any concerns about any of the postings on this email list please email me directly at rick....@gmail.com
---
You received this message because you are subscribed to the Google Groups "MostSignificantChange (MSC) email list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mostsignificantchange-msc-...@googlegroups.com.

Bernward Causemann

unread,
Jan 8, 2022, 6:58:21 AM1/8/22
to mostsignificantchang...@googlegroups.com

Dear Bob,

thanks for your interesting comments on the topic (and to others for their contributions).

One point I do not understand: Would you explain what you mean with 'confuse merit with worth', particularly in this context?

Kind regards

Bernward

--
Bernward Causemann
+49-7071-705801 / www.causemann.org

Fred van Leeuwen

unread,
Jan 8, 2022, 7:25:25 AM1/8/22
to mostsignificantchang...@googlegroups.com
And regrettably people are often reluctant to mention negative change. 
Often people seem to want to please the interviewer and therefore only mention what they think the interviewer wants to hear.

GHD (Fred) van Leeuwen



To view this discussion on the web visit https://groups.google.com/d/msgid/mostsignificantchange-msc-2020-email-list/0E3BFDD4-5747-47C9-9EF6-F7FB0C68E797%40bobwilliams.co.nz.

Bob

unread,
Jan 8, 2022, 11:36:39 AM1/8/22
to mostsignificantchang...@googlegroups.com
Hmm, yes, good question

Well as someone once said words have uses not definitions, but Michal Scriven tends to describe the difference between them as intrinsic vs extrinsic.  So an individual person may have the intrinsic quality of being short.  Being short allows you to do many excellent things (eg most champion snowboarders tend to below average height).  But that is of little worth if you were wanting to join the police say 50 years ago.  Today, in NZ at any rate, being short doesn’t not prevent you from joining the police.

So in evaluation jargon, merit describes the characteristics of the evaluand where worth describes whether those characteristics are subjected to some kind of assessment by someone or something (ie useful, good, bad).

Not entirely satisfied with this response, I did a quite search on the net.  

I found this reference useful


The evaluation example it gives is this :

Let say you’re evaluating two instances of the same program: Program Breakfast-for-all at Site A and Site B. While they may both have merits, the worth of the program at Site A may be different from Site B depending on its impact on the constituents. Worth between two comparable, but different programs may also differ if one is cheaper to run (so one is worth more than the other).

Or to return to the topic of change.  Much is made of the ability of organisations, individuals and groups to be ‘responsive’ (ie capable of adapting to altered circumstances).  This is a merit.  But whether those changes are good or bad, and to whom is a matter of worth.  

Most discussions on merit and worth in evaluation refer to Scriven’s Thesaurus which is on a shelf about 10,000 kilometres away.  But this paper is frequently referenced :


It’s an excellent read anyway and covers a lot of territory about the evaluation argument.

On 8/01/2022, at 8:40 AM, Bernward Causemann <r...@causemann.org> wrote:

Dear Bob,

thanks for your interesting comments on the topic (and to others for their contributions).

One point I do not understand: Would you explain what you mean with 'confuse merit with worth', particularly in this context?

Kind regards

Bernward



Bob

unread,
Jan 8, 2022, 11:53:25 AM1/8/22