"multimodal" posterior distribution/weak priors for GoNoGo-Task

[franzis...@gmail.com]

Dear everyone,

I have encountered the following (not surprising) situation when fitting one model over data from multiple groups that group differences on the parameters are very much blurred or not present at all, when however fitting the models separately over each group and then comparing the parameters the differences in the parameters are very prominent again.

Therefore I would either like to 

1) use deliberately weak group priors, or 

2) allow the fitting to decide *simultaneously* the difference between the means of potential different group distributions. 

In order to achieve (1) I am increasing the prior values from 1 to 10 for the individual parameters, is that correct? That should give me less informative, very weak priors, right?

// individual parameters w/ Matt trick

xi_pr ~ normal(0, 10);
ep_pr ~ normal(0, 10);
b_pr ~ normal(0, 10);
pi_pr ~ normal(0, 10);
rhoRew_pr ~ normal(0, 10);
rhoPun_pr ~ normal(0, 10);

How would I achieve (2), is there a distribution other than the Bernoulli that allows multiple peaks? Sorry if this is not very specific, I don't have a back ground in maths or computer science.

Thank you for your help in advance.

Franziska

[David Munoz]

I am really interested by this too since I have experienced the exact same "issue".

[Haines, Nathaniel B.]

Hi Franziska, 

For (1), you would need to modify the group-level means/SDs as opposed to the person-level parameters that you included in the code. e.g., the following: 

xi_pr ~ normal(0, 10);
ep_pr ~ normal(0, 10);
b_pr ~ normal(0, 10);
pi_pr ~ normal(0, 10);
rhoRew_pr ~ normal(0, 10);
rhoPun_pr ~ normal(0, 10);

Should be kept as:

xi_pr ~ normal(0, 1);
ep_pr ~ normal(0, 1);
b_pr ~ normal(0, 1);
pi_pr ~ normal(0, 1);
rhoRew_pr ~ normal(0, 1);
rhoPun_pr ~ normal(0, 1);

Then, you can change the group-level means/SDs by modifying the following to by more diffuse:

mu_pr ~ normal(0, 1);
sigma ~ normal(0, .2);

For (2), I am not sure I understand what your intention is—if you fit both groups separately, this is the same as estimating a different group-level mean and standard deviation within the same model. So, you are using groups come from different populations and estimating that difference. If you instead fit a single model to both groups, you are assuming that there is really a single group/population. Is your intention to take one of these approaches?

Let me know if that is clear! 

Best,

Nate

---

From: hbayesd...@googlegroups.com <hbayesd...@googlegroups.com> on behalf of franzis...@gmail.com <franzis...@gmail.com>
Sent: Friday, October 22, 2021 5:21 AM
To: hbayesdm-users <hbayesd...@googlegroups.com>
Subject: "multimodal" posterior distribution/weak priors for GoNoGo-Task

 

--
You received this message because you are subscribed to the Google Groups "hbayesdm-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbayesdm-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hbayesdm-users/65c103c2-874b-431f-90c9-8c763e8031e5n%40googlegroups.com.

[Haines, Nathaniel B.]

Oops, small typo: 

"So, you are using groups come from different populations and estimating that difference." 

-> 

So, you are assuming groups come from different populations and estimating that difference.

---

From: hbayesd...@googlegroups.com <hbayesd...@googlegroups.com> on behalf of Haines, Nathaniel B. <haine...@buckeyemail.osu.edu>
Sent: Friday, October 22, 2021 3:41 PM
To: franzis...@gmail.com <franzis...@gmail.com>; hbayesdm-users <hbayesd...@googlegroups.com>
Subject: Re: "multimodal" posterior distribution/weak priors for GoNoGo-Task

 

To view this discussion on the web visit https://groups.google.com/d/msgid/hbayesdm-users/CH2PR01MB57669E4FA563A33D623042318D809%40CH2PR01MB5766.prod.exchangelabs.com.

[Franziska Knolle]

Dear Nate,

Thank you for your response. Much appreciated!

I does make sense to change the change the group level parameters.

I believe you are referring to those, for the gng_m4:

// hyper parameters
	mu_pr[1] ~ normal(0, 1.0);
	mu_pr[2] ~ normal(0, 1.0);
	mu_pr[3] ~ normal(0, 10.0);
	mu_pr[4] ~ normal(0, 10.0);
	mu_pr[5] ~ normal(0, 1.0);
	mu_pr[6] ~ normal(0, 1.0);
	sigma[1:2] ~ normal(0, 0.2);
	sigma[3:4] ~ cauchy(0, 1.0);
	sigma[5:6] ~ normal(0, 0.2);

The mu_pr[3]/mu_pr[4] for go bias and pavlovian bias are already quite weak, does it make sense to lower the prior further, and similarly for all 6 parameters (say e.g. factor of 10) and accordingly for the the sigma? What value would correspond to a prior which has an SD say twice the observed SD?

Regarding the second question, I do understand what you are saying but the problem is that if you are modelling a sample for which is drawn from different populations, say at risk individuals and healthy controls, than modelling them together and assuming a group population distribution minimises the detectable differences when comparing parameters across the groups. Classification based on the parameters is not possible as differences are two weak. A separate modelling of groups is only then possible if you actually know the different groups, but if you want to use the modelled parameters for classification of patients and controls for example and you are only assuming that there are two different groups, the modelling of for example a bimodal posterior would be more sensitive to potential systematic group differences. Therefore fitting all together, but allow the fitting to decide between different means of group distributions, would be extremely helpful.

Thank you again!

Franziska

[Haines, Nathaniel B.]

Ahhh OK I think I am clear on the dilemma now. If you plan to use the person-level estimates in a secondary analysis (e.g., unsupervised clustering), then either assuming a single group or two (known/labelled) groups can negatively impact the unsupervised clustering. Is that the general idea, or is there something else I could be missing?

If that is the case, then it may be best to not use any prior information (or at least, very minimal prior information). To do that, you could make the priors uniform/near uniform, or alternatively set the group-level means/SDs to specific, diffuse values as opposed to estimating them from data. Then, you can ensure that the person-level estimate are minimally regularized/pooled for use in a secondary analysis. 

The only real alternative would be to do it in a fully generative way, which would require simultaneously doing the clustering and person-level parameter estimation in a single model. e.g., something like assuming a mixture model as the group-level model, as opposed to assuming each person-level parameter comes from a single normal distribution. But this is rather complex and would require a great deal of custom coding in Stan. 

That said, I would recommend taking the first approach (making the priors uninformative/near uniform). This way, information from one person does not influence estimates from another person. To get an idea of how to modify the priors, you can do some simulations in R. e.g., for the go bias, you can choose means/SDs, draw from a normal distribution, and plot things out:

hist(rnorm(100000, mu, sd))

For the bounded parameters, you would need to also account for the parameter transformations. e.g., for learning rate:

hist(pnorm(rnorm(100000, mu, sd)))

The `pnorm` function is the same as the `Phi_approx` function in the stan code (see the transformed parameters block). Note that you can get a uniform distribution on the learning by setting the mean to 0 and sd to 1. 

So, overall, I think just setting mu_pr and sigma parameters to specific values that ensure the person-level parameters they represent are distributed near uniformly should give you the results that you are looking for. You can set these values by replacing the mu_pr[1-6] and sigma[1-6] variables with specific values in the transformed parameters block (e.g., following from the example using the `pnorm` function above, below I changed the group-level ep parameterization to ensure that ep is distributed uniformly across people by replacing the group-level mean and SD parameters with specific values of 0 and 1):

"

transformed parameters {
	vector<lower=0, upper=1>[N] xi;
	vector<lower=0, upper=1>[N] ep;
	vector[N] b;
	vector[N] pi;
	vector<lower=0>[N] rhoRew;
	vector<lower=0>[N] rhoPun;
	for (i in 1:N) {
	xi[i]  = Phi_approx(mu_pr[1] + sigma[1] * xi_pr[i]);
	ep[i]  = Phi_approx(0 + 1 * ep_pr[i]);
	}
	b      = mu_pr[3] + sigma[3] * b_pr; // vectorization
	pi     = mu_pr[4] + sigma[4] * pi_pr;
	rhoRew = exp(mu_pr[5] + sigma[5] * rhoRew_pr);
	rhoPun = exp(mu_pr[6] + sigma[6] * rhoPun_pr);
	}

"

---

From: Franziska Knolle <franzis...@gmail.com>
Sent: Friday, October 22, 2021 4:23 PM
To: Haines, Nathaniel B. <haine...@buckeyemail.osu.edu>
Cc: hbayesdm-users <hbayesd...@googlegroups.com>
Subject: Re: "multimodal" posterior distribution/weak priors for GoNoGo-Task

 

[Lei Zhang]

hi Franziska (and Nate!)

I was thinking maybe fitting the model non-hierarchically, if that makes better sense for the subsequent classification?

cheers,

Lei

To view this discussion on the web visit https://groups.google.com/d/msgid/hbayesdm-users/CH2PR01MB57668AB692A2FC6D054899448D809%40CH2PR01MB5766.prod.exchangelabs.com.

[Franziska Knolle]

Dear Nate,
thank you so much for your detailed feedback.
That is exactly what I mean - you are not missing anything!

I will take your advice and go with the first approach of making the priors uninformative or near uniform.

Nevertheless, I have to ask these probably extremely basic questions, as I am simply not that familiar/skilled with general modelling approaches:

How do I generate sensible simulations for mu (for the sensitivity of reward e.g.), in order to then plot the histogram?
And what should guide my selection of the values when I am looking at the histogram in oder to choose the appropriate values? (To extend of my missing knowledge is now very obvious :S)
Finally, since it was the sensitivity to reward and punishment that showed extreme differences comparing parameters drawn from separate and combined group modelling - would I have to set all priors to uninformative/near uniform or just a selection for example of sensitivity to reward/punishment?

Thank you very much again for your help - I am aware that I am asking for very detailed, so your responses are very much appreciated!

Best wishes & thank you!
Franziska

[franzis...@gmail.com]

Dear Nate,

Thank you again for your help; we have been looking at this problem quite a bit.

I do understand how to transform the learning rate into a near uniform distribution, setting mu and sigma to 0 and 1 respectively makes total sense. That is the same for the lapse rate. For reward and punishment sensitivity (which is exponentially transformed), setting mu and sigma to 0 and 2 respectively produces a near uniform distribution. 

However, I don't understand how it set the values for the go bias and the pavlovian bias, both are normally distributed. The only way it seems I am able to transform a normal distribution into a uniform distribution is by setting the sigma to 0? Consequently does it make sense to set  mu and sigma to 0 and 0 respectively, for the go bias and the pavlovian bias?

Thank you

Best wishes

Franziska

[wooyou...@gmail.com]

Hi Franziska,

Many apologies for the long delay on this!

Go and Pavlovian bias parameters are not transformed, so we applied a weakly informative prior for these parameters. My understanding is that priors will have minimum influence on their parameter estimation while we set appropriate ranges for the parameters. Please let me know if you have further questions!

Best,

Young

[franzis...@gmail.com]

Dear Young 

Thank you for getting back.

This is a persisting issue, as when modelling patient group posterior distributions are vastly different when modelling groups separately, compared to a joint modelling approach. Therefor the question was whether it is possible to inform the model of the existence of two (or more) groups, letting the model choose whether to assign a subject to one or the other distribution?

Is that possible at all? 

Thank you

Franziska

[Eunhwi Lee]

Hi Franziska,

The current approach in hBayesDM assumes a single hyper group, so it might be inappropriate to use for what you've asked. Maybe you should consider nonparametric Bayesian or hierarchical mixture modeling? 

Please don't hesitate to ask me if you have further questions.

Best,

Eunhwi

2023년 6월 29일 목요일 오후 3시 42분 32초 UTC+9에 franzis...@gmail.com님이 작성: