generalizability

[Mohammad Khaleghi]

Hi everyone
In GIMME manual it has been said that we need to at least 10 people for subgroup or group generalization. Do we can be confident about generalization if a few people drop out? Is it still reliable? 
    

[Katie Gates]

This is a great question. 

The simulation studies show that GIMME can reliably detect paths that exist for everyone in the sample ("group-level paths") with as few as 10 people.

Whether or not this is generalizable to the population from which the sample is drawn is a perhaps separate question. It might be hard to say that any results from any analyses on such a small sample can generalize to the public. You would have to ensure your sample is truly representative of the population to which you are trying to make an inference about. Even so, small samples can be perhaps best viewed as hypothesis-generating or pilot studies to test a research question. 

So, the results will be reliable, but perhaps not generalizable with very small samples. 

I wonder if anyone else reading this has a perspective to share? 

[Mohammad Khaleghi]

Thanks dear Katie
Yes, that would be a relatively a pilot study. So, we can have reliable group or subgroups with participants < 10, apart from generalization? 
 

--
You received this message because you are subscribed to the Google Groups "gimme-r" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gimme-r+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gimme-r/9fe44633-2fea-4555-b557-4c24f9e837a0n%40googlegroups.com.

--

Best wishes
Mohammad Khaleghi
Licensed Clinical Psychologist, Tehran, Iran

PhD candidate in Clinical Psychology

[Jonathan J Park]

Hi all,

I actually feel like I can weigh in here.

While working on a revision for a separate paper on the GVAR, we encountered a reviewer question about clustering with subjects within a group versus the total sample size and how those are somewhat confounded.

We added an additional simulation where we increased total N but varied the within-subgroup sample sizes and found that parameter estimates remained largely the same; to the third decimal place in most cases.

While not GIMME, the algorithm we're using for subgrouping and the method for constructing the adjacency matrices are largely the same and our general conclusions were that the subgrouping is more sensitive to the number of differences between groups in terms of dynamics rather than the number of people within subgroups.

So, I'd echo Katie's point that it's likely a reliable result (i.e., the identified subgroups are indeed different) but would be cautious about the generalizability of those results given the relatively small samples.

All of this being dependent on satisfying power requirements and the like.

Best,

Jonathan J. Park

[Mohammad Khaleghi]

Thanks Jonathan for your long explanation. Your experience and comments will help me in this way.   

To view this discussion on the web visit https://groups.google.com/d/msgid/gimme-r/126c145e-31db-4678-bf29-bd2d2c0a759an%40googlegroups.com.