Re: Split CS-GIMME errors - are the results useable/do I need to resolve some of them before writing up?

[Gates, Kathleen Marie]

Hi Henry,

These errors can happen during various iterations. Usually the code catches them and they are not printed to the console. It is only if it is in specific "if" rules that they are displayed. 

In general these are only of concern if they stop the program from continuing. 

The main thing to report in any paper is the final outcome. In the summaryFit doc, you will find things like "converged normally" or "computationally singular". 

These errors tend to happen because of less than perfect data qualities, like lots of missingness, low variability, multicollinearity, and/or short data. We see them in ESM/EMA data much more than we do with fMRI, for instance (we rarely get these for fMRI data).

This is what can be reported (e.g., ##% converged normally). 

Katie

_______________________________________

Katie (Kathleen) Gates, Ph.D.

Associate Professor 

Psychology and Neuroscience
344A Davie Hall

University of North Carolina

https://unc.zoom.us/my/katiegates

---

From: Henry Whitfield <mindfuln...@gmail.com>
Sent: Thursday, March 14, 2024 2:39 PM
To: gim...@googlegroups.com <gim...@googlegroups.com>
Cc: brandon.sanford0030 <brandon.s...@gmail.com>; gimme <gi...@unc.edu>; he...@presentmind.org <he...@presentmind.org>
Subject: Split CS-GIMME errors - are the results useable/do I need to resolve some of them before writing up?

 

You don't often get email from mindfuln...@gmail.com. Learn why this is important

Dear Katie et al.,,

I managed to run it on my own machine and saw there were many errors (as usual). 

During subgroup-level search: Error in modindices(fit, standardized = FALSE, sort. = FALSE) : lavaan ERROR: could not compute modification indices; information matrix is singular

x2 Does this mean there is something inadequate in my dataset?

In addition: There were 36 warnings (use warnings() to see them)

When I run"warnings()" I only get 6 errors. Why not 36?

lavaan WARNING: Maximum number of iterations reached when computing the sample moments using EM; use the em.h1.iter.max= argument to increase the number of iterations

Do I need to increase the maximum number of iterations here?

We got this error multiple times. If I increase the number of iterations, what number might I use? I don't know what the default is. 

lavaan WARNING: The smallest eigenvalue of the EM estimated variance-covariance matrix (Sigma) is smaller than 1e-05; this may cause numerical instabilities; interpret the results with caution.

Is there anything I can do about this?

Then during individual-level search - again:

lavaan ERROR: could not compute modification indices; information matrix is singular In addition: There were 27 warnings (use warnings() to see them)

Many of the other errors above are also repeated during the individual level search. 

However we also get these twice each:

In addition: Warning messages:

1: In lav_fit_cfi_lavobject(lavobject = object, fit.measures = fit.measures, : lavaan WARNING: computation of robust CFI failed.

2: In lav_fit_rmsea_lavobject(lavobject = object, fit.measures = fit.measures, : lavaan WARNING: computation of robust RMSEA failed.

And also:

6: In (function (..., deparse.level = 1) : number of columns of result is not a multiple of vector length (arg 5)

So, my unifying question for all of these errors is this. Do I need to respond to these errors and run it again, or do I have to just write it up as it is, mentioning all these errors?

Thanks so much for your help,

Henry Whitfield

Just in case, here is the full text feed with errors from R-Studio:  

 subgroup-level search
Error in modindices(fit, standardized = FALSE, sort. = FALSE) : 
  lavaan ERROR: could not compute modification indices; information matrix is singular
In addition: There were 36 warnings (use warnings() to see them)
subgroup-level search
Error in modindices(fit, standardized = FALSE, sort. = FALSE) : 
  lavaan ERROR: could not compute modification indices; information matrix is singular
In addition: Warning messages:
1: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    Maximum number of iterations reached when computing the sample
    moments using EM; use the em.h1.iter.max= argument to increase the
    number of iterations
2: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    The smallest eigenvalue of the EM estimated variance-covariance
    matrix (Sigma) is smaller than 1e-05; this may cause numerical
    instabilities; interpret the results with caution.
3: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    Maximum number of iterations reached when computing the sample
    moments using EM; use the em.h1.iter.max= argument to increase the
    number of iterations
Error in modindices(fit, standardized = FALSE, sort. = FALSE) : 
  lavaan ERROR: could not compute modification indices; information matrix is singular
subgroup-level search
Error in modindices(fit, standardized = FALSE, sort. = FALSE) : 
  lavaan ERROR: could not compute modification indices; information matrix is singular
In addition: Warning messages:
1: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    Maximum number of iterations reached when computing the sample
    moments using EM; use the em.h1.iter.max= argument to increase the
    number of iterations
2: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    The smallest eigenvalue of the EM estimated variance-covariance
    matrix (Sigma) is smaller than 1e-05; this may cause numerical
    instabilities; interpret the results with caution.
3: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    Maximum number of iterations reached when computing the sample
    moments using EM; use the em.h1.iter.max= argument to increase the
    number of iterations
subgroup-level search
subgroup-level search
subgroup-level search
subgroup-level search
subgroup-level search
subgroup-level pruning, subject 1
subgroup-level pruning, subject 2
subgroup-level pruning, subject 3
subgroup-level pruning, subject 4
subgroup-level pruning, subject 5
subgroup-level pruning, subject 6
subgroup-level pruning, subject 7
subgroup-level pruning, subject 8
subgroup-level pruning, subject 9
subgroup-level pruning, subject 10
subgroup-level pruning, subject 1
subgroup-level pruning, subject 2
subgroup-level pruning, subject 3
subgroup-level pruning, subject 4
subgroup-level pruning, subject 5
subgroup-level pruning, subject 6
subgroup-level pruning, subject 7
subgroup-level pruning, subject 8
subgroup-level pruning, subject 9
subgroup-level pruning, subject 10
subgroup-level pruning, subject 11
subgroup-level pruning, subject 12
subgroup-level pruning, subject 13
subgroup-level pruning, subject 14
subgroup-level pruning, subject 15
subgroup-level pruning, subject 16
subgroup-level pruning, subject 17
subgroup-level pruning, subject 18
subgroup-level pruning, subject 19
subgroup-level pruning, subject 20
group-level pruning, subject 1 (119)
group-level pruning, subject 2 (223)
group-level pruning, subject 3 (227)
group-level pruning, subject 4 (299)
group-level pruning, subject 5 (301)
group-level pruning, subject 6 (302)
group-level pruning, subject 7 (304)
group-level pruning, subject 8 (308)
group-level pruning, subject 9 (313)
group-level pruning, subject 10 (446)
group-level pruning, subject 11 (528)
group-level pruning, subject 12 (530)
group-level pruning, subject 13 (531)
group-level pruning, subject 14 (541)
group-level pruning, subject 15 (559)
group-level pruning, subject 16 (575)
group-level pruning, subject 17 (580)
group-level pruning, subject 18 (586)
group-level pruning, subject 19 (590)
group-level pruning, subject 20 (593)
group-level pruning, subject 21 (599)
group-level pruning, subject 22 (621)
group-level pruning, subject 23 (627)
group-level pruning, subject 24 (647)
group-level pruning, subject 25 (681)
group-level pruning, subject 26 (694)
group-level pruning, subject 27 (697)
group-level pruning, subject 28 (765)
group-level pruning, subject 29 (889)
group-level pruning, subject 30 (995)
individual-level search, subject 1 (119)
individual-level search, subject 2 (223)
individual-level search, subject 3 (227)
individual-level search, subject 4 (299)
individual-level search, subject 5 (301)
individual-level search, subject 6 (302)
Error in modindices(fit, standardized = FALSE, sort. = FALSE) : 
  lavaan ERROR: could not compute modification indices; information matrix is singular
In addition: There were 27 warnings (use warnings() to see them)
individual-level search, subject 7 (304)
Error in modindices(fit, standardized = FALSE, sort. = FALSE) : 
  lavaan ERROR: could not compute modification indices; information matrix is singular
In addition: Warning messages:
1: In lav_fit_cfi_lavobject(lavobject = object, fit.measures = fit.measures,  :
  lavaan WARNING: computation of robust CFI failed.
2: In lav_fit_rmsea_lavobject(lavobject = object, fit.measures = fit.measures,  :
  lavaan WARNING: computation of robust RMSEA failed.
3: In lav_fit_cfi_lavobject(lavobject = object, fit.measures = fit.measures,  :
  lavaan WARNING: computation of robust CFI failed.
4: In lav_fit_rmsea_lavobject(lavobject = object, fit.measures = fit.measures,  :
  lavaan WARNING: computation of robust RMSEA failed.
individual-level search, subject 8 (308)
individual-level search, subject 9 (313)
individual-level search, subject 10 (446)
individual-level search, subject 11 (528)
individual-level search, subject 12 (530)
individual-level search, subject 13 (531)
Error in modindices(fit, standardized = FALSE, sort. = FALSE) : 
  lavaan ERROR: could not compute modification indices; information matrix is singular
In addition: There were 20 warnings (use warnings() to see them)
Error in modindices(fit, standardized = FALSE, sort. = FALSE) : 
  lavaan ERROR: could not compute modification indices; information matrix is singular
In addition: There were 14 warnings (use warnings() to see them)
individual-level search, subject 14 (541)
individual-level search, subject 15 (559)
individual-level search, subject 16 (575)
individual-level search, subject 17 (580)
individual-level search, subject 18 (586)
individual-level search, subject 19 (590)
individual-level search, subject 20 (593)
individual-level search, subject 21 (599)
individual-level search, subject 22 (621)
individual-level search, subject 23 (627)
individual-level search, subject 24 (647)
Error in modindices(fit, standardized = FALSE, sort. = FALSE) : 
  lavaan ERROR: could not compute modification indices; information matrix is singular
In addition: There were 38 warnings (use warnings() to see them)
individual-level search, subject 25 (681)
Error in modindices(fit, standardized = FALSE, sort. = FALSE) : 
  lavaan ERROR: could not compute modification indices; information matrix is singular
In addition: There were 29 warnings (use warnings() to see them)
individual-level search, subject 26 (694)
individual-level search, subject 27 (697)
individual-level search, subject 28 (765)
individual-level search, subject 29 (889)
individual-level search, subject 30 (995)
gimme finished running normally
output is stored in E:/Split GIMME output
Number of subgroups = 2
Modularity = 0.01517
Warning messages:
1: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    Maximum number of iterations reached when computing the sample
    moments using EM; use the em.h1.iter.max= argument to increase the
    number of iterations
2: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    Maximum number of iterations reached when computing the sample
    moments using EM; use the em.h1.iter.max= argument to increase the
    number of iterations
3: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    Maximum number of iterations reached when computing the sample
    moments using EM; use the em.h1.iter.max= argument to increase the
    number of iterations
4: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    Maximum number of iterations reached when computing the sample
    moments using EM; use the em.h1.iter.max= argument to increase the
    number of iterations
5: In lav_mvnorm_missing_h1_estimate_moments(Y = X[[g]], wt = WT[[g]],  :
  lavaan WARNING:
    Maximum number of iterations reached when computing the sample
    moments using EM; use the em.h1.iter.max= argument to increase the
    number of iterations
6: In (function (..., deparse.level = 1)  :
  number of columns of result is not a multiple of vector length (arg 5)

Henry J. Whitfield MSc
PhD Candidate
ACBS peer-reviewed ACT trainer, TIR Trainer
Mindfulness Training Ltd.

http://www.psychflex.co.uk
Tel: 020 7183 2485

Like us on facebook!

[Henry Whitfield]

Hi again Katie,

Looking a little deeper I see the 4 "computationally singular" don't have individual GIMME models. 

I assume this means they were discounted as too problematic to run? These four participants did not have this issue when we ran S-GIMME. This seems to only be such an issue with CS-GIMME. 

Is that just bad luck with how the filtering between individual-subgroup levels works? I'd like to understand why they run so differently.

I'm now worried the CS-GIMME was not a good option. This is because the two treatment groups were already unevenly sized: 10+20.

With 4 participants missing and 3 from one treatment group we now have  7+19. Is that a correct assessment?

Can I convincingly argue that we have measured the two treatments groups to have different subgroup (green) edges?

Best wishes,

Henry

--
You received this message because you are subscribed to the Google Groups "gimme-r" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gimme-r+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gimme-r/BL1PR03MB61015337DF35C1CDD3927120F8292%40BL1PR03MB6101.namprd03.prod.outlook.com.

[Henry Whitfield]

Thanks Katie,

In the SummaryFit, just 4 are computationally singular. Does that mean we have to be wary of the result with those ones?

Another 4 are "Last know convergence". What does that mean?

Thanks again,

Henry

[Henry Whitfield]

[Katie Gates]

"Last known convergence" means that it is the last model that converged. When adding person-specific paths for that person to try to get fit indices that indicate a good fit, the model failed to converge. So the final model likely does not have adequate fit according to fit indices used (see output).

Computationally singular can happen for a number of reason, from the model not being identifiable or fitting the data or the covariance matrix not being invertible. These people won't have results. These are usually reported as "computationally singular" in papers and the remaining the individuals' output is used. 

[Katie Gates]

Hi Henry, 

CS-GIMME is fine with 26 people. We have tested with simulation studies that subgroup-level paths can be obtained reliably with as few as 10 people, so I can't say as to how reliable the results are with 7 people in a subgroup. At worst, you may miss a subgroup level path or two. It certainly won't cause false positives. 

Generalizability is always an issue (in any study, with any type of analyses) with small sample sizes. So per your question regarding can you argue that tx groups have different edges - yes, if they each have different subgroup-level (green) edges, for the sample you have. 

See prior email re computationally singular models. The model being applied to those individuals does not fit. I don't know a fix. 

Best,

Katie

[Gates, Kathleen Marie]

Hi Henry, 

Usually those errors are hidden. They must have emerged in an area of the code where we didn't have suppression code surrounding it. 

They are not to worry about - if convergence or small eigenvalues emerge during the model search it only is a problem if they are still there at the final stage. 

Here, you look to the final solution to see if it converged normally, last known convergence, or computationally singular. These warnings and errors that don't stop gimme are simply an annoyance. 

To your question regarding what can be done - these issues tend to happen because of high collinearity, high amount of missing data, low variability (i.e., most responses across days are the same), short time series, and/or lots of NAs. 

Hope this helps,

Katie

_______________________________________

Katie (Kathleen) Gates, Ph.D.

Associate Professor 

Psychology and Neuroscience
344A Davie Hall

University of North Carolina

https://unc.zoom.us/my/katiegates

---

From: gim...@googlegroups.com <gim...@googlegroups.com> on behalf of Henry Whitfield <mindfuln...@gmail.com>

Sent: Thursday, March 14, 2024 2:39 PM
To: gim...@googlegroups.com <gim...@googlegroups.com>
Cc: brandon.sanford0030 <brandon.s...@gmail.com>; gimme <gi...@unc.edu>; he...@presentmind.org <he...@presentmind.org>
Subject: Split CS-GIMME errors - are the results useable/do I need to resolve some of them before writing up?

You don't often get email from mindfuln...@gmail.com. Learn why this is important

--
You received this message because you are subscribed to the Google Groups "gimme-r" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gimme-r+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gimme-r/CADbRyiwjzneBSStgXHWLyKPBid6ittu4Ge2PTzHN_vgQtUpi%3Dw%40mail.gmail.com.

[Henry Whitfield]

Hi again Katie,

Thanks so much for this. 

Would it be accurate to report in the findings  that the four  "computationally singular" participants (with no individual models established) were not confirmed to be part of the CS-Subgroup edge patterns?

For instance, when an a priori group of n=10 dropped 3 participants with a   "computationally singular" result. Does this mean that the other 7 that 'converged normally' were confirmed to have the edge pattern for that "sub_membership" collumn (in SummaryFit) group 1 (also attached as PDF).

Would that be a correct interpretation? So 7 out of 10 were confirmed in that subgroup and 19 out of 20 in the other?

Many thanks,

Henry

On Thursday 14 March 2024 at 21:05:05 UTC Gates, Kathleen Marie wrote:

[Henry Whitfield]

And in addition to the below could I also just check:

The CS-GIMME has no walktrap and this detracts from how robust the models are? In my paper I'll report both S and CS-GIMME and feel I'll need to contrast how they are different. If the S-GIMME subgroups act as a filter between signal and noise then CS has a less robust filter? 

The 4 participants that CS- fails to run, S-GIMME succeeds in running. Can I say this latter is partly due to the different filter, or also because they simply don't fit the CS-subgroup edges that define the a priori determined categories?

Both my CS and S GIMME results have positive modularity results. S-GIMME has a higher value (0.0152-CSGIMME, 0.0912-SGIMME) 

Green edges are thicker (stronger) in the S-GIMME results. Can this also be an indication that the S-GIMME subgroups are more robust than the CS-GIMME subgroup edges?

Many thanks,

Henry

--
You received this message because you are subscribed to the Google Groups "gimme-r" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gimme-r+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gimme-r/b61f0074-cb2f-4f52-80d0-ece1e303929en%40googlegroups.com.

[Henry Whitfield]

And one other thing - sorry I didn't get al lt his in one email. 

I just noticed the SummaryPathsPlot for the CS GIMME can two Green edges in it that neither of the separate subgroupplots have. What does this mean? Who do the additional green edges belong to.

Many thanks,

Henry 

[Katie Gates]

Hi Henry, 

Compiling the questions here: 

Would it be accurate to report in the findings  that the four  "computationally singular" participants (with no individual models established) were not confirmed to be part of the CS-Subgroup edge patterns?

The algorithm isn't really testing if an individual belongs to a specific group when arriving at subgroup-level paths. So you'd have to be careful and precise about wording here. The statistical interpretation is that the model could not converge for these individuals due to the subgroup-level paths. So the model does not seem to match the data for these individuals.

The CS-GIMME has no walktrap and this detracts from how robust the models are?

This is not accurate. Walktrap doesn't addd to the robustness; the addition of subgroup-level paths has been found to. So obtaining subgroup-level paths via CS or S-gimme should, in theory, improve reliable detection of paths. 

The 4 participants that CS- fails to run, S-GIMME succeeds in running. Can I say this latter is partly due to the different filter, or also because they simply don't fit the CS-subgroup edges that define the a priori determined categories?

The latter. 

Both my CS and S GIMME results have positive modularity results. S-GIMME has a higher value (0.0152-CSGIMME, 0.0912-SGIMME) 

Green edges are thicker (stronger) in the S-GIMME results. Can this also be an indication that the S-GIMME subgroups are more robust than the CS-GIMME subgroup edges?

There is no cutoff for how different two results need to be in terms of modularity to be 'significant' or not due to chance. Since the same similarity matrix is used to calculate both modularity, you can compare them and say it is higher but with the caveat that statistical testing wasn't done. If you want to say a statement about robustness of subgroups, can test robustness of subgroups in CS and S using a package pertubR. 

Would that be a correct interpretation? So 7 out of 10 were confirmed in that subgroup and 19 out of 20 in the other?

I would not make this statement since teh algorithm isn't explicitly testing who belongs in what subgroup. It is simply looking for paths that are consistent across people.

I just noticed the SummaryPathsPlot for the CS GIMME can two Green edges in it that neither of the separate subgroupplots have. What does this mean? Who do the additional green edges belong to.

Can you please share the plots you're referring to? 

Very thoughtful Qs.
Katie

[Henry Whitfield]

Or is there also a way to check that the individual subgroup paths are correct so that we might ignore the 'summary paths plot' that is apparently incorrect?

On Thu, Dec 5, 2024 at 2:10 PM Henry Whitfield <mindfuln...@gmail.com> wrote:

Many thanks again for this Katie, 

I'm finally back on this after getting sucked into another paper. 

Regarding the plotting error ('the summary paths plot' had extra green paths in the CS-GIMME that are not in its separate subgroup plots), how do we go about fixing this?

I saw that GIMME was upgraded. Do you know if this was fixed?

Many thanks,

Henry

On Tue, Apr 23, 2024 at 8:44 PM Katie Gates <katie...@gmail.com> wrote:

Thanks for sharing, it looks like a bug in the plotting. 

The attached paper may help in explaining when the CS results differ form S-GIMME, and some advantages of S-gimme. 

On Tue, Apr 23, 2024 at 3:25 PM Henry Whitfield <mindfuln...@gmail.com> wrote:

Thanks so much for this Katie,

I attach the two varied file sets for S and CS-GIMME respectively. Each set included the two separate subgroup path files and the "Summary Paths plot", which seems to combine in SGIMME but has extra green paths in the CS-GIMME that are not in its separate subgroup plots. 

Also, regarding thispoints - comments added after "--"

"The CS-GIMME has no walktrap and this detracts from how robust the models are?

This is not accurate. Walktrap doesn't add to the robustness; the addition of subgroup-level paths has been found to. So obtaining subgroup-level paths via CS or S-gimme should, in theory, improve reliable detection of paths."

--So there aren't any disadvantages of adding the a priori subgroups? I was thinking being more SGIMME data driven was somehow closer to the data. I'm trying to understand what I can say about the CS results. The reason we used CS was to try and get a clearer view of whether treatment could be linked to different edge processes. I worry that having only 7 (of 10) left in the smaller treatment group will be looked down on as too small. With the S-GIMME (n=30) we already got a Phi correlation (0.55 (0.2-0,8)) between S-GIMME-subgroup and treatment. So my main question is what does the CS-GIMME subgroups add to this question of treatment contributing to process?

And, what can I say about the variation in CS-Subgroup edges and S-GIMME subgroup edges?

Thanks so much again,

Henry

 

You received this message because you are subscribed to a topic in the Google Groups "gimme-r" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gimme-r/aeD3FW84tx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gimme-r+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gimme-r/e3b77906-2640-4d00-9d3d-2e70134c152cn%40googlegroups.com.

[Henry Whitfield]

Many thanks again for this Katie, 

I'm finally back on this after getting sucked into another paper. 

Regarding the plotting error ('the summary paths plot' had extra green paths in the CS-GIMME that are not in its separate subgroup plots), how do we go about fixing this?

I saw that GIMME was upgraded. Do you know if this was fixed?

Many thanks,

Henry

On Tue, Apr 23, 2024 at 8:44 PM Katie Gates <katie...@gmail.com> wrote:

Thanks for sharing, it looks like a bug in the plotting. 

The attached paper may help in explaining when the CS results differ form S-GIMME, and some advantages of S-gimme. 

On Tue, Apr 23, 2024 at 3:25 PM Henry Whitfield <mindfuln...@gmail.com> wrote:

Thanks so much for this Katie,

I attach the two varied file sets for S and CS-GIMME respectively. Each set included the two separate subgroup path files and the "Summary Paths plot", which seems to combine in SGIMME but has extra green paths in the CS-GIMME that are not in its separate subgroup plots. 

Also, regarding thispoints - comments added after "--"

"The CS-GIMME has no walktrap and this detracts from how robust the models are?

This is not accurate. Walktrap doesn't add to the robustness; the addition of subgroup-level paths has been found to. So obtaining subgroup-level paths via CS or S-gimme should, in theory, improve reliable detection of paths."

--So there aren't any disadvantages of adding the a priori subgroups? I was thinking being more SGIMME data driven was somehow closer to the data. I'm trying to understand what I can say about the CS results. The reason we used CS was to try and get a clearer view of whether treatment could be linked to different edge processes. I worry that having only 7 (of 10) left in the smaller treatment group will be looked down on as too small. With the S-GIMME (n=30) we already got a Phi correlation (0.55 (0.2-0,8)) between S-GIMME-subgroup and treatment. So my main question is what does the CS-GIMME subgroups add to this question of treatment contributing to process?

And, what can I say about the variation in CS-Subgroup edges and S-GIMME subgroup edges?

Thanks so much again,

Henry

 

On Tue, Apr 23, 2024 at 2:38 PM Katie Gates <katie...@gmail.com> wrote:

You received this message because you are subscribed to a topic in the Google Groups "gimme-r" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gimme-r/aeD3FW84tx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gimme-r+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gimme-r/e3b77906-2640-4d00-9d3d-2e70134c152cn%40googlegroups.com.

[Katie Gates]

Hi Henry, 

Did the error occur with the updated package? 

Best,

Katie

[Henry Whitfield]

Hi Katie, 

Now that I'm back on this i just ran it with the latest that R Studio GIMME: ver. 0.7-18 . I got the same summary paths plot for the CS-GIMME. I attach them. Is there definitely an error in the discrepancy between these? If so what can I do about it?

Another Question I have is why the same green subgroup pathway (Openness -> Personality in way) that is in both subgroups, is not in the overall (black) group pathways. I guess it's because it was just below .75 cut off even though it's detected in both?  

In the GIMME on the same dataset (with same cut offs) the group (black) pathways are 3 in a chaim, whilst the CS-GIMME only has one (the first in the chain) pathway. Is that normal that the group pathways on the same dataset are so different when the CS dataframe of a priori observations is added?

Another variation between the GIMME and CS-GIMME is that there are 5 'last known convergence' + 4 'computationally singular' results in the latter, whilst only 2' last known convergence' and zero  'computationally singular' in the GIMME. In the discussion on this would I say the GIMME model fit is significantly better than the CS-GIMME, as it alowed the data to speak more freely?   Reading back up the thread i see you already said:

 "The algorithm isn't really testing if an individual belongs to a specific group when arriving at subgroup-level paths. So you'd have to be careful and precise about wording here. The statistical interpretation is that the model could not converge for these individuals due to the subgroup-level paths. So the model does not seem to match the data for these individuals." So based on that I could say that the CS-GIMME confirmed the those that converged normally? That would be 6/10 of one treatment and 15/20 of the other (if I discount 5 'last known convergence' + 4 'computationally singular' results)? 

Any guidance of discussing the contrast between these results would be greatly appreciated. The GIMME subgroups correlated 0.5 with treatment groups.

Many thanks again for you help,

Henry

To view this discussion visit https://groups.google.com/d/msgid/gimme-r/c6290855-0762-4075-a6cf-cd2b09a57fbcn%40googlegroups.com.

[Katie Gates]

Hi Henry, 

Given time constraints, I can offer some quick general knowledge about the algorithm: 

- We have a check that makes a path group-level if it exists for >75% of the participants and is identified as an individual-level path. We don't do this for subgroup-level paths because it might be importatnt that some subgroups don't have this path. do you have subgroups = 1 in your data? 

- I can't say what is normal vs. not normal. In one case you got group-level paths, in another they were subgroup-level. Group-level paths are typically added at the front of the algorithm - however, if in the CS = FALSE case a path was found to exist for 75% of individuals after the indv. search then it will be bumped up to group level and all models re-estimated. So in the CS case it was justu found sooner than the CS=FALSE case. 

- these visuals are offered just to help. If they don't convey the information you think is relevant (i.e., the majority of people tend to have a path but in CS it was called "subgropu-level") that can always be changed. So you can make green paths black, etc for publication using whatever software you would like. 

- Sometimes specific models won't converge for some people. Sounds like different models were tried for different people in the two runs. Sometimes a person converged, sometimes they didn't. I'm not sure it's substantively meaningful but could be. 

Hope this helps, 
Katie