multiclass cluster correction

[LA]

Hi Nick,
I'm running a searchlight classification with 8 classes/subjects. I want to decode subject identity across different time points (i.e., 8 subs with 3 scans), so I extracted the confusion matrix to calculate the accuracy and the f1 score for each class. Finally, I'm using cosmo_montecarlo_cluster_stat (1 target, 8 chunks, chance level of 0.125) to perform the cluster correction using both the f1 score and the accuracy as my input (separately). The final maps look odd in two respects:

1. instead of a gradient of z-scores, I get a unique z-score across all voxels within the bigger clusters (z = 3.7). These voxels contain different f1 scores/accuracy values, so I would expect different z-scores (see attached).
2. The maximum z it's always the same = 3.7.

Any idea what the problem might be?
Thanks!

[Nick Oosterhof]

Greetings,

On 14 Jun 2023, at 21:16, LA <la...@georgetown.edu> wrote:

Hi Nick,

I did not study your maps, but I think you may have a highly significant effect. In that case, you may find a z-score with a corresponding p-score consistent with the inverse of the number of iterations. A maxed-out feature means that in all null iterations (random shuffles), the (absolute) TFCE score for that feature was less than in the original (non-shuffled) data. You may want to try a higher number of iterations, but even then it is possible to still get maxed-out features if the effect is very strong. If the effect is very strong and this makes localisation difficult, you may also consider Bonferroni correction, although that approach is very conservative for whole brain correction for multiple comparisons.