Generating group accuracy maps

169 views

Skip to first unread message

Nora Heinzelmann

unread,

Jan 7, 2020, 2:46:41 PM1/7/20

to cosm...@googlegroups.com

Dear members,

Following the approach mentioned here -- https://groups.google.com/forum/#!searchin/cosmomvpa/group$20map|sort:date/cosmomvpa/eFLcWqKwVPk/08X6fkWNBwAJ

-- I am stuck on step 2 because I am unsure how to generate the group accuracy maps. I have generated 100 .nii files (chance accuracy maps) for each of my N participants and randomly generated 10,000 lists listing N .nii file names each ("LIST").

What command(s) do I use for generating the group maps? Should it be something like

ttest(LIST)

? I do not have the Statistics and Machine Learning Toolbox.

Apologies if I have overlooked the obvious answer to my question -- this is all rather new to me. Thanks,

Nora

Faculty of Philosophy

Ludwig Maximilians University

Munich

Germany

Nick Oosterhof

unread,

Jan 8, 2020, 1:24:25 PM1/8/20

to Nora Heinzelmann, CoSMoMVPA

Greetings,

the null datasets should each be dataset struct, obtained using (in this case) cosmo_fmri_dataset, each with samples of size 1 x n_voxels. You should use the same common group mask for all participants (although data from some participants may contain NaN values).Â

It is not supported to you your own random list, cosmo_montecarlo_cluster_stat does that for you. However, by setting the seed value you can get pseudo-random, reproducible selections from the null.

I don't think the statistics and machine learning toolbox are required.

Does that answer your questions?

--
You received this message because you are subscribed to the Google Groups "CoSMoMVPA" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cosmomvpa+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/cosmomvpa/CAHe_OYHQf8tFXo%3DsCznrez1wOMTfWTYhX-NXhUx_3AzuphTTMw%40mail.gmail.com.

Nora Heinzelmann

unread,

Jan 10, 2020, 2:27:44 PM1/10/20

to Nick Oosterhof, CoSMoMVPA

Dear Nick,

Thank you so much for the quick response. I think I understand better now but would just like to double check with you: am I supposed to generate a new (group) data set from the individual (chance) accuracy maps, or from the original data? How do I input multiple nii files to cosmo_fmri_dataset?

Thanks,

Nora

Nick Oosterhof

unread,

Jan 15, 2020, 12:00:15 PM1/15/20

to Nora Heinzelmann, CoSMoMVPA

Hi Nora,

On Fri, 10 Jan 2020 at 20:27, Nora Heinzelmann <nora.hei...@googlemail.com> wrote:

Dear Nick,
Thank you so much for the quick response. I think I understand better now but would just like to double check with you: am I supposed to generate a new (group) data set from the individual (chance) accuracy maps

yes.

How do I input multiple nii files to cosmo_fmri_dataset?

You cannot, but you can use cosmo_stack to combine multiple datasets.

For example: see http://cosmomvpa.org/faq.html#make-an-intersection-mask-across-participants

Nora Heinzelmann

unread,

Jan 23, 2020, 5:40:41 AM1/23/20

to CoSMoMVPA

Dear Nick,

Thanks so much, this worked very well. Sorry I did not realise I should use cosmo_stack to form group datasets.

I created a group dataset as follows:

for p=1:numSubj % for each participant
actual_dataset{p,1} = cosmo_fmri_dataset(fullfile(sprintf('%s',p,'xVal_no0.nii')); % get accuracy map
for r=1:100 % get all the null maps of this participant
null_datasets{p,r} = cosmo_fmri_dataset(fullfile(sprintf('%s',p),sprintf('xVal_no%d.nii',r))); % get r-th nul map
end
end
ds_group=cosmo_stack(actual_dataset,1); % combine to group dataset
ds_group.sa.chunks=ones(size(ds_group.samples,1),1).*(1:size(ds_group.samples,1))'; % list of subj numbers
ds_group.sa.targets=ones(size(ds_group.samples,1),1); % for t-test

But now when I tried to use cosmo_montecarlo_cluster_stat, as follows...

nbrhood = cosmo_cluster_neighborhood(ds_group);
cosmo_montecarlo_cluster_stat(ds_group,nbrhood,'h0_mean',0,'null', null_datasets,'niter',10);

...I get the following error:

Error using cosmo_montecarlo_cluster_stat>check_null_datasets (line 746)
1-th null dataset: .samples size mismatch with dataset

And indeed, my null datasets are [1 153594] whilst the ds_group dataset is [numSubj 153594].

But when I do...

nbrhood2 = cosmo_cluster_neighborhood(actual_dataset{:});

cosmo_montecarlo_cluster_stat(actual_dataset{:},nbrhood2,'h0_mean',0,'null', null_datasets,'niter',10);

... I get

Error using cosmo_check_neighborhood (line 69)
check_basis: field 'samples' not allowed in neighborhood

Can you advise?

Thanks so much in advance,

Nora

Nick Oosterhof

unread,

Jan 23, 2020, 12:11:15 PM1/23/20

to Nora Heinzelmann, CoSMoMVPA

Hi Nora,

On Thu, 23 Jan 2020 at 11:40, 'Nora Heinzelmann' via CoSMoMVPA <cosm...@googlegroups.com> wrote:
>
> Dear Nick,
> Thanks so much, this worked very well. Sorry I did not realise I should use cosmo_stack to form group datasets.
> I created a group dataset as follows:
>>
>> for p=1:numSubj % for each participant
>> actual_dataset{p,1} = cosmo_fmri_dataset(fullfile(sprintf('%s',p,'xVal_no0.nii')); % get accuracy map
>> for r=1:100 % get all the null maps of this participant
>> null_datasets{p,r} = cosmo_fmri_dataset(fullfile(sprintf('%s',p),sprintf('xVal_no%d.nii',r))); % get r-th nul map
>> end
>> end
>> ds_group=cosmo_stack(actual_dataset,1); % combine to group dataset
>> ds_group.sa.chunks=ones(size(ds_group.samples,1),1).*(1:size(ds_group.samples,1))'; % list of subj numbers
>> ds_group.sa.targets=ones(size(ds_group.samples,1),1); % for t-test
>
>
> But now when I tried to use cosmo_montecarlo_cluster_stat, as follows...
>>
>> nbrhood = cosmo_cluster_neighborhood(ds_group);
>> cosmo_montecarlo_cluster_stat(ds_group,nbrhood,'h0_mean',0,'null', null_datasets,'niter',10);
>
>
> ...I get the following error:
>>
>> Error using cosmo_montecarlo_cluster_stat>check_null_datasets (line 746)
>> 1-th null dataset: .samples size mismatch with dataset
>
> And indeed, my null datasets are [1 153594] whilst the ds_group dataset is [numSubj 153594].

The null datasets should be placed in a cell and all have .samples of size numSubj x nVoxels.
Something like:

n_null=100;

n_subj=20;

null_cell=cell(n_null,1);

for i_null=1:n_null

subj_cell=cell(1,n_subj);

for i_subj=1:n_subj

fn=sprintf('s%02d_null%03d.nii',i_subj,i_null);

ds=cosmo_fmri_dataset(fn,'mask','common_group_mask.nii');

% (assume single volume)

ds.sa.targets=1;

ds.sa.chunks=i_subj;

subj_cell{i_subj}=ds;

end

null_cell{i_null} = cosmo_stack(subj_cell,1);

end

Does that help?

Nick Oosterhof

unread,

Jan 24, 2020, 12:27:12 PM1/24/20

to Nora Heinzelmann, CoSMoMVPA

On Fri, 24 Jan 2020 at 18:07, Nora Heinzelmann <nora.hei...@googlemail.com> wrote:
>
> Dear Nick,

> Thank you SO much! I am afraid this is a bit like a game of Whac-A-Mole, with one issue disappearing the next one popping up.
> For masking the group ds, I used ...
>>
>> [~,ds]=cosmo_mask_dim_intersect(ds);
>
>
> ... which worked well for the actual dataset but gives me an error for the null datasets:
>>
>> Error using cosmo_mask_dim_intersect (line 112)
>> Assertion failed.

Actually cosmo_mask_dim_intersect is supposed to work with cell inputs (where each element in the cell is a dataset struct), not with a dataset struct directly. Can you show the output of cosmo_disp(ds) assuming ds is the input that you provide in cosmo_mask_dim_intersect(ds) ?

>
>
> So I tried to just use the null datasets without masking but then I get this error (for cosmo_montecarlo_cluster_stat):

>>
>> Error using cosmo_check_neighborhood (line 69)
>> check_basis: field 'samples' not allowed in neighborhood

The second argument of cosmo_montecarlo_cluster_stat must be a neighbourhood, usually from cosmo_cluster_neighborhood. The null datasets must be provided using the 'null' option (or in a struct with a field named 'null').

Nick Oosterhof

unread,

Jan 25, 2020, 5:34:11 AM1/25/20

to Nora Heinzelmann, CoSMoMVPA

Hi Nora,

you should not use the ds{:} syntax, it expands the elements in the cell as arguments. For example, if

cell_elems = {1,2,'ab'}

then for a function f,

f(cell_elems{:})

expands to

f(1,2,'ab')

Can you try without the '{:}'-syntax? (And please cc cosm...@googlegroups.com)?

On Sat, 25 Jan 2020 at 09:29, Nora Heinzelmann <nora.hei...@googlemail.com> wrote:

Dear Nick,

> For masking the group ds, I used ...
>>
>> [~,ds]=cosmo_mask_dim_intersect(ds);
>
>
> ... which worked well for the actual dataset but gives me an error for the null datasets:
>>
>> Error using cosmo_mask_dim_intersect (line 112)
>> Assertion failed.

Actually cosmo_mask_dim_intersect is supposed to work with cell inputs (where each element in the cell is a dataset struct), not with a dataset struct directly. Can you show the output of cosmo_disp(ds) assuming ds is the input that you provide in cosmo_mask_dim_intersect(ds) ?

I get
{ .a
.vol .mat
[ -3 0 0 81       0 3 0 -115        0 0 3 -53
0 0 0 1 ]
.xform
'scanner_anat'
.dim
[ 53 63 46 ]
.fdim
.labels
{ 'i'
'j'
'k' }
.values
{ [ 1 2 3 ... 51 52 53 ]@1x53
[ 1 2 3 ... 61 62 63 ]@1x63
[ 1 2 3 ... 44 45 46 ]@1x46 }
.sa
struct (empty)
.samples
[ 0 0 0 ... 0 0 0 ]@1x153594
.fa
.i
[ 1 2 3 ... 51 52 53 ]@1x153594
.j
[ 1 1 1 ... 63 63 63 ]@1x153594
.k
[ 1 1 1 ... 46 46 46 ]@1x153594

etc., for each participant.
I also checked
iscell(ds)

ans =

logical

1

The second argument of cosmo_montecarlo_cluster_stat must be a neighbourhood, usually from cosmo_cluster_neighborhood. The null datasets must be provided using the 'null' option (or in a struct with a field named 'null').

I tried
nbrhood2 = cosmo_cluster_neighborhood(ds{:});
ds_z = cosmo_montecarlo_cluster_stat(ds{:},nbrhood2,'h0_mean',0,'null', null_datasets,'niter',10);

... which then gave me the error mentioned in the last email (field 'samples' not allowed in neighborhood).

Very best,
Nora

Nora Heinzelmann

unread,

Jan 26, 2020, 10:03:30 AM1/26/20

to Nick Oosterhof, CoSMoMVPA

Hi Nick and all,

you should not use the ds{:} syntax, it expands the elements in the cell as arguments.

Thanks for this explanation, I was not aware of this.

Because cosmo_cluster_neighborhood seems to require a struct as input, I have now tried generating a struct from the ds, as follows:

bla = struct('a',[],'sa',[],'samples',[],'fa',[]);
bla2 = repmat(bla,nsubj,1);
for i=1:nsubj
bla2(i)=ds{i};
end

When using ...

nbrhood2 = cosmo_cluster_neighborhood(bla2);

... I now get the following error:

Error using size
Too many input arguments.

I get this error for

size(bla2.samples)

bla2 =

27×1 struct array with fields:

a
sa
samples
fa

I guess I am creating this struct incorrectly?

Best,

Nora

PS: Apologies for hitting "reply" rather than "reply all" last time!

Nick Oosterhof

unread,

Jan 26, 2020, 12:06:42 PM1/26/20

to Nora Heinzelmann, CoSMoMVPA

Hi Nora

On Sun, 26 Jan 2020 at 16:03, Nora Heinzelmann <nora.hei...@googlemail.com> wrote:

you should not use the ds{:} syntax, it expands the elements in the cell as arguments.
Thanks for this explanation, I was not aware of this.

Because cosmo_cluster_neighborhood seems to require a struct as input, I have now tried generating a struct from the ds, as follows:
bla = struct('a',[],'sa',[],'samples',[],'fa',[]);
bla2 = repmat(bla,nsubj,1);
for i=1:nsubj
bla2(i)=ds{i};
end

When using ...
nbrhood2 = cosmo_cluster_neighborhood(bla2);

... I now get the following error:
Error using size
Too many input arguments.

I get this error for
size(bla2.samples)

bla2 =

27×1 struct array with fields:

a
sa
samples
fa

I guess I am creating this struct incorrectly?

Yes, the struct should be from

ds_group = cosmo_stack(ds_cell)

nbrhood = cosmo_cluster_neighborhood(ds_group)

assuming that ds_cell is a 27x1 cell with the datasets (Note: In your code 'ds' is a cell, which can be confusing as CoSMoMVPA in code and documentation uses the convention that ds is a dataset struct (of size 1x1) with a field .samples (of size n_samples x n_features)).

You wrote earlier that:

> For masking the group ds, I used ...
>>
>> [~,ds]=cosmo_mask_dim_intersect(ds);
>
>
> ... which worked well for the actual dataset but gives me an error for the null datasets:
>>
>> Error using cosmo_mask_dim_intersect (line 112)
>> Assertion failed.

That assertion failed is quite surprising to me. What the is input you use for the null datasets (full output of cosmo_disp)?

Nora Heinzelmann

unread,

Jan 26, 2020, 4:19:53 PM1/26/20

to Nick Oosterhof, CoSMoMVPA

Hi Nick,

Ah! Now I did

actual_dataset = cell(n_subj,1);
for p=1:n_subj
actual_dataset{p,1} = cosmo_fmri_dataset(fullfile(pathAnalysis,sprintf('%dxVal_no0.nii',p));
actual_dataset{p,1}.sa.targets = 1;
actual_dataset{p,1}.sa.chunks = p;
end
[~,ds_group]=cosmo_mask_dim_intersect(actual_dataset);

ds_group = cosmo_stack(ds_group);

null_cell=cell(n_null,1);
for i_null=1:n_null
subj_cell=cell(1,n_subj);
for i_subj=1:n_subj

ds = cosmo_fmri_dataset(fullfile(pathAnalysis,sprintf('%dxVal_no%d.nii',p,i_null)));
ds.sa.targets=1;
ds.sa.chunks=p;
subj_cell{p}=ds;

end
null_cell{i_null} = cosmo_stack(subj_cell,1);
end

nbrhood2 = cosmo_cluster_neighborhood(ds_group);
ds_z = cosmo_montecarlo_cluster_stat(ds_group,nbrhood2,'h0_mean',0,'null', null_datasets,'niter',10);

That seems to get around the error for cosmo_mask_dim_intersect that I mentioned earlier.

I get an error on the last line:

Error using cosmo_check_external>check_matlab_toolbox (line 266)
The matlab toolbox 'stats' seems absent

The full output of cosmo_disp is

cosmo_disp(null_datasets)
{ .samples

[ 0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

: : : : : :

0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

0 0 0 ... 0 0 0 ]@27x153594
.sa
.chunks
[ 1
2
3
:
25
26
27 ]@27x1
.targets
[ 1
1
1
:
1
1
1 ]@27x1

.fa
.i
[ 1 2 3 ... 51 52 53 ]@1x153594
.j
[ 1 1 1 ... 63 63 63 ]@1x153594
.k
[ 1 1 1 ... 46 46 46 ]@1x153594

.a

.fdim
.labels
{ 'i'
'j'
'k' }
.values
{ [ 1 2 3 ... 51 52 53 ]@1x53
[ 1 2 3 ... 61 62 63 ]@1x63
[ 1 2 3 ... 44 45 46 ]@1x46 }
.vol
.mat
[ -3 0 0 81
0 3 0 -115
0 0 3 -53
0 0 0 1 ]
.xform
'scanner_anat'
.dim
[ 53 63 46 ]

.samples
[ 0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

: : : : : :

0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

0 0 0 ... 0 0 0 ]@27x153594
.sa
.chunks
[ 1
2
3
:
25
26
27 ]@27x1
.targets
[ 1
1
1
:
1
1
1 ]@27x1

.fa
.i
[ 1 2 3 ... 51 52 53 ]@1x153594
.j
[ 1 1 1 ... 63 63 63 ]@1x153594
.k
[ 1 1 1 ... 46 46 46 ]@1x153594

.a

.fdim
.labels
{ 'i'
'j'
'k' }
.values
{ [ 1 2 3 ... 51 52 53 ]@1x53
[ 1 2 3 ... 61 62 63 ]@1x63
[ 1 2 3 ... 44 45 46 ]@1x46 }
.vol
.mat
[ -3 0 0 81
0 3 0 -115
0 0 3 -53
0 0 0 1 ]
.xform
'scanner_anat'
.dim
[ 53 63 46 ]

.samples
[ 0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

: : : : : :

0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

0 0 0 ... 0 0 0 ]@27x153594
.sa
.chunks
[ 1
2
3
:
25
26
27 ]@27x1
.targets
[ 1
1
1
:
1
1
1 ]@27x1

.fa
.i
[ 1 2 3 ... 51 52 53 ]@1x153594
.j
[ 1 1 1 ... 63 63 63 ]@1x153594
.k
[ 1 1 1 ... 46 46 46 ]@1x153594

.a

.fdim
.labels
{ 'i'
'j'
'k' }
.values
{ [ 1 2 3 ... 51 52 53 ]@1x53
[ 1 2 3 ... 61 62 63 ]@1x63
[ 1 2 3 ... 44 45 46 ]@1x46 }
.vol
.mat
[ -3 0 0 81
0 3 0 -115
0 0 3 -53
0 0 0 1 ]
.xform
'scanner_anat'
.dim
[ 53 63 46 ]

:

.samples
[ 0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

: : : : : :

0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

0 0 0 ... 0 0 0 ]@27x153594
.sa
.chunks
[ 1
2
3
:
25
26
27 ]@27x1
.targets
[ 1
1
1
:
1
1
1 ]@27x1

.fa
.i
[ 1 2 3 ... 51 52 53 ]@1x153594
.j
[ 1 1 1 ... 63 63 63 ]@1x153594
.k
[ 1 1 1 ... 46 46 46 ]@1x153594

.a

.fdim
.labels
{ 'i'
'j'
'k' }
.values
{ [ 1 2 3 ... 51 52 53 ]@1x53
[ 1 2 3 ... 61 62 63 ]@1x63
[ 1 2 3 ... 44 45 46 ]@1x46 }
.vol
.mat
[ -3 0 0 81
0 3 0 -115
0 0 3 -53
0 0 0 1 ]
.xform
'scanner_anat'
.dim
[ 53 63 46 ]

.samples
[ 0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

: : : : : :

0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

0 0 0 ... 0 0 0 ]@27x153594
.sa
.chunks
[ 1
2
3
:
25
26
27 ]@27x1
.targets
[ 1
1
1
:
1
1
1 ]@27x1

.fa
.i
[ 1 2 3 ... 51 52 53 ]@1x153594
.j
[ 1 1 1 ... 63 63 63 ]@1x153594
.k
[ 1 1 1 ... 46 46 46 ]@1x153594

.a

.fdim
.labels
{ 'i'
'j'
'k' }
.values
{ [ 1 2 3 ... 51 52 53 ]@1x53
[ 1 2 3 ... 61 62 63 ]@1x63
[ 1 2 3 ... 44 45 46 ]@1x46 }
.vol
.mat
[ -3 0 0 81
0 3 0 -115
0 0 3 -53
0 0 0 1 ]
.xform
'scanner_anat'
.dim
[ 53 63 46 ]

.samples
[ 0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

: : : : : :

0 0 0 ... 0 0 0
0 0 0 ... 0 0 0

0 0 0 ... 0 0 0 ]@27x153594
.sa
.chunks
[ 1
2
3
:
25
26
27 ]@27x1
.targets
[ 1
1
1
:
1
1
1 ]@27x1

.fa
.i
[ 1 2 3 ... 51 52 53 ]@1x153594
.j
[ 1 1 1 ... 63 63 63 ]@1x153594
.k
[ 1 1 1 ... 46 46 46 ]@1x153594

.a

.fdim
.labels
{ 'i'
'j'
'k' }
.values
{ [ 1 2 3 ... 51 52 53 ]@1x53
[ 1 2 3 ... 61 62 63 ]@1x63
[ 1 2 3 ... 44 45 46 ]@1x46 }
.vol
.mat
[ -3 0 0 81
0 3 0 -115
0 0 3 -53
0 0 0 1 ]
.xform
'scanner_anat'
.dim
[ 53 63 46 ]

Best wishes,

Nora

Nick Oosterhof

unread,

Jan 27, 2020, 1:34:42 PM1/27/20

to Nora Heinzelmann, CoSMoMVPA

Hi Nora,

it seems you are making progress :-).

The error message indicates that cosmo_montecarlo_cluster_stat needs the Matlab statistics toolbox. This is used to compute p values based on t statistics.
If you don't have that toolbox: I just noticed that fieldtrip has an alternative implementation of the tcdf function (which is required by cosmo_montecarlo_cluster_stat in your case). So that may be an alternative option to compute t statistics p values.

Please let me know if you would like to try this avenue. To do so, you could temporarily comment out cosmo_stat.m line 455 (add a percent sign so that it reads "% ensure_has_stats_functions();"). This should work (I think) if you have tcdf in your Matlab path. To install Fieldtrip: download it, the run ft_defaults - do not add all of FieldTrip's subdirectories to the Matlab search path, as it will add many undesired directories as well.

If this works for you, I would be happy to change a bit of code in CoSMoMVPA so that it works out of the box with FieldTrip's tcdf.

Nora Heinzelmann

unread,

Jan 28, 2020, 3:20:26 AM1/28/20

to Nick Oosterhof, CoSMoMVPA

Hi Nick,

This worked! Thank you so much, I am truly happy. It was really straightforward and easy with your description.

May I ask, as the procedure was very slow for me (for niter=100, it took me 64 minutes, so I guesstimate that 10,000 iterations would take me almost 45 days), is there a way to interrupt the process and to resume overnight, say?

Best wishes,

Nora

--

You received this message because you are subscribed to the Google Groups "CoSMoMVPA" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cosmomvpa+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/cosmomvpa/CAJXY1fJQztWARo1kYvjfFW%3D0tVSCLtFOF6%2BkRP5Dh%3Dk30-g%3DnA%40mail.gmail.com.

Nick Oosterhof

unread,

Jan 28, 2020, 2:02:35 PM1/28/20

to Nora Heinzelmann, CoSMoMVPA

Hi Nora,

You could run bits in parts (generate multiple datasets of say niter=100), then combine these. Since the output of cosmo_montecarlo_cluster_stat are z scores, it seems fine to take their average (over n datasets) and multiply by sqrt(n) to get a combined z-score.

Note: if you use the seed option, you should use different seed values for the different datasets.

best,

Nick

Nora Heinzelmann

unread,

Jan 29, 2020, 2:19:17 AM1/29/20

to Nick Oosterhof, CoSMoMVPA

Hi Nick,

Brilliant, I'll give that a try.

Thanks so much for all your help with this!!

Nora

Nick Oosterhof

unread,

Jan 30, 2020, 2:25:38 PM1/30/20

to Nora Heinzelmann, CoSMoMVPA

On Wed, 29 Jan 2020 at 08:19, Nora Heinzelmann <nora.hei...@googlemail.com> wrote:

Hi Nick,

Brilliant, I'll give that a try.
Thanks so much for all your help with this!!

Excellent, good luck with the analysis!

Nora Heinzelmann

unread,

Feb 8, 2020, 8:30:56 AM2/8/20

to Nick Oosterhof, CoSMoMVPA

Dear Nick,

Sorry to pester you again. The analysis ran successfully but in the resulting ds_z, all my .samples are 0. Does that mean that my t-tests are not significant, or am I doing something wrong? The .samples of the null_datasets and ds_group are not 0, and the code I use is:

n_null=100;
n_subj=30;

actual_dataset = cell(n_subj,1);
for p=1:n_subj

j=j+1;
actual_dataset{j,1} = cosmo_fmri_dataset('no0.mat'));
actual_dataset{j,1}.sa.targets = 1;
actual_dataset{j,1}.sa.chunks = j;
end
[~,ds_group]=cosmo_mask_dim_intersect(actual_dataset);
ds_group = cosmo_stack(ds_group,1,'drop_nonunique');

null_cell=cell(n_null,1);
for i_null=1:n_null
subj_cell=cell(1,27);
for i_subj=1:n_subj
rng('shuffle');
r = randi([1 100],1,1);
ds = cosmo_fmri_dataset(fullfile(sprintf('subj%s','no%.0f.mat',i_subj,r)));
ds.sa.targets=1;
ds.sa.chunks=i_subj;
subj_cell{1,i_subj}=ds;
end
[~,subj_cell2] = cosmo_mask_dim_intersect(subj_cell);
null_cell{i_null,1} = cosmo_stack(subj_cell2,1,'drop_nonunique');
end
[~,null_datasets]=cosmo_mask_dim_intersect(null_cell);

nbrhood2 = cosmo_cluster_neighborhood(ds_group);
ds_z = cosmo_montecarlo_cluster_stat(ds_group,nbrhood2,'h0_mean',0,'null', null_datasets,'niter',10000);

Many thanks in advance for any advice.

All best,

Nora

--

You received this message because you are subscribed to the Google Groups "CoSMoMVPA" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cosmomvpa+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/cosmomvpa/CAJXY1fLWfpdBscbfGdB0jR4MudbwtHMSYh5%2Bji_Ey5eU6qxprA%40mail.gmail.com.

Nick Oosterhof

unread,

Feb 10, 2020, 1:09:24 PM2/10/20

to Nora Heinzelmann, CoSMoMVPA

Hi Nora,

On Sat, 8 Feb 2020 at 14:30, Nora Heinzelmann <nora.hei...@googlemail.com> wrote:
>
> Dear Nick,
> Sorry to pester you again. The analysis ran successfully but in the resulting ds_z, all my .samples are 0. Does that mean that my t-tests are not significant, or am I doing something wrong? The .samples of the null_datasets and ds_group are not 0, and the code I use is:

It could well be that t-tests are not significant after multiple comparison correction. Things to check:
1) that you did not mask out accidentally large parts of the brain. To test this, you could fill the brain with random data and make sure that all voxels in the brain have a value. For example: ds_z.samples=randn(size(ds_z.samples));

2) you could use cosmo_stat(ds, 't', 'z') for the univariate tests without multiple comparison correction. If all uncorrected z scores are less than, say, 3, this indicates a weak or no effect.

3) try cosmo_montecarlo_stat without the 'null' option, this should give similar (not identical) results as with the null option.

4) your h0_mean=0 indicates you are doing RSA or correlation-based analysis, not accuracy (unless you subtracted chance level yourself). Is that correct in your analysis pipeline?

best,

Nick

Nora Heinzelmann

unread,

Feb 11, 2020, 3:34:44 AM2/11/20

to Nick Oosterhof, CoSMoMVPA

Hi Nick,

Thanks for the fast reply! Related to your point (1), I am wondering whether the problem is with the individual-participants level because the files I get from there and feed into my group-level analysis using

actual_dataset{j,1} = cosmo_fmri_dataset('no0.mat'));

are rather small (just 150 KB). Do you think that could be the case?

Thanks,

Nora

PS: This is my searchlight analysis:
niter=100;

subs = 1:30;
subs = cellstr(num2str(subs(:),'%02d'))';
for subj = 1:length(subs)
data_path = fullfile(pathAnalysis,sprintf('%s',...
strjoin(subs(subj))),’model’);
data_fn=fullfile(data_path,'SPM.mat');
mask_fn = fullfile(pathAnalysis,sprintf('%s',...
strjoin(subs(subj))),'mask.hdr');
ds=cosmo_fmri_dataset(data_fn,'mask',mask_fn);
output_data_path = fullfile(data_path,'Cosmo');
bit1 = [1 2 3 4 5 6 7 8];
bit2 = [1 2 3 4 5 6 7 8 9];
bit3 = [1 2 3 4 5 6 7 8];
ds = cosmo_fmri_dataset(data_fn,'mask',mask_fn, 'targets',[bit1 bit2 bit3]);
ds=cosmo_remove_useless_data(ds);
msk_cond1 = ds.sa.targets == 5;
msk_cond2 = ds.sa.targets == 7;
msk = msk_cond1+msk_cond2;
ds=cosmo_slice(ds,logical(msk));
acc0=zeros(niter,1);
ds0=ds;
for k=0:niter
if k>0
ds0.sa.targets=cosmo_randomize_targets(ds);
end
nvoxels_per_searchlight=4;
nbrhood=cosmo_spherical_neighborhood(ds0,'count',nvoxels_per_searchlight);
measure=@cosmo_crossvalidation_measure;
measure_args=struct();
measure_args.partitions=cosmo_nfold_partitioner(ds0);
measure_args.classifier=@cosmo_classify_lda;
ds_cfy=cosmo_searchlight(ds0,nbrhood,measure,measure_args);
output_fn=fullfile(output_data_path,...
sprintf('no%.0f.mat',k));
save(output_fn,'ds_cfy');
end
end

To view this discussion on the web, visit https://groups.google.com/d/msgid/cosmomvpa/CAJXY1fKkRxGHeoYc8YhnN56heGoj%3Dv-%2BsSY3%2BEvqvx46SCEJVg%40mail.gmail.com.

Nora Heinzelmann

unread,

Feb 11, 2020, 10:00:30 AM2/11/20

to Nick Oosterhof, CoSMoMVPA

Dear Nick,

Apologies -- your last point was spot on! I changed h0 from 0 to 1/2 and now I am starting to see results (just tried niter=100 quickly and expect niter=10000 to be even better).

Many thanks again,

Nora

Reply all

Reply to author

Forward

0 new messages