Creating balanced exemplars

[MS Al-Rawi]

Creating balanced exemplars is very important to prevent classifier
biasing and can be accomplished using
create_balanced_xvalid_selectors.m which calls
balance_active_selector.m. To this I have two points:
1- balance_active_selector calls a function which is called sample,
as follows
% pick a random subset to remove, bringing the number of active
% timepoints for this condition down to MIN_COUNTS
remove_idx = sample(cur_cond_idx,cur_surplus); (line 47) % has only
two inputs!!
however, sample.m should have four inputs or more, its prototype is
{sample(A,B,C,D,K,T,inters,sample) using MATLAB 8} and this parameter
passing is obligatory since C and D are used inside the function (they
don't have default parameters).
2- Wouldn't be better to remove chunks of surplus (adjacent timepoints
resulted from one stimulus) instead of randomly selecting what idx to
remove. This is due to the HRF, a timepoint in the middle doesn’t have
the same weight (and effect) as another at the end of the stimulus
presentation. Then we have few timepoints here and there to randomly
remove.

Regards
Al-Rawi

[Greg Detre]

MS Al-Rawi wrote:
> Creating balanced exemplars is very important to prevent classifier
> biasing and can be accomplished using
> create_balanced_xvalid_selectors.m which calls
> balance_active_selector.m. To this I have two points:
> 1- balance_active_selector calls a function which is called sample,
> as follows
> % pick a random subset to remove, bringing the number of active
> % timepoints for this condition down to MIN_COUNTS
> remove_idx = sample(cur_cond_idx,cur_surplus); (line 47) % has only
> two inputs!!
> however, sample.m should have four inputs or more, its prototype is
> {sample(A,B,C,D,K,T,inters,sample) using MATLAB 8} and this parameter
> passing is obligatory since C and D are used inside the function (they
> don't have default parameters).

ah, rats. it looks like matlab introduced a 'sample' function. we should
have chosen a less obvious name.

> 2- Wouldn't be better to remove chunks of surplus (adjacent timepoints
> resulted from one stimulus) instead of randomly selecting what idx to
> remove. This is due to the HRF, a timepoint in the middle doesn’t have
> the same weight (and effect) as another at the end of the stimulus
> presentation. Then we have few timepoints here and there to randomly
> remove.

yes! that would be better, but i don't think that's an easy function to
write.

if you felt like having a go, i'd love to include it in the toolbox.
because it's hard though, i think we'd want to have a unit test too.

you could use the unit test for create_balanced_xvalid_selectors.m as a
template, but it's not as comprehensive as it should be (e.g. it doesn't
test when you include actives selectors)

trunk/mvpa/tests/unit_create_balanced_xvalid_selectors.m.

g