How to cope with missing values

[Daniel Schwarz]

Hi!

I've had your tool random jungle on my ToDo list for a long time, and today I wanted to try it out. Unfortunately, it always ends with an error and the documentation isn't that helpful.

Situation:
I have GWAS SNP dataset and want to have random jungle help me select the relevant features/SNPs. The data contains missing values and I don't wont to impute them, because I want to be comparable with other approaches I used.

I used PLINK to convert my data set to the .raw format and even tried replacing all occurences of "NA" with "3".

I'm running Ubuntu 9.10 on an Intel Core2 Duo on 32bit and have 4GB of RAM.

This is my output:
./rjunglesparse -f plink.raw -p -v -o study_DE_filtered
Start: Tue Mar 23 12:17:22 2010
+---------------------+-----------------+-------------------+
| RandomJungle | 0.8.6.333 | 2010 |
+---------------------+-----------------+-------------------+
| (C) 2008-2010 Daniel F Schwarz et al., GNU GPL, v2 |
| dan...@randomjungle.com |
+-----------------------------------------------------------+

Output to: study_DE_filtered.*
Loading data...
Read 1652 row(s) and 452871 column(s).
ERROR:
Missings in Data
terminate called after throwing an instance of 'Exception'
Aborted

=====

So I tried letting rjungle impute it, and this was the result:
rjungle -f plink.raw -p -v -U 2 -M 2 -x 3 -I 100 -o study_DE_filtered

Output to: study_DE_filtered.*
Loading data...
Read 1652 row(s) and 452871 column(s).
Use 1652 row(s) and 452867 column(s).
Dependent variable name: PHENOTYPE
Growing jungle...
Number of variables: 452867 (mtry = 672)
ERROR:
RJungleCtrl::make:std::bad_alloc
terminate called after throwing an instance of 'Exception'
Aborted

Any help is appreciated ;)

 

[Daniel Schwarz]

Hi Florian!

Thanks for using RJ!

Do you want to compare the imputation ability of RJ with other
approaches? Or do you
investigate several approaches for selecting top SNPs?

For testing the work flow, perform a test analysis as follows:
- plink --bfile gwadata --recodeA --out mypedfile
- rjunglesparse -f mypedfile.raw -p -t1 -I1 -o test1a
- rjunglesparse -f test1a.imputed.dat -p -o test1b

Does the error occur again?

Cheers,
Daniel

On 5 Mrz., 22:23, "Daniel Schwarz" <dafre...@googlemail.com> wrote:
> Hi!
>
> I've had your tool random jungle on my ToDo list for a long time, and today
> I wanted to try it out. Unfortunately, it always ends with an error and the
> documentation isn't that helpful.
>
> Situation:
> I have GWAS SNP dataset and want to have random jungle help me select the
> relevant features/SNPs. The data contains missing values and I don't wont to
> impute them, because I want to be comparable with other approaches I used.
>
> I used PLINK to convert my data set to the .raw format and even tried
> replacing all occurences of "NA" with "3".
>
> I'm running Ubuntu 9.10 on an Intel Core2 Duo on 32bit and have 4GB of RAM.
>
> This is my output:
> ./rjunglesparse -f plink.raw -p -v -o study_DE_filtered
> Start: Tue Mar 23 12:17:22 2010
> +---------------------+-----------------+-------------------+
> | RandomJungle | 0.8.6.333 | 2010 |
> +---------------------+-----------------+-------------------+
> | (C) 2008-2010 Daniel F Schwarz et al., GNU GPL, v2 |

> |  <mailto:dan...@randomjungle.com> dan...@randomjungle.com |

[Daniel Schwarz]

First of all, thanks for the quick answer.

No, I don't want to compare the imputation ability, I just want to
investigate several approaches for selecting top SNPs and my other
tests already used the non-imputed data, so I don't want to change
this.

Surprisingly, the error does not occur again. Everything runs fine.

Is this because it now runs on data without missing values? Or because
the number of trees is limited (out of memory?) ?

[Daniel Schwarz]

It looked like a memory issue (see bad_alloc).

For yielding reliable results, use many trees (at least 10,000), a
high mtry value (at least a 10th of number of predictors) and backward
elimination (option -B2). And maybe consider to use unscaled
conditional importance (option -K) .

Cheers,
Daniel

On 5 Mrz., 22:24, Daniel Schwarz <dafre...@googlemail.com> wrote:

[Daniel Schwarz]

Well, one of the error messages said "Missings in Data", so I'm
guessing that it just can't cope with missing values. Or is there a
way to make this importance calculation with missing values in the
data?

As for the other issue, I ran the same command on our 64bit server
with lots of RAM and the bad_alloc error hasn't occurred so far (but
it's not yet finished computing), so it really could be a memory
issue.

The parameters you suggested, for which task are they supposed?
Imputing or calculating the importance of features? Because it is the
latter one I want.

Regards
Florian

[Daniel Schwarz]

The options are suggested for feature selection.

Cheers,
Daniel