Memory RAM minimum requirements

[andrefcjp]

Dear All,

Just one simple question, I am trying to run the general tutorial for 16S samples. When I am in this step below.....

!pick_open_reference_otus.py -o otus/ -i slout/seqs.fna -p ../uc_fast_params.txt

....

 
It seems that the memory run out and the computer freezes. My laptop has only 4Gb of RAM and it worked well with the ITS tutorial. Is there any alternative to run the 16S and use less memory? Like an alternative command that could reach the same results?

I could run the ITS tutorial with this command below and everything was OK. Why with the 16S the computer freezes?

!pick_open_reference_otus.py -i its-soils-tutorial/seqs.fna -r its_12_11_otus/rep_set/97_otus.fasta -o otus/ -p its-soils-tutorial/params.txt --suppress_align_and_tree

Thanks,

Andre

[Embriette]

Hi Andre,

You're right-you're running out of memory. With only 4GB of memory, it will be really difficult for you to go from raw data to biom table as you start applying QIIME to life-size datasets. You could get more memory for your machine; alternatively, do you have access to a cluster on which you could run your analyses?

Best,

Embriette

[Colin Brislawn]

Hello Andre,

I think 4 GB of RAM will be a pretty big limit on these first steps, just like Embriette said.

If you don't have access to a computer cluster through you institution or a more powerful laptop or desktop, you could also consider renting a supercomputer from Amazon. For $2.66 an hour, Amazon EC2 offers an 32 core computer with 244 GB of RAM. Although this is a littler harder to set up, this is a great way to get the computer power you need for the initial analysis. Once you process your .fastq files into a .biom table, you can make graphs and do statistical tests in a less powerful computer, like your current laptop.

http://www.ec2instances.info/?selected=r3.8xlarge

Colin Brislawn 

[andrefcjp]

Dear Colin, Embriete,

In fact I just added 4 gb of memory and could run the protocol for 16S in my laptop, thanks for the advise.
However, I believe that I should register and install the amazon EC because this year I am planing to run 180 samples and I guess that my laptop wont be able to do that. To rent the Amazon computer we must follow the guidlines to install the qiime using the amazon protocol, right? http://qiime.org/tutorials/working_with_aws.html

Thanks again,

Andre

[Colin Brislawn]

Hello Andre,

Yep, that tutorial is the perfect place to start. It's one of the most comprehensive tutorials we have. And of course, all the qiime commands will be the same on AWS as they are on your laptop, so you can try workflows on your laptop using your test data set, then scale it up on EC2. 

When using Amazon before, I also found this pricing info really helpful:

http://www.ec2instances.info/ 

Keep in touch!

Colin