MACS 2.0.9 uses a very large amount of RAM

[Ivan Gregoretti]

Hello Tao,

I think that a MACS2 user has reported this before.

With MACS 1.4, I have routinely used sets of more than 100 million
tags for both treatment and control. It always used very little
memory. So little that I don't even how much. Perhaps a 1GB RAM.

Now, with MACS2, I try to analyse a set of 50 million tags in
treatment and 80 million tags in control and I am running out of my
16GB of RAM. Actually, my Fedora linux is swapping so persistently now
that the analysis is advancing very slowly.

I read very few posts about MACS2. Is it because people are having
trouble with it?

Thank you,

Ivan

Ivan Gregoretti, PhD

[Tao Liu]

Hi Ivan,

MACS2 will use more memory than before since it will remember all treatment pileup, control lambda, pvalue, qvalue tracks for every genomic location in memory. So far in all my MACS2 runs with 100million reads of both treatment and control, I haven't seen out-of-memory issue, but I am mainly working on small genome ~100Mb, and I am using a cluster with 48G mem each node.

There are many places needed to be optimized including memory usage and speed. If anyone in the group is interest in the source code and has any suggestion, please let me know.

Best,
Tao Liu

Research Fellow
Dept of Biostats and Comp Bio, DFCI / HSPH
450 Brookline Ave., Boston, MA 02215

[Tao Liu]

Hi Ivan and other people interested in MACS2,

I want to follow up with the discussion of the memory usage issue for MACS2. In some of my local tests, MACS2 (the recent github codes)  will use about 3 times the memory of MACS1.4. Use an ENCODE dataset CTCF as example, it has 22 million ChIP reads and 13 million input control reads. MACS1.4 uses about 1.3G mem whereas MACS2 will use 3.5G. Another example is the old FoxA1 data used in original MACS paper, which has 3.6 million ChIP reads and 4.4 million control reads, MACS1.4 uses 320M mem comparing to MACS2 730M.  Memory usage of MACS is almost linearly increasing with sequencing depth, and related to data complexity which has a small effect. If you want, you can keep track of your memory usage by using ps command ( an example on github: https://gist.github.com/1572440) or qstat for HPC cluster. 

BTW, currently MACS2 (the recent github codes) running speed is 2 times slower than MACS1.4 if you don't need signal tracks, and slightly faster than MACS1.4 if the output for signal files is turned on. 

Best wishes,

Tao

[Yuliang Tan]

Hi tao,

Can I use MACS2 to run the chip-seq data for s-cerevisiae? when i run y chip-seq by MACS1.4, it seems that the data is too big for yeast, and i only get one peak when i finished it.

Julian Tan

Laboratory of Michael Grunstein
Department of Biological Chemistry
David Geffen School of Medicine
University of California, Los Angeles
611 Charles E. Young Drive East
Boyer Hall #320
Los Angeles, CA 90095

Tel: (310) 954 7673
Email: yulia...@gmail.com

[Ivan Gregoretti]

Hi Tao,

Thank you for your follow up on memory usage; and happy ChIP-seqing.

Ivan

Ivan Gregoretti, PhD

> --
> You received this message because you are subscribed to the Google Groups
> "MACS announcement" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/macs-announcement/-/r_VxONU8uRsJ.
>
> To post to this group, send email to macs-ann...@googlegroups.com.
> To unsubscribe from this group, send email to
> macs-announcem...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/macs-announcement?hl=en.

[Nash]

Hey Tao,

It was very useful to read your earlier email about MACS2. I am using
a Mac computer with 16GB memory and trying to do chip seq analysis on
control and sample with about ~50 million reads for each. For this the
16gb memory is not enough and program ends. Could you suggest the size
of memory required to analyze the sample of this size with MACS2. I
work with mouse and human genomes

Thanks Nash

[芋岐 何]

Dear Tao,

I am new in data analysis. Could you please give a simple instruction for the use of MACS, I just use the windows system. Thank you very much

Yuqi

> Date: Thu, 2 Feb 2012 19:21:13 -0800
> Subject: [macs-announscement] Re: MACS 2.0.9 uses a very large amount of RAM
> From: avinash...@gmail.com
> To: macs-ann...@googlegroups.com

[Tao Liu]

Hi Yuqi,

The question is out of topic of this thread. However, I am happy to answer... I would recommend you to read the README document within MACS, and a recent paper published in the Current Protocol Bioinformatics: http://www.ncbi.nlm.nih.gov/pubmed/21633945

Best,
Tao

Tao Liu

[Tao Liu]

Hi Nash,

Thanks for the followup!

Is it for the newest MACS2 codes or for the MACS2.0.9 package I uploaded to github?

Best,
Tao

Tao Liu

[avinash waghray]

Hey Tao,

I just realised it may have been the older MACS2.0.9. I will install the newer version, try again and give you the feedback.

Thanks
Nash

[芋岐 何]

Dear Tao,

Thanks a lot for your guidance.

Yuqi

> Subject: Re: [macs-announscement] MACS 2.0.9 uses a very large amount of RAM
> From: vladim...@gmail.com
> Date: Fri, 3 Feb 2012 14:08:52 -0500
> To: macs-ann...@googlegroups.com

>
> Hi Yuqi,
>
> The question is out of topic of this thread. However, I am happy to answer... I would recommend you to read the README document within MACS, and a recent paper published in the Current Protocol Bioinformatics: http://www.ncbi.nlm.nih.gov/pubmed/21633945
>
> Best,
> Tao
>

[mali]

Hi
Can you please tell me where I can find the newest MACS2 version?
In github I see only MACS-2.0.9-1.tar.gz version
Thanks
Mali

On Feb 3, 9:30 pm, avinash waghray <avinash.wagh...@gmail.com> wrote:
> Hey Tao,
>
> I just realised it may have been the older MACS2.0.9. I will install the
> newer version, try again and give you the feedback.
>
> Thanks
> Nash
>