Re: [spctools-discuss] COMET searches with QExactive data

[Jimmy Eng]

Philip,

I'm guessing you're specifying a small "fragment_bin_tol" value for the high-res ms/ms spectra.  This causes Comet (not all capitalized) to use a ton of memory and you're just running out of memory.

On the following UWPR SEQUEST page, there's a table correlating a set of fragment_bin_tol settings vs. # input spectra vs. memory used:

https://proteomicsresource.washington.edu/sequest_release/release_201201.php

These numbers apply to Comet as well.

To address the problem, run smaller searches i.e. run Comet on a subset of your input spectra.  You can do this either with the "scan_range" parameter in the params file or simply invoke the searches such as

   comet yourfile.mzXML:1-3000

   comet yourfile.mzXML:3001-6000

   comet yourfile.mzXML:6001-9000

Your going to get multiple outputs if you do this which will need to be handled.  Splitting searches and managing results is something that some script should do for you; some day I'll throw some simple windows and linux example scripts on the Comet website for users.

Hopefully we'll have a workaround to address the memory use for these small fragment_bin_tol values in the semi-near future (thanks to Mike Hoopmann here who has implemented sparse matrix support in the code which I'm actively working with now).

- Jimmy

On Wed, Jan 16, 2013 at 11:05 AM, Philip Brownridge <philip.b...@gmail.com> wrote:

Hello all, please can I apologise if I have posted this in the wrong group, but I can't find a COMET group and there are other COMET postings here so I hope someone can help me! I'm trying to use some QExactive data with Comet and I keep getting either a calloc error message or a message saying there is no search to perform. I can get COMET searches to work on LTQ Orbitrap data but not QExactive. I have been using Proteowizard to convert the RAW files to mzML. I'm using the default params file with high resolution fragment ion parameters. 

Please if anybody is using COMET with QExactive data please could they let me know where i'm wrong!

thanks in advance,

Philip

--
You received this message because you are subscribed to the Google Groups "spctools-discuss" group.
To view this discussion on the web visit https://groups.google.com/d/msg/spctools-discuss/-/Jr5k5f4wcY0J.
To post to this group, send email to spctools...@googlegroups.com.
To unsubscribe from this group, send email to spctools-discu...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/spctools-discuss?hl=en.

[Philip Brownridge]

Hello Jimmy, thank you very much for your quick reply! You're completely correct about the fragment bin value, when I ran Comet (thank you for name guidance) using the LTQ settings it worked fine! When I ran it on the high resolution settings with task mamager on, I saw that I was getting the calloc error at about 2Gb of memory. I'm running Comet on a win7-64bit machine with 24Gb of memory and the program is called comet-win32, please could you tell me whether it would be possible to recompile Comet for win64 and perform larger searches? Sorry if this is a naive question, i'm very much a beginner when it comes to code.

thank you again,

Philip    

[Jimmy Eng]

I just emailed you off-list regarding the x64 binary.

[Dave Trudgian]

Hi Philip,

Sorry to drag up an old thread, but I wondered if you had info about the effects on results of limiting Comet memory usage by using a large bin, vs using the 64-bit binary and small bin? 

We are primarily working with QExactive data here. I was thinking of adding Comet to our pipeline, and checking on the bin-size effect on results - but cheekily wondered if you or anyone else have any feeling for this?

Cheers,

Dave Trudgian

[Jimmy Eng]

Dave,

Mike Hoopmann implemented a nice fix for searches using small bins.  The sources current in Comet SourceForge trunk should be fully functional.  One new parameter entry is titled "use_sparse_matrix" (see Comet version 2013.01 params in link below) which controls whether or not the sparse matrix data representation is used.  For small bins, this not only addresses the memory use but those searches are also a little faster. 

http://comet-ms.sourceforge.net/parameters/parameters_201301/

There's one other little feature I'd like to add before putting out another official release but the sparse matrix data representation to deal with this issue (memory use and small bin sizes) is the major feature in the next version.  When the next release happens, I'll document memory use and compare small vs. large bins on high res data (among other analysis).

- Jimmy

To unsubscribe from this group and stop receiving emails from it, send an email to spctools-discu...@googlegroups.com.

To post to this group, send email to spctools...@googlegroups.com.

Visit this group at http://groups.google.com/group/spctools-discuss?hl=en.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Dave Trudgian]

Jimmy,

Thanks for that info. Sounds great. I'll proceed based on the knowledge that'll be available.

Cheers,

DT