ASAPRatio - multi-threading

[khogue]

Hi,

I have finally made it through the process of ASAPRatio for 38 files
generated using an Orbitrap. It took me 7 days to get the results.
This seemed a little slow to me

I have been able to test out a server with 24 cores, 2.66 GHz, 64 MB
RAM using a Windows Server 2003 Enterprise OS on the same dataset. I
have come to realize that ASAPRatio (and likely all the functions of
TPP) uses 1 core and only 2 MB RAM for processing. Again it will take
7 days to finish data crunching.

Is there any chance that ASAPRatio can utilize multi-threading to take
advantage of the multiple cores?

I will apologize in advance to the hard-core IT people on the list if
I am using threads and cores incorrectly. Basically I want to know if
there is a way to speed up the ASAPRatio process?

Also, is there a way to change the ratio from L/H to H/L?

Best Regards,
Kelly

[Brian Pratt]

Hi Kelly,

That seems a little slow to me too. How big are these files? And are you
certain that ASAPRatio is the bottleneck, as opposed to some other part of
the TPP pipeline?

Assuming it is ASAPRatio, the place to begin would be profiling ASAPRatio
while it operates on one of these files. Conceivably there's a simple code
optimization that would fix the performance issues. Have done some
performance work on this code in the past I'd be curious to see if a new
performance issue has arisen, if you can send me one of the files (and
accompanying mzXML and FASTA file) I'd be happy to investigate.

Brian Pratt

[Kelly Hogue]

I am fairly certain it is ASAPRatio that is the bottleneck but don't quote me on that.

Here are the commands from the run so far:
# Commands for session DPBYHSST7 on Thu Jan 15 12:19:15 2009
# BEGIN COMMAND BLOCK
###### BEGIN Command Execution ######
[Thu Jan 15 12:19:15 2009] EXECUTING: run_in c:/Inetpub/wwwroot/ISB/data/G041L10; xinteract  -NG041L10_arthur_interact.pep.xml -p0.05 -l7 -x20 -OANp -dREV_ -X-m0.05-nK,8.014199 -A-lK-F-B-r0.05-mK136.109161 c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_35_G041L10_F11.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_36_G041L10_F12.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_37_G041L10_G01.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_38_G041L10_G02.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_39_G041L10_G03.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_40_G041L10_G04.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_41_G041L10_G05.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_42_G041L10_G06.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_43_G041L10_G07.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_44_G041L10_G08.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_45_G041L10_G09.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_46_G041L10_G10.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_47_G041L10_G11.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_48_G041L10_G12.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_49_G041L10_H01.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_50_G041L10_H02.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_51_G041L10_H03.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_52_G041L10_H04.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_53_G041L10_H05.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_51_G041L10_F11_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_52_G041L10_F12_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_53_G041L10_G01_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_54_G041L10_G02_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_55_G041L10_G03_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_56_G041L10_G04_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_57_G041L10_G05_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_58_G041L10_G06_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_59_G041L10_G07_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_60_G041L10_G08_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_61_G041L10_G09_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_62_G041L10_G10_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_63_G041L10_G11_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_64_G041L10_G12_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_65_G041L10_H01_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_66_G041L10_H02_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_67_G041L10_H03_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_68_G041L10_H04_R2.tandem.pep.xml c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_69_G041L10_H05_R2.tandem.pep.xml
OUTPUT:

xinteract (TPP v4.1 JETSTREAM rev 1, Build 200811121809 (MinGW))

running: "C:/Inetpub/tpp-bin/InteractParser "G041L10_arthur_interact.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_35_G041L10_F11.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_36_G041L10_F12.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_37_G041L10_G01.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_38_G041L10_G02.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_39_G041L10_G03.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_40_G041L10_G04.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_41_G041L10_G05.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_42_G041L10_G06.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_43_G041L10_G07.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_44_G041L10_G08.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_45_G041L10_G09.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_46_G041L10_G10.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_47_G041L10_G11.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_48_G041L10_G12.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_49_G041L10_H01.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_50_G041L10_H02.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_51_G041L10_H03.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_52_G041L10_H04.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_53_G041L10_H05.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_51_G041L10_F11_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_52_G041L10_F12_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_53_G041L10_G01_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_54_G041L10_G02_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_55_G041L10_G03_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_56_G041L10_G04_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_57_G041L10_G05_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_58_G041L10_G06_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_59_G041L10_G07_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_60_G041L10_G08_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_61_G041L10_G09_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_62_G041L10_G10_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_63_G041L10_G11_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_64_G041L10_G12_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_65_G041L10_H01_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_66_G041L10_H02_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_67_G041L10_H03_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_68_G041L10_H04_R2.tandem.pep.xml" "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_69_G041L10_H05_R2.tandem.pep.xml" -L"7""
 file 1: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_35_G041L10_F11.tandem.pep.xml
 file 2: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_36_G041L10_F12.tandem.pep.xml
 file 3: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_37_G041L10_G01.tandem.pep.xml
 file 4: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_38_G041L10_G02.tandem.pep.xml
 file 5: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_39_G041L10_G03.tandem.pep.xml
 file 6: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_40_G041L10_G04.tandem.pep.xml
 file 7: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_41_G041L10_G05.tandem.pep.xml
 file 8: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_42_G041L10_G06.tandem.pep.xml
 file 9: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_43_G041L10_G07.tandem.pep.xml
 file 10: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_44_G041L10_G08.tandem.pep.xml
 file 11: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_45_G041L10_G09.tandem.pep.xml
 file 12: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_46_G041L10_G10.tandem.pep.xml
 file 13: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_47_G041L10_G11.tandem.pep.xml
 file 14: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_48_G041L10_G12.tandem.pep.xml
 file 15: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_49_G041L10_H01.tandem.pep.xml
 file 16: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_50_G041L10_H02.tandem.pep.xml
 file 17: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_51_G041L10_H03.tandem.pep.xml
 file 18: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_52_G041L10_H04.tandem.pep.xml
 file 19: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_53_G041L10_H05.tandem.pep.xml
 file 20: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_51_G041L10_F11_R2.tandem.pep.xml
 file 21: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_52_G041L10_F12_R2.tandem.pep.xml
 file 22: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_53_G041L10_G01_R2.tandem.pep.xml
 file 23: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_54_G041L10_G02_R2.tandem.pep.xml
 file 24: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_55_G041L10_G03_R2.tandem.pep.xml
 file 25: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_56_G041L10_G04_R2.tandem.pep.xml
 file 26: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_57_G041L10_G05_R2.tandem.pep.xml
 file 27: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_58_G041L10_G06_R2.tandem.pep.xml
 file 28: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_59_G041L10_G07_R2.tandem.pep.xml
 file 29: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_60_G041L10_G08_R2.tandem.pep.xml
 file 30: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_61_G041L10_G09_R2.tandem.pep.xml
 file 31: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_62_G041L10_G10_R2.tandem.pep.xml
 file 32: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_63_G041L10_G11_R2.tandem.pep.xml
 file 33: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_64_G041L10_G12_R2.tandem.pep.xml
 file 34: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_65_G041L10_H01_R2.tandem.pep.xml
 file 35: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_66_G041L10_H02_R2.tandem.pep.xml
 file 36: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_67_G041L10_H03_R2.tandem.pep.xml
 file 37: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_68_G041L10_H04_R2.tandem.pep.xml
 file 38: c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_69_G041L10_H05_R2.tandem.pep.xml
 processed altogether 569337 results


 results written to file c:/Inetpub/wwwroot/ISB/data/G041L10/G041L10_arthur_interact.pep.xml

 direct your browser to http://localhost/ISB/data/G041L10/G041L10_arthur_interact.pep.shtml



command completed in 229 sec

running: "C:/Inetpub/tpp-bin/PeptideProphetParser "G041L10_arthur_interact.pep.xml" MINPROB=0.05 EXTRAITRS=20 ACCMASS NONTT DECOY=REV_"
using Accurate Mass Bins
Using Decoy Label "REV_".
Not using ntt model
 (X! Tandem (k-score))
adding Accurate Mass mixture distr
init with X! Tandem (k-score) trypsin
MS Instrument info: Manufacturer: Thermo Scientific, Model: LTQ Orbitrap XL, Ionization: NSI, Analyzer: FTMS, Detector: unknown

 PeptideProphet  (TPP v4.1 JETSTREAM rev 1, Build 200811121809 (MinGW)) AKeller@ISB
 read in 0 1+, 286618 2+, 200027 3+, 38660 4+, and 8752 5+ spectra
Initialising statistical models ...
Iterations: .........10.........20.........30...
model complete after 34 iterations
command completed in 1676 sec

running: "C:/Inetpub/tpp-bin/ProphetModels.pl -i G041L10_arthur_interact.pep.xml"
Analyzing G041L10_arthur_interact.pep.xml ...
Reading Accurate Mass Model model +1 ...
Reading Accurate Mass Model model +2 ...
Reading Accurate Mass Model model +3 ...
Reading Accurate Mass Model model +4 ...
Reading Accurate Mass Model model +5 ...
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_35_G041L10_F11 (X! Tandem (k-score))"...
  => Total of 6471 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_36_G041L10_F12 (X! Tandem (k-score))"...
  => Total of 3584 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_37_G041L10_G01 (X! Tandem (k-score))"...
  => Total of 5466 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_38_G041L10_G02 (X! Tandem (k-score))"...
  => Total of 6745 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_39_G041L10_G03 (X! Tandem (k-score))"...
  => Total of 7833 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_40_G041L10_G04 (X! Tandem (k-score))"...
  => Total of 7522 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_41_G041L10_G05 (X! Tandem (k-score))"...
  => Total of 7787 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_42_G041L10_G06 (X! Tandem (k-score))"...
  => Total of 7611 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_43_G041L10_G07 (X! Tandem (k-score))"...
  => Total of 7529 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_44_G041L10_G08 (X! Tandem (k-score))"...
  => Total of 8088 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_45_G041L10_G09 (X! Tandem (k-score))"...
  => Total of 8567 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_46_G041L10_G10 (X! Tandem (k-score))"...
  => Total of 8335 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_47_G041L10_G11 (X! Tandem (k-score))"...
  => Total of 8109 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_48_G041L10_G12 (X! Tandem (k-score))"...
  => Total of 7471 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_49_G041L10_H01 (X! Tandem (k-score))"...
  => Total of 7439 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_50_G041L10_H02 (X! Tandem (k-score))"...
  => Total of 7859 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_51_G041L10_H03 (X! Tandem (k-score))"...
  => Total of 7178 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_52_G041L10_H04 (X! Tandem (k-score))"...
  => Total of 7540 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_53_G041L10_H05 (X! Tandem (k-score))"...
  => Total of 6501 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_51_G041L10_F11_R2 (X! Tandem (k-score))"...
  => Total of 6370 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_52_G041L10_F12_R2 (X! Tandem (k-score))"...
  => Total of 3710 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_53_G041L10_G01_R2 (X! Tandem (k-score))"...
  => Total of 5450 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_54_G041L10_G02_R2 (X! Tandem (k-score))"...
  => Total of 6638 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_55_G041L10_G03_R2 (X! Tandem (k-score))"...
  => Total of 7851 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_56_G041L10_G04_R2 (X! Tandem (k-score))"...
  => Total of 7854 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_57_G041L10_G05_R2 (X! Tandem (k-score))"...
  => Total of 7459 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_58_G041L10_G06_R2 (X! Tandem (k-score))"...
  => Total of 7719 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_59_G041L10_G07_R2 (X! Tandem (k-score))"...
  => Total of 7387 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_60_G041L10_G08_R2 (X! Tandem (k-score))"...
  => Total of 7611 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_61_G041L10_G09_R2 (X! Tandem (k-score))"...
  => Total of 8017 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_62_G041L10_G10_R2 (X! Tandem (k-score))"...
  => Total of 7982 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_63_G041L10_G11_R2 (X! Tandem (k-score))"...
  => Total of 7864 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_64_G041L10_G12_R2 (X! Tandem (k-score))"...
  => Total of 7297 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_65_G041L10_H01_R2 (X! Tandem (k-score))"...
  => Total of 7404 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_66_G041L10_H02_R2 (X! Tandem (k-score))"...
  => Total of 7681 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_67_G041L10_H03_R2 (X! Tandem (k-score))"...
  => Total of 7104 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_68_G041L10_H04_R2 (X! Tandem (k-score))"...
  => Total of 7461 hits.
Parsing search results "c:/Inetpub/wwwroot/ISB/data/G041L10/B_081103_69_G041L10_H05_R2 (X! Tandem (k-score))"...
  => Total of 6469 hits.
command completed in 32 sec

running: "C:/Inetpub/tpp-bin/XPressPeptideParser "G041L10_arthur_interact.pep.xml" -m0.05 -nK,8.014199"
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
WARNING: Found more than one variable mod on 'K'.
.................................................. 1k
.................................................. 2k
.................................................. 3k
.................................................. 4k
.................................................. 5k
.................................................. 6k
.................................................. 7k
.................................................. 8k
.................................................. 9k
.................................................. 10k
.................................................. 11k
.................................................. 12k
.................................................. 13k
.................................................. 14k
.................................................. 15k
.................................................. 16k
.................................................. 17k
.................................................. 18k
.................................................. 19k
.................................................. 20k
.................................................. 21k
.................................................. 22k
.................................................. 23k
.................................................. 24k
.................................................. 25k
.................................................. 26k
.................................................. 27k
.................................................. 28k
.................................................. 29k
.................................................. 30k
.................................................. 31k
.................................................. 32k
.................................................. 33k
.................................................. 34k
.................................................. 35k
.................................................. 36k
.................................................. 37k
.................................................. 38k
.................................................. 39k
.................................................. 40k
.................................................. 41k
.................................................. 42k
.................................................. 43k
.................................................. 44k
.................................................. 45k
.................................................. 46k
.................................................. 47k
.................................................. 48k
.................................................. 49k
.................................................. 50k
.................................................. 51k
.................................................. 52k
.................................................. 53k
.................................................. 54k
.................................................. 55k
.................................................. 56k
.................................................. 57k
.................................................. 58k
.................................................. 59k
.................................................. 60k
.................................................. 61k
.................................................. 62k
.................................................. 63k
.................................................. 64k
.................................................. 65k
.................................................. 66k
.................................................. 67k
.................................................. 68k
.................................................. 69k
.................................................. 70k
.................................................. 71k
.................................................. 72k
.................................................. 73k
.................................................. 74k
.................................................. 75k
.................................................. 76k
.................................................. 77k
.................................................. 78k
.................................................. 79k
.................................................. 80k
.................................................. 81k
.................................................. 82k
.................................................. 83k
.................................................. 84k
.................................................. 85k
.................................................. 86k
.................................................. 87k
.................................................. 88k
.................................................. 89k
.................................................. 90k
.................................................. 91k
.................................................. 92k
.................................................. 93k
.................................................. 94k
.................................................. 95k
.................................................. 96k
.................................................. 97k
.................................................. 98k
.................................................. 99k
.................................................. 100k
.................................................. 101k
.................................................. 102k
.................................................. 103k
.................................................. 104k
.................................................. 105k
.................................................. 106k
.................................................. 107k
.................................................. 108k
.................................................. 109k
.................................................. 110k
.................................................. 111k
.................................................. 112k
.................................................. 113k
.................................................. 114k
.................................................. 115k
.................................................. 116k
.................................................. 117k
.................................................. 118k
.................................................. 119k
.................................................. 120k
.................................................. 121k
.................................................. 122k
.................................................. 123k
.................................................. 124k
.................................................. 125k
.................................................. 126k
.................................................. 127k
.................................................. 128k
.................................................. 129k
.................................................. 130k
.................................................. 131k
.................................................. 132k
.................................................. 133k
.................................................. 134k
.................................................. 135k
.................................................. 136k
.................................................. 137k
.................................................. 138k
.................................................. 139k
.................................................. 140k
.................................................. 141k
.................................................. 142k
.................................................. 143k
.................................................. 144k
.................................................. 145k
.................................................. 146k
.................................................. 147k
.................................................. 148k
.................................................. 149k
.................................................. 150k
.................................................. 151k
.................................................. 152k
.................................................. 153k
.................................................. 154k
.................................................. 155k
.................................................. 156k
.................................................. 157k
.................................................. 158k
.................................................. 159k
.................................................. 160k
.................................................. 161k
.................................................. 162k
.................................................. 163k
.................................................. 164k
.................................................. 165k
.................................................. 166k
.................................................. 167k
.................................................. 168k
.................................................. 169k
.................................................. 170k
.................................................. 171k
.................................command completed in 1707 sec

running: "C:/Inetpub/tpp-bin/ASAPRatioPeptideParser "G041L10_arthur_interact.pep.xml" -lK -F -B -r0.05 -mK136.109161"

The last command has been running for 6 days so far. I have a Process Explorer program and I can see that ASAPRatioPeptideParser.exe is running. I can also see that is is only using 4.17% CPU on my 24 core.

When I open the interact.pep.xml file the xpress ratios are there. That file was last updated on Jan 15 when I started this process.

The file sizes are large. This is Orbitrap data generating 1 MS in FT and 10 MS/MS in the ion trap in parallel with a 2 hour gradient. One raw file is around 600 MB and that can generate a 1 - 1.5 GB mzXML file.

I can send you a file or two for you to play with if you wish. Just let me know how to upload the data to you.

Best regards,
Kelly

[Oded]

Hi Brian,
I'm also experiencing very long analysis times with ASAPratio (On dual-
core Mac with 4G RAM). For me it can go up to 48 hours using 12 mzXML
files (or 700M to 1G each) of Orbitrap data.
This is still better than what it used to be with TPP before the
integration of IPP - but still this is very slow processes. I wonder
if it is possible to make ASAPratio do less calculations in order to
speed it up. For example I prefer to calculate the ratio only where
the CID occur - if this option is selected ASAPratio present the ratio
of the CID state but still calculate (or try to calculate it for all
the other states and they can be added later on. This is something
that makes the calculation 3 or 4 times slower (depending on the
number of charge states) and usually provide not very essential input.
If you want I can send you one of my files etc.
Thanks,
Oded

> ...
>
> read more »

[Brian Pratt]

Hi Kelly,

 

It certainly looks like ASAPRatio is the bottleneck.  Go ahead and zip up a couple of the smaller pepXML files and corresponding mzXML files (throw in the FASTA file, too), and FTP to ftp://insilicos.serveftp.net/pub and I’ll see what I can find out.

 

Brian

 

---

[Brian Pratt]

Hi Oded,

This would seem to be a simple enough code change, but there's presumably
some reason for the current behavior (why have a "used=0" tag when you could
just omit the entries for the non-CID charge states altogether?). I'd feel
better if someone closer to the code would weigh in on this.

Brian

-----Original Message-----
From: spctools...@googlegroups.com
[mailto:spctools...@googlegroups.com] On Behalf Of Oded
Sent: Wednesday, January 21, 2009 7:18 AM
To: spctools-discuss
Subject: [spctools-discuss] Re: ASAPRatio - multi-threading

[David Shteynberg]

Hi Brian,

Yes, this would be a good optimization to make, but I think it would
make sense to create an additional separate option for this. So in
addition to having an option to compute the ratio from only the CID
charge state, we would have an option to not try to compute the other
ratios at all. Does this sound reasonable?

-David

[Brian Pratt]

I guess so, although it seems a little tricky explaining the difference.
What's a scenario for wanting one vs the other?

[Brian Pratt]

Kelly,

 

Looks like your data FTP’d fine.  I’ll look into these performance issues, but I note with just a quick eyeballing of the files that you aren’t using data compression on the peaklists.  This can’t be helping since it greatly increases disk IO, which is of course slow.  You might try reconverting those files with ReAdW and the –z option (why this isn’t the default, I cannot say – I think maybe because X!Tandem didn’t used to deal with it, but it does now).

 

Brian

 

---

From: spctools...@googlegroups.com [mailto:spctools...@googlegroups.com] On Behalf Of Kelly Hogue
Sent: Wednesday, January 21, 2009 1:09 AM
To: spctools...@googlegroups.com
Subject: [spctools-discuss] Re: ASAPRatio - multi-threading

 

I am fairly certain it is ASAPRatio that is the bottleneck but don't quote me on that.

[Kelly Hogue]

Thanks Brian.

BTW, my server has 64 GB RAM not 64 MB RAM. Small typo...

Is there any chance that ASAPRatio can be written to run the files in parallel? I am sure this is not trivial.

Kelly

[shygza]

Kelly,
    It may be helpful to use ramdisk tool to map your data into your huge RAM, and because TPP web interface can only use files in its root directory, you have to run ASAPRatio in command mode.

Chengpin

2009/1/22 Kelly Hogue <kelly...@gmail.com>

--
Chengpin Shen

[Walter]

Hi,
Useful fixes, but the original question seems unanswered. Is it
correct that only one core and a single thread are utilised? Is this
true for TPP in general? (No criticism implied BTW, just need to plan
the hardware).
Thanks
Walter

On Jan 23, 10:43 am, shygza <shy...@gmail.com> wrote:
> Kelly,
>     It may be helpful to use ramdisk tool to map your data into your huge
> RAM, and because TPP web interface can only use files in its root directory,
> you have to run ASAPRatio in command mode.
>
> Chengpin
>

> 2009/1/22 Kelly Hogue <kelly.ho...@gmail.com>

>
> > Thanks Brian.
>
> > BTW, my server has 64 GB RAM not 64 MB RAM. Small typo...
>
> > Is there any chance that ASAPRatio can be written to run the files in
> > parallel? I am sure this is not trivial.
>
> > Kelly
>

> > On Fri, Jan 23, 2009 at 3:33 AM, Brian Pratt <brian.pr...@insilicos.com>wrote:
>
> >>  Kelly,
>
> >> Looks like your data FTP'd fine.  I'll look into these performance issues,
> >> but I note with just a quick eyeballing of the files that you aren't using
> >> data compression on the peaklists.  This can't be helping since it greatly
> >> increases disk IO, which is of course slow.  You might try reconverting
> >> those files with ReAdW and the –z option (why this isn't the default, I
> >> cannot say – I think maybe because X!Tandem didn't used to deal with it, but
> >> it does now).
>
> >> Brian
>

> >>  ------------------------------
>
> >> *From:* spctools...@googlegroups.com [mailto:
> >> spctools...@googlegroups.com] *On Behalf Of *Kelly Hogue
> >> *Sent:* Wednesday, January 21, 2009 1:09 AM
> >> *To:* spctools...@googlegroups.com
> >> *Subject:* [spctools-discuss] Re: ASAPRatio - multi-threading

> ...
>
> read more »

[Oded]

Hi Walter,
The actual performance are of course dependent on the data and
analysis but it is possible to use one core and single thread for most
of TPP functions. There is a specific issue with ASAPratio that make
the calculations rather slow but using the same files and similar
analysis with Xpress are much faster (although this is not always
possible to switch ASAPratio and Xpress).
In any case if you plan your hardware I suggest to aim higher than
single core - we use routinely 2 years old dual core intel with 1G
RAM and it works just fine (although it would be better to have more
RAM).
Cheers,
Oded

> ...
>
> read more »

[Natalie Tasman]

Hello,

To be clear, the TPP code is quite straightforward, and only single threaded.  The TPP is fairly I/O-heavy, so a multicore system *might* see a (slight) improvement if the OS balances system calls to the other core(s).  But in general, the TPP is not multithreaded nor written to take advantage of multicore architectures.

Natalie

[Brian Pratt]

Hey All,

re ASAPRatio speed: the problem was just a bunch of redundant disk reads and
base64 decoding due to poor caching of the scans read from mzXML. I've
checked the fix into trunk and it is back to taking only about 4 times as
long as Xpress (certainly an improvement over the reported 300+ times as
long).

re Multithreading / multicore / parallelization ideas: this should be less
necessary with this fix. But as Natalie has already noted, even though TPP
isn't multithreaded, having at least one more core for the OS to do the
stuff that isn't TPP is a Good Thing.

Brian

> > >> OUTPUT:
>
> > >> xinteract (TPP v4.1 JETSTREAM rev 1, Build 200811121809 (MinGW))
>
> > >> running: "C:/Inetpub/tpp-bin/InteractParser
> > >> "G041L10_arthur_interact.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_35_G041L10_F11.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_36_G041L10_F12.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_37_G041L10_G01.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_38_G041L10_G02.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_39_G041L10_G03.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_40_G041L10_G04.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_41_G041L10_G05.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_42_G041L10_G06.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_43_G041L10_G07.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_44_G041L10_G08.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_45_G041L10_G09.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_46_G041L10_G10.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_47_G041L10_G11.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_48_G041L10_G12.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_49_G041L10_H01.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_50_G041L10_H02.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_51_G041L10_H03.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_52_G041L10_H04.tandem.pep.xml"
> > >>
"c:/Inetpub/wwwroot/ISB/data/G041L10/B_081030_53_G041L10_H05.tandem.pep.xml"
> > >>