Re: New Version of Peptideshaker

77 views
Skip to first unread message

Pratik Jagtap

unread,
Jun 4, 2015, 8:07:55 AM6/4/15
to Harald Barsnes, Marc Vaudel, Ira Cooke, Gerben Menschaert, Björn Grüning, James Johnson, Trevor Wennblom, Elvis Ndah, Galaxy for Proteomics, Timothy Griffin
Thanks Ira, Bjoern and Gerben !

Hello JJ / Trevor, Can you please in have the SearchGUI/PeptideShaker installed on GalaxyP under test category? It would be a good idea to have both the earlier and new version on galaxyp.msi - so that we can start testing it.

Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Thu, Jun 4, 2015 at 3:47 AM, Harald Barsnes <Harald....@uib.no> wrote:

Hi Ira,

Which versions of SearchGUI and PeptideShaker are you using? I'd recommend updating to the latest releases (SearchGUI v1.28.0 and PeptideShaker v0.39.1), as we've fixed a lot of issues recently. For example the PTM scoring has become a lot more robust and the parsing of big mzIdentML result files is now a lot faster and uses less memory.

And regarding the MS Amanda issues, where does the “The search failed" message come from? Is it a message from SearchGUI or from MS Amanda? If it's from PeptideShaker there should be a stack trace, and if it's from MS Amanda there should be a log file. On a normal setup the MS Amanda log file is located here: C:\ProgramData\MSAmanda\Logs. Not sure how that translates to the Galaxy setup though?

And as always, if a certain file does not work in the Galaxy context, I'd recommend testing it outside Galaxy to see if the error is reproducible there, as that would make it so much easier to debug. Have you tried this with your test files?

Best regards,
Harald





Den 2015-06-04 10:28, skrev Marc Vaudel:
Hi Ira,

Can you forward me the file creating the problem with Comet? It indeed
seems to be a problem with the number of hits it produces, but this
should be handled already.

Best regards,

Marc

2015-06-04 2:12 GMT+02:00 Ira Cooke <I.C...@latrobe.edu.au>:

Hi Pratik,

The new searchgui and peptideshaker are now up on the testtoolshed.
I was able to get my small test file to run OK .. but not running
all search engines yet. You may have different degrees of success.

On my file I am able to search with X!Tandem , OMSSA, MS-GF+,
Myrimatch and Amanda!! :)!

Unfortunately when I add Comet I get an error "Attempting to
estimate the percentile of an empty list” which I assume is
because comet didn’t produce enough results? My files are quite
small.

I can get Amanda to run by installing mono … but then it fails
later without giving a sensible error message “The search
failed”.

Anyway, these are not new problems … and the new wrapper is
better than the old in my opinion.

Of course, I’d say that it’s likely that the user interface
will still change slightly before it finally settles down. For
example, maybe we should expose some “Advanced” parameters
outside the advanced tab?

I will also try to add some better help text in coming days

Cheers
Ira

Note:
When you install the wrappers take care to install updated
“Proteomics datatypes” repository because it needs the
searchgui_zip type (no longer included with the toolshed package)

Also follow the notes on getting myrimatch to work. Specifically I
needed to do

export LC_ALL=C

Pratik Jagtap

unread,
Jun 4, 2015, 9:08:23 AM6/4/15
to Björn Grüning, Ira Cooke, Marc Vaudel, Harald Barsnes, Gerben Menschaert, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hello Marc, Harald, Ira and Bjoern,

Thanks for your inputs.

Just to give you an overview - we have a Galaxy Community Conference Workshop on July 5th wherein we will be highlighting the use of SearchGUI/PeptideShaker and downstream data processing. The current version works well (with MS-GF+, X!tandem and OMSSA). 

The new version (after merging with commits from Gerben) supports a few more search algorithms and has a few new parameters. Ideally, we would like to test and us this version.

However, given that we have only a month to test and use a robust software - our plan is to test the new version and see if it works in our current workflows for the workshop. If it does - we will test it multiple times and get it to work while the development for the next version continues.If it does not work - or needs a lot more time, we will use the current version. 

We will of course move on to the new version after the GCC workshop.

Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Thu, Jun 4, 2015 at 8:01 AM, Björn Grüning <bjoern....@gmail.com> wrote:
Hi Marc, Ira,


> Hi Marc,
>
> I think this would work … the search engine binaries are not
> particularly large right?  So it would not be a big copy operation?
>
> I’m in favour of this change.

+1 for me as well. Maybe linking would be also possible?


> Cheers Ira
>
>
> On 4 Jun 2015, at 7:53 am, Marc Vaudel
> <mva...@gmail.com<mailto:mva...@gmail.com>> wrote:
>
> Hi Ira,
>
> Concerning the writing in the global resources directory, these files
> are used by the search engines. In the specific case of X!Tandem I
> could move the paramters to the temp folder but some search engines
> need to edit files in their folders and I cannot change this, so no
> simple generic fix. A solution would be to move all the search
> engines to the temp folder, and run them there. Would it be an idea?
>
> Best regards,
>
> Marc
>
> 2015-06-04 14:29 GMT+02:00 Ira Cooke
> <I.C...@latrobe.edu.au<mailto:I.C...@latrobe.edu.au>>: Hi Harald,
>
> Thanks for getting back to us on this.
>
> We are using the latest versions of searchgui and peptideshaker (as
> of yesterday).
>
> With regard to MSAmanda I should say that actually it works fine
> (sorry for the confusion) … I just hadn’t fully installed mono.  I
> followed the instructions on your website and have added those
> instructions to our readme.
>
> With regard to Comet. I believe this may be an issue with my specific
> search.  In the past I’ve seen the same error cropping up .. often in
> cases where I would guess that very few spectra or very few peptides
> would be matched?  It would be great to get it fixed so that
> searchgui and peptideshaker gracefully deal with that situation.
>
> I’ve uploaded my searchgui zip output and a text file with the
> commands used by galaxy to process this with peptideshaker. You will
> need to set the PEPTIDESHAKER_JAR_PATH to an appropriate value to run
> this script.
>
> Those files are on dropbox
>
> https://dl.dropboxusercontent.com/u/226794/ForHarald.zip
>
> One other thing I noticed is that searchgui still seems to write
> search parameters to its global resources directory?  For example, in
> the searchgui output I see (for example).
>
>
> xtandem command:
> /home/vagrant/tool_dependencies/searchgui/1.28/iracooke/package_searchgui_1_28/ec9887977acc/resources/XTandem/linux/linux_64bit/tandem
> /home/vagrant/tool_dependencies/searchgui/1.28/iracooke/package_searchgui_1_28/ec9887977acc/resources/XTandem/linux/linux_64bit/input_searchGUI.xml
>
>
>
> Which indicates that it is reading from the global resources
> directory?  Presumably it writes parameters there prior to a search
> and then reads them back?  For Galaxy it would be much better to do
> all this within the directory specified by -temp_dir because this
> will prevent problems where multiple users are running searches
> simultaneously.
>
> Many thanks for your efforts and all your improvements
>
> Cheers Ira
>
>
>
> On 4 Jun 2015, at 3:47 am, Harald Barsnes

Pratik Jagtap

unread,
Jun 4, 2015, 11:03:52 AM6/4/15
to Gerben Menschaert, Jim Johnson, Ira Cooke, Harald Barsnes, Marc Vaudel, Björn Grüning, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hello Gerben,

Thanks - I will be testing the new version on our galaxy instance and might use that (with a few minor changes) for the workshop (to avoid any last minute surprises).

Thanks a lot for all the changes - it would be great to use the latest and the best available version for the workshop.

Its great that Elvis is going to be attending. We will keep in touch on whether we will need his help.

Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Thu, Jun 4, 2015 at 9:48 AM, Gerben Menschaert <gerben.m...@gmail.com> wrote:
Hi all,

@Ira, Björn,
Thanks for merging our SearchGUI tool wrapper.

@Marc,Harald,
Thanks for fixing the last minor issues and of course the almost always split-second responsiveness.

@Pratik,JJ,
I would try to work with this latest version within the workshop @GCC. I guess within a month all minor issues should be tackled. A such, new users can immediately start off with the latest version. Elvis will possibly also come over, so maybe he could help out in the workshop, if that’s an idea.

Next in line will be the submission of a separate report tool that comprises both the ReportCLI and FollowUpCLI.
We can keep the basic level output in the current PeptideShaker tool wrapper and just add a full report tool separately.
We’ll get it out asap.

Gerben & Elvis






On Jun 4, 2015, at 9:02 AM, Jim Johnson <john...@umn.edu> wrote:

Sorry.   My blurred eyes missed that this AM.  

On 6/4/15 8:02 AM, Ira Cooke wrote:
Hi JJ, 

At the top of searchgui there is now a selector like this.  If the tool is not selected here it won’t run?

Is that what you mean?

Cheers
Ira


<Mail Attachment.png>

On 4 Jun 2015, at 7:51 am, Jim Johnson <john...@umn.edu> wrote:


Is it possible to select which search applications to use in SearchGUI and PeptideShaker?  
If so, the Galaxy tool forms options selector for each application could be:  "Default", "Advanced", "Don't Run".
-- 
James E. Johnson Minnesota Supercomputing Institute University of Minnesota 



-- 
James E. Johnson Minnesota Supercomputing Institute University of Minnesota 


Björn Grüning

unread,
Jun 4, 2015, 11:17:50 AM6/4/15
to Ira Cooke, Pratik Jagtap, Gerben Menschaert, Jim Johnson, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hi all!

This is really awesome! Thanks for all the hard work, I guess this
conference is really inspiring :)


> Hi Pratik,
>
> Just one further thought. I wonder if you are planning on releasing
> your tutorials in some kind of form we could put online after the
> workshop? We could link to it from the peptideshaker github page …
> eg here
>
> http://galaxyproteomics.github.io/peptideshaker/

+1000!

One small question regarding the Mono dependency. Anyone thought about
how to get this integrated into the TS?

Ciao,
Bjoern

> Also … I totally agree with your cautious attitude when it comes to
> last minute surprises :)! You don’t want that in a workshop.
>
> Hopefully we have enough time to test the new version though. Once I
> am back at LTU I would be happy to test using your tutorial
> datasets.
>
> Cheers Ira
>
>
> On 4 Jun 2015, at 10:03 am, Pratik Jagtap
> <pja...@umn.edu<mailto:pja...@umn.edu>> wrote:
>
> Hello Gerben,
>
> Thanks - I will be testing the new version on our galaxy instance and
> might use that (with a few minor changes) for the workshop (to avoid
> any last minute surprises).
>
> Thanks a lot for all the changes - it would be great to use the
> latest and the best available version for the workshop.
>
> Its great that Elvis is going to be attending. We will keep in touch
> on whether we will need his help.
>
> Regards,
>
> Pratik
>
> Pratik Jagtap, Managing Director, Center for Mass Spectrometry and
> Proteomics, 43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN
> 55108 Phone: 612-624-9275
>
> On Thu, Jun 4, 2015 at 9:48 AM, Gerben Menschaert
> <gerben.m...@gmail.com<mailto:gerben.m...@gmail.com>>
> wrote: Hi all,
>
> @Ira, Björn, Thanks for merging our SearchGUI tool wrapper.
>
> @Marc,Harald, Thanks for fixing the last minor issues and of course
> the almost always split-second responsiveness.
>
> @Pratik,JJ, I would try to work with this latest version within the
> workshop @GCC. I guess within a month all minor issues should be
> tackled. A such, new users can immediately start off with the latest
> version. Elvis will possibly also come over, so maybe he could help
> out in the workshop, if that’s an idea.
>
> Next in line will be the submission of a separate report tool that
> comprises both the ReportCLI and FollowUpCLI. We can keep the basic
> level output in the current PeptideShaker tool wrapper and just add a
> full report tool separately. We’ll get it out asap.
>
> Gerben & Elvis
>
>
>
>
>
>
> On Jun 4, 2015, at 9:02 AM, Jim Johnson
> <john...@umn.edu<mailto:john...@umn.edu>> wrote:
>
> Sorry. My blurred eyes missed that this AM.
>
> On 6/4/15 8:02 AM, Ira Cooke wrote: Hi JJ,
>
> At the top of searchgui there is now a selector like this. If the
> tool is not selected here it won’t run?
>
> Is that what you mean?
>
> Cheers Ira
>
>
> <Mail Attachment.png>
>
> On 4 Jun 2015, at 7:51 am, Jim Johnson
> <I.C...@latrobe.edu.au<mailto:I.C...@latrobe.edu.au>>:

Björn Grüning

unread,
Jun 4, 2015, 3:46:02 PM6/4/15
to Ira Cooke, Pratik Jagtap, Gerben Menschaert, Jim Johnson, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hi Ira,

Am 04.06.2015 um 17:22 schrieb Ira Cooke:
> Hi Bjoern,
>
> Thanks for your enthusiasm :). The conference is great.
>
> As far as I can see it hasn’t been possible to put java in the TS, so
> I’m assuming something like mono will have a similar level of
> difficulty.

https://github.com/galaxyproject/tools-iuc/pull/182

It seems to be pretty easy!
Would be nice if anyone can test this.

Thanks,
Bjoern

> I think it’s fine to include something in the README.
>
> Cheers Ira
>
>
>> One small question regarding the Mono dependency. Anyone thought
>> about how to get this integrated into the TS?
>
>
>> On 4 Jun 2015, at 10:17 am, Björn Grüning

Pratik Jagtap

unread,
Jun 5, 2015, 9:52:03 AM6/5/15
to Björn Grüning, Ira Cooke, Gerben Menschaert, Jim Johnson, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Had a good talk with Ira last night.

JJ and Trevor - please let us know once the latest version of SearchGUI / PeptideShaker is installed and available for testing on galaxyp.msi

Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

Pratik Jagtap

unread,
Jun 5, 2015, 10:25:10 AM6/5/15
to Jim Johnson, Björn Grüning, Ira Cooke, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Thanks JJ,

Let us know once they are available and I will start testing.

Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Fri, Jun 5, 2015 at 9:18 AM, Jim Johnson <john...@umn.edu> wrote:
I installed the Galaxy Tools Thurs morning.  
I haven't set up any additional search engines, so I'll need to see what needs to be done for that. 

JJ

Ira Cooke

unread,
Jun 5, 2015, 10:54:26 AM6/5/15
to Pratik Jagtap, Jim Johnson, Björn Grüning, Ira Cooke, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hi JJ, 

As far as I know the search engines should all just be included with searchGUI.  With luck they will just work. 

Cheers
Ira


On Fri, Jun 5, 2015 at 9:25 AM, Pratik Jagtap <pja...@umn.edu> wrote:
Thanks JJ,

Let us know once they are available and I will start testing.

Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Fri, Jun 5, 2015 at 9:18 AM, Jim Johnson <john...@umn.edu> wrote:
I installed the Galaxy Tools Thurs morning.  
I haven't set up any additional search engines, so I'll need to see what needs to be done for that. 

JJ


On 6/5/15 8:52 AM, Pratik Jagtap wrote:

Pratik Jagtap

unread,
Jun 5, 2015, 10:56:43 AM6/5/15
to Ira Cooke, Jim Johnson, Björn Grüning, Ira Cooke, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Thanks Ira -

I will test them all then (except TIDE).

- Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

Ira Cooke

unread,
Jun 5, 2015, 11:55:29 AM6/5/15
to Pratik Jagtap, Jim Johnson, Björn Grüning, Ira Cooke, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
OK.  I forgot to say that Amanda will not work unless mono is installed.  See instructions in the peptideshaker readme.



On Fri, Jun 5, 2015 at 9:56 AM, Pratik Jagtap <pja...@umn.edu> wrote:
Thanks Ira -

I will test them all then (except TIDE).

- Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Fri, Jun 5, 2015 at 9:54 AM, Ira Cooke <irac...@gmail.com> wrote:

Björn Grüning

unread,
Jun 5, 2015, 12:04:05 PM6/5/15
to Ira Cooke, Pratik Jagtap, Jim Johnson, Ira Cooke, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics


Am 05.06.2015 um 17:55 schrieb Ira Cooke:
> OK. I forgot to say that Amanda will not work unless mono is installed.
> See instructions in the peptideshaker readme.

Maybe this experimental TS package is of any use:

https://testtoolshed.g2.bx.psu.edu/view/galaxyp/package_mono_4_0

Pratik Jagtap

unread,
Jun 5, 2015, 12:29:42 PM6/5/15
to Björn Grüning, Ira Cooke, Jim Johnson, Ira Cooke, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Running with MS-AMANDA and TIDE off the list. The earlier ones had failed.



Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

Marc Vaudel

unread,
Jun 5, 2015, 12:41:00 PM6/5/15
to Pratik Jagtap, Björn Grüning, Ira Cooke, Jim Johnson, Ira Cooke, Gerben Menschaert, Harald Barsnes, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hi all,

As a short hint, I would recommend not running Comet and Tide, but one or the other. Both are the same algorithm with differences in the implementation (a larger index for Tide which makes it faster). Putting the results of both search engines in PeptideShaker will thus not provide additional hits but just give more weight to the Comet/Tide approach. In our experience Tide fails when running out of space for its index.

Best regards and thank you for all this interesting input.

Marc

Björn Grüning

unread,
Jun 5, 2015, 5:47:22 PM6/5/15
to j...@umn.edu, Ira Cooke, Pratik Jagtap, Ira Cooke, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hi JJ,

this are very good news! Thanks for testing!

> I installed package_mono_4_0 on our galaxyp server,
> and then doctored the: package_searchgui_1_28/ec9887977acc/env.sh
> to include the environment variable set in:
> package_mono_4_0/fee35eef75ac/env.sh
>
> I ran SearchGUI with all search engine but TIDE, and it completed
> successfully.
>
> Can we include package_mono_4_0 as a prior install and add its
> set_environment
> in the package_searchgui_1_28 tool_dependencies.xml similar to what
> you'll done with the python_2_7 pacakges?

You mean with the $ENV[] trick? This should work, yes.

> Or do we add a tool_dependency and requirement for mono in peptideshaker?

This would work as well.
Let me know if I should do this. Btw. I wrongly put this package under
the user "galaxyp" it should probably under the IUC account.

I have corrected this now. Please use the IUC one if possible.
Thanks for all,
Bjoern



>
> JJ

Pratik Jagtap

unread,
Jun 5, 2015, 6:23:38 PM6/5/15
to Marc Vaudel, Björn Grüning, Ira Cooke, Jim Johnson, Ira Cooke, Gerben Menschaert, Harald Barsnes, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
SearchGUI worked but PeptideShaker failed.


JJ and Tom - can you please look at items # 14-19 in the history?

Thanks,

Pratik

Pratik Jagtap

unread,
Jun 5, 2015, 8:30:11 PM6/5/15
to Marc Vaudel, Björn Grüning, Ira Cooke, Jim Johnson, Ira Cooke, Gerben Menschaert, Harald Barsnes, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Second attempt worked ! I will keep testing this...

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

Ira Cooke

unread,
Jun 8, 2015, 4:21:23 PM6/8/15
to j...@umn.edu, Björn Grüning, Pratik Jagtap, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hi JJ, 

Thanks for doing this.  I’ll run some tests and can update the testtoolshed versions today if you like.  I won’t update the main toolshed until Pratik is sure he will use the new versions in his workshop though.

Cheers
Ira


On 9 Jun 2015, at 5:42 am, Jim Johnson <john...@umn.edu> wrote:

I updated https://github.com/galaxyproteomics/ :
https://github.com/galaxyproteomics/tools-galaxyp/tree/master/packages/package_peptideshaker_0_40
https://github.com/galaxyproteomics/tools-galaxyp/tree/master/packages/package_searchgui_1_28
https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/peptideshaker
Updated to peptideshaker version 0.40.0, searchgui version 0.28.1
Package mono dependency added to:  package_searchgui_1_28/tool_dependencies.xml

I haven't updated the toolshed versions as yet.  

JJ

Pratik Jagtap

unread,
Jun 8, 2015, 4:22:24 PM6/8/15
to Jim Johnson, Björn Grüning, Ira Cooke, Ira Cooke, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Thanks a lot JJ,

As we discussed today, please let us know once the latest version of PeptideShaker and SearchGUI is in toolshed and on galaxyp.msi

The first tests for the current version worked (all except MSAmanda and TIDE) - but it will be a good idea to use the latest version (0.40.0 for PeptideShaker and 0.28.1 for SearchGUI) and MSAmanda for the GCC Workshop.

Once the tests for latest version work - I will start having documentation, screenshots and intermediate histories.

Thanks and Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Mon, Jun 8, 2015 at 2:42 PM, Jim Johnson <john...@umn.edu> wrote:
I updated https://github.com/galaxyproteomics/ :
https://github.com/galaxyproteomics/tools-galaxyp/tree/master/packages/package_peptideshaker_0_40
https://github.com/galaxyproteomics/tools-galaxyp/tree/master/packages/package_searchgui_1_28
https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/peptideshaker
Updated to peptideshaker version 0.40.0, searchgui version 0.28.1
Package mono dependency added to:  package_searchgui_1_28/tool_dependencies.xml

I haven't updated the toolshed versions as yet.  

JJ


On 6/5/15 4:47 PM, Björn Grüning wrote:

Ira Cooke

unread,
Jun 8, 2015, 11:49:11 PM6/8/15
to Pratik Jagtap, Jim Johnson, Björn Grüning, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hi Pratik, 

All tests pass for the new version.  I have uploaded it to the testtoolshed

Bjoern.  Would you mind adding the mono package to the main toolshed.  Once pratik is happy with the new version I would like to upload it to the main toolshed … but of course, I need all the dependencies to be there.  I have added them all except mono.

Cheers
Ira


On 9 Jun 2015, at 6:16 am, Pratik Jagtap <pja...@umn.edu> wrote:

Thanks a lot JJ,

As we discussed today, please let us know once the latest version of PeptideShaker and SearchGUI is in toolshed and on galaxyp.msi

The first tests for the current version worked (all except MSAmanda and TIDE) - but it will be a good idea to use the latest version (0.40.0 for PeptideShaker and 0.28.1 for SearchGUI) and MSAmanda for the GCC Workshop.

Once the tests for latest version work - I will start having documentation, screenshots and intermediate histories.

Thanks and Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Mon, Jun 8, 2015 at 2:42 PM, Jim Johnson <john...@umn.edu> wrote:
I updated https://github.com/galaxyproteomics/ :
https://github.com/galaxyproteomics/tools-galaxyp/tree/master/packages/package_peptideshaker_0_40
https://github.com/galaxyproteomics/tools-galaxyp/tree/master/packages/package_searchgui_1_28
https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/peptideshaker
Updated to peptideshaker version 0.40.0, searchgui version 0.28.1
Package mono dependency added to:  package_searchgui_1_28/tool_dependencies.xml

I haven't updated the toolshed versions as yet.  

JJ


On 6/5/15 4:47 PM, Björn Grüning wrote:

Pratik Jagtap

unread,
Jun 8, 2015, 11:52:15 PM6/8/15
to Ira Cooke, Jim Johnson, Björn Grüning, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Thanks Ira,

JJ/Trevor - can you please install the latest version to galaxyp.msi - so that I can test it?

Thanks and Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

Björn Grüning

unread,
Jun 9, 2015, 1:56:20 AM6/9/15
to Ira Cooke, Pratik Jagtap, Jim Johnson, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Good morning!

> Hi Pratik,
>
> All tests pass for the new version. I have uploaded it to the
> testtoolshed
>
> Bjoern. Would you mind adding the mono package to the main toolshed.
> Once pratik is happy with the new version I would like to upload it
> to the main toolshed … but of course, I need all the dependencies to
> be there. I have added them all except mono.

Done! :)

Ciao,
Bjoern

> Cheers Ira
>
>
>> On 9 Jun 2015, at 6:16 am, Pratik Jagtap <pja...@umn.edu> wrote:
>>
>> Thanks a lot JJ,
>>
>> As we discussed today, please let us know once the latest version
>> of PeptideShaker and SearchGUI is in toolshed and on galaxyp.msi
>>
>> The first tests for the current version worked (all except MSAmanda
>> and TIDE) - but it will be a good idea to use the latest version
>> (0.40.0 for PeptideShaker and 0.28.1 for SearchGUI) and MSAmanda
>> for the GCC Workshop.
>>
>> Once the tests for latest version work - I will start having
>> documentation, screenshots and intermediate histories.
>>
>> Thanks and Regards,
>>
>> Pratik
>>
>> Pratik Jagtap, Managing Director, Center for Mass Spectrometry and
>> Proteomics, 43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul,
>> MN 55108 Phone: 612-624-9275
>>
>> On Mon, Jun 8, 2015 at 2:42 PM, Jim Johnson <john...@umn.edu
>> <mailto:john...@umn.edu>> wrote: I updated
>> https://github.com/galaxyproteomics/
>> <https://github.com/galaxyproteomics/> :
>> https://github.com/galaxyproteomics/tools-galaxyp/tree/master/packages/package_peptideshaker_0_40
>> <https://github.com/galaxyproteomics/tools-galaxyp/tree/master/packages/package_peptideshaker_0_40>
>>
>>
https://github.com/galaxyproteomics/tools-galaxyp/tree/master/packages/package_searchgui_1_28
<https://github.com/galaxyproteomics/tools-galaxyp/tree/master/packages/package_searchgui_1_28>
>> https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/peptideshaker
>>>>>>> <tel:612-624-9275>
>>>>>>>
>>>>>>> On Fri, Jun 5, 2015 at 9:54 AM, Ira Cooke
>>>>>>> <irac...@gmail.com> <mailto:irac...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi JJ,
>>>>>>>>
>>>>>>>> As far as I know the search engines should all just be
>>>>>>>> included with searchGUI. With luck they will just
>>>>>>>> work.
>>>>>>>>
>>>>>>>> Cheers Ira
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jun 5, 2015 at 9:25 AM, Pratik Jagtap
>>>>>>>> <pja...@umn.edu> <mailto:pja...@umn.edu> wrote:
>>>>>>>>
>>>>>>>>> Thanks JJ,
>>>>>>>>>
>>>>>>>>> Let us know once they are available and I will start
>>>>>>>>> testing.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Pratik
>>>>>>>>>
>>>>>>>>> Pratik Jagtap, Managing Director, Center for Mass
>>>>>>>>> Spectrometry and Proteomics, 43 Gortner Laboratory,
>>>>>>>>> 1479 Gortner Avenue, St. Paul, MN 55108 Phone:
>>>>>>>>> 612-624-9275 <tel:612-624-9275>
>>>>>>>>>
>>>>>>>>> On Fri, Jun 5, 2015 at 9:18 AM, Jim Johnson
>>>>>>>>> <john...@umn.edu> <mailto:john...@umn.edu> wrote:
>>>>>>>>>
>>>>>>>>>> I installed the Galaxy Tools Thurs morning. I
>>>>>>>>>> haven't set up any additional search engines, so
>>>>>>>>>> I'll need to see what needs to be done for that.
>>>>>>>>>>
>>>>>>>>>> JJ
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 6/5/15 8:52 AM, Pratik Jagtap wrote:
>>>>>>>>>>
>>>>>>>>>> Had a good talk with Ira last night.
>>>>>>>>>>
>>>>>>>>>> JJ and Trevor - please let us know once the latest
>>>>>>>>>> version of SearchGUI / PeptideShaker is installed
>>>>>>>>>> and available for testing on galaxyp.msi
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Pratik
>>>>>>>>>>
>>>>>>>>>> Pratik Jagtap, Managing Director, Center for Mass
>>>>>>>>>> Spectrometry and Proteomics, 43 Gortner Laboratory,
>>>>>>>>>> 1479 Gortner Avenue, St. Paul, MN 55108 Phone:
>>>>>>>>>> 612-624-9275 <tel:612-624-9275>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jun 4, 2015 at 2:45 PM, Björn Grüning <
>>>>>>>>>> bjoern....@gmail.com
>>>>>>>>>> <mailto:bjoern....@gmail.com>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Ira,
>>>>>>>>>>>
>>>>>>>>>>> Am 04.06.2015 um 17:22 schrieb Ira Cooke:
>>>>>>>>>>>> Hi Bjoern,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for your enthusiasm :). The conference
>>>>>>>>>>>> is great.
>>>>>>>>>>>>
>>>>>>>>>>>> As far as I can see it hasn’t been possible to
>>>>>>>>>>>> put java in the TS, so I’m assuming something
>>>>>>>>>>>> like mono will have a similar level of
>>>>>>>>>>>> difficulty.
>>>>>>>>>>> https://github.com/galaxyproject/tools-iuc/pull/182
>>>>>>>>>>> <https://github.com/galaxyproject/tools-iuc/pull/182>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
It seems to be pretty easy!
>>>>>>>>>>> Would be nice if anyone can test this.
>>>>>>>>>>>
>>>>>>>>>>> Thanks, Bjoern
>>>>>>>>>>>
>>>>>>>>>>>> I think it’s fine to include something in the
>>>>>>>>>>>> README.
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers Ira
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> One small question regarding the Mono
>>>>>>>>>>>>> dependency. Anyone thought about how to get
>>>>>>>>>>>>> this integrated into the TS? On 4 Jun 2015,
>>>>>>>>>>>>> at 10:17 am, Björn Grüning
>>>>>>>>>>>>> <bjoern....@gmail.com>
>>>>>>>>>>>>>> <mailto:pja...@umn.edu><mailto:pja...@umn.edu>
>>>>>>>>>>>>>> <mailto:pja...@umn.edu>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello Gerben,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks - I will be testing the new version
>>>>>>>>>>>>>> on our galaxy instance and might use that
>>>>>>>>>>>>>> (with a few minor changes) for the
>>>>>>>>>>>>>> workshop (to avoid any last minute
>>>>>>>>>>>>>> surprises).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks a lot for all the changes - it would
>>>>>>>>>>>>>> be great to use the latest and the best
>>>>>>>>>>>>>> available version for the workshop.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Its great that Elvis is going to be
>>>>>>>>>>>>>> attending. We will keep in touch on whether
>>>>>>>>>>>>>> we will need his help.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Pratik
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Pratik Jagtap, Managing Director, Center
>>>>>>>>>>>>>> for Mass Spectrometry and Proteomics, 43
>>>>>>>>>>>>>> Gortner Laboratory, 1479 Gortner Avenue,
>>>>>>>>>>>>>> St. Paul, MN 55108 Phone: 612-624-9275
>>>>>>>>>>>>>> <tel:612-624-9275>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jun 4, 2015 at 9:48 AM, Gerben
>>>>>>>>>>>>>> Menschaert <gerben.m...@gmail.com
>>>>>>>>>>>>>> <mailto:gerben.m...@gmail.com><mailto:gerben.m...@gmail.com>
>>>>>>>>>>>>>> <mailto:john...@umn.edu><mailto:john...@umn.edu>
>>>>>>>>>>>>>> <mailto:john...@umn.edu>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry. My blurred eyes missed that this
>>>>>>>>>>>>>> AM.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 6/4/15 8:02 AM, Ira Cooke wrote: Hi JJ,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> At the top of searchgui there is now a
>>>>>>>>>>>>>> selector like this. If the tool is not
>>>>>>>>>>>>>> selected here it won’t run?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Is that what you mean?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers Ira
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <Mail Attachment.png>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 4 Jun 2015, at 7:51 am, Jim Johnson
>>>>>>>>>>>>>> <john...@umn.edu
>>>>>>>>>>>>>> <mailto:john...@umn.edu><mailto:john...@umn.edu>
>>>>>>>>>>>>>> <mailto:Harald....@uib.no><mailto:Harald....@uib.no>
>>>>>>>>>>>>>> <mailto:I.C...@latrobe.edu.au><mailto:I.C...@latrobe.edu.au>

Pratik Jagtap

unread,
Jun 9, 2015, 11:46:27 PM6/9/15
to Jim Johnson, Ira Cooke, Björn Grüning, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hello Ira,

My tests for latest version in galaxyp.msi worked with all tools except TIDE. JJ also ran a few tests - if his tests are positive we can place this version in galaxy tool shed and we will use these versions for the GCC workshop.

Thanks everyone for a quick turnaround !

Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Tue, Jun 9, 2015 at 8:38 AM, Jim Johnson <john...@umn.edu> wrote:
Pratik,

New versions are installed on galaxyp server in section " Beta Test Tools".  I'm rerunning searchgui and peptideshaker now. 

If you select the rerun button on a previous peptideshaker job,
remember to use "Versions" button at top of the tool form to switch to version: 0.40.0

JJ

Pratik Jagtap

unread,
Jun 10, 2015, 6:24:19 AM6/10/15
to Jim Johnson, Ira Cooke, Björn Grüning, Gerben Menschaert, Harald Barsnes, Marc Vaudel, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Thanks JJ,

Just to add to JJ's point. My first PeptideShaker searches always fail with an error. They succeed later once on a rerun or when used in a workflow.

Regards,

Pratik


Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Wed, Jun 10, 2015 at 5:13 AM, Jim Johnson <john...@umn.edu> wrote:
My 2 run of Searchgui and PeptideShaker both succeeded.  
Lets promote these to the main toolshed. 

I looked at the log of Pratik's PeptideShaker job that failed.   If the 8 stacktraces in the stderr represent the 8 threads,   4 were:
java.io.IOException: Input/output error
	at java.io.RandomAccessFile.readBytes0(Native Method)
	at java.io.RandomAccessFile.readBytes(RandomAccessFile.java:350)
	at java.io.RandomAccessFile.read(RandomAccessFile.java:385)
	at uk.ac.ebi.pride.tools.braf.BufferedRandomAccessFile.fillBuffer(BufferedRandomAccessFile.java:80)
	at uk.ac.ebi.pride.tools.braf.BufferedRandomAccessFile.getNextLine(BufferedRandomAccessFile.java:165)
	at com.compomics.util.experiment.io.massspectrometry.MgfReader.getSpectrum(MgfReader.java:800)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:784)   <-  This is a synchronized method
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
Given the synchronized method, I presume I'm not interpreting the stderr stacktrace correctly. 

3 more were;
Caused by: java.sql.SQLException: Cannot create log file at directory /panfs/roc/groups/1/mscls/apps/galaxyp2014/job_working_directory/085/85336/PeptideShakerCLI/matches/Galaxy_Experiment_2015060911181433866719_Sample_2015060911181433866719_1_id/log.
	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
	... 20 more
Caused by: ERROR XSLAQ: Cannot create log file at directory /panfs/roc/groups/1/mscls/apps/galaxyp2014/job_working_directory/085/85336/PeptideShakerCLI/matches/Galaxy_Experiment_2015060911181433866719_Sample_2015060911181433866719_1_id/log.
	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
	at org.apache.derby.impl.store.raw.log.LogToFile.getLogDirectory(Unknown Source)
	at org.apache.derby.impl.store.raw.log.LogToFile.getLogFileName(Unknown Source)
JJ

Marc Vaudel

unread,
Jun 10, 2015, 6:31:17 AM6/10/15
to Jim Johnson, Pratik Jagtap, Ira Cooke, Björn Grüning, Gerben Menschaert, Harald Barsnes, James Johnson, Trevor Wennblom, Elvis Ndah, Timothy Griffin, Galaxy for Proteomics
Hi JJ,

Could it be that four threads were waiting to execute the synchronized method and sequentially failed? It seems that the tool lost access to an mgf file, which would explain why all threads fail, any idea why this would happen?
Could you send me the entire stderr so that I see if a previous error is closing the connection to the file?

Thank you,

Marc

Pratik Jagtap

unread,
Jun 10, 2015, 5:26:05 PM6/10/15
to Marc Vaudel, Jim Johnson, Harald Barsnes, Thomas McGowan, Ira Cooke, Bjoern Gruening, Timothy Griffin, Galaxy for Proteomics
Hello JJ,

My second attempt at running SearchGUI 0.28 and PeptideShaker 0.4 failed as well.


I am trying the run with older version (SearchGUI 0.26 and PeptideShaker 0.37).

Regards,
Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

On Wed, Jun 10, 2015 at 1:04 PM, Marc Vaudel <mva...@gmail.com> wrote:
Hi JJ,

As said, when a file is not available the tool tries multiple times during ~10 seconds (the timeout), and then cancels the loading. What is unclear to me is why the access to the file is interrupted in the first place. Is it something specific to this dataset or do you see it on all projects running on this particular system?

Best regards,

Marc

2015-06-10 19:43 GMT+02:00 Jim Johnson <john...@umn.edu>:
Marc,

Is this what may be sporadically happening for us:

One of threads reading the MGF files times out:
java.lang.IllegalArgumentException: Error while loading spectrum pjagtap_20150211_14631_ABRFprg_25ug10fm_13_23_33.07998.07998.2 of file Mascot formatted MGF of data 6.mgf.
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:794)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
        ...
        at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:723)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:741)
	at eu.isas.peptideshaker.scoring.psm_scoring.BestMatchSelection.getBestHit(BestMatchSelection.java:593)
	at eu.isas.peptideshaker.fileimport.PsmImporter.importPsm(PsmImporter.java:822)
	at eu.isas.peptideshaker.fileimport.PsmImporter.access$000(PsmImporter.java:62)
	at eu.isas.peptideshaker.fileimport.PsmImporter$PsmImporterRunnable.run(PsmImporter.java:1184)

 com/compomics/util/experiment/massspectrometry/SpectrumFactory.java
 783             try {
 784                 currentSpectrum = MgfReader.getSpectrum(mgfFilesMap.get(spectrumFile), mgfIndexesMap.get(spectrumFile).getIndex(spectrumTitle), spectrumFile);
 785             } catch (Exception e) {
 786                 if (waitingTime < timeOut) {
 787                     try {
 788                         wait(waitingTime);
 789                     } catch (InterruptedException ie) {
 790                     }
 791                     return getSpectrum(spectrumFile, spectrumTitle, 2 * waitingTime);
 792                 } else {
 793                     e.printStackTrace();
 794                     throw new IllegalArgumentException("Error while loading spectrum " + spectrumTitle + " of file " + spectrumFile + ".");
 795                 }
 796             }
 
The IllegalArgumentException propagates up to :

eu/isas/peptideshaker/fileimport/PsmImporter.java
1174         public PsmImporterRunnable(SpectrumMatch spectrumMatch, WaitingHandler waitingHandler) {
1175             this.spectrumMatch = spectrumMatch;
1176             this.waitingHandler = waitingHandler;
1177         }
1178
1179         @Override
1180         public void run() {
1181
1182             try {
1183                 if (!waitingHandler.isRunCanceled()) {
1184                     importPsm(spectrumMatch, peptideSpectrumAnnotator, waitingHandler);
1185                 }
1186             } catch (Exception e) {
1187                 if (!waitingHandler.isRunCanceled()) {
1188                     exceptionHandler.catchException(e);
1189                     waitingHandler.setRunCanceled();
1190                 }
1191             }
1192         }
1193     }


eu/isas/peptideshaker/fileimport/PsmImporter.java
 
267     public void importPsmsMultipleThreads(LinkedList<SpectrumMatch> idFileSpectrumMatches, int nThreads, WaitingHandler waitingHandler)
 268             throws IOException, SQLException, InterruptedException, ClassNotFoundException, MzMLUnmarshallerException {
 269
 270         ExecutorService pool = Executors.newFixedThreadPool(nThreads);
 271         while (!idFileSpectrumMatches.isEmpty()) {
 272             SpectrumMatch match = idFileSpectrumMatches.pollLast();
 273             PsmImporterRunnable psmImporterRunnable = new PsmImporterRunnable(match, waitingHandler);
 274             pool.submit(psmImporterRunnable);
 275             if (waitingHandler.isRunCanceled()) {
 276                 pool.shutdownNow();
 277                 return;
 278             }
 279         }
 280         pool.shutdown();
 281         if (!pool.awaitTermination(12, TimeUnit.HOURS)) {
 282             throw new InterruptedException("PSM import timed out. Please contact the developers.");
 283         }
 284     }

Execution continues until
PeptideShakerCLI notices waitingHandler.runCanceled is true, then it deletes the temp dir and the other threads fail trying to read/write files that had been in the temp dir.
 
eu/isas/peptideshaker/cmd/PeptideShakerCLI.java
 183             // see if the project was created or canceled
 184             if (waitingHandler.isRunCanceled()) {
 185                 try {
 186                     closePeptideShaker(identification);
 187                 } catch (Exception e) {
 188                     waitingHandler.appendReport("An error occurred while closing PeptideShaker.", true, true);
 189                     e.printStackTrace();
 190                 }

 760     public static void closePeptideShaker(Identification identification) throws IOException, SQLException {
 ...
 808         try {
 809             TempFilesManager.deleteTempFolders();
 810         } catch (Exception e) {
 811             e.printStackTrace();
 812         }


On 6/10/15 7:55 AM, Marc Vaudel wrote:
Hi again,

Thank you for all the details. It indeed seems that the connection to the files is lost, multiple threads attempting to read an mgf file thus throw an exception the one after the other, and the tool attempts to cancel the loading and close. It is only then that derby is complaining, it seems that the connection to the derby files is lost as well, so it fails at closing the connection to the back-end database, and also throws an exception when attempting to log it. 

I can try to handle the crash of derby more nicely but the problem of the connection to the files remains. Note that in order to account for unstable connections whenever the connection to an mgf file is lost the tool tries to reconnect multiple times during 10s. This means that the connection was dropped during this entire time. Any idea where this could come from?

Hope that helps!

Marc

2015-06-10 14:14 GMT+02:00 Jim Johnson <john...@umn.edu>:
Marc,

Your interpretation of the stacktraces in the stderr sounds plausible.  I attached the peptide shaker stderr output and the derby log. 
Below is more info on the environment in which was run.  3 other identical runs completed successfully. 

The bash script galaxy generated to run peptideshaker:

mkdir output_reports; cwd=`pwd`; ln -s '/home/mscls/apps/galaxyp2014/files/097/dataset_97210.dat' searchgui_input.zip && (java -cp $PEPTIDESHAKER_JAR_PATH eu.isas.peptideshaker.cmd.PeptideShakerCLI -temp_folder $cwd/PeptideShakerCLI -experiment 'Galaxy_Experiment_2015060911181433866719' -sample 'Sample_2015060911181433866719' -replicate 1 -identification_files $cwd/searchgui_input.zip -out $cwd/peptideshaker_output.cps -zip $cwd/peptideshaker_output.zip -threads "${GALAXY_SLOTS:-12}" 2>> peptideshaker_stderr) && echo "Running Reports"; echo "Generating mzIdentML"; (java -cp $PEPTIDESHAKER_JAR_PATH eu.isas.peptideshaker.cmd.MzidCLI -in $cwd/peptideshaker_output.zip -output_file output.mzid -contact_first_name "Proteomics" -contact_last_name "Galaxy" -contact_email "gal...@umn.edu" -contact_address "gal...@umn.edu" -organization_name "University of Minnesota" -organization_email "gal...@umn.edu" -organization_address "Minneapolis, MN 55455, Vereinigte Staaten" 2>> peptideshaker_stderr) && (java -cp $PEPTIDESHAKER_JAR_PATH eu.isas.peptideshaker.cmd.ReportCLI -temp_folder $cwd/ReportCLI -in $cwd/peptideshaker_output.zip -out_reports $cwd/output_reports -reports 3,5,7,0 2>> peptideshaker_stderr) && find $cwd/output_reports -name '*Certificate_of_Analysis*' -exec bash -c 'mv "$0" "certificate.txt"' {} \; ; find $cwd/output_reports -name '*PSM_Report*' -exec bash -c 'mv "$0" "psm.txt"' {} \; ; find $cwd/output_reports -name '*Peptide_Report*' -exec bash -c 'mv "$0" "peptides.txt"' {} \; ; find $cwd/output_reports -name '*Protein_Report*' -exec bash -c 'mv "$0" "proteins.txt"' {} \; ; exit_code_for_galaxy=$?; cat peptideshaker_stderr 2>&1; (exit $exit_code_for_galaxy)


The job was allocated 1 node with 4 cores and 15gb RAM

PBS torque log entry:

        <Job_Name>g85336_peptide_shaker_pjagtap_msi_umn_edu</Job_Name>
        <Job_Owner>galax...@galaxyp.msi.umn.edu</Job_Owner>
        <resources_used>
                <cput>3257</cput>
                <mem>13294156kb</mem>
                <vmem>31549092kb</vmem>
                <walltime>1092</walltime>
        </resources_used>
        <Resource_List>
                <mem>15gb</mem>
                <neednodes>1:ppn=4</neednodes>
                <nodect>1</nodect>
                <nodes>1:ppn=4</nodes>
                <procct>0</procct>
                <walltime>86400</walltime>
        </Resource_List>
        <etime>1433866721</etime>
        <exit_status>0</exit_status>

stdout from galaxy bash script:

Fatal error: Java Exception
Path configuration completed.
Tue Jun 09 11:18:48 CDT 2015 Unzipping searchgui_input.zip.
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Tue Jun 09 11:19:20 CDT 2015 Import process for Galaxy_Experiment_2015060911181433866719 (Sample: Sample_2015060911181433866719, Replicate: 1)

Tue Jun 09 11:19:21 CDT 2015 Importing sequences from input_database.fasta.
Reindexing: input_database.fasta.
10% 20% 30% 40% 50% 60% 70% 80% 90%
Tue Jun 09 11:19:29 CDT 2015 Estimated import time: 24 minutes.
Tue Jun 09 11:19:29 CDT 2015     See http://code.google.com/p/compomics-utilities/wiki/ProteinInference.
10% 20% 30% 40% 50% 60% 70% 80% 90%
Tue Jun 09 11:26:33 CDT 2015 FASTA file import completed.
Tue Jun 09 11:26:33 CDT 2015 Establishing local database connection.
Tue Jun 09 11:26:34 CDT 2015 Reading identification files.
Tue Jun 09 11:26:34 CDT 2015 Parsing Mascot formatted MGF of data 6.comet.pep.xml.
Tue Jun 09 11:26:36 CDT 2015 Loading spectra for Mascot formatted MGF of data 6.comet.pep.xml.
Tue Jun 09 11:26:36 CDT 2015 Importing Mascot formatted MGF of data 6.mgf
10% 20% 30% 40% 50% 60% 70% 80% 90%
Tue Jun 09 11:26:38 CDT 2015 Mascot formatted MGF of data 6.mgf imported.
10% 20% 30% 40% 50% 60% 70% 80% 90%
Tue Jun 09 11:26:38 CDT 2015 Mapping peptides to proteins.
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Tue Jun 09 11:30:38 CDT 2015 Importing PSMs from Mascot formatted MGF of data 6.comet.pep.xml
10% 20% 30% 40% 50% 60% 70% 80% 90%
Tue Jun 09 11:30:39 CDT 2015 72 PSMs (1.2%) excluded by the import filters:
Tue Jun 09 11:30:39 CDT 2015     - 51.9% peptide mapping to both target and decoy.
Tue Jun 09 11:30:39 CDT 2015     - 48.1% peptide length less than 4 or greater than 30.
Tue Jun 09 11:30:39 CDT 2015 Parsing Mascot formatted MGF of data 6.msgf.mzid.
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Tue Jun 09 11:30:42 CDT 2015 Loading spectra for Mascot formatted MGF of data 6.msgf.mzid.
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Tue Jun 09 11:30:42 CDT 2015 Mapping peptides to proteins.
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Tue Jun 09 11:33:58 CDT 2015 Importing PSMs from Mascot formatted MGF of data 6.msgf.mzid
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Tue Jun 09 11:33:59 CDT 2015 12 PSMs (0.2%) excluded by the import filters:
Tue Jun 09 11:33:59 CDT 2015     - 100.0% peptide mapping to both target and decoy.
Tue Jun 09 11:33:59 CDT 2015 Parsing Mascot formatted MGF of data 6.myrimatch.mzid.
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Tue Jun 09 11:34:01 CDT 2015 Loading spectra for Mascot formatted MGF of data 6.myrimatch.mzid.
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Tue Jun 09 11:34:01 CDT 2015 Mapping peptides to proteins.
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Tue Jun 09 11:36:33 CDT 2015 Importing PSMs from Mascot formatted MGF of data 6.myrimatch.mzid
An error occurred while creating table long_key_table
Tue Jun 09 11:36:50 CDT 2015 PeptideShaker Processing Canceled.
PeptideShaker processing canceled. See the PeptideShaker log for details.
Running Reports
Generating mzIdentML

PeptideShaker project (.cps or zip file) '/home/mscls/apps/galaxyp2014/job_working_directory/085/85336/peptideshaker_output.zip' not found.


========================================
PeptideShaker mzid - Command Line
========================================

The PeptideShaker mzid command line takes a cps file and export the identification results in the mzIdentML format.

For further help see http://peptide-shaker.googlecode.com and http://code.google.com/p/peptide-shaker/wiki/PeptideShakerCLI.

Or contact the developers at https://groups.google.com/group/peptide-shaker.

----------------------
OPTIONS
----------------------

Mandatory parameters:

-in                                 PeptideShaker project (.cps or zip file)
-contact_first_name                 Contact first name.
-contact_last_name                  Contact last name.
-contact_email                      Contact e-mail.
-contact_address                    Contact address.
-organization_name                  Organization name.
-organization_email                 Organization e-mail.
-organization_address               Organization address.
-output_file                        Output file.


Optional annotation parameters:
-contact_url                        Contact URL.
-organization_url                   Organization URL.


Optional temporary folder:
-temp_folder                        A folder for temporary file storage. Use only if you encounter problems with the default configuration.

PeptideShaker project (.cps or zip file) '/home/mscls/apps/galaxyp2014/job_working_directory/085/85336/peptideshaker_output.zip' not found.


===============================================
PeptideShaker Report Exporter - Command Line
===============================================

The PeptideShaker report command line takes a cps file and generates various types of reports.

For further help see http://peptide-shaker.googlecode.com and http://code.google.com/p/peptide-shaker/wiki/PeptideShakerCLI.

Or contact the developers at https://groups.google.com/group/peptide-shaker.

----------------------
OPTIONS
----------------------

Mandatory parameters:

-in                                 PeptideShaker project (.cps or zip file)
-out_reports                        Output folder for report files. (Existing files will be overwritten.)


Optional output parameters:

Report export:

-reports                            Comma separated list of types of report to export. 0: Certificate of Analysis, 1: Default Hierarchical Report, 2: Default PSM Phosphorylation Report, 3: Default PSM Report, 4: Default Peptide Phosphorylation Report, 5: Default Peptide Report, 6: Default Protein Phosphorylation Report, 7: Default Protein Report

Report Documentation export:

-documentation                      Comma separated list of types of report documentation to export. 0: Certificate of Analysis, 1: Default Hierarchical Report, 2: Default PSM Phosphorylation Report, 3: Default PSM Report, 4: Default Peptide Phosphorylation Report, 5: Default Peptide Report, 6: Default Protein Phosphorylation Report, 7: Default Protein Report


Optional temporary folder:

-temp_folder                        A folder for temporary file storage. Use only if you encounter problems with the default configuration.
java.io.IOException: Input/output error
	at java.io.RandomAccessFile.readBytes0(Native Method)
	at java.io.RandomAccessFile.readBytes(RandomAccessFile.java:350)
	at java.io.RandomAccessFile.read(RandomAccessFile.java:385)
	at uk.ac.ebi.pride.tools.braf.BufferedRandomAccessFile.fillBuffer(BufferedRandomAccessFile.java:80)
	at uk.ac.ebi.pride.tools.braf.BufferedRandomAccessFile.getNextLine(BufferedRandomAccessFile.java:165)
	at com.compomics.util.experiment.io.massspectrometry.MgfReader.getSpectrum(MgfReader.java:800)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:784)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:723)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:741)
	at eu.isas.peptideshaker.scoring.psm_scoring.BestMatchSelection.getBestHit(BestMatchSelection.java:593)
	at eu.isas.peptideshaker.fileimport.PsmImporter.importPsm(PsmImporter.java:822)
	at eu.isas.peptideshaker.fileimport.PsmImporter.access$000(PsmImporter.java:62)
	at eu.isas.peptideshaker.fileimport.PsmImporter$PsmImporterRunnable.run(PsmImporter.java:1184)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
java.lang.IllegalArgumentException: Error while loading spectrum pjagtap_20150211_14631_ABRFprg_25ug10fm_13_23_33.07998.07998.2 of file Mascot formatted MGF of data 6.mgf.
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:794)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:723)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:741)
	at eu.isas.peptideshaker.scoring.psm_scoring.BestMatchSelection.getBestHit(BestMatchSelection.java:593)
	at eu.isas.peptideshaker.fileimport.PsmImporter.importPsm(PsmImporter.java:822)
	at eu.isas.peptideshaker.fileimport.PsmImporter.access$000(PsmImporter.java:62)
	at eu.isas.peptideshaker.fileimport.PsmImporter$PsmImporterRunnable.run(PsmImporter.java:1184)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
java.io.IOException: Input/output error
	at java.io.RandomAccessFile.readBytes0(Native Method)
	at java.io.RandomAccessFile.readBytes(RandomAccessFile.java:350)
	at java.io.RandomAccessFile.read(RandomAccessFile.java:385)
	at uk.ac.ebi.pride.tools.braf.BufferedRandomAccessFile.fillBuffer(BufferedRandomAccessFile.java:80)
	at uk.ac.ebi.pride.tools.braf.BufferedRandomAccessFile.getNextLine(BufferedRandomAccessFile.java:165)
	at com.compomics.util.experiment.io.massspectrometry.MgfReader.getSpectrum(MgfReader.java:800)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:784)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:723)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:741)
	at eu.isas.peptideshaker.scoring.psm_scoring.BestMatchSelection.getBestHit(BestMatchSelection.java:593)
	at eu.isas.peptideshaker.fileimport.PsmImporter.importPsm(PsmImporter.java:822)
	at eu.isas.peptideshaker.fileimport.PsmImporter.access$000(PsmImporter.java:62)
	at eu.isas.peptideshaker.fileimport.PsmImporter$PsmImporterRunnable.run(PsmImporter.java:1184)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
java.io.IOException: Input/output error
	at java.io.RandomAccessFile.readBytes0(Native Method)
	at java.io.RandomAccessFile.readBytes(RandomAccessFile.java:350)
	at java.io.RandomAccessFile.read(RandomAccessFile.java:385)
	at uk.ac.ebi.pride.tools.braf.BufferedRandomAccessFile.fillBuffer(BufferedRandomAccessFile.java:80)
	at uk.ac.ebi.pride.tools.braf.BufferedRandomAccessFile.getNextLine(BufferedRandomAccessFile.java:165)
	at com.compomics.util.experiment.io.massspectrometry.MgfReader.getSpectrum(MgfReader.java:800)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:784)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:723)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:741)
	at eu.isas.peptideshaker.scoring.psm_scoring.BestMatchSelection.getBestHit(BestMatchSelection.java:593)
	at eu.isas.peptideshaker.fileimport.PsmImporter.importPsm(PsmImporter.java:822)
	at eu.isas.peptideshaker.fileimport.PsmImporter.access$000(PsmImporter.java:62)
	at eu.isas.peptideshaker.fileimport.PsmImporter$PsmImporterRunnable.run(PsmImporter.java:1184)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
java.io.IOException: Input/output error
	at java.io.RandomAccessFile.readBytes0(Native Method)
	at java.io.RandomAccessFile.readBytes(RandomAccessFile.java:350)
	at java.io.RandomAccessFile.read(RandomAccessFile.java:385)
	at uk.ac.ebi.pride.tools.braf.BufferedRandomAccessFile.fillBuffer(BufferedRandomAccessFile.java:80)
	at uk.ac.ebi.pride.tools.braf.BufferedRandomAccessFile.getNextLine(BufferedRandomAccessFile.java:165)
	at com.compomics.util.experiment.io.massspectrometry.MgfReader.getSpectrum(MgfReader.java:800)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:784)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:791)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:723)
	at com.compomics.util.experiment.massspectrometry.SpectrumFactory.getSpectrum(SpectrumFactory.java:741)
	at eu.isas.peptideshaker.scoring.psm_scoring.BestMatchSelection.getBestHit(BestMatchSelection.java:593)
	at eu.isas.peptideshaker.fileimport.PsmImporter.importPsm(PsmImporter.java:822)
	at eu.isas.peptideshaker.fileimport.PsmImporter.access$000(PsmImporter.java:62)
	at eu.isas.peptideshaker.fileimport.PsmImporter$PsmImporterRunnable.run(PsmImporter.java:1184)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
java.sql.SQLException: Cannot create log file at directory /panfs/roc/groups/1/mscls/apps/galaxyp2014/job_working_directory/085/85336/PeptideShakerCLI/matches/Galaxy_Experiment_2015060911181433866719_Sample_2015060911181433866719_1_id/log.
	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
	at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
	at com.compomics.util.db.ObjectsDB.addTable(ObjectsDB.java:210)
	at com.compomics.util.db.ObjectsDB.saveLongKeys(ObjectsDB.java:1198)
	at com.compomics.util.db.ObjectsDB.close(ObjectsDB.java:1236)
	at com.compomics.util.experiment.identification.IdentificationDB.close(IdentificationDB.java:1362)
	at com.compomics.util.experiment.identification.Identification.close(Identification.java:1800)
	at eu.isas.peptideshaker.fileimport.FileImporter$IdProcessorFromFile.importFiles(FileImporter.java:453)
	at eu.isas.peptideshaker.fileimport.FileImporter.importFiles(FileImporter.java:153)
	at eu.isas.peptideshaker.PeptideShaker.importFiles(PeptideShaker.java:225)
	at eu.isas.peptideshaker.cmd.PeptideShakerCLI.createProject(PeptideShakerCLI.java:708)
	at eu.isas.peptideshaker.cmd.PeptideShakerCLI.call(PeptideShakerCLI.java:177)
	at eu.isas.peptideshaker.cmd.PeptideShakerCLI.main(PeptideShakerCLI.java:1025)
Caused by: java.sql.SQLException: Cannot create log file at directory /panfs/roc/groups/1/mscls/apps/galaxyp2014/job_working_directory/085/85336/PeptideShakerCLI/matches/Galaxy_Experiment_2015060911181433866719_Sample_2015060911181433866719_1_id/log.
	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
	... 20 more
Caused by: ERROR XSLAQ: Cannot create log file at directory /panfs/roc/groups/1/mscls/apps/galaxyp2014/job_working_directory/085/85336/PeptideShakerCLI/matches/Galaxy_Experiment_2015060911181433866719_Sample_2015060911181433866719_1_id/log.
	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
	at org.apache.derby.impl.store.raw.log.LogToFile.getLogDirectory(Unknown Source)
	at org.apache.derby.impl.store.raw.log.LogToFile.getLogFileName(Unknown Source)
	at org.apache.derby.impl.store.raw.log.LogToFile.flush(Unknown Source)
	at org.apache.derby.impl.store.raw.log.LogToFile.flush(Unknown Source)
	at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.flush(Unknown Source)
	at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.addContainer(Unknown Source)
	at org.apache.derby.impl.store.raw.xact.Xact.addContainer(Unknown Source)
	at org.apache.derby.impl.store.access.heap.Heap.create(Unknown Source)
	at org.apache.derby.impl.store.access.heap.HeapConglomerateFactory.createConglomerate(Unknown Source)
	at org.apache.derby.impl.store.access.RAMTransaction.createConglomerate(Unknown Source)
	at org.apache.derby.impl.sql.execute.CreateTableConstantAction.executeConstantAction(Unknown Source)
	at org.apache.derby.impl.sql.execute.MiscResultSet.open(Unknown Source)
	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
	... 14 more

========================================
PeptideShaker mzid - Command Line
========================================

The PeptideShaker mzid command line takes a cps file and export the identification results in the mzIdentML format.

For further help see http://peptide-shaker.googlecode.com and http://code.google.com/p/peptide-shaker/wiki/PeptideShakerCLI.

Or contact the developers at https://groups.google.com/group/peptide-shaker.

----------------------
OPTIONS
----------------------

Mandatory parameters:

-in                                 PeptideShaker project (.cps or zip file)
-contact_first_name                 Contact first name.
-contact_last_name                  Contact last name.
-contact_email                      Contact e-mail.
-contact_address                    Contact address.
-organization_name                  Organization name.
-organization_email                 Organization e-mail.
-organization_address               Organization address.
-output_file                        Output file.


Optional annotation parameters:
-contact_url                        Contact URL.
-organization_url                   Organization URL.


Optional temporary folder:
-temp_folder                        A folder for temporary file storage. Use only if you encounter problems with the default configuration.

PeptideShaker project (.cps or zip file) '/home/mscls/apps/galaxyp2014/job_working_directory/085/85336/peptideshaker_output.zip' not found.


===============================================
PeptideShaker Report Exporter - Command Line
===============================================

The PeptideShaker report command line takes a cps file and generates various types of reports.

For further help see http://peptide-shaker.googlecode.com and http://code.google.com/p/peptide-shaker/wiki/PeptideShakerCLI.

Or contact the developers at https://groups.google.com/group/peptide-shaker.

----------------------
OPTIONS
----------------------

Mandatory parameters:

-in                                 PeptideShaker project (.cps or zip file)
-out_reports                        Output folder for report files. (Existing files will be overwritten.)


Optional output parameters:

Report export:

-reports                            Comma separated list of types of report to export. 0: Certificate of Analysis, 1: Default Hierarchical Report, 2: Default PSM Phosphorylation Report, 3: Default PSM Report, 4: Default Peptide Phosphorylation Report, 5: Default Peptide Report, 6: Default Protein Phosphorylation Report, 7: Default Protein Report

Report Documentation export:

-documentation                      Comma separated list of types of report documentation to export. 0: Certificate of Analysis, 1: Default Hierarchical Report, 2: Default PSM Phosphorylation Report, 3: Default PSM Report, 4: Default Peptide Phosphorylation Report, 5: Default Peptide Report, 6: Default Protein Phosphorylation Report, 7: Default Protein Report


Optional temporary folder:

-temp_folder                        A folder for temporary file storage. Use only if you encounter problems with the default configuration.

Marc Vaudel

unread,
Jun 11, 2015, 12:40:40 AM6/11/15
to Pratik Jagtap, Jim Johnson, Harald Barsnes, Thomas McGowan, Ira Cooke, Bjoern Gruening, Timothy Griffin, Galaxy for Proteomics
Hi Pratik,

I am sorry but I don't think using earlier versions will help. This code was untouched for over a year, and the latest modification was the implementation of a longer timeout to avoid IO issues on your system (see my email from 12.05.14 in the thread "Non-deterministic errors on tiny dataset"). It is unclear to me why these IO issues occur, and there is very little information for the debugging. Would it be possible for you to run with a single thread and see if the problem persists?

It somehow seems that the random access file reader we use to access spectra looses the connection to the file along the way. I will try to restore this connection and come back to you with a beta version later this week or next week. Even if this fixes the problem, I am afraid it would make the tool very slow (or even slower ;) ) to regularly restore the connection to mgf files... So any input on how to make this connection stable is more than welcome!

Best regards and sorry for the inconvenience,

Marc

Pratik Jagtap

unread,
Jun 11, 2015, 9:23:59 AM6/11/15
to Marc Vaudel, Jim Johnson, Harald Barsnes, Thomas McGowan, Ira Cooke, Bjoern Gruening, Timothy Griffin, Galaxy for Proteomics
Hello Marc,

Thanks ! I will be meeting with JJ and Tom today and see what best we can do. I will get back to you after the discussion.

Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

...

[Message clipped]  

Ira Cooke

unread,
Jun 16, 2015, 11:13:52 PM6/16/15
to Pratik Jagtap, Marc Vaudel, Jim Johnson, Harald Barsnes, Thomas McGowan, Bjoern Gruening, Timothy Griffin, Galaxy for Proteomics
Hi Pratik & JJ,

I was just wondering if you decided to use the new version of peptideshaker in your tutorial or not?

If you will use the new version then I will upload it to the main toolshed. 

Cheers
Ira

Pratik Jagtap

unread,
Jun 16, 2015, 11:36:46 PM6/16/15
to Ira Cooke, Marc Vaudel, Jim Johnson, Harald Barsnes, Thomas McGowan, Bjoern Gruening, Timothy Griffin, Galaxy for Proteomics
Hello Ira,

From the beta test tools that JJ has installed - we will be using SearchGUI version 1.28.0 and PeptideShaker version 0.40 respectively. JJ will have information on where the latest builds have been deposited. He has made a few changes to them and will be able to provide most up to date information.

Thanks and Regards,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

Ira Cooke

unread,
Jun 16, 2015, 11:46:29 PM6/16/15
to Pratik Jagtap, Marc Vaudel, Jim Johnson, Harald Barsnes, Thomas McGowan, Bjoern Gruening, Timothy Griffin, Galaxy for Proteomics
Hi Pratik, 

OK … I updated the main and test toolsheds to both have the latest versions on the github repository. 

We can easily add JJ’s changes whenever they are ready.

Cheers
Ira

Pratik Jagtap

unread,
Jun 16, 2015, 11:52:32 PM6/16/15
to Ira Cooke, Jim Johnson, Galaxy for Proteomics
Thanks Ira,

We are having our first mock session tomorrow.

Thanks again,

Pratik

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

Ira Cooke

unread,
Jun 17, 2015, 12:01:13 AM6/17/15
to Pratik Jagtap, Jim Johnson, Galaxy for Proteomics
Hi Pratik, 

Awesome. Hope it goes well. 

In case you forgot … we have the project web page


Where the students can go to learn more.  If you want me to add anything there please let me know.

Cheers
Ira

Pratik Jagtap

unread,
Jun 17, 2015, 12:05:12 AM6/17/15
to Ira Cooke, Thomas McGowan, Timothy Griffin, Jim Johnson, Galaxy for Proteomics
We are thinking of having a Word Doc that we will convert to a PDF or html file that can be used as a document for future users to access and use. The document will need a lot of help though (screenshots and instructions in simple language) - going to be a challenge !

Pratik Jagtap,
Managing Director, Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 1479 Gortner Avenue, St. Paul, MN 55108      

Ira Cooke

unread,
Jun 17, 2015, 12:08:06 AM6/17/15
to Pratik Jagtap, Thomas McGowan, Timothy Griffin, Jim Johnson, Galaxy for Proteomics
Hi Pratik, 

No worries … whatever you are happy to post … I am happy to add it.

Cheers
Ira

Pratik Jagtap

unread,
Oct 13, 2015, 1:18:48 PM10/13/15
to Galaxy for Proteomics, Timothy Griffin
FYI -

Pratik Jagtap,
Managing Director, 
Center for Mass Spectrometry and Proteomics,
43 Gortner Laboratory, 
1479 Gortner Avenue, 
St. Paul, MN 55108      

---------- Forwarded message ----------
From: Marc Vaudel <mva...@gmail.com>
Date: Tue, Oct 13, 2015 at 9:52 PM
Subject: Re: New Version of Peptideshaker
To: Björn Grüning <bjoern....@gmail.com>
Cc: Ira Cooke <I.C...@latrobe.edu.au>, Pratik Jagtap <pja...@umn.edu>, Gerben Menschaert <gerben.m...@gmail.com>, James Johnson <j...@msi.umn.edu>, Trevor Wennblom <tre...@umn.edu>, Elvis Ndah <elvis...@ugent.be>, Harald Barsnes <Harald....@uib.no>


Dear Galaxy friends,

We have released a new version of PeptideShaker which should be faster, more stable, and use less memory. We thus recommend updating the one you are using in Galaxy :)

Best regards,

Marc

2015-08-07 22:53 GMT+02:00 Björn Grüning <bjoern....@gmail.com>:
Hi,

is there any way we can compile tandem on an older architecture.
Currently, I get this error if I run SearchGUI on SL6.

> tandem: /lib64/libc.so.6: version `GLIBC_2.14' not found

We have very good experience with using Docker to compile our binaries
for Galaxy: please have a look at:
https://github.com/galaxyproject/docker-build

We already have compiled TPP with it.
The same error I got for crux:

/crux: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found

Thanks,
Bjoern


Marc Vaudel

unread,
Oct 13, 2015, 1:32:52 PM10/13/15
to Pratik Jagtap, Galaxy for Proteomics, Timothy Griffin
Apologizes Tim and Pratik, I did a quick answer all and did not check whether you and the mailing list were included!

Best regards,

Marc

--
You received this message because you are subscribed to the Google Groups "Galaxy for Proteomics" group.
To post to this group, send email to gal...@umn.edu.
Visit this group at http://groups.google.com/a/umn.edu/group/galaxyp/.
To view this discussion on the web visit https://groups.google.com/a/umn.edu/d/msgid/galaxyp/CAFMfZ42rMf_4upO8R7eXjCO_x6%2BqJKQt40OEHa%3DBrm-Gwzd3ng%40mail.gmail.com.

To unsubscribe from this group and stop receiving emails from it, send an email to galaxyp+u...@umn.edu.

Tim Griffin

unread,
Oct 13, 2015, 3:10:38 PM10/13/15
to Marc Vaudel, Thomas McGowan, James Johnson, Candace Guerrero, Pratik Jagtap, Galaxy for Proteomics
Thanks Marc.  We are in fact aware that a new release came out.  We are starting to test it in our test Galaxy instance and hoping we can get it implemented soon and make it our main database searching platform.

We will keep you updated if we run into any issues.

- Tim
--
Tim Griffin, Ph.D.
Associate Professor, and

Director, Center for Mass Spectrometry and Proteomics
University of Minnesota
Dept. of Biochemistry, Molecular Biology and Biophysics
6-155 Jackson Hall
321 Church Street SE
Minneapolis, MN  55455
USA

Office: 7-144 Molecular Cellular Biology (MCB)

Tel: 612-624-5249
Fax: 612-624-0432
Email: tgri...@umn.edu

https://www.cbs.umn.edu/bmbb/contacts/timothy-j-griffin
Center for Mass Spectrometry and Proteomics website: http://www.cbs.umn.edu/msp/

Marc Vaudel

unread,
Oct 13, 2015, 3:26:23 PM10/13/15
to Tim Griffin, Thomas McGowan, James Johnson, Candace Guerrero, Pratik Jagtap, Galaxy for Proteomics
Hi Tim,

You guys are fast, the version is not 24 hours old! We will be happy to help if anything does not go as wanted during testing.

Best regards,

Marc

Tim Griffin

unread,
Oct 13, 2015, 3:29:26 PM10/13/15
to Marc Vaudel, Thomas McGowan, James Johnson, Candace Guerrero, Pratik Jagtap, Galaxy for Proteomics
We probably aren't that fast -- I think we had started working with the update prior to this one.  We'll make sure we have the newest release and keep you updated if we have problems.

Cheers,
- Tim

Marc Vaudel

unread,
Dec 9, 2015, 2:26:23 PM12/9/15
to Ira Cooke, Galaxy for Proteomics, Pratik Jagtap, Gerben Menschaert, Björn Grüning, James Johnson, Trevor Wennblom, Elvis Ndah, Harald Barsnes
Dear all,

We have recently released new versions of SearchGUI and PeptideShaker which are faster with larger databases, include numerous bug fixes, and give much more control to the users over the algorithms. The main changes are that many more parameters are available from the GUI and the command line. The parameters do not need to be provided as separate files anymore and all can now be passed to the command line directly. Finally, the parameter files are now in an open format, json, which can be generated by third party applications.

The documentation has been updated:
http://compomics.github.io/projects/searchgui.html
http://compomics.github.io/peptide-shaker/wiki/peptideshakercli.html
http://compomics.github.io/compomics-utilities/wiki/identificationparameterscli.html

As usual, should you encounter problems with the tools or the documentation, we will be happy to help and do the necessary corrections.

Best regards,

Marc

Marc Vaudel

unread,
Dec 9, 2015, 6:14:49 PM12/9/15
to Gerben Menschaert, Ira Cooke, Galaxy for Proteomics, Pratik Jagtap, Björn Grüning, James Johnson, Trevor Wennblom, Elvis Ndah, Harald Barsnes
Hi Gerben,

The changes are essentially on Comet itself, and our parsing of ms-gf+. These changes only apply to N-terminal proteomics datasets with combinations of peptide N-terminal peptide modifications. There will be more updates to come :)

Best regards,

Marc

2015-12-09 21:25 GMT+01:00 Gerben Menschaert <gerben.m...@gmail.com>:
Hi Marc,

Thanks for the update.
Has there been a lot of updates/changes on SE-specific advanced params or default values? I’ve seen some email traffic on the peptide shaker google-group on changes to Comet and MSGF+.
Should I best proceed in verifying all advanced settings for all SE?

Best regards,
Gerben
Reply all
Reply to author
Forward
0 new messages