Re: [maker-devel] Maker issues

[Carson Holt]

Thanks. Could you also run with the --debug flag set on the command line for a few minutes and send me that.

--Carson

From: Daniel Standage <daniel....@gmail.com>
Date: Monday, 5 November, 2012 10:05 AM
To: Carson Holt <cars...@gmail.com>, Maker Mailing List <maker...@yandell-lab.org>
Subject: Maker issues

Carson,

I updated to the latest development version, made sure the TMP directory is on native disk space, and relaunched. I have attached the output of the job that failed in <5 minutes. It looks pretty similar to the errors I got the last time I used the dev version.

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

[Carson Holt]

1.006902        Bio::Root::Version      /N/u/dstandag/Mason/local/src/PerlLibs/bioperl-live/Bio/Root/Version.pm

One thing I noticed, in the debug output is that you are using Bioperl live (here -->  /N/u/dstandag/Mason/local/src/PerlLibs/bioperl-live).  It's fasta indexer is broken.  I have an open bug I am trying to resolve with the Bioperl developers, but for now use the CPAN version of Bioperl.

Thanks,

Carson

From: Daniel Standage <daniel....@gmail.com>
Date: Monday, 5 November, 2012 10:14 AM
To: Carson Holt <cars...@gmail.com>

Cc: Maker Mailing List <maker...@yandell-lab.org>
Subject: Re: Maker issues

Debug output attached (bzip2 compressed).

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

[Daniel Standage]

[Daniel Standage]

Done.

Test job has successfully cleared the preliminary Fasta indexing steps and is repeat masking. I'll let you know if there are any problems. Thanks!

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

[Daniel Standage]

Looked good for a while, but came across this error.

total clusters:20 now processing 0

flattening EST clusters

doing tblastx of alt-ESTs

Undefined subroutine &GI::loalize_file called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 2648.

--> rank=NA, hostname=c4

ERROR: Failed while doing tblastx of alt-ESTs

ERROR: Chunk failed at level:4, tier_type:2

FAILED CONTIG:scaffold_58

ERROR: Chunk failed at level:5, tier_type:0

FAILED CONTIG:scaffold_58

examining contents of the fasta file and run log

Calling Datastore::MD5::mkdir at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

Calling uri_escape at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

Calling File::Path::mkpath at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

--Next Contig--

It seems pretty clear that there is a typo in GI.pm. I changed loalize to localize and relaunched.

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

[Carson Holt]

Thanks.  Typo now fixed on my end too ;-)

[Daniel Standage]

Scaling up to whole-genome annotation, things seem to be going well. However, there are some intermittent issues. I've seen a couple occurrences of the following error...

#-------------------------------#

Calling out to FastaSeq::convert at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 1480.

running  est2genome search.

#--------- command -------------#

Widget::exonerate::est2genome:

/N/hd01/dstandag/Mason/local/bin/exonerate  -q /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/comp58983_c0_seq101.for.716125-721460.0.fasta -t /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/scaffold_23.716125-721460.0.fasta -Q dna -T dna --model est2genome  --minintron 20 --maxintron 10000 --showcigar --percent 20 > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/scaffold_23.716125-721460.comp58983_c0_seq101.est_exonerate.0

#-------------------------------#

Calling out to FastaSeq::convert at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 1480.

couldn't close /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/comp58983_c0_seq37.for.716125-723330.0.fasta at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/FastaFile.pm line 60.

--> rank=NA, hostname=c4

ERROR: Failed while polishig ESTs

ERROR: Chunk failed at level:2, tier_type:2

FAILED CONTIG:scaffold_23

ERROR: Chunk failed at level:5, tier_type:0

FAILED CONTIG:scaffold_23

examining contents of the fasta file and run log

Calling Datastore::MD5::mkdir at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

Calling uri_escape at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

Calling File::Path::mkpath at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

...as well as one occurrence of this error.

#-------------------------------#

Calling out to FastaSeq::convert at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 1480.

running  est2genome search.

#--------- command -------------#

Widget::exonerate::est2genome:

/N/hd01/dstandag/Mason/local/bin/exonerate  -q /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.3.mason.maker.output/maker.pd

om.3.mason_datastore/scaffold_7/theVoid.scaffold_7/comp59027_c1_seq93.for.1869077-1869882.0.fasta -t /N/dc/scratch/dstandag/PdomGenomic/Anno

tation/output/maker.pdom.3.mason.maker.output/maker.pdom.3.mason_datastore/scaffold_7/theVoid.scaffold_7/scaffold_7.1869077-1869882.0.fasta 

-Q dna -T dna --model est2genome  --minintron 20 --maxintron 10000 --showcigar --percent 20 > /N/dc/scratch/dstandag/PdomGenomic/Annotation/

output/maker.pdom.3.mason.maker.output/maker.pdom.3.mason_datastore/scaffold_7/theVoid.scaffold_7/scaffold_7.1869077-1869882.comp59027_c1_se

q93.est_exonerate.0

#-------------------------------#

------------- EXCEPTION: Bio::Root::Exception -------------

MSG: Sequence is a protein. Cannot revcom

STACK: Error::throw

STACK: Bio::Root::Root::throw /N/u/dstandag/Mason/local/src/PerlLibs/lib/perl5/Bio/Root/Root.pm:368

STACK: Bio::PrimarySeqI::revcom /N/u/dstandag/Mason/local/src/PerlLibs/lib/perl5/Bio/PrimarySeqI.pm:381

STACK: Bio::LocatableSeq::revcom /N/u/dstandag/Mason/local/src/PerlLibs/lib/perl5/Bio/LocatableSeq.pm:577

STACK: exonerate::splice_info::needs_to_be_revcomped /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/exonerate/splice_info.pm:86

STACK: Widget::exonerate::est2genome::assemble /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Widget/exonerate/est2genome.pm:686

STACK: Widget::exonerate::est2genome::parse /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Widget/exonerate/est2genome.pm:961

STACK: polisher::exonerate::est::e_exonerate /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/polisher/exonerate/est.pm:82

STACK: polisher::exonerate::est::polish /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/polisher/exonerate/est.pm:44

STACK: GI::to_polisher /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm:1670

STACK: GI::polish_exonerate /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm:1517

STACK: Process::MpiChunk::_go /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm:1663

STACK: Process::MpiChunk::run /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm:335

STACK: Process::MpiChunk::run_all /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm:351

STACK: Process::MpiTiers::run_all /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm:286

STACK: Process::MpiTiers::run_all /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm:286

STACK: /N/u/dstandag/Mason/local/src/maker-dev/bin/maker:644

-----------------------------------------------------------

--> rank=NA, hostname=c4

ERROR: Failed while polishig ESTs

ERROR: Chunk failed at level:2, tier_type:2

FAILED CONTIG:scaffold_7

ERROR: Chunk failed at level:5, tier_type:0

FAILED CONTIG:scaffold_7

examining contents of the fasta file and run log

Calling Datastore::MD5::mkdir at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

Calling uri_escape at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

Calling File::Path::mkpath at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

I'll let you know if I see anything else.

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

[Mark Yandell]

Hi Daniel,

is it possible you have some proteins in your EST files?

'------------- EXCEPTION: Bio::Root::Exception -------------

MSG: Sequence is a protein. Cannot revcom

STACK: Error::throw'



Mark Yandell
Professor of Human Genetics
H.A. & Edna Benning Presidential Endowed Chair
Eccles Institute of Human Genetics
University of Utah
15 North 2030 East, Room 2100
Salt Lake City, UT 84112-5330
ph:801-587-7707

________________________________________
From: maker-dev...@yandell-lab.org [maker-dev...@yandell-lab.org] on behalf of Daniel Standage [daniel....@gmail.com]
Sent: Thursday, November 08, 2012 7:32 AM
To: Carson Holt
Cc: Maker Mailing List
Subject: Re: [maker-devel] Maker issues

STACK: exonerate::splice_info::needs_to_be_revcomped /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/exonerate/splice_info.pm:86<http://splice_info.pm:86>
STACK: Widget::exonerate::est2genome::assemble /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Widget/exonerate/est2genome.pm:686<http://est2genome.pm:686>
STACK: Widget::exonerate::est2genome::parse /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Widget/exonerate/est2genome.pm:961<http://est2genome.pm:961>
STACK: polisher::exonerate::est::e_exonerate /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/polisher/exonerate/est.pm:82<http://est.pm:82>
STACK: polisher::exonerate::est::polish /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/polisher/exonerate/est.pm:44<http://est.pm:44>

On Wed, Nov 7, 2012 at 11:46 AM, Carson Holt <cars...@gmail.com<mailto:cars...@gmail.com>> wrote:
Thanks. Typo now fixed on my end too ;-)

Thanks,
Carson

From: Daniel Standage <daniel....@gmail.com<mailto:daniel....@gmail.com>>
Date: Wednesday, 7 November, 2012 11:43 AM

To: Carson Holt <cars...@gmail.com<mailto:cars...@gmail.com>>
Cc: Maker Mailing List <maker...@yandell-lab.org<mailto:maker...@yandell-lab.org>>

Subject: Re: Maker issues

Looked good for a while, but came across this error.

total clusters:20 now processing 0
flattening EST clusters
doing tblastx of alt-ESTs
Undefined subroutine &GI::loalize_file called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 2648.
--> rank=NA, hostname=c4
ERROR: Failed while doing tblastx of alt-ESTs
ERROR: Chunk failed at level:4, tier_type:2
FAILED CONTIG:scaffold_58

ERROR: Chunk failed at level:5, tier_type:0
FAILED CONTIG:scaffold_58

examining contents of the fasta file and run log
Calling Datastore::MD5::mkdir at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.
Calling uri_escape at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.
Calling File::Path::mkpath at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.



--Next Contig--

It seems pretty clear that there is a typo in GI.pm. I changed loalize to localize and relaunched.


--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

On Wed, Nov 7, 2012 at 9:30 AM, Daniel Standage <daniel....@gmail.com<mailto:daniel....@gmail.com>> wrote:
Done.

Test job has successfully cleared the preliminary Fasta indexing steps and is repeat masking. I'll let you know if there are any problems. Thanks!


--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

On Wed, Nov 7, 2012 at 9:00 AM, Carson Holt <cars...@gmail.com<mailto:cars...@gmail.com>> wrote:
1.006902 Bio::Root::Version /N/u/dstandag/Mason/local/src/PerlLibs/bioperl-live/Bio/Root/Version.pm

One thing I noticed, in the debug output is that you are using Bioperl live (here --> /N/u/dstandag/Mason/local/src/PerlLibs/bioperl-live). It's fasta indexer is broken. I have an open bug I am trying to resolve with the Bioperl developers, but for now use the CPAN version of Bioperl.

Thanks,
Carson

From: Daniel Standage <daniel....@gmail.com<mailto:daniel....@gmail.com>>
Date: Monday, 5 November, 2012 10:14 AM
To: Carson Holt <cars...@gmail.com<mailto:cars...@gmail.com>>
Cc: Maker Mailing List <maker...@yandell-lab.org<mailto:maker...@yandell-lab.org>>
Subject: Re: Maker issues

Debug output attached (bzip2 compressed).


--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

On Mon, Nov 5, 2012 at 10:08 AM, Carson Holt <cars...@gmail.com<mailto:cars...@gmail.com>> wrote:
Thanks. Could you also run with the --debug flag set on the command line for a few minutes and send me that.

--Carson

From: Daniel Standage <daniel....@gmail.com<mailto:daniel....@gmail.com>>
Date: Monday, 5 November, 2012 10:05 AM
To: Carson Holt <cars...@gmail.com<mailto:cars...@gmail.com>>, Maker Mailing List <maker...@yandell-lab.org<mailto:maker...@yandell-lab.org>>
Subject: Maker issues

Carson,

I updated to the latest development version, made sure the TMP directory is on native disk space, and relaunched. I have attached the output of the job that failed in <5 minutes. It looks pretty similar to the errors I got the last time I used the dev version.

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

_______________________________________________
maker-devel mailing list
maker...@box290.bluehost.com
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

[Daniel Standage]

Based on Mark's suggestion, I took a look at the EST files. Luckily there is no protein sequence contamination.

[dstandag@mason Transcriptome] grep -v '^>' Pdom.Trinity.Trimmomatic.fasta | grep -o . | sort | uniq -c

79400764 A

39834991 C

40702954 G

77980105 T

[dstandag@mason Transcriptome] grep -v '^>' Pmet.Trinity.R.fasta | grep -o . | sort | uniq -c

18294708 A

9108213 C

9449127 G

17756470 T

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

[Carson Holt]

The first error is an IO error with your system.  I've added some more detail to the errors in the development version if you do an 'svn update'.  Then you will know the system specific reason why close or opened failed.  For the other error, could you send me this file  --> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.3.mason.maker.output/maker.pdom.3.mason_datastore/scaffold_7/theVoid.scaffold_7/scaffold_7.1869077-1869882.comp59027_c1_seq93.est_exonerate.0

This one --> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/scaffold_23.716125-721460.0.fasta

And this one --> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/comp58983_c0_seq101.for.716125-721460.0.fasta

thanks,

[Daniel Standage]

Thanks for your reply, and sorry for my delayed response.

I have attached the first file you requested, but the other two do not exist. I have attached a listing of the files in that directory. Let me know if you need anything else.

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University