problem indexing, process running without any progress

[alban mathieu]

I tried to index, but the program has run overnight without any success.

I tried with the dubug option, here is the ouput:

./phylosift index --config phylosiftrc --debug

Reading config file : phylosiftrc

Found 741 markers

... and nothing more happens

I was thinking that maybe i don't need to index, I tried to start a all programm with debug but no marker were retrieved despite I know that it has work in a previous version of phylosift that i had..

Any idea, please?

Thanks,

Alban.

[alban mathieu]

Ok, I tried with markers folder "markers_20140913.tgz" instead of using "markers.tgz" and it seems to have worked:
########

./phylosift index --config phylosiftrc --debug
Reading config file : phylosiftrc
Found 741 markers

Requesting Indexed_markers : 0
Indexing 741 in PhyloSift/markers2/markers2
Inside index_marker_db
###########

But am I right using folder dated of the year 2014?

[Guillaume Jospin]

I’d recommend grabbing the more recent version of the markers from the figshare repository.  

https://figshare.com/articles/PhyloSift_markers_database/5755404

use the file markers.tgz. The date file from 2014 is just there to have a record of what the DB used to look like.

I'm also working on a newer version of the DB that should fix a couple few automation mistakes. It only affects a very small subset of the data.

Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "PhyloSift" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phylosift+...@googlegroups.com.
To post to this group, send email to phyl...@googlegroups.com.
Visit this group at https://groups.google.com/group/phylosift.
For more options, visit https://groups.google.com/d/optout.

[alban mathieu]

Ok, I did, thanks. It works now, 

It looks that it runs with the redownloaded markers.tgz. But only 80 markers seems to be used, do you agree with that?

./phylosift index --config phylosiftrc --debug

Reading config file : phylosiftrc

Use of uninitialized value $timestamp in scalar chomp at /mnt/gpfs/pt2/ama/phylosift/phylosift_v1.0.1/bin/../lib/Phylosift/Utilities.pm line 365, <$FH> line 1.

Found 80 markers

Requesting Indexed_markers : 0

Indexing 80 in /mnt/gpfs/pt2/ama/phylosift/phylosift_v1.0.1/markers

Inside index_marker_db

Thanks Guillaume,

[Guillaume Jospin]

Yes 80 is about right. We removed the viral markers because we weren’t very confident in those. 

Sent from my iPhone

[alban mathieu]

Ok, thanks for your advises, I have new issues (sorry to complain in real time):

Error: Failed to open covariance model save file /mnt/gpfs/pt2/ama/phylosift/phylosift_v1.0.1/markers/16s_reps_bac/16s_reps_bac.cm

Running /mnt/gpfs/pt2/ama/phylosift/phylosift_v1.0.1/bin/pplacer --groups 15 -o /mnt/gpfs/pt2/ama/all_genomesPROCHICK_withphylosift_v1.0.1_and_dbmarker80/treeDir/concat.updated.1.jplace --verbosity 0 -j 10 -c /mnt/gpfs/pt2/ama/phylosift/phylosift_v1.0.1/markers/concat.updated "/mnt/gpfs/pt2/ama/all_genomesPROCHICK_withphylosift_v1.0.1_and_dbmarker80/alignDir/concat.updated.1.fasta"

query 15.1.2208073.2208810.1.1442628.1442933.1.503192.503881.1.3233865.3235919.1.3084317.3086371.1.3084308.3086362.1.2061359.2062261.1.1459684.1460673.1.1439369.1439701.1.1441272.1441886.1.1440035.1440865.1.992405.992794.1.1441909.1442538.1.2927185.2929548.1.1437346.14377

11.1.1434370.1434906.1.1439732.1440004.1.1993287.1993754.1.1438251.1438655.1.1429979.1430344.1.2926133.2927149.1.1433720.1434154.1.1440973.1441254.1.1435277.1435804.1.503945.504364.1.1436450.1436986.1.1992735.1993142.1.1438036.1438215.1.1438664.1439344.1.1429558.1429923.1

.502404.502874.1.1435821.1436216.1.3237377.3238270.1.3081957.3082850.1.3239401.3239664.1.3080563.3080826.1.557264.558115.1.1437748.1437993.1.991951.992376.1.1913250.1913837.1.1437006.1437323 is not the same length as the reference alignment (got 9023; expected 7109)

no output in /mnt/gpfs/pt2/ama/all_genomesPROCHICK_withphylosift_v1.0.1_and_dbmarker80/treeDir/concat.updated.1.jplace

Do you know what it means? Because it ended with other markers, but not concat..

[alban mathieu]

and the HTML page produces nothing..

[Guillaume Jospin]

Ok so for the 16s thing, check to make sure the proper permissions are set for the cm file reported. You should be able to read the file from your account. I think I never fixed that part. 

For the other issue, are you running contigs ? This usually happens when you have 2 hits to the same marker (one of 37 that get concatenated ) and it strings an extra copy of the marker for the same sequence header. A quick way to get around that is to add the —besthit flag to ensure it keeps the best matching candidate sequence for each marker. It’s kind of a dirty hackish workaround. 

PS was originally developed for processing reads but it takes a very very long time now that read counts have increased tremendously. 

I hope this helps. 

Sent from my iPhone

[Guillaume Jospin]

The html is empty because it only gets populated with concatenated data. If your concat market failed then the. Subsequent steps will fail also. 

FYI, All the markers are reported in the text files. Only the concat is reported in the krona plot. 

Sent from my iPhone

[alban mathieu]

Ok it makes sense, I tried to show the all picture of my problems.

For the 16 thing:

There are 2 files and I have the capacity to read both:

16s_reps_bac.cm

16s_reps_bac.cm_backup

So I don't know whats wrong.

I'll try what you propose with the best hit option,

Thanks a lot for your suggestions.

Alban.

[alban mathieu]

And yes you'r right it is contigs,

[Guillaume Jospin]

Hrm strange about the cm file.

I'm not sure what the problem would be. 

I would also recommend you use a different tool if you really are interested in the 16S reads. We developed the 16S marker to try to have a direct comparison with other tools (like qiime)  but those other tools are so much better at it than PS. If you ignore the 16s error, you will get results just not for that particular marker.

I hope that's an acceptable solution for your study.