sample2markers.py crashes because of handling of the fastq piped as standard input

[Christophe Lemetre]

Hi again,

It seems that now the process with sample2markers.py crashes because of the SeqIO package, with FastqGeneralIterator not able to recognize the first character as properly fastq format:

$ sample2markers.py --ifn_samples SRR6000873.sam.bz2 --input_type sam --output_dir strainphlan_output/ --bcftools_exe /Data-2/bin/bcftools/bcftools
ooSubprocess: dump_file.py --input_file SRR6000873.sam.bz2 | samtools view -bS - | samtools sort - -o strainphlan_output/SRR6000873.sam.bz2.bam.sorted | samtools mpileup -u - | /Data-2/bin/bcftools/bcftools view -c -g -p 1.1 - | fix_AF1.py --input_file - | vcfutils.pl vcf2fq
2019-12-04 15:43:19,477 | ERROR | __main__ | sam2markers | 357 | sam2markers failed on file SRR6000873.sam.bz2
Traceback (most recent call last):
  File "/Data-2/anaconda3/envs/metag/bin/ooSubprocess.py", line 246, in wrapper
    return f(*args, **kwargs)
  File "/Data-2/anaconda3/envs/metag/bin/sample2markers.py", line 414, in run_sample
    bcftools_exe=args['bcftools_exe'])
  File "/Data-2/anaconda3/envs/metag/bin/sample2markers.py", line 353, in sam2markers
    for rec in SeqIO.parse(p7, 'fastq'):
  File "/Data-2/anaconda3/envs/metag/lib/python3.6/site-packages/Bio/SeqIO/__init__.py", line 661, in parse
    for r in i:
  File "/Data-2/anaconda3/envs/metag/lib/python3.6/site-packages/Bio/SeqIO/QualityIO.py", line 1027, in FastqPhredIterator
    for title_line, seq_string, quality_string in FastqGeneralIterator(handle):
  File "/Data-2/anaconda3/envs/metag/lib/python3.6/site-packages/Bio/SeqIO/QualityIO.py", line 897, in FastqGeneralIterator
    raise ValueError("Is this handle in binary mode not text mode?")
ValueError: Is this handle in binary mode not text mode?

I tried to read a regular fastq that is saved in the tmp folder to test the SeqIO FastqGeneralIterator, and it works fine to read it... Any idea how to get around this?

Many thanks

Christophe

[Christophe Lemetre]

No one has any idea?

[Aitor Blanco-Miguez]

Hi Christophe,

Both errors were already reported and we already uploaded a fix for them on the last MetaPhlAn verion. I will check the last version in conda in order to ensure it has this issues fixed.
Thank you very much for reporting these problems. I will contact again after this verification.

Best,

Aitor

[Christophe Lemetre]

Hi Aitor,

Ok thank you! I installed metaphlan with conda and have the 2.9.5.3 (21 Nov 2019) version. Is there a new one then?

Many thanks!
Christophe

[Aitor Blanco-Miguez]

Hi Christophe,
I just checked and the commit fixing this issue is from 25 Nov. We will try to upload the fix in conda asap. When the new version will be available, I will notify you.

Best,
Aitor

[Christophe Lemetre]

Awesome! Looking forward to it, let me know.

Thank you

Christophe

[Aitor Blanco-Miguez]

In the meanwhile, I recomend you to get the last Metaphlan2.9 version from the Bitbucket repository: https://bitbucket.org/biobakery/metaphlan2/src/2.9/

Best, 

Aitor

[Christophe Lemetre]

Thanks Aitor, I really appreciate the help, just trying to figure all this out so I can make progress.

So I am confused, Which version of both Metaphlan and Strainphlan should be used then? (bug fixed)
And which are available on conda and which are on bitbucket?

Are you advising to remove the conda install of both Metaphlan and Strainphlan and re-install only with the source in the bitbucket repo?

Many thanks
Christophe

[Aitor Blanco-Miguez]

Hi Christophe, 

The best thing would be removing the MetaPhlAn and StrainPhlAn conda installations and get the Bitbucket version.

You should clone the repository: https://bitbucket.org/biobakery/metaphlan2/src/2.9/ and use the branch 2.9 (MetaPhlAn and StrainPhlAn are both together).

Also you must have this libraries installed for python: argparse, tempfile, numpy, and Biopython; and also BowTie2 version 2.0.0 or higher and Perl.

Best,

Aitor

[Christophe Lemetre]

Hi Aitor, 

Thank you for your help. I am trying to install via the clone of the repo (2.9).

Any update on the conda install status?

[Aitor Blanco-Miguez]

Hi Christophe, 

Sorry, not yet. I will have you updated.

Best, 

Aitor

[Aitor Blanco-Miguez]

Hi Christophe,
Metaphlan2.96 is already available through conda.

Best,

Aitor