metaphlan2.py running error

[adrian...@inia.es]

Hi!

I have recently started using MetaPhlAn2, and I am having some trouble for setting up the configuration.
I have manually downloaded and installed Metaphlan2 in my session of a remote server. Following the tutorial instructions, I included the export PATH=$MDIR: $MDIR/utils/:$PATH command in bashrc file and manually downloaded and installed bowtie2.
I suspect that I am not installing bowtie2 correctly, because when I run a command:

$[Path]/metaphlan2.py $[Path_data]/0.15.01610.001_R1.fastq.gz --input_type fastq --bowtie2_exe $[Path]/bowtie2-2.3.4-mingw-x86_64/bowtie2 > $[Path_data]/../0.15.01610.001_R1_profile.txt

My output is:

Downloading https://bitbucket.org/biobakery/metaphlan2/downloads/mpa_v20_m200.tar
Downloading file of size: 241.78 MB
241.78 MB 100.00 % 16.66 MB/sec 0 min -0 sec
Downloading https://bitbucket.org/biobakery/metaphlan2/downloads/mpa_v20_m200.md5
Downloading file of size: 0.00 MB
0.01 MB 16062.75 % 25.44 MB/sec 0 min -0 sec

Decompressing /home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200.fna.bz2 into /home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200.fna

Building Bowtie2 indexes
Building a SMALL index
Could not open file for reading a reference graph: "/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200.0.sa"
Error: Encountered internal Bowtie 2 exception (#1)
Command: bowtie2-build --wrapper basic-0 --quiet --threads 4 -f /home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200.fna /home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200
Deleting "/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200.3.bt2" file written during aborted indexing attempt.
Deleting "/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200.4.bt2" file written during aborted indexing attempt.
Deleting "/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200.1.bt2" file written during aborted indexing attempt.
Deleting "/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200.2.bt2" file written during aborted indexing attempt.
Fatal error running 'bowtie2-build --quiet --threads 4 -f /home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200.fna /home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200'
Error message: 'Command '['bowtie2-build', '--quiet', '--threads', '4', '-f', '/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200.fna', '/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/databases/mpa_v20_m200']' returned non-zero exit status 1'

Also, when I rerun the command:

No MetaPhlAn BowTie2 database found (--index option)!
Expecting location bowtie2db
Exiting...

What seems to be the problem?

Thanks a lot!

[Francesco Asnicar]

Hi,

I briefly check your problem and I think it is nothing wrong with your system/installation, but I think it is related to bowtie2 (https://github.com/BenLangmead/bowtie2/issues/36). That issue in GitHub is quite old, but it seems to be exactly your problem.

From the error you posted, I think you ran MetaPhlAn2 with "--nproc 4", can I ask you to re-run it without specifying the "--nproc" param?

In this way, MetaPhlAn2 will run in single-core mode and it won't use the "--threads" option of Bowtie2 for building the indexed database.

Please, let me know if this solves the problem so that I will then remove the portion of code that uses the multi-threading Bowtie2 for building the database.

Many thanks,

Francesco

--
You received this message because you are subscribed to the Google Groups "MetaPhlAn-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metaphlan-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[adrian...@inia.es]

Hi,
Thank you for your answer. But this is quite strange, because I did not specified the parameter --nproc in my script, although bowtie2 seems to be using that multi-thread option yopu mention... Would it work if I specify "--nproc 1"?

[Francesco Asnicar]

You're right... nproc in MetaPhlAn2 is set to 4 by default, sorry.

Yes, if you specified --nproc 1 it should then disable the multi-threaded bowtie2 execution.

Thanks,

Francesco

On Tue, Apr 17, 2018 at 9:48 AM <adrian...@inia.es> wrote:

Hi,
Thank you for your answer. But this is quite strange, because I did not specified the parameter --nproc in my script, although bowtie2 seems to be using that multi-thread option yopu mention... Would it work if I specify "--nproc 1"?

[adrian...@inia.es]

Ok, with --nproc 1 this error disappeared, but another issue came out:
Running the same command, the output now is:

Help message for read_fastx.py
: No such file or directory
OSError: "Command '['/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/bowtie2-2.3.4-mingw-x86_64/bowtie2', '-h']' returned non-zero exit status 127"
Fatal error running BowTie2. Is BowTie2 in the system path?

Apparently, I have 2 different paths for bowtie2:
I discovered one using:
(pythonenv) [otinialg@fs6801 ~]$ which bowtie2
~/.conda/envs/pythonenv/bin/bowtie2

The other one is installed manually at this path:
'/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/bowtie2-2.3.4-mingw-x86_64/bowtie2'

And when using:
(pythonenv) [otinialg@fs6801 ~]$ vi ~/.bashrc
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

# User specific aliases and functions
R_LIBS_USER=$HOME/R/
export R_LIBS_USER
export PATH=/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801:/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/utils/:$PATH
export BTPATH=$BTPATH:~/.conda/envs/pythonenv/bin/bowtie2

Is there anything wrong with my bowtie installs?
Many thanks for your help.
Adrian

[Francesco Asnicar]

Hi Adrian,

Can you check that the following command is executing correctly (it should print all the Bowtie2 command line options)?

/home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/bowtie2-2.3.4-mingw-x86_64/bowtie2 -h

Many thanks,

Francesco

[adrian...@inia.es]

No, it is not working correctly:

(pythonenv) [otinialg@fs6801 cleaned]$ /home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/bowtie2-2.3.4-mingw-x86_64/bowtie2 -h

: No such file or directory

But, if I cd to that path "-h" seems to function correctly:

(pythonenv) [otinialg@fs6801 ~]$ cd /home/otras/ini/alg/biobakery-metaphlan2-5175d8783801/bowtie2-2.3.4-mingw-x86_64/
(pythonenv) [otinialg@fs6801 bowtie2-2.3.4-mingw-x86_64]$ ls
AUTHORS bowtie2-align-s.exe bowtie2-build-l.exe bowtie2-inspect-l-debug.exe example scripts
bowtie2 bowtie2.bat bowtie2-build-s-debug.exe bowtie2-inspect-l.exe LICENSE TUTORIAL
bowtie2-align-l-debug.exe bowtie2-build bowtie2-build-s.exe bowtie2-inspect-s-debug.exe MANUAL VERSION
bowtie2-align-l.exe bowtie2-build.bat bowtie2-inspect bowtie2-inspect-s.exe MANUAL.markdown
bowtie2-align-s-debug.exe bowtie2-build-l-debug.exe bowtie2-inspect.bat doc NEWS
(pythonenv) [otinialg@fs6801 bowtie2-2.3.4-mingw-x86_64]$ bowtie2 -h
Bowtie 2 version 2.2.8 by Ben Langmead (lan...@cs.jhu.edu, www.cs.jhu.edu/~langmea)
Usage:
[...]

[Francesco Asnicar]

Hi Adrian,

Ok, so that's the problem.

If you first "cd" to the directory and then execute "bowtie2 -h", that will not run the bowtie2 executable in the local directory, but the one you have exported in your path (the one from the conda env from your previous email).

If you want to run the bowtie2 executable present in the local directory you should type "./bowtie2 -h", and I think it will give you the exact same error.

I suggest you to check that you have the executables permission for that bowtie2, you can see if you have the "x" by issuing a "ls -l".

Otherwise, I think you should use the bowtie2 you already have available form the conda environment.

Many thanks,

Francesco

[adrian...@inia.es]

I follow your advice and used the bowtie2 version from the conda environment. Now it seems to work!

Thank you very much for your time.
Adrián

[Nhung Doan]

I run the command: metaphlan2.py /Users/Bioinformatics/Desktop/metaphlan/acropora.fastq --input_type fasta --mpa_pkl ./metaphlan_databases/mpa_v20_m200.pkl --bowtie2db ./metaphlan_databases/mpa_v20_m200 --bowtie2_exe /data/program/bowtie2-2.3.4.1 --bowtie2out stool.out --tmp_dir 

when I remove --tmp_dir at the end 

OSError: "[Errno 2] No such file or directory: '/data/program/bowtie2-2.3.4.1'"

Fatal error running BowTie2. Is BowTie2 in the system path?

what should I do to fix

Vào 18:11:56 UTC+7 Thứ Hai, ngày 16 tháng 4 năm 2018, adria...@inia.es đã viết:

[Francesco Beghini]

When you use --tmp_dir  you have to provide a folder in which the temporary files are then saved. Regarding the BowTie2 error, you have to provide to --bowtie2_exe the exact path to the bowtie2 executable (/data/program/bowtie2-2.3.4.1/bowtie2).

Let me know if you have any problems.

Best,

Francesco

Francesco Beghini

PhD Student - Laboratory of Computational Metagenomics

Department of Cellular, Computational and Integrative Biology - CIBIO

University of Trento

--

You received this message because you are subscribed to the Google Groups "MetaPhlAn-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metaphlan-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/metaphlan-users/2c9af37e-4f81-473a-84ca-d443f8e92727%40googlegroups.com.