metaphlan2 database error with demo

[wesle...@gmail.com]

Hello,

I recently installed HUMAnN2 (v0.11.1) along with metaphlan2 (v2.7.6) and am currently trying to run the demo. Based on a previous forum topic, I included the metaphlan options as recommended due to the change in metaphlan default database location (command below). However, the process stilll gets stuck on "Running metaphlan2.py" with no additional output or error message. Do you have any recommendations for troubleshooting this issue?

humann2 --input examples/demo.fastq --output demo_out --metaphlan-options='--mpa_pkl /n/home09/wloo/biobakery-metaphlan2-7df8753d7e5e/databases/mpa_v20_m200.pkl --bowtie2db /n/home09/wloo/biobakery-metaphlan2-7df8753d7e5e/databases/mpa_v20_m200'

Thank you!
Wesley

[Lauren McIver]

Hi Wesley - The first time the new MetaPhlAn2 version runs it downloads, builds, and installs its database. This can take a couple minutes. Is it possible this is what you might be seeing? If you don't see any errors I would let it run for a bit and see if it completes the demo. If this is the case, the next time you run the demo it will be much faster. If you think this is not the case, can you check out the humann2 log for the run to see if there is any additional information?

Thanks!

Lauren

[wesle...@gmail.com]

Hi Lauren,

I did run MetaPhlAn2 by itself and it successfully downloaded, built, and installed the database. I can run the metaphlan2.py command by itself with example data and get the expected output.

I tried the humann2 command again and let it run overnight. It is still stuck on the metaphlan2 command and the log file doesn't show any errors. I've copied it below.

Happy to hear any other suggestions for getting this running.

Thanks!
Wesley

04/03/2018 09:36:21 PM - humann2.humann2 - INFO: Running humann2 v0.11.1
04/03/2018 09:36:21 PM - humann2.humann2 - INFO: Output files will be written to: /n/regal/cavanaugh_lab/wesleyloo/df_metagenomics/humann_demo/humann2-0.11.1/demo_out
04/03/2018 09:36:21 PM - humann2.humann2 - INFO: Writing temp files to directory: /n/regal/cavanaugh_lab/wesleyloo/df_metagenomics/humann_demo/humann2-0.11.1/demo_out/demo_humann2_temp
04/03/2018 09:36:21 PM - humann2.utilities - INFO: File ( /n/regal/cavanaugh_lab/wesleyloo/df_metagenomics/humann_demo/humann2-0.11.1/examples/demo.fastq ) is of format: fastq
04/03/2018 09:36:21 PM - humann2.utilities - DEBUG: Check software, bowtie2, for required version, 2.2
04/03/2018 09:36:22 PM - humann2.utilities - INFO: Using bowtie2 version 2.3
04/03/2018 09:36:22 PM - humann2.humann2 - INFO: Search mode set to uniref90 because a uniref90 translated search database is selected
04/03/2018 09:36:22 PM - humann2.utilities - DEBUG: Check software, diamond, for required version, 0.8.22
04/03/2018 09:36:22 PM - humann2.utilities - INFO: Using diamond version 0.8.24
04/03/2018 09:36:22 PM - humann2.config - INFO:
Run config settings:

DATABASE SETTINGS
nucleotide database folder = /n/home09/wloo/.conda/envs/humann2/lib/python2.7/site-packages/humann2/data/chocophlan_DEMO
protein database folder = /n/home09/wloo/.conda/envs/humann2/lib/python2.7/site-packages/humann2/data/uniref_DEMO
pathways database file 1 = /n/home09/wloo/.conda/envs/humann2/lib/python2.7/site-packages/humann2/data/pathways/metacyc_reactions_level4ec_only.uniref.bz2
pathways database file 2 = /n/home09/wloo/.conda/envs/humann2/lib/python2.7/site-packages/humann2/data/pathways/metacyc_pathways_structured_filtered
utility mapping database folder = /n/home09/wloo/.conda/envs/humann2/lib/python2.7/site-packages/humann2/data/misc

RUN MODES
resume = False
verbose = False
bypass prescreen = False
bypass nucleotide index = False
bypass nucleotide search = False
bypass translated search = False
translated search = diamond
pick frames = off
threads = 1

SEARCH MODE
search mode = uniref90
identity threshold = 90.0

ALIGNMENT SETTINGS
evalue threshold = 1.0
prescreen threshold = 0.01
translated subject coverage threshold = 50.0
translated query coverage threshold = 90.0

PATHWAYS SETTINGS
minpath = on
xipe = off
gap fill = on

INPUT AND OUTPUT FORMATS
input file format = fastq
output file format = tsv
output max decimals = 10
remove stratified output = False
remove column description output = False
log level = DEBUG

04/03/2018 09:36:22 PM - humann2.store - DEBUG: Initialize Alignments class instance to minimize memory use
04/03/2018 09:36:22 PM - humann2.store - DEBUG: Initialize Reads class instance to minimize memory use
04/03/2018 09:36:28 PM - humann2.humann2 - INFO: Load pathways database part 1: /n/home09/wloo/.conda/envs/humann2/lib/python2.7/site-packages/humann2/data/pathways/metacyc_reactions_level4ec_only.uniref.bz2
04/03/2018 09:36:42 PM - humann2.humann2 - INFO: Load pathways database part 2: /n/home09/wloo/.conda/envs/humann2/lib/python2.7/site-packages/humann2/data/pathways/metacyc_pathways_structured_filtered
04/03/2018 09:36:42 PM - humann2.search.prescreen - INFO: Running metaphlan2.py ........
04/03/2018 09:36:42 PM - humann2.utilities - DEBUG: Using software: /n/home09/wloo/biobakery-metaphlan2-7df8753d7e5e/metaphlan2.py
04/03/2018 09:36:42 PM - humann2.utilities - INFO: Execute command: /n/home09/wloo/biobakery-metaphlan2-7df8753d7e5e/metaphlan2.py /n/regal/cavanaugh_lab/wesleyloo/df_metagenomics/humann_demo/humann2-0.11.1/examples/demo.fastq --mpa_pkl /n/home09/wloo/biobakery-metaphlan2-7df8753d7e5e/databases/mpa_v20_m200.pkl --bowtie2db /n/home09/wloo/biobakery-metaphlan2-7df8753d7e5e/databases/mpa_v20_m200 -o /n/regal/cavanaugh_lab/wesleyloo/df_metagenomics/humann_demo/humann2-0.11.1/demo_out/demo_humann2_temp/demo_metaphlan_bugs_list.tsv --input_type multifastq --bowtie2out /n/regal/cavanaugh_lab/wesleyloo/df_metagenomics/humann_demo/humann2-0.11.1/demo_out/demo_humann2_temp/demo_metaphlan_bowtie2.txt

[Lauren McIver]

Hi Wesley, Would you try running just the exact metaphlan2 command (the last command found in the humann2 log)? I am hoping this will give us more information by maybe printing an error message. I have not seen this behavior before and am not able to replicate it so any additional information would be helpful in debugging.

Thanks,

Lauren

[wesle...@gmail.com]

Hi Lauren,

I have tried running the exact metaphlan2 command as written in the humann2 log overnight. It also does not produce any output and hangs after printing 'Help message for read_fastx.py'.

Since I can run metaphlan2 with default options, I'm guessing there's something wrong with the additional arguments '--mpa_pkl' and '--bowtie2db'.

To clarify, you are unable to replicate this error using metaphlan2 v2.7.6 as called by humann2 v0.11.1 with python 2.7.14?

I'm also not sure how to further troubleshoot the issue, so any input you can give would be very helpful!

Thank you,
Wesley

[Lauren McIver]

Hi Wesley - Thanks for trying the command just for metaphlan2. I have been unable to replicate the hanging behavior you see with the latest metaphlan2 v2.7.6 (setting different paths, databases, etc). I am trying to think of what might be different in your environment that could cause this issue. Is it possible you have multiple versions of metaphlan2 installed and when you run it uses an older version of read_fastx.py? I don't think the bowtie2 version should matter but just to double check I have run with bowtie2 v2.2.3 and v2.3.4.1 and they both work well. Also maybe can you double check the demo input file and databases are the originals and did not get modified at any point.

Thanks,

Lauren