Error in snakemake metagenomics tutorial

[Hilary Morrison]

At this step:

anvi-run-workflow -w metagenomics \

>                   -c config-idba_ud.json

I changed threads from 11 to 8

multiple errors of this sort:

fq2fa --merge 01_QC/sample_03-QUALITY_PASSED_R1.fastq 01_QC/sample_03-QUALITY_PASSED_R2.fastq 01_QC/sample_03-merged-reads.fa >> 00_LOGS/sample_03-fq2fa.log 2>&1

Error in rule fq2fa:

    jobid: 42

    output: 01_QC/sample_03-merged-reads.fa

    log: 00_LOGS/sample_03-fq2fa.log

RuleException:

CalledProcessError in line 432 of /users/morrison/virtual-envs/anvio-5.1/lib/python3.6/site-packages/anvio/workflows/metagenomics/Snakefile:

Command ' set -euo pipefail;  fq2fa --merge 01_QC/sample_03-QUALITY_PASSED_R1.fastq 01_QC/sample_03-QUALITY_PASSED_R2.fastq 01_QC/sample_03-merged-reads.fa >> 00_LOGS/sample_03-fq2fa.log 2>&1 ' returned non-zero exit status 127.

  File "/users/morrison/virtual-envs/anvio-5.1/lib/python3.6/site-packages/anvio/workflows/metagenomics/Snakefile", line 432, in __rule_fq2fa

  File "/bioware/python-3.6.5-201804120909/lib/python3.6/concurrent/futures/thread.py", line 56, in run

Shutting down, this might take some time.

Exiting because a job execution failed. Look above for error message

Complete log: /groups/sequencing/Illumina/Singh_MEGAHITassembly/WORKFLOW_TUTORIAL_DATA/.snakemake/log/2018-08-07T131213.087684.snakemake.log

I have attached the log...I cannot interpret the error. 

[Alon Shaiber]

Could yoy please also send this log file:

00_LOGS/sample_03-fq2fa.log

Best,

Alon

--
Anvi'o Paper: https://peerj.com/articles/1319/
Project Page: http://merenlab.org/projects/anvio/
Code Repository: https://github.com/meren/anvio
---
You received this message because you are subscribed to the Google Groups "Anvi'o" group.
To unsubscribe from this group and stop receiving emails from it, send an email to anvio+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/45602269-0a9b-409d-9bff-36e1b9b00910%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hilary Morrison]

[Alon Shaiber]

Hi Hilary,

The problem looks quite clear from this log file. You don't have fq2fa on your system. You need to either install it, or load a module that provides it on your system. Since I'm probably familiar with the system you are using, I will write more info in a private email, but I wanted to have this response here, so anyone who finds this issue relevant for them, could see the answer.

Also, I want to mention that the first log file that you attached is the general log file kept by snakemake, while it is sometimes useful, most of the time, it would just let you know which "rule" is the one that caused the error, and would point you to the relevant log file that was created by that rule. It is this log file that you should be looking at (the one created by the rule) in order to understand what went wrong.

Best,

Alon

To unsubscribe from this group and stop receiving emails from it, send an email to anvio+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/d0ef2bef-811c-497d-9327-9c084132fb6f%40googlegroups.com.

[A. Murat Eren]

I thought we were checking all programs involved with a given set of rules prior to running the workflow and let the user know if they are missing someting.

Why do you think that procedure did not engage to catch this one, Alon?

Best,

--

A. Murat Eren (Meren)
http://merenlab.org :: twitter :: gpg

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/d0ef2bef-811c-497d-9327-9c084132fb6f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Anvi'o Paper: https://peerj.com/articles/1319/
Project Page: http://merenlab.org/projects/anvio/
Code Repository: https://github.com/meren/anvio
---
You received this message because you are subscribed to the Google Groups "Anvi'o" group.
To unsubscribe from this group and stop receiving emails from it, send an email to anvio+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/CADGk4-D9ZQDqw_iyH9UrVDN5e158jq7_58X1jwVkkuA8qnDWqw%40mail.gmail.com.

[Alon Shaiber]

We are currently not checking those programs. We can discuss this, but I removed that steps due to a number of problems with the current implementation of it.

To unsubscribe from this group and stop receiving emails from it, send an email to anvio+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/d0ef2bef-811c-497d-9327-9c084132fb6f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Anvi'o Paper: https://peerj.com/articles/1319/
Project Page: http://merenlab.org/projects/anvio/
Code Repository: https://github.com/meren/anvio
---
You received this message because you are subscribed to the Google Groups "Anvi'o" group.

To unsubscribe from this group and stop receiving emails from it, send an email to anvio+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/CADGk4-D9ZQDqw_iyH9UrVDN5e158jq7_58X1jwVkkuA8qnDWqw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Anvi'o Paper: https://peerj.com/articles/1319/
Project Page: http://merenlab.org/projects/anvio/
Code Repository: https://github.com/meren/anvio
---
You received this message because you are subscribed to the Google Groups "Anvi'o" group.

To unsubscribe from this group and stop receiving emails from it, send an email to anvio+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/CAK5_LadLVzNTTO3efP46S_X88wKyjvjFNVTC_6%2B7jpzwxMUuVQ%40mail.gmail.com.