Metatrans Error 1:

[Christopher Ray Hughes]

Good afternoon,

I'm attempting to setup and use Metatrans for a few gut microbiome samples. I'm using the Samples-Test and Databases-Test files. I have followed the instructions in the test/sample README. Briefly, I have downloaded all files and unpacked in accordance with the README instructions. I have run ./config and then ./metatrans. The first sample (C1JB3ACXX_1_5) is run through quality control where the pipeline fails, exits, and throws Error 1. I have read all the previous threads and do not seem to have the same circumstances (i.e.--not using Red Hat).

The QC log output indicates that:

mv: cannot stat ‘/home/manager/Downloads/1-PROCESSED_SAMPLES/C1JB3ACXX_1_5/1-QC/m1-temp/kraken_results/krakened_C1JB3ACXX_1_5_1/krakened_C1JB3ACXX_1_5_1_reaper_record.txt’: No such file or directory

Indeed this file does not exists and does not appear to ever be created.

System information:
I am running Bio-Linux-8.0.7 (Ubuntu 14.04) in Oracle VM VirtualBox.

Terminal output:
#Checking pipeline files' integrity (Date: '26-06-17' Time: '19h:22m:22s UTC')...
#Checking sequence files' integrity (Date: '26-06-17' Time: '19h:22m:22s UTC')...
#Checking if R libraries are installed and accessible (Date: '26-06-17' Time: '19h:22m:26s UTC')...
#Running Metatrans Pipeline (by default will perform a funcional analysis with differential expression) (Date: '26-06-17' Time: '19h:22m:45s UTC')...
#Using 1 threads as default (this value can be modified per module)
#and a maximum of 1 GB of memory for the Sortmerna task.
#Running sample 'C1JB3ACXX_1_5':
#Running Module1 >>Quality Control<< ... (Date: '26-06-17' Time: '19h:22m:47s UTC')
#Error# Pipeline exited with error code 1.
#HELP# An error may lead files in an intermediate state. To re-run the pipeline some folder might be deleted.

#HELP# Please, read the troubleshooting section of the manual for help.


Any help would be greatly appreciated.

Cheers,
Chris

[metatr...@gmail.com]

Hi Chris,

That error is produced because some files are missing as you already said. To identify the cause, we should investigate a bit more.

According to the info you provided the problem is just at the beginning in the QC module with the test dataset. Seems like a problem with
Kraken tools. We should know at which step the pipeline stopped. Could you check the file:
1-PROCESSED_SAMPLES/C1JB3ACXX_1_5/1-QC/m1-log/m1-time.tsv
That file provides all important commands launched by the pipeline, the last one in the file is the last executed.

Furthermore, the following files will provide also more detailed info. Sort them by date. Check the last modified file (ls -ltr):
output_kraken_stage_organise.log
output_kraken_stage_reaper.log
output_kraken_stage_tally.log
output_kraken_stage_tally.ops.log
krakened_reaper_log_C1JB3ACXX_1_5_1_reaper_record.txt
krakened_reaper_log_C1JB3ACXX_1_5_2_reaper_record.txt
krakened_tally_C1JB3ACXX_1_5_2_and_krakened_C1JB3ACXX_1_5_1_filter_record.txt

This should provide valuable info to track down the problem. The test dataset should work fine in Ubuntu 14.04, we already tested MetaTrans with that environment.

Greetings!

[metatr...@gmail.com]

Hi Chris,

After revising your issue more in detail we detected a bug in the code which made the pipeline to stop
in the "1-QC" step when using only 1 thread for the analysis. This was caused by the different behavior
in output files by Kraken tools when using 1 or more than 1 threads.

    1-PROCESSED_SAMPLES/${SAMPLE}/1-QC/m1-log/output_kraken_stage_reaper.log  # 1 thread

   -->
    ./m1-log/krakened_reaper_log_${SAMPLE}_1_reaper_record.txt  # >1 thread
    ./m1-log/krakened_reaper_log_${SAMPLE}_2_reaper_record.txt  # >1 thread

    1-PROCESSED_SAMPLES/${SAMPLE}/1-QC/m1-log/output_kraken_stage_tally.log  # 1 thread
    -->
    ./m1-log/krakened_${SAMPLE}_1_and_krakened_${SAMPLE}_2_filter_record.txt  # >1 thread

Since this is a major issue, the file "run_qc.sh" was updated with a protection for such case, which
is included in the updated "1-Scripts.tar.gz" file as well.

This change is already updated in the Download section of the website.

Thank your Chris for pointing this issue out and taking the time to report it!

Metatrans team

[monsie...@gmail.com]

Hi Metatrans team,

I am having a similar issue. I am testing my install by running the test samples/database.

Here is the terminal output:

---

[sgibbons@login-0 metatrans]$ ./metatrans

#Checking pipeline files' integrity (Date: '26-10-17' Time: '09h:48m:14s CST')...

#Checking sequence files' integrity (Date: '26-10-17' Time: '09h:48m:15s CST')...

#Checking if R libraries are installed and accessible (Date: '26-10-17' Time: '09h:48m:17s CST')...

#Running Metatrans Pipeline (by default will perform a funcional analysis with differential expression) (Date: '26-10-17' Time: '09h:48m:30s CST')...

#Using 20 threads as default (this value can be modified per module)
#and a maximum of 5 GB of memory for the Sortmerna task.


#Running sample 'C1JB3ACXX_1_5':
    #Running Module1 >>Quality Control<< ... (Date: '26-10-17' Time: '09h:48m:31s CST')

real    0m20.599s
user    0m12.165s
sys    0m0.431s

real    0m21.394s
user    0m12.925s
sys    0m0.436s

real    0m2.105s
user    0m0.049s
sys    0m0.022s

real    0m57.752s
user    1m36.275s
sys    0m1.032s

real    0m25.089s
user    0m19.876s
sys    0m0.449s

 #Error# Pipeline exited with error code 1.

 #HELP# An error may lead files in an intermediate state. To re-run the pipeline some folder might be deleted.
 #HELP# Please, read the troubleshooting section of the manual for help.

---

The </home/sgibbons/metatrans/1-PROCESSED_SAMPLES/C1JB3ACXX_1_5/1-QC/m1-log> folder has the following contents:

---

[sgibbons@login-0 m1-log]$ ls -l
total 32
-rw-r--r-- 1 sgibbons cluster 2675 Oct 26 09:49 krakened_reaper_log_C1JB3ACXX_1_5_1_reaper_record.txt
-rw-r--r-- 1 sgibbons cluster 2676 Oct 26 09:49 krakened_reaper_log_C1JB3ACXX_1_5_2_reaper_record.txt
-rw-r--r-- 1 sgibbons cluster   70 Oct 26 09:48 m1-time.tsv
-rw-r--r-- 1 sgibbons cluster 1099 Oct 26 09:48 output_fastqc_raw.log
-rw-r--r-- 1 sgibbons cluster 3726 Oct 26 09:48 output_kraken_stage_organise.log
-rw-r--r-- 1 sgibbons cluster 3987 Oct 26 09:48 output_kraken_stage_reaper.log
-rw-r--r-- 1 sgibbons cluster 3698 Oct 26 09:50 output_kraken_stage_tally.log
-rw-r--r-- 1 sgibbons cluster  214 Oct 26 09:50 output_kraken_stage_tally.ops.log

---

The output_kraken_stage_tally.ops.log file looks like this:

---

mv: cannot stat ‘/home/sgibbons/metatrans/1-PROCESSED_SAMPLES/C1JB3ACXX_1_5/1-QC/m1-temp/kraken_results/krakened_C1JB3ACXX_1_5_1/REAPER/krakened_reaper_C1JB3ACXX_1_5_1.lane.clean.gz’: No such file or directory

---

The contents of </home/sgibbons/metatrans/1-PROCESSED_SAMPLES/C1JB3ACXX_1_5/1-QC/m1-temp/kraken_results/krakened_C1JB3ACXX_1_5_1/REAPER/> are as follows:

---

[sgibbons@login-0 REAPER]$ ls -l
total 91320
-rw-r--r-- 1 sgibbons cluster 93176240 Oct 26 09:49 krakened_C1JB3ACXX_1_5_1.lane.clean.gz
-rw-r--r-- 1 sgibbons cluster      520 Oct 26 09:49 krakened_C1JB3ACXX_1_5_1.lane.report.clean.len
-rw-r--r-- 1 sgibbons cluster     2528 Oct 26 09:49 krakened_C1JB3ACXX_1_5_1.lane.report.clean.nt
-rw-r--r-- 1 sgibbons cluster     2222 Oct 26 09:49 krakened_C1JB3ACXX_1_5_1.lane.report.clean.trinucl
-rw-r--r-- 1 sgibbons cluster     2533 Oct 26 09:49 krakened_C1JB3ACXX_1_5_1.lane.report.input.nt
-rw-r--r-- 1 sgibbons cluster     1847 Oct 26 09:49 krakened_C1JB3ACXX_1_5_1.lane.report.input.q
-rw-r--r-- 1 sgibbons cluster   303756 Oct 26 09:49 krakened_C1JB3ACXX_1_5_1.lint.gz
-rw-r--r-- 1 sgibbons cluster      517 Oct 26 09:49 krakened_C1JB3ACXX_1_5_1.sumstat

---

So...it looks like the files are getting named improperly? i.e. 'krakened_C1JB3ACXX_1_5_1.lane.clean.gz' rather than 'krakened_reaper_C1JB3ACXX_1_5_1.lane.clean.gz'.

Any ideas on what might be going on and how to implement a quick fix?

thanks,
Sean Gibbons

[metatr...@gmail.com]

Hi Sean,

What platform are you using? is it RedHat?, the errors shown recall use those produced by running
the pipeline in CentOS/RedHat 's. If so please read the thread " Is the Metatrans pipeline works now in the RedHat/CentOS ? "

Greetings

[bob nichols]

Hey Guys, 

I hate to keep bringing this error up but I am having the same type error as Sean. I tried the troubleshooting steps in "Is the Metatrans pipeline works now in the RedHat/CentOS ?" but it did not work for me. During the build step it said my GCC version was either out of date or missing. I looked at the requirements and it said that it is recommended to have gcc version 4.1. Currently I have GCC version 5.2 so I was thinking that my version of GCC was too upgraded for that fix. I am just curious if an newer version of GLIBC would do the same fix as GLIBC 2.14. Or should I downgrade my GCC version. Let me know what would work.

Thanks,

Bob

[metatr...@gmail.com]

Hi Bob,

Unfortunately we don't have time/resources right now to test/fix your use case.

As an easy fix/solution we can only recommend you to use a virtual machine (via VirtualBox or Docker) with an Ubuntu 14.04 image
and install Metatrans on it.

Sorry for the inconvenience.

Cheers