PEAR timeout issue through HPC

26 views

Skip to first unread message

Daniel Revillini

unread,

Mar 26, 2022, 3:34:05 PM3/26/22

to SAMSA bioinformatics group

HI Sam,

First, thanks for SAMSA2, this is super helpful to someone who has almost entirely done targeted amplicon sequencing with little experience in the meta- world.

Ok, I am getting a timeout issue on my first sample for the PEAR step. I have tried messing with my SBATCH commands to match the threads (-c 28), but maybe I just don't know what I'm doing with the SBATCH matching the 'Threads.' ???

Here is the timeout error:

Step 0.1: Checking for the presence of the checkpoint file.

The file 'checkpoints' already exists in the input directory.

The variable TRIMMO is in the checkpoint file. STEP 1 will be skipped.

____ _____ _ ____

| _ \| ____| / \ | _ \

| |_) | _| / _ \ | |_) |

| __/| |___ / ___ \| _ <

|_| |_____/_/ \_\_| \_\

PEAR v0.9.10 [May 30, 2016]

Citation - PEAR: a fast and accurate Illumina Paired-End reAd mergeR

Zhang et al (2014) Bioinformatics 30(5): 614-620 | doi:10.1093/bioinformatics/btt593

Forward reads file.................: /projects/soil_ecology/dr359/mimench/samsa2/mime/step_1_output/ambient_fertilized_m307.cleaned.forward

Reverse reads file.................: /projects/soil_ecology/dr359/mimench/samsa2/mime/step_1_output/ambient_fertilized_m307.cleaned.reverse

PHRED..............................: 33

Using empirical frequencies........: YES

Statistical method.................: OES

Maximum assembly length............: 999999

Minimum assembly length............: 50

p-value............................: 0.010000

Quality score threshold (trimming).: 0

Minimum read size after trimming...: 1

Maximal ratio of uncalled bases....: 1.000000

Minimum overlap....................: 10

Scoring method.....................: Scaled score

Threads............................: 28

Allocating memory..................: 200,000,000 bytes

Computing empirical frequencies....: DONE

A: 0.198124

C: 0.301876

G: 0.303239

T: 0.196761

14475 uncalled bases

Assemblying reads:srun: Force Terminated job 50656638

srun: Job step aborted: Waiting up to 32 seconds for job step to finish.

slurmstepd: error: *** STEP 50656638.0 ON cn68 CANCELLED AT 2022-03-26T11:56:02 DUE TO TIME LIMIT ***

Assemblying reads: 48%srun: error: cn68: task 0: Terminated

----

Here is the top of my modified master_script.sh:

#!/bin/bash

#SBATCH --job-name=samsa2_mime

#SBATCH ––mem=150000

#SBATCH ––cpus-per-task=28

#SBATCH --time=700

----

So, it times out in 2 hours at about 50% assembly and not sure why? Also, it doesn't actually look like it is using 28 CPUS (htop shows 7), but also says 196 threads are started so that checks out with (28 x 7). I guess this is likely a problem with my parallelism SBATCH calls, or perhaps not enough memory requested?

Please help when you get the chance, thanks very much!