HiC-Pro Output folder alreads exists

[Ray Oq]

Hi Group,

     I am trying to get HiC-Pro up and running for our Univ project.   I was able to run the complete Test Dataset and receive the 

correct output as per the HiC-Pro website.

I then proceed to run HiC-Pro with our data.   I received the following message after several days of processing:

Run HiC-Pro 2.10.0

--------------------------------------------

Fri May 18 08:29:10 EDT 2018

Bowtie2 alignment step1 ...

/isg/shared/apps/HiC-Pro/2.10.0/scripts/bowtie_wrap.sh -c /home/CAM/aromd/HiCPro_Config_Prod.txt -u >> hicpro.log

--------------------------------------------

Thu May 24 00:59:08 EDT 2018

Bowtie2 alignment step2 ...

/isg/shared/apps/HiC-Pro/2.10.0/scripts/bowtie_wrap.sh -c /home/CAM/aromd/HiCPro_Config_Prod.txt -l >> hicpro.log

/home/CAM/aromd/HiCPro_Prod_008_Output folder alreads exists. Do you want to delete it before running ?(y/n) [n] : 

real 0m0.418s

user 0m0.050s

sys 0m0.090s

My batch file has the following execution statement:

time HiC-Pro -c HiCPro_Config_Prod.txt -i rawdata -o HiCPro_Prod_008_Output

I did notice some additional post with this error message but they did not provide any clarification for my problem.

Any feedback would be appreciated.

Thanks in advance.

Ray

[Moshe Olshansky]

Hi Ray,

If you created HiCPro_Prod_008_Output folder so that HiCPro output goes there, you shouldn't have done this. HiCPro will create that folder. So if this is an empty folder just delete it (rmdir HiCPro_Prod_008_Output) and run HiCPro again.

Hope this helps,

Moshe.

[nservant]

Hi guys,
Could you send me you config file and input folder organization.
I would be interesting to try to reproduce this message ...
Thanks

[Ray Oq]

Hi Nicolas,

     I am away from the office and will post the files on Monday, 6/11.

Thanks for your response.

Ray

[nservant]

Hi Ray
Sure no worries. In practice, I don't think I need the 'real' input files. Their name and how they are organized should be enough.
Best

[Ray Oq]

Hi Nicolas,

    Here is my config file:

# Please change the variable settings below if necessary

#########################################################################

## Paths and Settings  - Do not edit !

#########################################################################

TMP_DIR = tmp

LOGS_DIR = logs

BOWTIE2_OUTPUT_DIR = bowtie_results

MAPC_OUTPUT = hic_results

RAW_DIR = rawdata

#######################################################################

## SYSTEM - PBS - Start Editing Here !!

#######################################################################

N_CPU = 64

LOGFILE = hicpro.log

JOB_NAME = IMR90_split

JOB_MEM = 100gb

JOB_WALLTIME = 25-00:00:00

JOB_QUEUE = general

JOB_MAIL = xxxxxxxxxxxxx...@uchc.edu

#########################################################################

## Data

#########################################################################

PAIR1_EXT = _R1

PAIR2_EXT = _R2

#######################################################################

## Alignment options

#######################################################################

FORMAT = phred33

MIN_MAPQ = 0

BOWTIE2_IDX_PATH = /isg/shared/databases/alignerIndex/animal/hg19_ucsc/hg19_bowtie2

BOWTIE2_GLOBAL_OPTIONS = --very-sensitive -L 30 --score-min L,-0.6,-0.2 --end-to-end --reorder

BOWTIE2_LOCAL_OPTIONS =  --very-sensitive -L 20 --score-min L,-0.6,-0.2 --end-to-end --reorder

#######################################################################

## Annotation files

#######################################################################

REFERENCE_GENOME = Human_genome

GENOME_SIZE = chrom_hg19.sizes

#######################################################################

## Allele specific

#######################################################################

ALLELE_SPECIFIC_SNP = 

#######################################################################

## Digestion Hi-C

#######################################################################

GENOME_FRAGMENT = HindIII_resfrag_hg19.bed

LIGATION_SITE = AAGCTAGCTT

MIN_FRAG_SIZE = 100

MAX_FRAG_SIZE = 100000

MIN_INSERT_SIZE = 100

MAX_INSERT_SIZE = 600

#######################################################################

## Hi-C processing

#######################################################################

MIN_CIS_DIST =

GET_ALL_INTERACTION_CLASSES = 1

GET_PROCESS_SAM = 1

RM_SINGLETON = 1

RM_MULTI = 1

RM_DUP = 1

#######################################################################

## Contact Maps

#######################################################################

BIN_SIZE = 500000 1000000

MATRIX_FORMAT = upper

#######################################################################

## ICE Normalization

#######################################################################

MAX_ITER = 100

FILTER_LOW_COUNT_PERC = 0.02

FILTER_HIGH_COUNT_PERC = 0

EPS = 0.1

Here is my batch file:

#!/bin/bash

#SBATCH -J AROMD06032018_008

#SBATCH -N 1

#SBATCH -n 1

#SBATCH -c 16

#SBATCH --mail-user=xxxxx...@uchc.edu

#SBATCH --mail-type=ALL

#SBATCH -o HiCPro-%j.output

##SBATCH -e jobname-%j.error

#SBATCH --partition=general

#SBATCH --mem=64G

# code starts here

module load HiC-Pro

time HiC-Pro -c HiCPro_Config_Prod_060318.txt -i rawdata -o HiCPro_Prod_008_Output_060318

This is the file directory structure:

/home/CAM/aromd/rawdata/S1_008

The S1_008 directory has our fastq files R1 and R2.

Thanks for your feedback in advance.

Ray