Re: HiC-Pro command not found

[nservant]

Hi,
Could you please give me the content of the config-install.txt file.
I think that the PREFIX was not set.
Thanks

On Wednesday, 8 March 2017 13:30:50 UTC+1, babita singh wrote:

hi,

I have performed following command for installing the HiC-Pro:

tar -zxvf HiC-Pro-2.8.0.tar.gz
cd HiC-Pro-master
## Edit config-install.txt file if necessary
make configure
make install

After install:

(g++ -Wall -O2 -std=c++0x -o build_matrix /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0/scripts/src/build_matrix.cpp; mv build_matrix /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0/scripts)
(g++ -Wall -O2 -std=c++0x -o cutsite_trimming /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0/scripts/src/cutsite_trimming.cpp; mv cutsite_trimming /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0/scripts)
(cp /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0/scripts/src/ice_mod/iced/scripts/ice /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0/scripts; cd /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0/scripts/src/ice_mod/; /usr/bin/python setup.py install --user;)
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'include_package_data'
  warnings.warn(msg)
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'zip_safe'
  warnings.warn(msg)
Warning: Assuming default configuration (iced/utils/{setup_utils,setup}.py was not found)Appending iced.utils configuration to iced
Ignoring attempt to set 'name' (from 'iced' to 'iced.utils')
Appending iced.datasets configuration to iced
Ignoring attempt to set 'name' (from 'iced' to 'iced.datasets')
non-existing path in 'iced/io': '../src/cblas'
non-existing path in 'iced/io': '../src/cblas'
Appending iced.io configuration to iced
Ignoring attempt to set 'name' (from 'iced' to 'iced.io')
non-existing path in 'iced': '../src/cblas'
non-existing path in 'iced': '../src/cblas'
Appending iced configuration to 
Ignoring attempt to set 'name' (from '' to 'iced')
running install
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "iced.io.fastio_" sources
building extension "iced.io.read" sources
building extension "iced._normalization_" sources
building extension "iced._filter_" sources
building data_files sources
build_src: building npy-pkg config files
running build_py
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
running build_scripts
running install_lib
running install_scripts
changing mode of /home/samir/.local/bin/ice to 775
running install_data
running install_egg_info
Removing /home/samir/.local/lib/python2.7/site-packages/iced-0.4.0_git.egg-info
Writing /home/samir/.local/lib/python2.7/site-packages/iced-0.4.0_git.egg-info
running install_clib
customize UnixCCompiler
cp -Ri /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0 /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0
HiC-Pro installed in  !

Then run command and shown following error:

HiC-Pro -i HiC_data/Input_raw/ -o HiC_data/Output_result/ -c config -s -h 

HiC-Pro: command not found

Please anyone help me to solve this problem.

[nservant]

Did you have any error during the 'make configure' ?
Can you also give me the content of the 'config-system.txt' file ?


On Wednesday, 8 March 2017 13:43:57 UTC+1, babita singh wrote:

#########################################################################

## Paths and Settings  - Start editing here !

#########################################################################

PREFIX = /home/samir/Desktop/archana/B/HiC_analysis/ 

BOWTIE2_PATH = /home/samir/bowtie2

SAMTOOLS_PATH = /home/samir/samtools 0.1.19-96b5f2294a

R_PATH = /home/samir/R 

PYTHON_PATH = /home/samir/Python 2.7.10

CLUSTER_SYS = SLURM

[nservant]

So HiC-Pro should be installed in

INSTALL_PATH = /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0

Can you please check by doing

/home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/bin/HiC-Pro -h

However, there is something strange in the installation message ;
It says ;

cp -Ri /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0 /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0
HiC-Pro installed in  !

Meaning that the INSTALL_PATH is known (as HiC-Pro is copied in the right place)
But in theory, you should have the message :

HiC-Pro installed in /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0 !

Could you check if the command 'realpath' is install on your system ?
Thanks

On Wednesday, 8 March 2017 13:57:27 UTC+1, babita singh wrote:

After make configure:

make -f ./scripts/install/Makefile CONFIG_SYS=./config-install.txt

make[1]: Entering directory '/home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0'

./scripts/install/install_dependencies.sh -c ./config-install.txt -o  /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0

Make sure internet connection works for your shell prompt under current user's privilege ...

Starting HiC-Pro installation ...

export /home/samir/bowtie2 in PATH

export /home/samir/samtools0.1.19-96b5f2294a in PATH

export /home/samir/R in PATH

export /home/samir/Python2.7.10 in PATH

Where should missing software prerequisites be installed ? [/home/samir/bin] 

Checking dependencies ... 

Checking Python libraries ...

The required Python libraries appear to be already installed. 

Checking R installation ...

The required R packages appear to be already installed. 

Bowtie2 Aligner appears to be already installed. 

./scripts/install/install_dependencies.sh: line 43: 10#19-96b5f2294a: value too great for base (error token is "96b5f2294a")

./scripts/install/install_dependencies.sh: line 47: 10#19-96b5f2294a: value too great for base (error token is "96b5f2294a")

Samtools appears to be already installed. 

Dependencies checked !

Check HiC-Pro configuration ... 

Configuration for SLURM system.

make[1]: Leaving directory '/home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro-2.8.0'

Following are the content of the 'config-system.txt' file:

#######################################################################
## HiC-Pro - System settings
#######################################################################
#######################################################################
## Required Software - Specified the DIRECTORY name of the executables
## If not specified, the program will try to locate the executable
## using the 'which' command
#######################################################################
R_PATH = /usr/bin
BOWTIE2_PATH = /usr/bin
SAMTOOLS_PATH = /usr/bin
PYTHON_PATH = /usr/bin
INSTALL_PATH = /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0
SCRIPTS = /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/scripts
SOURCES = /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/scripts/src
ANNOT_DIR = /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/annotation
CLUSTER_SCRIPT = /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/scripts/make_slurm_script.sh

[nservant]

HiCPro is installed in the PREFIX path from the config-install.txt file.
So in your case in /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/bin/HiC-Pro

Then, regarding your error, how did you run HiC-Pro ?
It seems that you specified the "-h" option for help, and the "-s" option for stepwise analysis.

Please, have a look at the manual to see how to run HiC-Pro.
Data must be organized in a folder, with one sub-folder per sample.

* mydata
** sample1
*** f_R1.fastq
*** f_R2.fastq

Then run
 /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/bin/HiC-Pro -i mydata -o myoutput -c myconfig

Hope it helps
Nicolas
On Wednesday, 8 March 2017 14:36:51 UTC+1, babita singh wrote:

Thank you so much for solve my problem.

I have install HiC-Pro-2.8.0 software. I don't understand, how  HiC-Pro_2.8.0 folder generated after installing the HiC-Pro.

After using  this path  /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/bin/HiC-Pro -h, HiC-Pro do working but showing following error:

Unknown step option ("-s -h"). Use bin/HiC-Pro --help for usage information.

Directory Hierarchy of rawdata '/home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/HiC_data/Input_raw/' is not correct. No '.fastq(.gz)' files detected.

What I can do for same.

 

[babita singh]

yes I have organized data in a folder like:

* HiC_data
** Input_raw
*** f_R1.fastq

*** f_R2.fastq 

what is something wrong here

[nservant]

ok, then, how do you run HiC-Pro ?

[nservant]

Which command line did you used ??? how do you set the -i option ??
It seems that you have an extra level in your input folder organization.
HiC_data/Input_raw/Sample_1/


On Wednesday, 8 March 2017 15:04:31 UTC+1, babita singh wrote:

I don't know. Please give me any idea, where are do something wrong.

Same error shown  like:

Error: Directory Hierarchy of rawdata '/home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/HiC_data/Input_raw/Sample_1/' is not correct. No '.fastq(.gz)' files detected

 

[nservant]

As your input is

* HiC_data
** Input_raw
*** f_R1.fastq

*** f_R2.fastq

You should run

 bin/HiC-Pro -i HiC_data -o output_result -c config

 
Is it better ?

On Wednesday, 8 March 2017 15:33:17 UTC+1, babita singh wrote:

Hi,

                  Thank you very much for solving my problem.

                   Please give me any suggestion for input file format error.

[nservant]

Can you have a look in the log folder. You should find the bowtie log with a clear error message ...

On Wednesday, 8 March 2017 15:46:01 UTC+1, babita singh wrote:

Hi

Thank you so much. Its working but other error shown like :

Exit: Error in Bowtie alignment

/home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/bin/../scripts//Makefile:85: recipe for target 'bowtie_global' failed

make: *** [bowtie_global] Error 1

[nservant]

Hi
Could you tell me which bowtie2 version your are using please ?
In addition, you should have a bowtie2 log file in the logs folder. Could you look at it pleae ?
Thanks

On Thursday, 9 March 2017 07:18:12 UTC+1, babita singh wrote:

Hi

After run the HiC-Pro, shown some following error:

  Warning: Output file 'thaliana.bwt2glob.unmap.fastq' was specified without -S.  This will not work in future Bowtie 2 versions.  Please use -S instead.

Extra parameter(s) specified: "thaliana", "thaliana.log"

Error: Encountered internal Bowtie 2 exception (#1)

Command: /usr/bin/bowtie2-align-s --wrapper basic-0 --very-sensitive -L 20 --score-min L,-0.6,-0.2 --end-to-end --reorder --rg-id BMG --rg SM:BHC-1_R1 --phred33-quals -p 2 -x /Home/samir/Desktop/archana/B/HiC_analysis/TAIR/Arabidopsis --passthrough -U rawdata/Sample_1/BHC-1_R1.fastq thaliana.bwt2glob.unmap.fastq thaliana thaliana.log 

(ERR): bowtie2-align exited with value 1

Anyone please help me.

Thank you in advance

[nservant]

what did you put in your config_hicpro.txt file ?
The BOWTIE2_IDX_PATH is not correct. This should be the path to your bowtie2 indexes.
Not that the indexes are found using ${BOWTIE2_IDX_PATH}/${REFERENCE_GENOME}.*.bt2

Then me you configuration file if you are not sure ...

On Thursday, 9 March 2017 10:57:43 UTC+1, babita singh wrote:

Hi,

Yes, Bowtie2 version : 2.2.5. 

Error in log folder:

Could not locate a Bowtie index corresponding to basename "/Home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/TAIR/TAIR"

Error: Encountered internal Bowtie 2 exception (#1)

Command: /usr/bin/bowtie2-align-s --wrapper basic-0 --very-sensitive -L 30 --score-min L,-0.6,-0.2 --end-to-end --rg-id BMG --rg SM:BHC-1_R1 --phred33-quals -p 2 -x /Home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/TAIR/TAIR --passthrough -U rawdata/Sample_1/BHC-1_R1.fastq 

(ERR): bowtie2-align exited with value 1

Thanks

[babita singh]

I am sending our config file. Please correct the file.

Thanks 

[nservant]

As I told you before, your BOWTIE2_IDX_PATH is not correct.
You need to specifiy where you bowtie indexes are.
Then, be sure that the file chrom_Ara.sizes and DpnII_resfrag_Ara.bed are on the annotation folder of HiC-Pro.
The rest should be fine

On Thursday, 9 March 2017 12:18:59 UTC+1, babita singh wrote:

Hi,

I am waiting for your suggestions. can you please correct congi file.

Thank you

[babita singh]

Hi, 

 HiC-Pro is running. Thank you so much for helping me..

[babita singh]

Hi,

Please give me any suggestion for solve the problem.

Thanks

[nservant]

On which system are you ?
Your sort version does not look like a standard unix sort. Option -T is not recognized.
I think that you will have to update or install a new sort version

sort: invalid option -- 'T'

sort: invalid option -- 'T'

[babita singh]

Hi,

My sort version is 8.23 and ubuntu version 15.10.

Please tell, should have update or install a new sort version.

Thanks 

[nservant]

Hi
Could check if your sort is compatible with ;
-T option
- with -V option to sort by "version"

For instance, take a BED file, and run
sort -k1,1V -k2,2n MYBED
sort -k1,1V -k2,2n -T /tmp MYBED

And let me know if it works.
Then to solve your issue, you can either install a sort version which is compatible with these two options.
Or directly modify the scripts merge_valid_interactions.sh and mapped_2hic_fragments.sh to remove these options.
Several users have the same issues with the sort command. I will try to fix it in the current development version very soon.
best
N

[babita singh]

Hi

For your suggestion, I have run following command

sort -k1,1V -k2,2n MYBED
sort -k1,1V -k2,2n -T /tmp MYBED

Both command run properly. Now what I can do.

Thanks

[nservant]

This is not possible, as you previously get the error

sort: invalid option -- 'T'

Are you sure that you are really checking the good sort version ?? on the good node ?

[babita singh]

I Don't Known my sort version is compatible with ;

-T option.

But both sort command (With -T and without -T) running in bed file.

Could modify the scripts merge_valid_interactions.sh and mapped_2hic_fragments.sh.

Thanks

[nservant]

ah sorry, i read again your messages. The error comes from samtools ! not the sort command.
It says that samtools does not have a -T option.
What is your samtools version ?
Here is mine ;

>>samtools sort
Usage: samtools sort [options...] [in.bam]
Options:
  -l INT     Set compression level, from 0 (uncompressed) to 9 (best)
  -m INT     Set maximum memory per thread; suffix K/M/G recognized [768M]
  -n         Sort by read name
  -o FILE    Write final output to FILE rather than standard output
  -O FORMAT  Write output as FORMAT ('sam'/'bam'/'cram')   (either -O or
  -T PREFIX  Write temporary files to PREFIX.nnnn.bam       -T is required)
  -@ INT     Set number of sorting and compression threads [1]

[nservant]

yes. I told you that the error does not come from the unix sort
The error comes from your samtools version which is too old.
Please update it.
N

On Friday, 10 March 2017 12:25:22 UTC+1, babita singh wrote:

My samtools version is 0.1.19. I have modify the scripts merge_valid_interactions.sh and mapped_2hic_fragments.sh to remove -T ${TMP_DIR} options. But unfortunately same error come again.

Combine both alignment ...

/home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/scripts/bowtie_combine.sh -c /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/config >> hicpro.log

sort: invalid option -- 'T'

sort: invalid option -- 'T'

[bam_sort_core] merging from 10 files...

[bam_sort_core] merging from 10 files...

mv: cannot stat ‘bowtie_results/bwt2/Sample_1/BHC-1_R1_TAIR.bwt2merged.sorted.bam’mv: : No such file or directory

cannot stat ‘bowtie_results/bwt2/Sample_1/BHC-1_R2_TAIR.bwt2merged.sorted.bam’: No such file or directory

--------------------------------------------

Fri Mar 10 15:04:02 IST 2017

Bowtie2 mapping statistics for R1 and R2 tags ...

/home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/scripts/mapping_stat.sh -c /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/config >> hicpro.log

--------------------------------------------

Fri Mar 10 15:06:03 IST 2017

Pairing of R1 and R2 tags ...

/home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/scripts/bowtie_pairing.sh -c /home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/config >> hicpro.log

/home/samir/Desktop/archana/B/HiC_analysis/HiC-Pro_2.8.0/bin/../scripts//Makefile:136: recipe for target 'bowtie_pairing' failed

make: *** [bowtie_pairing] Error 1

I don't understand what doing wrong. Can you please solve the problem. I am waiting for your suggestions.

Thanks

[nservant]

http://www.htslib.org/

On Friday, 10 March 2017 12:52:29 UTC+1, babita singh wrote:

Sorry for silly question. Can you tell which command used for update the samtools.

Thank you for your response in advance. 

[nservant]

Sorry I did not get your point ?
Is HiC-Pro running now ?

On Friday, 10 March 2017 14:47:06 UTC+1, babita singh wrote:

I have install new samtools version 1.2 in different directory and give updated samtools version path in config-install.txt file. Then run again make configure amd make install.

After that again run HiC-Pro. Can you please tell, hic-pro is working or give the paired alignment.

Please help me again.