Unable to build cleaned binary

[Lynsey Whitacre]

Hello,

I am using Cortex_var to do variant calling on 10 genomes without a reference. I wrote commands to do the initial data processing of the 10 samples in parallel, as below:

[lwwvd@MUgenomics06 madtom]$ perl CORTEX_release_v1.0.5.21/scripts/calling/run_calls.pl --first_kmer 31 --last_kmer 61 -kmer_step 30 --fastaq_index index_93675 --auto_cleaning yes --bc no --pd no --outdir ./variants --ploidy 2 --genome_size 1000000000 --qthresh 5 --mem_height 25 --mem_width 100 --vcftools_dir /usr/local/bin/vcftools_0.1.12b --do_union no --logfile crtx_93675.log --workflow joint --ref Absent

For 9 of the samples, this worked. However, for 1 sample, I get the error:  Unable to build /data/lwwvd/madtom/variants/binaries/cleaned/k31/93675.kmer31.q5cleaned_6.ctx

Then, in the cleaned binaries log file for sample 93675:

[lwwvd@MUgenomics06 k31]$ more 93675.kmer31.q5cleaned_6.ctx.log 

-----

Tue Jun 30 07:35:45 2015

Starting Cortex, version 1.0.5.21

Command: /data/lwwvd/madtom/CORTEX_release_v1.0.5.21//bin/cortex_var_31_c1 --kmer_size 31 --mem_height 25 --mem_width 100 --dump_binary /data/l

wwvd/madtom/variants/binaries/cleaned/k31/93675.kmer31.q5cleaned_6.ctx --remove_low_coverage_supernodes 6 --multicolour_bin /data/lwwvd/madtom/

variants/binaries/uncleaned/31/93675.unclean.kmer31.q5.ctx 

Maximum k-mer size (compile-time setting): 31

Actual K-mer size: 31

Hash table created, number of buckets: 33554432

-----

Tue Jun 30 07:35:45 2015

Error: Failed to read covg in db_node_read_multicolour_binary

A perfectly fine looking covg and covg.pdf file is located in the uncleaned/31/ directory so I'm unsure what covg it failed to read or why. Do you have any suggestions?

Thanks,

Lynsey

[Zamin Iqbal]

Hi Lynsey

Sorry to hear this. Let's do some basic checks first

1. Did you compile cortex on the same machine that you ran it on?

2. Could you download and compile this app for checking Cortex binaries, and see if this one is corrupted/malformed?

Do

git clone g...@github.com:noporpoise/cortex_binary_reader.git

then cd in and type 

make

and then run the following to see if it throws any warnings

cortex_bin_reader --print_info /data/lwwvd/madtom/variants/binaries/uncleaned/31/93675.unclean.kmer31.q5.ctx 

[Lynsey Whitacre]

Zam,

Thanks for the suggestions. I did compile on the same machine where the program was run, and when I run the cortex_binary_reader it doesn't appear that there are any big red flags.

[lwwvd@MUgenomics06 cortex_binary_reader]$ ./cortex_bin_reader --print_info /data/lwwvd/madtom/variants/binaries/uncleaned/31/93675.unclean.kmer31.q5.ctx

Loading file: /data/lwwvd/madtom/variants/binaries/uncleaned/31/93675.unclean.kmer31.q5.ctx

File size: 35GB

----

binary version: 6

kmer size: 31

bitfields: 1

colours: 1

-- Colour 0 --

  sample name: '93675'

  mean read length: 91

  total sequence loaded: 39,932,965,930

  sequence error rate: 0.010000

  tip clipping: no

  remove low coverage supernodes: no [threshold: -1]

  remove low coverage kmers: no [threshold: -1]

  cleaned against graph: no [against: 'undefined']

--

Expected number of kmers: 2,909,089,325

----

mem_width: 28; mem_height: 27;

Memory required: 43.3GB

Memory suggested: --mem_width 28 --mem_height 27

  [3,758,096,384 entries; 56.0GB memory]

[Zamin Iqbal]

Hi Lynsey

OK, good to remove that possibility. So, the binary of the sample seems fine. So if you now type 

 /data/lwwvd/madtom/CORTEX_release_v1.0.5.21//bin/cortex_var_31_c1 --kmer_size 31 --mem_height 25 --mem_width 100 --multicolour_bin /data/lwwvd/madtom/variants/binaries/uncleaned/31/93675.unclean.kmer31.q5.ctx 

do you still get the error?

regards

Zam

[Lynsey Whitacre]

No error with that command. Appears to be running thru now. 

Thanks!

Lynsey

[Zamin Iqbal]

OK, well that command was almost identical to yours. Your error message was about reading a binary file, so I removed all the options except those pertaining to reading the binary file.

I basically don't understand what has changed. If you re-run your original command does it work? I don't think I have fixed or changed anything!