ERROR: Total amount of bytes read not equal to Starch v2 metadata length

[Margaret H]

Hi all,

I am using a program called HOTSPOT (http://www.uwencode.org/proj/hotspot-ptih/) that uses the starch function from BEDOPS and I get a buffer overflow wherein starch is terminated. In addition, I get the two following errors from BEDOPS:

ERROR: Total amount of bytes read not equal to Starch v2 metadata length!
ERROR: Could not read metadata

Has anyone else encountered this type of error (I'm not familiar with how to solve buffer overflows) or does anyone know have any ideas about how to solve this problem? Is it hardware (i.e. not enough memory causing the buffer overflow?) or is it just a problem with calling starch?

Forgive my lack on knowledge, I'm still relatively new to analyzing NGS and using these kinds of tools.


More detailed error output:

run_make_lib: creating bed file...
*** buffer overflow detected ***: starch terminated
======= Backtrace: =========
[0x80a8224]
[0x80a804d]
[0x80a7bb8]
[0x807d789]
[0x80b6f38]
[0x80a7c75]
[0x80a7b9f]
[0x804e1ea]
[0x804efdd]
[0x8048e33]
[0x806a794]
[0x80491a9]
======= Memory map: ========
08048000-08123000 r-xp 00000000 00:19 1573103
  /woldlab/castor/data00/home/

margaret.ho/bin/bedops_linux_i386-v2.1.1/bin/starch
08123000-08126000 rw-p 000da000 00:19 1573103
  /woldlab/castor/data00/home/margaret.ho/bin/bedops_linux_i386-v2.1.1/bin/starch
08126000-08128000 rw-p 00000000 00:00 0
08c4e000-08cbf000 rw-p 00000000 00:00 0                                  [heap]
f704c000-f704d000 rw-p 00000000 00:00 0
f776b000-f7774000 rw-p 00000000 00:00 0
f7774000-f7775000 r-xp 00000000 00:00 0                                  [vdso]
ff9a3000-ffea6000 rw-p 00000000 00:00 0                                  [stack]
./run_make_lib.tok: line 44: 20964 Done                    samtools view $tags
     20965                       | awk 'BEGIN{FS="\t";
OFS="\t"}{len=length($10); s=$4-1; if(and(16,$2) == 16) s+=len-1;
chr=$3; print chr,s,s+1}'
     20966                       | sort-bed -
     20967 Aborted                 | starch - > $tagsb
run_make_lib: creating lib file...
ERROR: Total amount of bytes read not equal to Starch v2 metadata length!
ERROR: Could not read metadata

Thanks,
Margaret

[Alex Reynolds]

Hi Margaret,

If you are using a 32-bit system, it is possible to encounter problems with compressing datasets larger than 2 GB in size, due to limitations in how 32-bit systems handle so-called "large files" (those greater than 2 GB in size).

If possible, try to perform your analysis on a 64-bit system, using the 64-bit version of BEDOPS. This is the "x86_64" installer download available from our site. You will need a 64-bit-capable (Intel) system to use this version of starch.

If your analysis would not produce large files, one option is to provide an archive with your data and a description of your analysis. This would be a reproducible setup that we could use to try to debug on this end. Dropbox is a useful (and free) service to store and share this test case. Feel free to contact off-list if you want to make this available.

Alternatively, you may be able to edit the hotspot pipeline to disable starch creation, thus getting around this error entirely. I'm cc-ing Bob in case he has advice or insight on this.

Regards,
Alex

[Margaret Ho]

Hi Alex and Bob,

Thank you both for your responses. Switching to the 64-bit version of
Bedops solved the problem for me!

Margie