not sure if this is a bug per se, but...

[Nathan Schneider]

(on workhorse)

$ ducttape nli.tape
#
# There is insufficient memory for the Java Runtime Environment to continue.
# pthread_getattr_np
# An error report file with more information is saved as:
# /home/nschneid/l1id/dtsystem/hs_err_pid22630.log

Editing the ducttape executable to reduce the -Xmx argument from 1g to 256m did the trick. :P

Nathan

[Jonathan Clark]

That's a very severe memory limitation on workhorse. Most supercomputers and grid systems I've worked on allow 1-2 GB on head nodes. For example, many of my dissertation workflows might not run with that little memory.

That said, it would be nice if we could detect both how much memory is available (e.g. using ulimit -v) and then make a guess from there. Balancing what's available with what a workflow might need gets a bit tricky though, so it's not just a quick fix. If anyone has suggestions or wants to hack something into the ./ducttape shell script, I'd be happy to see improvements here. :)

--
You received this message because you are subscribed to the Google Groups "ducttape-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ducttape-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Nathan Schneider]

I was able to get the static analysis working with a lower memory threshold, but it would hang before listing realizations to be run. Now if I request more memory (-Xmx1g -Xss2M) with qsub -I, it is still failing:

$ qsub -I -l"vmem=2g,mem=2g"

$ ducttape nli.tape                 
#
# There is insufficient memory for the Java Runtime Environment to continue.
# pthread_getattr_np
# An error report file with more information is saved as:
# /home/nschneid/l1id/dtsystem/hs_err_pid30724.log

The error log is attached. Any ideas?

Nathan

[Jonathan Clark]

How much memory did you request with qsub -I?

[Nathan Schneider]

2g—see the shell command.

[Jonathan Clark]

Ah, I see it now.

Try this to get more info:

$ qsub -I -l"vmem=2g,mem=2g"

$ ulimit -a # Show current memory limits

$ bash -x $(which ducttape) nli.tape # Show JVM invocation of ducttape with all memory args

[Nathan Schneider]

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) 2097152
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 257789
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) 2097152
open files                      (-n) 4096
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 257789
virtual memory          (kbytes, -v) 2097152
file locks                      (-x) unlimited

$ bash -x $(which ducttape) nli.tape
+ set -ueo pipefail
+++ dirname /home/nschneid/ducttape-0.3/ducttape
++ cd /home/nschneid/ducttape-0.3
++ pwd
+ scriptDir=/home/nschneid/ducttape-0.3
+ JAVA=java
+ java -Xmx1g -Xss2M -cp /home/nschneid/ducttape-0.3/ducttape.jar -Dfile.encoding=UTF-8 Ducttape nli.tape
#
# There is insufficient memory for the Java Runtime Environment to continue.
# pthread_getattr_np
# An error report file with more information is saved as:
# /home/nschneid/l1id/dtsystem/hs_err_pid30856.log

[Nathan Schneider]

OK, it looks like part of the problem was specifying both vmem and mem in qsub. At least now ducttape starts....

[Nathan Schneider]

$ qsub -I -l"mem=2g"

$ bash -x $(which ducttape) nli.tape 
+ set -ueo pipefail
+++ dirname /home/nschneid/ducttape-0.3/ducttape
++ cd /home/nschneid/ducttape-0.3
++ pwd
+ scriptDir=/home/nschneid/ducttape-0.3
+ JAVA=java
+ java -Xmx2g -Xss2M -cp /home/nschneid/ducttape-0.3/ducttape.jar -Dfile.encoding=UTF-8 Ducttape nli.tape
ducttape 0.3
by Jonathan Clark
Loading workflow version history...
Have 0 previous workflow versions
No plans specified in workflow -- Using default one-off realization plan: Each realization will have no more than 1 non-baseline branch

At this point it hangs.

[Nathan Schneider]

I thought I'd try some other ducttape modes:

[nschneid@compute-0-23 dtsystem]$ ducttape nli.tape versions
ducttape 0.3
by Jonathan Clark
Checking for package versions in workflow output directory...
Loading workflow version history...
Have 0 previous workflow versions
Already built package versions:

Previously executed workflow versions:

[nschneid@compute-0-23 dtsystem]$ ducttape nli.tape list    
ducttape 0.3
by Jonathan Clark
No plans specified in workflow -- Using default one-off realization plan: Each realization will have no more than 1 non-baseline branch

The second one hangs. So I guess the problem is where it is computing the realizations?

Incidentally, this is my first time attempting to use packages. If I am doing something wonky with a package could it screw things up at this stage?