storm-yarn master fail due to enormous usage of virtual memory.

292 views
Skip to first unread message

Sean Zhong

unread,
Jan 27, 2014, 8:21:36 PM1/27/14
to storm...@googlegroups.com
storm-yarn master fail to start due to enormous usage of virtual memory.

ERROR:
================

Application application_1390831241102_0001 failed 2 times due to AM Container for appattempt_1390831241102_0001_000002 exited with exitCode: 143 due to: Container [pid=100407,containerID=container_1390831241102_0001_02_000001] is running beyond virtual memory limits. Current usage: 524.0 MB of 5 GB physical memory used; 21.7 GB of 10.5 GB virtual memory used. Killing container. 

Here is the dump of the container proc tree:
================

|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE 
|- 100412 100407 100407 100407 (java) 321 11 17834373120 47011 /usr/java/latest/bin/java -Dstorm.home=./storm/storm/ -Dlogfile.name=/var/log/hadoop-yarn/containers/application_1390831241102_0001/container_1390831241102_0001_02_000001/master.log com.yahoo.storm.yarn.MasterServer 
|- 100456 100412 100407 100407 (java) 202 7 1680891904 28301 /usr/java/jdk1.6.0_31/jre/bin/java -server -Dstorm.home=./storm/storm/ -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64 -Dstorm.conf.file=storm.yaml -cp conf:./storm/storm/:./storm/storm/storm-starter.jar:...:./storm/storm/storm-core.jar -Xmx768m -Dlogfile.name=drpc.log backtype.storm.daemon.drpc 
|- 100453 100412 100407 100407 (java) 213 8 1950203904 29480 /usr/java/jdk1.6.0_31/jre/bin/java -server -Dstorm.home=./storm/storm/ -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64 -Dstorm.conf.file=storm.yaml -cp conf:./storm/storm/:./storm/storm/storm-starter.jar:...:./storm/storm/storm-core.jar -Xmx1024m -Dlogfile.name=/var/log/hadoop-yarn/containers/application_1390831241102_0001/container_1390831241102_0001_02_000001/nimbus.log backtype.storm.daemon.nimbus 
|- 100407 99919 100407 100407 (bash) 0 0 108703744 310 /bin/bash -c /usr/java/latest/bin/java -Dstorm.home=./storm/storm/ -Dlogfile.name=/var/log/hadoop-yarn/containers/application_1390831241102_0001/container_1390831241102_0001_02_000001/master.log com.yahoo.storm.yarn.MasterServer 1>/var/log/hadoop-yarn/containers/application_1390831241102_0001/container_1390831241102_0001_02_000001/stdout 2>/var/log/hadoop-yarn/containers/application_1390831241102_0001/container_1390831241102_0001_02_000001/stderr 
|- 100454 100412 100407 100407 (java) 209 10 1679749120 29051 /usr/java/jdk1.6.0_31/jre/bin/java -server -Dstorm.home=./storm/storm/ -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64 -Dstorm.conf.file=storm.yaml -cp conf:./storm/storm/:./storm/storm/storm-starter.jar:...:./storm/storm/storm-core.jar -Xmx768m -Dstorm.options=nimbus.host=localhost -Dlogfile.name=/var/log/hadoop-yarn/containers/application_1390831241102_0001/container_1390831241102_0001_02_000001/ui.log backtype.storm.ui.core 


From the dump:
================================
Master consumes 17GB+ virtual memory 


pmap of MasterServer
================================
00001/master.log com.yahoo.storm.yarn.MasterServer
0000000040000000     36K r-x--  /usr/java/jdk1.6.0_31/bin/java
0000000040108000      8K rwx--  /usr/java/jdk1.6.0_31/bin/java
0000000041e5b000    288K rwx--    [ anon ]
000000040d000000  28544K rwx--    [ anon ]
000000040ebe0000  55424K rwx--    [ anon ]
0000000412200000 686720K rwx--    [ anon ]
000000043c0a0000 10300160K rwx--    [ anon ]
00000006b0b60000 343296K rwx--    [ anon ]
00000006c5aa0000 5150080K rwx--    [ anon ]
0000003adb800000    128K r-x--  /lib64/ld-2.12.so
0000003adba1f000      4K r-x--  /lib64/ld-2.12.so
0000003adba20000      4K rwx--  /lib64/ld-2.12.so
0000003adba21000      4K rwx--    [ anon ]
0000003adbc00000      8K r-x--  /lib64/libdl-2.12.so
0000003adbc02000   2048K -----  /lib64/libdl-2.12.so
0000003adbe02000      4K r-x--  /lib64/libdl-2.12.so
0000003adbe03000      4K rwx--  /lib64/libdl-2.12.so
0000003adc000000   1628K r-x--  /lib64/libc-2.12.so
0000003adc197000   2048K -----  /lib64/libc-2.12.so
0000003adc397000     16K r-x--  /lib64/libc-2.12.so
0000003adc39b000      4K rwx--  /lib64/libc-2.12.so
0000003adc39c000     20K rwx--    [ anon ]
0000003adc400000     92K r-x--  /lib64/libpthread-2.12.so
0000003adc417000   2044K -----  /lib64/libpthread-2.12.so
0000003adc616000      4K r-x--  /lib64/libpthread-2.12.so
0000003adc617000      4K rwx--  /lib64/libpthread-2.12.so
0000003adc618000     16K rwx--    [ anon ]
0000003adc800000    524K r-x--  /lib64/libm-2.12.so
0000003adc883000   2044K -----  /lib64/libm-2.12.so
0000003adca82000      4K r-x--  /lib64/libm-2.12.so
0000003adca83000      4K rwx--  /lib64/libm-2.12.so
0000003add000000     28K r-x--  /lib64/librt-2.12.so
0000003add007000   2044K -----  /lib64/librt-2.12.so
0000003add206000      4K r-x--  /lib64/librt-2.12.so
0000003add207000      4K rwx--  /lib64/librt-2.12.so
0000003aec800000     88K r-x--  /lib64/libnsl-2.12.so
0000003aec816000   2044K -----  /lib64/libnsl-2.12.so
0000003aeca15000      4K r-x--  /lib64/libnsl-2.12.so
0000003aeca16000      4K rwx--  /lib64/libnsl-2.12.so
0000003aeca17000      8K rwx--    [ anon ]
00007f9cfdcc7000     32K r-xs-  /usr/java/jdk1.6.0_31/jre/lib/resources.jar
00007f9cfdccf000     12K -----    [ anon ]
00007f9cfdcd2000   1016K rwx--    [ anon ]
00007f9cfddd0000     12K -----    [ anon ]
00007f9cfddd3000   1016K rwx--    [ anon ]
00007f9cfded1000     12K -----    [ anon ]
00007f9cfded4000   1016K rwx--    [ anon ]
00007f9cfdfd2000     12K -----    [ anon ]
00007f9cfdfd5000   1016K rwx--    [ anon ]
00007f9cfe0d3000     12K -----    [ anon ]
00007f9cfe0d6000   1016K rwx--    [ anon ]
00007f9cfe1d4000     12K -----    [ anon ]
00007f9cfe1d7000   1016K rwx--    [ anon ]
00007f9cfe2d5000     12K -----    [ anon ]
00007f9cfe2d8000   1016K rwx--    [ anon ]
00007f9cfe3d6000     12K -----    [ anon ]
00007f9cfe3d9000   1016K rwx--    [ anon ]
00007f9cfe4d7000     12K -----    [ anon ]
00007f9cfe4da000   1016K rwx--    [ anon ]
00007f9cfe5d8000     12K r-xs-  /usr/java/jdk1.6.0_31/jre/lib/ext/sunjce_provider.jar
00007f9cfe5db000     28K r-x--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libnio.so
00007f9cfe5e2000   1020K -----  /usr/java/jdk1.6.0_31/jre/lib/amd64/libnio.so
00007f9cfe6e1000      8K rwx--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libnio.so
00007f9cfe6e3000     12K -----    [ anon ]
00007f9cfe6e6000   1016K rwx--    [ anon ]
00007f9cfe7e4000     12K -----    [ anon ]
00007f9cfe7e7000   1016K rwx--    [ anon ]
00007f9cfe8e5000     12K -----    [ anon ]
00007f9cfe8e8000   1016K rwx--    [ anon ]
00007f9cfe9e6000     12K -----    [ anon ]
00007f9cfe9e9000   1016K rwx--    [ anon ]
00007f9cfeae7000     12K -----    [ anon ]
00007f9cfeaea000   1016K rwx--    [ anon ]
00007f9cfebe8000     12K -----    [ anon ]
00007f9cfebeb000   1016K rwx--    [ anon ]
00007f9cfece9000     12K -----    [ anon ]
00007f9cfecec000   1016K rwx--    [ anon ]
00007f9cfedea000     12K -----    [ anon ]
00007f9cfeded000   1016K rwx--    [ anon ]
00007f9cfeeeb000     12K -----    [ anon ]
00007f9cfeeee000   1016K rwx--    [ anon ]
00007f9cfefec000     12K -----    [ anon ]
00007f9cfefef000   1016K rwx--    [ anon ]
00007f9cff0ed000     24K r-xs-  /usr/java/jdk1.6.0_31/jre/lib/jsse.jar
00007f9cff0f3000     16K r-xs-  /usr/java/jdk1.6.0_31/jre/lib/ext/sunpkcs11.jar
00007f9cff0f7000     12K r-xs-  /usr/java/jdk1.6.0_31/jre/lib/jce.jar
00007f9cff0fa000      4K r-x--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libjaas_unix.so
00007f9cff0fb000   1020K -----  /usr/java/jdk1.6.0_31/jre/lib/amd64/libjaas_unix.so
00007f9cff1fa000      8K rwx--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libjaas_unix.so
00007f9cff1fc000     88K r-x--  /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0
00007f9cff212000   2044K -----  /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0
00007f9cff411000      4K rwx--  /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0
00007f9cff412000     24K r-x--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libmanagement.so
00007f9cff418000   1020K -----  /usr/java/jdk1.6.0_31/jre/lib/amd64/libmanagement.so
00007f9cff517000      8K rwx--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libmanagement.so
00007f9cff519000     12K -----    [ anon ]
00007f9cff51c000   1016K rwx--    [ anon ]
00007f9cff61a000     12K -----    [ anon ]
00007f9cff61d000   2556K rwx--    [ anon ]
00007f9cff89c000     76K r-x--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libnet.so
00007f9cff8af000   1028K -----  /usr/java/jdk1.6.0_31/jre/lib/amd64/libnet.so
00007f9cff9b0000     12K rwx--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libnet.so
00007f9cff9b3000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/ring-devel-0.3.11.jar
00007f9cff9b4000     24K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/libthrift7-0.7.0-2.jar
00007f9cff9ba000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/jzmq-2.1.0.jar
00007f9cff9bb000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/carbonite-1.5.0.jar
00007f9cff9bd000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/ring-jetty-adapter-0.3.11.jar
00007f9cff9be000     28K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/snakeyaml-1.11.jar
00007f9cff9c5000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/core.incubator-0.1.0.jar
00007f9cff9c6000     16K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/jetty-util-6.1.26.jar
00007f9cff9ca000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/compojure-1.1.3.jar
00007f9cff9cb000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/slf4j-api-1.6.5.jar
00007f9cff9cd000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/json-simple-1.1.jar
00007f9cff9ce000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/objenesis-1.2.jar
00007f9cff9d0000     96K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/netty-3.6.3.Final.jar
00007f9cff9e8000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/hiccup-0.3.6.jar
00007f9cff9e9000     60K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/joda-time-2.0.jar
00007f9cff9f8000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/clojure-complete-0.2.3.jar
00007f9cff9f9000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/swank-clojure-1.4.3.jar
00007f9cff9fb000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/jline-0.9.94.jar
00007f9cff9fd000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/disruptor-2.10.1.jar
00007f9cff9ff000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/tools.macro-0.1.0.jar
00007f9cffa00000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/log4j-over-slf4j-1.6.6.jar
00007f9cffa02000     20K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/curator-framework-1.3.2.jar
00007f9cffa07000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/cdt-1.2.6.2.jar
00007f9cffa08000     36K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/logback-core-1.0.6.jar
00007f9cffa11000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/clout-1.0.1.jar
00007f9cffa12000     20K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/commons-lang-2.5.jar
00007f9cffa17000    132K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/mockito-all-1.9.5.jar
00007f9cffa38000    240K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/clojure-1.4.0.jar
00007f9cffa74000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/servlet-api-2.5.jar
00007f9cffa76000     20K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/httpcore-4.1.jar
00007f9cffa7b000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/reflectasm-1.07-shaded.jar
00007f9cffa7d000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/tools.logging-0.2.3.jar
00007f9cffa7e000     12K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/commons-io-2.1.jar
00007f9cffa81000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/commons-codec-1.4.jar
00007f9cffa83000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/tools.cli-0.2.2.jar
00007f9cffa84000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/ring-servlet-0.3.11.jar
00007f9cffa85000     16K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/kryo-2.17.jar
00007f9cffa89000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/debug-repl-0.3.1.jar
00007f9cffa8b000     32K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/httpclient-4.1.1.jar
00007f9cffa93000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/curator-client-1.3.2.jar
00007f9cffa95000     28K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/jetty-6.1.26.jar
00007f9cffa9c000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/clj-stacktrace-0.2.4.jar
00007f9cffa9d000     28K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/jgrapht-0.8.3.jar
00007f9cffaa4000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/clj-time-0.4.1.jar
00007f9cffaa5000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/minlog-1.2.jar
00007f9cffaa7000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/tools.nrepl-0.2.3.jar
00007f9cffaa9000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/ring-core-1.1.5.jar
00007f9cffaab000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/asm-4.0.jar
00007f9cffaad000     12K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/commons-fileupload-1.2.1.jar
00007f9cffab0000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/servlet-api-2.5-20081211.jar
00007f9cffab2000     60K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/netty-3.2.2.Final.jar
00007f9cffac1000    144K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/guava-13.0.1.jar
00007f9cffae5000     24K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/logback-classic-1.0.6.jar
00007f9cffaeb000     48K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/zookeeper-3.4.5.jar
00007f9cffaf7000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/math.numeric-tower-0.0.1.jar
00007f9cffaf9000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/commons-logging-1.1.1.jar
00007f9cffafb000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/lib/commons-exec-1.1.jar
00007f9cffafd000     12K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/storm-yarn.jar
00007f9cffb00000    236K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/storm-kafka-spout-example.jar
00007f9cffb3b000    388K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/storm-starter.jar
00007f9cffb9c000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/storm-netty.jar
00007f9cffb9e000      8K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/storm-kafka-spout.jar
00007f9cffba0000    364K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/storm-core.jar
00007f9cffbfb000      4K r-xs-  /mnt/disk8/hadoop/yarn/filecache/10/storm.zip/storm/storm-console-logging.jar
00007f9cffbfc000     28K r-xs-  /usr/lib/hbase/lib/protobuf-java-2.5.0.jar
00007f9cffc03000     16K r-xs-  /usr/lib/tez/lib/xz-1.0.jar
00007f9cffc07000      4K r-xs-  /usr/lib/tez/lib/tez-common-0.2.0.jar
00007f9cffc08000     12K r-xs-  /usr/lib/tez/lib/tez-mapreduce-examples-0.2.0.jar
00007f9cffc0b000     16K r-xs-  /usr/lib/tez/lib/jackson-core-asl-1.8.8.jar
00007f9cffc0f000      8K r-xs-  /usr/lib/tez/lib/aopalliance-1.0.jar
00007f9cffc11000     20K r-xs-  /usr/lib/tez/lib/commons-compress-1.4.1.jar
00007f9cffc16000     60K r-xs-  /usr/lib/tez/lib/guice-3.0.jar
00007f9cffc25000     52K r-xs-  /usr/lib/tez/lib/tez-dag-0.2.0.jar
00007f9cffc32000      8K r-xs-  /usr/lib/tez/lib/commons-cli-1.2.jar
00007f9cffc34000     48K r-xs-  /usr/lib/tez/lib/hadoop-mapreduce-client-common-2.2.0.jar
00007f9cffc40000      8K r-xs-  /usr/lib/tez/lib/snappy-java-1.0.4.1.jar
00007f9cffc42000     96K r-xs-  /usr/lib/tez/lib/hadoop-mapreduce-client-core-2.2.0.jar
00007f9cffc5a000      8K r-xs-  /usr/lib/tez/lib/hadoop-mapreduce-client-shuffle-2.2.0.jar
00007f9cffc5c000     16K r-xs-  /usr/lib/tez/lib/tez-mapreduce-0.2.0.jar
00007f9cffc60000     12K r-xs-  /usr/lib/tez/lib/tez-runtime-internals-0.2.0.jar
00007f9cffc63000      8K r-xs-  /usr/lib/tez/lib/paranamer-2.3.jar
00007f9cffc65000     28K r-xs-  /usr/lib/tez/lib/tez-runtime-library-0.2.0.jar
00007f9cffc6c000     28K r-xs-  /usr/lib/tez/lib/avro-1.7.4.jar
00007f9cffc73000     52K r-xs-  /usr/lib/tez/lib/jackson-mapper-asl-1.8.8.jar
00007f9cffc80000      4K r-xs-  /usr/lib/tez/lib/javax.inject-1.jar
00007f9cffc81000     36K r-xs-  /usr/lib/tez/tez-api-0.2.0.jar
00007f9cffc8a000    312K r-xs-  /usr/lib/hbase/hbase-0.94.7-Intel.jar
00007f9cffcd8000      4K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-datajoin-2.2.0-Intel.jar
00007f9cffcd9000      4K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-Intel.jar
00007f9cffcda000     48K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.2.0-Intel.jar
00007f9cffce6000     16K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-nativetask-2.2.0-Intel.jar
00007f9cffcea000     16K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-gridmix-2.2.0-Intel.jar
00007f9cffcee000     40K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.2.0-Intel.jar
00007f9cffcf8000     12K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-streaming-2.2.0-Intel.jar
00007f9cffcfb000      8K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-distcp-2.2.0-Intel.jar
00007f9cffcfd000     92K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.2.0-Intel.jar
00007f9cffd14000      4K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.2.0-Intel.jar
00007f9cffd15000     24K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-rumen-2.2.0-Intel.jar
00007f9cffd1b000      8K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-extras-2.2.0-Intel.jar
00007f9cffd1d000      8K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-archives-2.2.0-Intel.jar
00007f9cffd1f000     12K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.2.0-Intel.jar
00007f9cffd22000      8K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.2.0-Intel.jar
00007f9cffd24000     88K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-Intel-tests.jar
00007f9cffd3a000     20K r-xs-  /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.2.0-Intel.jar
00007f9cffd3f000     16K r-xs-  /usr/lib/hadoop-mapreduce/lib/xz-1.0.jar
00007f9cffd43000      8K r-xs-  /usr/lib/hadoop-mapreduce/lib/hadoop-annotations-2.2.0-Intel.jar
00007f9cffd45000     32K r-xs-  /usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar
00007f9cffd4d000     36K r-xs-  /usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar
00007f9cffd56000     16K r-xs-  /usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.8.8.jar
00007f9cffd5a000     28K r-xs-  /usr/lib/hadoop-mapreduce/lib/junit-4.10.jar
00007f9cffd61000      8K r-xs-  /usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar
00007f9cffd63000     92K r-xs-  /usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar
00007f9cffd7a000     20K r-xs-  /usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar
00007f9cffd7f000     60K r-xs-  /usr/lib/hadoop-mapreduce/lib/guice-3.0.jar
00007f9cffd8e000      4K r-xs-  /usr/lib/hadoop-mapreduce/lib/hadoop-crypto-1.0.0-Intel.jar
00007f9cffd8f000     24K r-xs-  /usr/lib/hadoop-mapreduce/lib/avro-1.7.4-Intel.jar
00007f9cffd95000     72K r-xs-  /usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar
00007f9cffda7000      8K r-xs-  /usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar
00007f9cffda9000      8K r-xs-  /usr/lib/hadoop-mapreduce/lib/imkey-2.2.0-Intel.jar
00007f9cffdab000     12K r-xs-  /usr/lib/hadoop-mapreduce/lib/commons-io-2.1.jar
00007f9cffdae000      8K r-xs-  /usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.1.jar
00007f9cffdb0000      8K r-xs-  /usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar
00007f9cffdb2000      4K r-xs-  /usr/lib/hadoop-mapreduce/lib/commons-logging-api-1.1.jar
00007f9cffdb3000      8K r-xs-  /usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar
00007f9cffdb5000      4K r-xs-  /usr/lib/hadoop-mapreduce/lib/asm-3.2.jar
00007f9cffdb6000      4K r-xs-  /usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar
00007f9cffdb7000     52K r-xs-  /usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.8.8.jar
00007f9cffdc4000     40K r-xs-  /usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar
00007f9cffdce000      4K r-xs-  /usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar
00007f9cffdcf000      4K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.2.0-Intel.jar
00007f9cffdd0000     52K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.2.0-Intel.jar
00007f9cffddd000     48K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.2.0-Intel.jar
00007f9cffde9000      4K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.2.0-Intel.jar
00007f9cffdea000     12K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-client-2.2.0-Intel.jar
00007f9cffded000     16K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.2.0-Intel.jar
00007f9cffdf1000      8K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.2.0-Intel.jar
00007f9cffdf3000     68K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-api-2.2.0-Intel.jar
00007f9cffe04000      8K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.2.0-Intel.jar
00007f9cffe06000      4K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-site-2.2.0-Intel.jar
00007f9cffe07000     72K r-xs-  /usr/lib/hadoop-yarn/hadoop-yarn-common-2.2.0-Intel.jar
00007f9cffe19000     16K r-xs-  /usr/lib/hadoop-yarn/lib/xz-1.0.jar
00007f9cffe1d000      8K r-xs-  /usr/lib/hadoop-yarn/lib/hadoop-annotations-2.2.0-Intel.jar
00007f9cffe1f000     32K r-xs-  /usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar
00007f9cffe27000     36K r-xs-  /usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar
00007f9cffe30000     16K r-xs-  /usr/lib/hadoop-yarn/lib/jackson-core-asl-1.8.8.jar
00007f9cffe34000     28K r-xs-  /usr/lib/hadoop-yarn/lib/junit-4.10.jar
00007f9cffe3b000      8K r-xs-  /usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar
00007f9cffe3d000     92K r-xs-  /usr/lib/hadoop-yarn/lib/netty-3.6.2.Final.jar
00007f9cffe54000     20K r-xs-  /usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar
00007f9cffe59000     60K r-xs-  /usr/lib/hadoop-yarn/lib/guice-3.0.jar
00007f9cffe68000      4K r-xs-  /usr/lib/hadoop-yarn/lib/hadoop-crypto-1.0.0-Intel.jar
00007f9cffe69000     24K r-xs-  /usr/lib/hadoop-yarn/lib/avro-1.7.4-Intel.jar
00007f9cffe6f000     72K r-xs-  /usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar
00007f9cffe81000      8K r-xs-  /usr/lib/hadoop-yarn/lib/snappy-java-1.0.4.1.jar
00007f9cffe83000     12K r-xs-  /usr/lib/hadoop-yarn/lib/commons-io-2.1.jar
00007f9cffe86000      8K r-xs-  /usr/lib/hadoop-yarn/lib/hamcrest-core-1.1.jar
00007f9cffe88000      8K r-xs-  /usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar
00007f9cffe8a000      4K r-xs-  /usr/lib/hadoop-yarn/lib/commons-logging-api-1.1.jar
00007f9cffe8b000      8K r-xs-  /usr/lib/hadoop-yarn/lib/paranamer-2.3.jar
00007f9cffe8d000      4K r-xs-  /usr/lib/hadoop-yarn/lib/asm-3.2.jar
00007f9cffe8e000      4K r-xs-  /usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar
00007f9cffe8f000     52K r-xs-  /usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar
00007f9cffe9c000     40K r-xs-  /usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar
00007f9cffea6000      4K r-xs-  /usr/lib/hadoop-yarn/lib/javax.inject-1.jar
00007f9cffea7000     88K r-xs-  /usr/lib/hadoop-hdfs/hadoop-hdfs-2.2.0-Intel-tests.jar
00007f9cffebd000      8K r-xs-  /usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.2.0-Intel.jar
00007f9cffebf000    276K r-xs-  /usr/lib/hadoop-hdfs/hadoop-hdfs-2.2.0-Intel.jar
00007f9cfff04000    176K r-xs-  /usr/lib/hadoop-hdfs/hadoop-hdfs-bkjournal-2.2.0-Intel.jar
00007f9cfff30000      8K r-xs-  /usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar
00007f9cfff32000     28K r-xs-  /usr/lib/hadoop-hdfs/lib/jetty-6.1.26.jar
00007f9cfff39000     32K r-xs-  /usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar
00007f9cfff41000      4K r-xs-  /usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar
00007f9cfff42000     36K r-xs-  /usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar
00007f9cfff4b000      8K r-xs-  /usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar
00007f9cfff4d000     16K r-xs-  /usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar
00007f9cfff51000     92K r-xs-  /usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar
00007f9cfff68000     12K r-xs-  /usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar
00007f9cfff6b000     20K r-xs-  /usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar
00007f9cfff70000     12K r-xs-  /usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar
00007f9cfff73000      8K r-xs-  /usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar
00007f9cfff75000    132K r-xs-  /usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar
00007f9cfff96000     72K r-xs-  /usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar
00007f9cfffa8000     16K r-xs-  /usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.jar
00007f9cfffac000     12K r-xs-  /usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar
00007f9cfffaf000     12K r-xs-  /usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar
00007f9cfffb2000      8K r-xs-  /usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar
00007f9cfffb4000     52K r-xs-  /usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar
00007f9cfffc1000     20K r-xs-  /usr/lib/hadoop/lib/jets3t-0.6.1.jar
00007f9cfffc7000     12K r-xs-  /usr/lib/hadoop/lib/jsch-0.1.42.jar
00007f9cfffca000     16K r-xs-  /usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar
00007f9cfffce000     20K r-xs-  /usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar
00007f9cfffd3000     92K r-xs-  /usr/lib/hadoop/lib/netty-3.6.2.Final.jar
00007f9cfffea000     16K r-xs-  /usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar
00007f9cfffee000      8K r-xs-  /usr/lib/hadoop/lib/jettison-1.1.jar
00007f9cffff0000     20K r-xs-  /usr/lib/hadoop/lib/commons-compress-1.4.1.jar
00007f9cffff5000     12K r-xs-  /usr/lib/hadoop/lib/commons-el-1.0.jar
00007f9cffff8000      4K r-xs-  /usr/lib/hadoop/lib/jackson-xc-1.8.8.jar
00007f9cffff9000     20K r-xs-  /usr/lib/hadoop/lib/commons-lang-2.5.jar
00007f9cffffe000      4K r-xs-  /usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar
00007f9cfffff000     12K r-xs-  /usr/lib/hadoop/lib/jsp-api-2.1.jar
00007f9d00002000      8K r-xs-  /usr/lib/hadoop/lib/commons-cli-1.2.jar
00007f9d00004000      8K r-xs-  /usr/lib/hadoop/lib/slf4j-api-1.7.5.jar
00007f9d00006000      8K r-xs-  /usr/lib/hadoop/lib/hadoop-auth-2.2.0-Intel.jar
00007f9d00008000     72K r-xs-  /usr/lib/zookeeper/zookeeper-3.4.5-Intel.jar
00007f9d0001a000     24K r-xs-  /usr/lib/hadoop/lib/commons-httpclient-3.1.jar
00007f9d00020000      4K r-xs-  /usr/lib/hadoop/lib/hadoop-crypto-1.0.0-Intel.jar
00007f9d00021000      8K r-xs-  /usr/lib/hadoop/lib/activation-1.1.jar
00007f9d00023000    132K r-xs-  /usr/lib/hadoop/lib/guava-11.0.2.jar
00007f9d00044000     24K r-xs-  /usr/lib/hadoop/lib/avro-1.7.4-Intel.jar
00007f9d0004a000     72K r-xs-  /usr/lib/hadoop/lib/jersey-server-1.9.jar
00007f9d0005c000     16K r-xs-  /usr/lib/hadoop/lib/jetty-util-6.1.26.jar
00007f9d00060000      8K r-xs-  /usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar
00007f9d00062000      8K r-xs-  /usr/lib/hadoop/lib/imkey-2.2.0-Intel.jar
00007f9d00064000     20K r-xs-  /usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar
00007f9d00069000     12K r-xs-  /usr/lib/hadoop/lib/commons-io-2.1.jar
00007f9d0006c000     12K r-xs-  /usr/lib/hadoop/lib/jsr305-1.3.9.jar
00007f9d0006f000      4K r-xs-  /usr/lib/hadoop/lib/commons-logging-api-1.1.jar
00007f9d00070000     20K r-xs-  /usr/lib/hadoop/lib/commons-net-3.1.jar
00007f9d00075000      8K r-xs-  /usr/lib/hadoop/lib/paranamer-2.3.jar
00007f9d00077000      8K r-xs-  /usr/lib/hadoop/lib/stax-api-1.0.1.jar
00007f9d00079000      8K r-xs-  /usr/lib/hadoop/lib/servlet-api-2.5.jar
00007f9d0007b000     56K r-xs-  /usr/lib/hadoop/lib/commons-collections-3.2.1.jar
00007f9d00089000     12K r-xs-  /usr/lib/hadoop/lib/jaxb-api-2.2.2.jar
00007f9d0008c000      4K r-xs-  /usr/lib/hadoop/lib/asm-3.2.jar
00007f9d0008d000     16K r-xs-  /usr/lib/hadoop/lib/jersey-json-1.9.jar
00007f9d00091000     52K r-xs-  /usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar
00007f9d0009e000     24K r-xs-  /usr/lib/hadoop/lib/junit-4.8.2.jar
00007f9d000a4000     16K r-xs-  /usr/lib/hadoop/lib/commons-digester-1.8.jar
00007f9d000a8000     40K r-xs-  /usr/lib/hadoop/lib/jersey-core-1.9.jar
00007f9d000b2000      8K r-xs-  /usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar
00007f9d000b4000      4K -----    [ anon ]
00007f9d000b5000   1024K rwx--    [ anon ]
00007f9d001b5000     12K -----    [ anon ]
00007f9d001b8000   1016K rwx--    [ anon ]
00007f9d002b6000     12K -----    [ anon ]
00007f9d002b9000   1016K rwx--    [ anon ]
00007f9d003b7000     12K -----    [ anon ]
00007f9d003ba000   1016K rwx--    [ anon ]
00007f9d004b8000     12K -----    [ anon ]
00007f9d004bb000   1016K rwx--    [ anon ]
00007f9d005b9000  96836K r-x--  /usr/lib/locale/locale-archive
00007f9d0644a000     12K -----    [ anon ]
00007f9d0644d000   1016K rwx--    [ anon ]
00007f9d0654b000     12K -----    [ anon ]
00007f9d0654e000   1016K rwx--    [ anon ]
00007f9d0664c000      4K -----    [ anon ]
00007f9d0664d000  24668K rwx--    [ anon ]
00007f9d07e64000   1648K r-xs-  /usr/java/jdk1.6.0_31/jre/lib/rt.jar
00007f9d08000000  29620K rwx--    [ anon ]
00007f9d09ced000  35916K -----    [ anon ]
00007f9d0c000000    132K rwx--    [ anon ]
00007f9d0c021000  65404K -----    [ anon ]
00007f9d10000000    132K rwx--    [ anon ]
00007f9d10021000  65404K -----    [ anon ]
00007f9d14000000  22592K rwx--    [ anon ]
00007f9d15610000  42944K -----    [ anon ]
00007f9d18000000    676K rwx--    [ anon ]
00007f9d180a9000  64860K -----    [ anon ]
00007f9d1c000000    132K rwx--    [ anon ]
00007f9d1c021000  65404K -----    [ anon ]
00007f9d20000000    132K rwx--    [ anon ]
00007f9d20021000  65404K -----    [ anon ]
00007f9d24001000     40K r-xs-  /usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar
00007f9d2400b000      8K r-xs-  /usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar
00007f9d2400d000     20K r-xs-  /usr/lib/hadoop/hadoop-nfs-2.2.0-Intel.jar
00007f9d24012000      8K r-xs-  /usr/lib/hadoop/hadoop-auth-2.2.0-Intel.jar
00007f9d24014000    168K r-xs-  /usr/lib/hadoop/hadoop-common-2.2.0-Intel.jar
00007f9d2403e000     88K r-xs-  /usr/lib/hadoop/hadoop-common-2.2.0-Intel-tests.jar
00007f9d24054000      8K r-xs-  /usr/lib/hadoop/hadoop-annotations-2.2.0-Intel.jar
00007f9d24056000      8K r-xs-  /usr/lib/hadoop/lib/commons-codec-1.4.jar
00007f9d24058000     76K r-xs-  /usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar
00007f9d2406b000     16K r-xs-  /usr/lib/hadoop/lib/xz-1.0.jar
00007f9d2406f000     28K r-xs-  /usr/lib/hadoop/lib/jetty-6.1.26.jar
00007f9d24076000     32K r-xs-  /usr/lib/hadoop/lib/protobuf-java-2.5.0.jar
00007f9d2407e000     20K r-xs-  /usr/lib/hadoop/lib/aspectjrt-1.7.4.jar
00007f9d24083000     36K r-xs-  /usr/lib/hadoop/lib/log4j-1.2.17.jar
00007f9d2408c000   1028K rwx--    [ anon ]
00007f9d2418d000      4K -----    [ anon ]
00007f9d2418e000   1024K rwx--    [ anon ]
00007f9d2428e000      4K -----    [ anon ]
00007f9d2428f000   1024K rwx--    [ anon ]
00007f9d2438f000      4K -----    [ anon ]
00007f9d24390000   1024K rwx--    [ anon ]
00007f9d24490000      4K -----    [ anon ]
00007f9d24491000   1024K rwx--    [ anon ]
00007f9d24591000      4K -----    [ anon ]
00007f9d24592000   1024K rwx--    [ anon ]
00007f9d24692000      4K -----    [ anon ]
00007f9d24693000   1024K rwx--    [ anon ]
00007f9d24793000      4K -----    [ anon ]
00007f9d24794000   1024K rwx--    [ anon ]
00007f9d24894000      4K -----    [ anon ]
00007f9d24895000   1024K rwx--    [ anon ]
00007f9d24995000      4K -----    [ anon ]
00007f9d24996000   1024K rwx--    [ anon ]
00007f9d24a96000      4K -----    [ anon ]
00007f9d24a97000   1024K rwx--    [ anon ]
00007f9d24b97000      4K -----    [ anon ]
00007f9d24b98000   1024K rwx--    [ anon ]
00007f9d24c98000      4K -----    [ anon ]
00007f9d24c99000   1024K rwx--    [ anon ]
00007f9d24d99000      4K -----    [ anon ]
00007f9d24d9a000   1024K rwx--    [ anon ]
00007f9d24e9a000      4K -----    [ anon ]
00007f9d24e9b000   1024K rwx--    [ anon ]
00007f9d24f9b000      4K -----    [ anon ]
00007f9d24f9c000   1024K rwx--    [ anon ]
00007f9d2509c000      4K -----    [ anon ]
00007f9d2509d000   1024K rwx--    [ anon ]
00007f9d2519d000      4K -----    [ anon ]
00007f9d2519e000   1024K rwx--    [ anon ]
00007f9d2529e000      4K -----    [ anon ]
00007f9d2529f000   1024K rwx--    [ anon ]
00007f9d2539f000      4K -----    [ anon ]
00007f9d253a0000   1024K rwx--    [ anon ]
00007f9d254a0000      4K -----    [ anon ]
00007f9d254a1000   1024K rwx--    [ anon ]
00007f9d255a1000      4K -----    [ anon ]
00007f9d255a2000   1024K rwx--    [ anon ]
00007f9d256a2000      4K -----    [ anon ]
00007f9d256a3000   1024K rwx--    [ anon ]
00007f9d257a3000      4K -----    [ anon ]
00007f9d257a4000   2368K rwx--    [ anon ]
00007f9d259f4000  20116K rwx--    [ anon ]
00007f9d26d99000     56K rwx--    [ anon ]
00007f9d26da7000    108K rwx--    [ anon ]
00007f9d26dc2000   1344K rwx--    [ anon ]
00007f9d26f12000  20112K rwx--    [ anon ]
00007f9d282b6000    676K rwx--    [ anon ]
00007f9d2835f000  10056K rwx--    [ anon ]
00007f9d28d31000     44K rwx--    [ anon ]
00007f9d28d3c000    728K rwx--    [ anon ]
00007f9d28df2000   2496K rwx--    [ anon ]
00007f9d29062000  46656K rwx--    [ anon ]
00007f9d2bdf2000     48K r-x--  /lib64/libnss_files-2.12.so
00007f9d2bdfe000   2048K -----  /lib64/libnss_files-2.12.so
00007f9d2bffe000      4K r-x--  /lib64/libnss_files-2.12.so
00007f9d2bfff000      4K rwx--  /lib64/libnss_files-2.12.so
00007f9d2c000000  16492K rwx--    [ anon ]
00007f9d2d01b000  49044K -----    [ anon ]
00007f9d30000000      4K r-xs-  /usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar
00007f9d30001000      4K r-xs-  /usr/lib/hadoop-hdfs/lib/asm-3.2.jar
00007f9d30002000    116K r-xs-  /usr/lib/hadoop/lib/mockito-all-1.8.5.jar
00007f9d3001f000     68K r-xs-  /usr/lib/hadoop/lib/commons-math-2.1.jar
00007f9d30030000    424K rwx--    [ anon ]
00007f9d3009a000    108K rwx--    [ anon ]
00007f9d300b5000     56K r-x--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libzip.so
00007f9d300c3000   1032K -----  /usr/java/jdk1.6.0_31/jre/lib/amd64/libzip.so
00007f9d301c5000     12K rwx--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libzip.so
00007f9d301c8000      4K rwx--    [ anon ]
00007f9d301c9000    164K r-x--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libjava.so
00007f9d301f2000   1020K -----  /usr/java/jdk1.6.0_31/jre/lib/amd64/libjava.so
00007f9d302f1000     28K rwx--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libjava.so
00007f9d302f8000     52K r-x--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libverify.so
00007f9d30305000   1020K -----  /usr/java/jdk1.6.0_31/jre/lib/amd64/libverify.so
00007f9d30404000     12K rwx--  /usr/java/jdk1.6.0_31/jre/lib/amd64/libverify.so
00007f9d30407000     12K -----    [ anon ]
00007f9d3040a000   1016K rwx--    [ anon ]
00007f9d30508000   9320K r-x--  /usr/java/jdk1.6.0_31/jre/lib/amd64/server/libjvm.so
00007f9d30e22000   1032K -----  /usr/java/jdk1.6.0_31/jre/lib/amd64/server/libjvm.so
00007f9d30f24000   1748K rwx--  /usr/java/jdk1.6.0_31/jre/lib/amd64/server/libjvm.so
00007f9d310d9000    244K rwx--    [ anon ]
00007f9d31116000     28K r-x--  /usr/java/jdk1.6.0_31/jre/lib/amd64/jli/libjli.so
00007f9d3111d000   1028K -----  /usr/java/jdk1.6.0_31/jre/lib/amd64/jli/libjli.so
00007f9d3121e000      8K rwx--  /usr/java/jdk1.6.0_31/jre/lib/amd64/jli/libjli.so
00007f9d31220000      4K rwx--    [ anon ]
00007f9d31221000      8K r-xs-  /usr/lib/hadoop/lib/hadoop-annotations-2.2.0-Intel.jar
00007f9d31223000      4K r-xs-  /usr/lib/hadoop/lib/xmlenc-0.52.jar
00007f9d31224000      8K r-xs-  /usr/lib/hadoop/lib/commons-logging-1.1.1.jar
00007f9d31226000     24K r-xs-  /usr/lib/hadoop/lib/commons-configuration-1.6.jar
00007f9d3122c000     32K rwxs-  /tmp/hsperfdata_yarn/24041
00007f9d31234000      4K rwx--    [ anon ]
00007f9d31235000      4K r-x--    [ anon ]
00007f9d31236000      4K rwx--    [ anon ]
00007fff1f5fe000     88K rwx--    [ stack ]
00007fff1f73b000      4K r-x--    [ anon ]
ffffffffff600000      4K r-x--    [ anon ]
 total         17420680K


Analysis of pmap:
=======================
There are two anno block which consumes over 15GB Virtual memory:
000000043c0a0000 10300160K rwx--    [ anon ]
00000006c5aa0000 5150080K rwx--    [ anon ]


Anno can represents heap space, do you have ideas why it take that much?


Sean Zhong

unread,
Jan 27, 2014, 9:02:17 PM1/27/14
to storm...@googlegroups.com
I didn't find the exact reason, but this issue get fixed after I specified a max heap limit.

"-server -Xmx1024m"

Should I make a PR for this?

Sean

Bobby Evans

unread,
Jan 28, 2014, 11:23:56 AM1/28/14
to Sean Zhong, storm...@googlegroups.com
Basically the issue is with YARN.  The NodeManager enfaces two different memory limits.  The amount of physical memory being used and the amount of virtual memory being used, which by default is set to be 2.5x the physical memory requested.  Another user complained about this same thing, and your log below shows the same thing.  For some reason the AM has 17GB of virtual memory. DRPC is using 1.6, nimbus is using 1.9, and the ui is using 1.6.  If you want to look into this that would be great, but I would prefer to understand a bit more what is causing the massive amount of virtual memory to be used in the AM especially when there is only 183MB resident.

—Bobby

--
You received this message because you are subscribed to the Google Groups "storm-yarn" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-yarn+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply all
Reply to author
Forward
0 new messages