[Hadoop-studio-users] java.lang.OutOfMemoryError: Java heap space while running simple job in netbeans(mc is high end having plenty of memory)
papu bhattacharya
Hello.. I’m testing simple WordCount code with Netbeans plugin.
I’m facing some memory error everytime.
I’m running it on RHEL5
I’m using hadoop 0.20.2
My m/c has 4 GB RAM. I have allocated netbeans almost 3 GB.
No other process is taking any memory.
Thanks for your help in advance first.Could you please help.
This is shown in hadoop deployment window:
10/08/24 17:33:12 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
10/08/24 17:33:13 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
10/08/24 17:33:13 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
10/08/24 17:33:13 INFO mapred.FileInputFormat: Total input paths to process : 2
10/08/24 17:33:13 INFO mapred.JobClient: Running job: job_local_0001
10/08/24 17:33:13 INFO mapred.FileInputFormat: Total input paths to process : 2
10/08/24 17:33:14 INFO mapred.MapTask: numReduceTasks: 1
10/08/24 17:33:14 INFO mapred.MapTask: io.sort.mb = 100
10/08/24 17:33:14 WARN mapred.LocalJobRunner: job_local_0001
java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:781)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
10/08/24 17:33:14 INFO mapred.JobClient: map 0% reduce 0%
10/08/24 17:33:14 INFO mapred.JobClient: Job complete: job_local_0001
10/08/24 17:33:14 INFO mapred.JobClient: Counters: 0
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
at WordCount.main(WordCount.java:52)
Java Result: 1
Tools debug window output:
Welcome to Karmasphere Studio.
A message (even a warning or stack trace) in this window does not necessarily mean that anything broke; but if anything DID break, this window will help Karmasphere to work out what broke, and why.
FINE com.karmasphere.studio.common.persist.PersistentObjectRegistry <init>: Loaded PersistenceManager implementation com.karmasphere.studio.common.netbeans.persist.NetbeansPersistenceManager
FINE com.karmasphere.studio.common.decorator.DecoratorMap <init>: Loading decorators for HadoopLocalCluster
FINE com.karmasphere.studio.common.decorator.DecoratorMap <init>: Loaded 0 decorators for HadoopLocalCluster
FINE com.karmasphere.studio.common.decorator.DecoratorMap <init>: Loading decorators for HadoopLocalFilesystem
FINE com.karmasphere.studio.common.decorator.DecoratorMap <init>: New decorator BrowseDecorator(HadoopLocalFilesystem) using com.karmasphere.studio.hadoop.netbeans.decorator.NetbeansFilesystemBrowseDecorator@1ce1791(com.karmasphere.studio.hadoop.filesystem.HadoopFilesystem)
FINE com.karmasphere.studio.common.decorator.DecoratorMap <init>: Loaded 1 decorators for HadoopLocalFilesystem
FINE com.karmasphere.studio.common.persist.PersistentObjectReference setId: PersistentObjectReference@23934629 owner=In-Process Thread property=defaultFilesystem setId=00000000-0000-0000-377a-99710000002f
FINE com.karmasphere.studio.common.persist.PersistentObjectReference setId: PersistentObjectReference@18779762 owner=In-Process Thread property=defaultFilesystem setId=00000000-0000-0000-377a-99710000002f
FINE com.karmasphere.studio.common.persist.PersistentObjectReference setId: PersistentObjectReference@18475052 owner=In-Process Thread property=defaultFilesystem setId=00000000-0000-0000-377a-99710000002f
FINE com.karmasphere.studio.common.netbeans.persist.NetbeansPersistenceManager getRootFolder: Configuration root is MultiFileObject@1ae53e9[com-karmasphere-studio-hadoop-target]
FINE com.karmasphere.studio.common.netbeans.persist.NetbeansPersistenceManager load: Loading MultiFileObject@1ae53e9[com-karmasphere-studio-hadoop-target]
FINE com.karmasphere.studio.common.netbeans.persist.NetbeansPersistenceManager load: Loading MultiFileObject@c19aae[com-karmasphere-studio-hadoop-target/HadoopJarJob_442c3399-cd1f-434a-87a2-406606c4b300.xml]
FINE com.karmasphere.studio.common.decorator.DecoratorMap <init>: Loading decorators for HadoopJarJob
FINE com.karmasphere.studio.common.decorator.DecoratorMap <init>: Loaded 0 decorators for HadoopJarJob
FINE com.karmasphere.studio.common.persist.PersistentObjectReference setId: PersistentObjectReference@25628793 owner=Job#4 property=defaultCluster setId=00000000-0000-0000-ffff-fffff0356687
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-cli-2.0-SNAPSHOT.jar!/ => AbstractFileObject@171e076[root of /root/.netbeans/6.8/modules/ext/commons-cli-2.0-SNAPSHOT.jar[org.openide.filesystems.JarFileSystem@14789ac]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-codec-1.4.jar!/ => AbstractFileObject@19f8c45[root of /root/.netbeans/6.8/modules/ext/commons-codec-1.4.jar[org.openide.filesystems.JarFileSystem@1b2370e]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-httpclient-3.1.jar!/ => AbstractFileObject@13cab44[root of /root/.netbeans/6.8/modules/ext/commons-httpclient-3.1.jar[org.openide.filesystems.JarFileSystem@1022ec]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-logging-1.1.1.jar!/ => AbstractFileObject@164cb94[root of /root/.netbeans/6.8/modules/ext/commons-logging-1.1.1.jar[org.openide.filesystems.JarFileSystem@7174b9]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-net-1.4.1.jar!/ => AbstractFileObject@a65d82[root of /root/.netbeans/6.8/modules/ext/commons-net-1.4.1.jar[org.openide.filesystems.JarFileSystem@4541bd]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/oro-2.0.8.jar!/ => AbstractFileObject@dee69c[root of /root/.netbeans/6.8/modules/ext/oro-2.0.8.jar[org.openide.filesystems.JarFileSystem@1f63f80]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.log4j/modules/ext/log4j-1.2.15.jar!/ => AbstractFileObject@1c320cc[root of /root/.netbeans/6.8/modules/ext/log4j-1.2.15.jar[org.openide.filesystems.JarFileSystem@19e3615]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.hadoop.common/modules/ext/xmlenc-0.52.jar!/ => AbstractFileObject@1f0d2df[root of /root/.netbeans/6.8/modules/ext/xmlenc-0.52.jar[org.openide.filesystems.JarFileSystem@1a437ac]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.hadoop.common/modules/ext/jets3t-0.7.1.jar!/ => AbstractFileObject@432b2e[root of /root/.netbeans/6.8/modules/ext/jets3t-0.7.1.jar[org.openide.filesystems.JarFileSystem@1c7efb5]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.hadoop.v20/modules/ext/hadoop-0.20.2-core.jar!/ => AbstractFileObject@14a8c61[root of /root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar[org.openide.filesystems.JarFileSystem@146662d]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.hadoop.v20/modules/ext/hadoop-0.20.2-streaming.jar!/ => AbstractFileObject@de484c[root of /root/.netbeans/6.8/modules/ext/hadoop-0.20.2-streaming.jar[org.openide.filesystems.JarFileSystem@15caf78]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.hadoop.v20/modules/ext/hadoop-0.20-karmasphere-extras.jar!/ => AbstractFileObject@3f82f[root of /root/.netbeans/6.8/modules/ext/hadoop-0.20-karmasphere-extras.jar[org.openide.filesystems.JarFileSystem@9b04db]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.jsr305/modules/ext/jsr305.jar!/ => AbstractFileObject@15bca46[root of /root/.netbeans/6.8/modules/ext/jsr305.jar[org.openide.filesystems.JarFileSystem@13cf7e8]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-logging-1.1.1.jar!/ => AbstractFileObject@164cb94[root of /root/.netbeans/6.8/modules/ext/commons-logging-1.1.1.jar[org.openide.filesystems.JarFileSystem@7174b9]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-beanutils-core-1.8.2.jar!/ => AbstractFileObject@1b7de87[root of /root/.netbeans/6.8/modules/ext/commons-beanutils-core-1.8.2.jar[org.openide.filesystems.JarFileSystem@727a27]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-lang-2.4.jar!/ => AbstractFileObject@1b1b678[root of /root/.netbeans/6.8/modules/ext/commons-lang-2.4.jar[org.openide.filesystems.JarFileSystem@42a821]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-io-1.4.jar!/ => AbstractFileObject@2c36aa[root of /root/.netbeans/6.8/modules/ext/commons-io-1.4.jar[org.openide.filesystems.JarFileSystem@b9118f]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-vfs-1.0.jar!/ => AbstractFileObject@2ffb0c[root of /root/.netbeans/6.8/modules/ext/commons-vfs-1.0.jar[org.openide.filesystems.JarFileSystem@b074a]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/commons-cli-2.0-SNAPSHOT.jar!/ => AbstractFileObject@171e076[root of /root/.netbeans/6.8/modules/ext/commons-cli-2.0-SNAPSHOT.jar[org.openide.filesystems.JarFileSystem@14789ac]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/collections-generic-4.01.jar!/ => AbstractFileObject@bc6cb[root of /root/.netbeans/6.8/modules/ext/collections-generic-4.01.jar[org.openide.filesystems.JarFileSystem@1283973]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.apache/modules/ext/karmasphere-client-apache.jar!/ => AbstractFileObject@1d3afaa[root of /root/.netbeans/6.8/modules/ext/karmasphere-client-apache.jar[org.openide.filesystems.JarFileSystem@b1cc11]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.amazon/modules/ext/karmasphere-client-amazon.jar!/ => AbstractFileObject@cc880a[root of /root/.netbeans/6.8/modules/ext/karmasphere-client-amazon.jar[org.openide.filesystems.JarFileSystem@1f75644]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.amazon/modules/ext/aws-java-sdk-1.0.003-ioe.jar!/ => AbstractFileObject@140c0b5[root of /root/.netbeans/6.8/modules/ext/aws-java-sdk-1.0.003-ioe.jar[org.openide.filesystems.JarFileSystem@2be31c]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.libs.jcraft/modules/ext/jsch-0.1.42-patched.jar!/ => AbstractFileObject@14b5f0c[root of /root/.netbeans/6.8/modules/ext/jsch-0.1.42-patched.jar[org.openide.filesystems.JarFileSystem@14d4c11]]
FINE com.karmasphere.studio.hadoop.netbeans.job.NetbeansClassPathProvider toClassPath: Resolving library entry jar:nbinst://com.karmasphere.studio.hadoop.client/modules/ext/karmasphere-client.jar!/ => AbstractFileObject@1bcf8f6[root of /root/.netbeans/6.8/modules/ext/karmasphere-client.jar[org.openide.filesystems.JarFileSystem@17d14b5]]
INFO com.karmasphere.studio.hadoop.executor.HadoopExecutor setMainClassName: Setting main class name to WordCount
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing WordCount
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/tmp/karmasphere-assembler-6619023549332779720.dir/hadoop-job-554234890548660146.jar!/WordCount.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector$Visitor test: Found Hadoop class org/apache/hadoop/mapred/JobConf
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/FileInputFormat
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/FileOutputFormat
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/FileInputFormat.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/util/ReflectionUtils
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/FileInputFormat$MultiPathFilter
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/InvalidInputException
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/FileSplit
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/util/StringUtils
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/FileInputFormat$2
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/FileInputFormat$NodeInfo
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/FileInputFormat$1
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/FileOutputFormat.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/InvalidJobConfException
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/FileAlreadyExistsException
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/TaskAttemptID
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/TaskAttemptContext
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/FileOutputCommitter
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/util/ReflectionUtils.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/conf/Configurable
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/util/ReflectionUtils$CopyInCopyOutBuffer
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/util/ReflectionUtils$1
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/FileInputFormat$MultiPathFilter.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/InvalidInputException.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/FileSplit.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapreduce/InputSplit
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/util/StringUtils.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/util/VersionInfo
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/util/StringUtils$1
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/FileInputFormat$2.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/FileInputFormat$NodeInfo.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/FileInputFormat$1.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/InvalidJobConfException.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/FileAlreadyExistsException.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/TaskAttemptID.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapreduce/TaskAttemptID
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/TaskID
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/TaskAttemptContext.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapreduce/TaskAttemptContext
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/util/Progressable
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/FileOutputCommitter.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/OutputCommitter
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/JobContext
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/conf/Configurable.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/util/ReflectionUtils$CopyInCopyOutBuffer.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/util/ReflectionUtils$1.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapreduce/InputSplit.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/util/VersionInfo.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/HadoopVersionAnnotation
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/util/StringUtils$1.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapreduce/TaskAttemptID.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapred/ID
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapreduce/TaskID
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/TaskID.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapreduce/TaskAttemptContext.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapreduce/JobContext
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/util/Progressable.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/OutputCommitter.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapreduce/OutputCommitter
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/JobContext.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/HadoopVersionAnnotation.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapred/ID.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector add: Enqueueing org/apache/hadoop/mapreduce/ID
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapreduce/TaskID.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapreduce/JobContext.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapreduce/OutputCommitter.class
FINE com.karmasphere.studio.hadoop.executor.HadoopClientInspector call: Processing jar:file:/root/.netbeans/6.8/modules/ext/hadoop-0.20.2-core.jar!/org/apache/hadoop/mapreduce/ID.class
FINE com.karmasphere.studio.common.decorator.DecoratorMap getDecorator: Looking for interface com.karmasphere.studio.hadoop.target.decorator.JobMonitorDecorator in {}
FINE com.karmasphere.studio.hadoop.executor.HadoopExecutorAssembler cleanup: Cleaning up
Shevek
> Hello.. I’m testing simple WordCount code with Netbeans plugin.
>
>
> I’m facing some memory error everytime.
> I’m running it on RHEL5
> I’m using hadoop 0.20.2
>
> My m/c has 4 GB RAM. I have allocated netbeans almost 3 GB.
It's not NetBeans that's running out of memory. It's the child JVM
that's running the Hadoop-local job.
Have you set the map output buffer to something inordinately large?
That's where it died.
I'll have to remind myself of some details to work out where the memory
parameters for the sub-jvm come from.
> 10/08/24 17:33:14 INFO mapred.MapTask: numReduceTasks: 1
> 10/08/24 17:33:14 INFO mapred.MapTask: io.sort.mb = 100
> 10/08/24 17:33:14 WARN mapred.LocalJobRunner: job_local_0001
> java.lang.OutOfMemoryError: Java heap space
> at org.apache.hadoop.mapred.MapTask
> $MapOutputBuffer.<init>(MapTask.java:781)
> at
> org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.LocalJobRunner
> $Job.run(LocalJobRunner.java:177)
> 10/08/24 17:33:14 INFO mapred.JobClient: map 0% reduce 0%
> 10/08/24 17:33:14 INFO mapred.JobClient: Job complete: job_local_0001
> 10/08/24 17:33:14 INFO mapred.JobClient: Counters: 0
> Exception in thread "main" java.io.IOException: Job failed!
> at
> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
> at WordCount.main(WordCount.java:52)
S.
--
http://www.karmasphere.com/
Karmasphere Studio for Hadoop - An intuitive visual interface to Big Data
------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users
worldwide. Take advantage of special opportunities to increase revenue and
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
Hadoop-studio-users mailing list
Hadoop-st...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/hadoop-studio-users