Faunus Out of memory Error in EdgeCopy Reduce

[Janarthanam Ramalingam]

Hi,

We compiled Faunus for Hadoop-0.23 and running this on our grid. We are running in to an Out of Memory error during write. I am fairly sure that we don't have a super node vertex. How do we figure out the root cause of this?

FATAL[main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:2271)
	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:122)
	at java.io.DataOutputStream.writeByte(DataOutputStream.java:153)
	at org.apache.hadoop.io.WritableUtils.writeVLong(WritableUtils.java:296)
	at com.thinkaurelius.faunus.FaunusEdge.writeCompressed(FaunusEdge.java:112)
	at com.thinkaurelius.faunus.FaunusVertex$EdgeMap.write(FaunusVertex.java:396)
	at com.thinkaurelius.faunus.FaunusVertex.write(FaunusVertex.java:281)
	at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:100)
	at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:84)
	at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1268)
	at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:83)
	at org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat$LazyRecordWriter.write(LazyOutputFormat.java:112)
	at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:568)
	at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
	at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
	at com.thinkaurelius.faunus.formats.EdgeCopyMapReduce$Reduce.reduce(EdgeCopyMapReduce.java:93)
	at com.thinkaurelius.faunus.formats.EdgeCopyMapReduce$Reduce.reduce(EdgeCopyMapReduce.java:69)
	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:170)
	at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:636)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:396)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)

Thanks,
Janar

[Marko Rodriguez]

Hello,

Can you validate that everything works on the small "Graph of the Gods" example distributed with Faunus?

Marko.

http://markorodriguez.com

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Janarthanam Ramalingam]

Yup!

It tested on Graph of Gods. I was even able to load a couple of my load files on that, other than GoG. It works fine with mapreduce 0.23 (yarn goodnes).

Thanks,

Janar

[Janarthanam Ramalingam]

Updating with what i found:

Our vertices have a few "Obama" vertices, with a lot of incoming edges aka super nodes. I was hoping to work around the problem by script loading in to faunus, by only creating out edges. Apparently, faunus generates in edges and out edges, internally.

I can't increase memory of my map reduce as we are running 32 bit jvms. Increasing mapper doesn't help either.

[Marko Rodriguez]

Hello,

Yes, a vertex maintains incoming and outgoing adjacencies and a vertex (with its edges and properties) must be able to be held in main memory.

https://github.com/thinkaurelius/faunus/wiki/Gotchas-and-Limitations#a-vertex-and-its-incident-edges-must-fit-in-memory

Marko.

http://markorodriguez.com