java.lang.OutOfMemoryError: Java heap space

[Daniel SA]

Hi there,

I'm running CDAP 6.5.1 sandbox in a docker image, in a 16gb ram machine and still facing some memory issues when running a single pipeline with several joins put in series --> container exits with "java.lang.OutOfMemoryError: Java heap space"

Already tried to increase resources in cdap (from 1gb to 16gb) and docker (also 16gb), but issue still remains.

Since afterwards the pipeline should run in a server instance with even greater resources, I wonder if there is a chance to increase jvm heap space right in docker for a sandbox?

Thanks in advance for your support and kind regards, Daniel

[Albert Shau]

Hi Daniel,

The 'resources' configuration for a pipeline in sandbox doesn't actually control anything since everything in sandbox runs in memory. Java requires you to set the max memory when the process is started, so it can't be a dynamic thing. It only impacts things when the pipeline is run on a distributed cluster. If you would like to increase the memory given to the sandbox, you can do so by setting an environment variable like:

JAVA_OPTS="-Xmx16G" bin/cdap sandbox start

Do also note that the sandbox is meant for development purposes and not for processing real datasets.

Regards,

Albert

--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/f2401b65-5548-4950-b0fa-218568475ce4n%40googlegroups.com.