Issue running jobs in Jupyter

iamhasib

unread,

Jul 5, 2017, 7:45:10 PM7/5/17

to Project Jupyter

We just deployed Jupyter with EMR in AWS and running it memory issues whenever we run any job in Jupyter.

We have pretty large servers (3 servers, all m4.2xlarge servers) and test data volume is pretty tiny.

The error messages are shown on ecnlosed images.

If someone could shed lights, I appreciate a lot.

Pasted image at 2017_07_06 09_17 AM.png

Pasted image at 2017_07_06 09_18 AM.png

iamhasib

unread,

Jul 5, 2017, 8:24:38 PM7/5/17

to Project Jupyter

More detail -

```17/07/03 04:08:15 WARN YarnAllocator: Container marked as failed: container_1499050738160_0001_01_000011 on host: ip-10-0-24-85.ap-southeast-2.compute.internal. Exit status: 52. Diagnostics: Exception from container-launch.

Container id: container_1499050738160_0001_01_000011

Exit code: 52

Stack trace: ExitCodeException exitCode=52:

at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)

at org.apache.hadoop.util.Shell.run(Shell.java:479)

at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)

at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:748)```

Roland Weber

unread,

Jul 6, 2017, 1:51:44 AM7/6/17

to Project Jupyter

On Thursday, July 6, 2017 at 1:45:10 AM UTC+2, iamhasib wrote:

We have pretty large servers (3 servers, all m4.2xlarge servers) and test data volume is pretty tiny.

Have you checked the configuration of the containers that are started?

If a container is limited to x MB of RAM, it doesn't matter how much memory the machine has.

hope that helps,

Roland

iamhasib

unread,

Jul 6, 2017, 4:14:51 AM7/6/17

to Project Jupyter

Appreciate if you could elaborate bit more........

Why would container and related config so important if the input data set is only 3 MB?

Reply all

Reply to author

Forward