Unable to pass the flags option

13 views
Skip to first unread message

Buntu Dev

unread,
Jan 15, 2016, 3:11:33 AM1/15/16
to cdk...@cloudera.org
I'm attempting to run a 'copy' command to copy from one large dataset to a new dataset with parquet format with different partition strategy. This keeps failing with this message:
 
Container [pid=1546,containerID=container_1452278047864_45579_01_107350] is running beyond physical memory limits. Current usage: 2.8 GB of 2 GB physical memory used; 3.4 GB of 4.2 GB virtual memory used. Killing container.


In order to set the java options via flags, I'm running into this error: 
~~~~~~~~~~~~~~~~
$ flags="-Xmx4096m" kite-dataset copy src_table dest_table

Exception in thread "main" java.lang.ClassNotFoundException: -Xmx4096m
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:270)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:214)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
~~~~~~~~~~~~~~~~~~~

I'm on "1.0.0-cdh5.5.1" Kite version, please let me know how to fix these errors, thanks!

Esteban Roncancio

unread,
Sep 26, 2016, 5:15:30 PM9/26/16
to CDK Development
Buntu,

did you ever find a solution for this?
Reply all
Reply to author
Forward
0 new messages