How to pass parameters to Scala script launched via sparkling-shell?

1,446 views
Skip to first unread message

Michal Malohlava

unread,
Jan 22, 2016, 4:13:03 PM1/22/16
to h2ostream
Question:
    How to pass parameters to Scala script launched via sparkling-shell?

Answer:
Right now there is no support to pass parameters via command line in sparkling-shell.
In more details: sparkling-shell is a simple wrapper around Spark's spark-shell and it just configures `--jars` parameter to point to sparkling water libraries.
And spark-shell does not accept any parameters to pass to scala script.

However, there are several workarounds:
  - you can setup environment property (export prop1=value1) and access it directly from you Scala code via sys.env
  - you can setup system property (-Dprop2=value2 option for JVM) and access it via sys.pros

Nice summary is here: http://stackoverflow.com/questions/29928999/passing-command-line-arguments-to-spark-shell

Michal

Reply all
Reply to author
Forward
0 new messages