External jar in python
Arpit Goel
Hi
Under my project , I was exploring livy with spark.
For scala and java, we can convert them into fat jar along with our external dependent jars. But for python I am unable to provide my external dependent jars with livy submit command.
curl -X POST --data '{"jars":"/tmp/mqtt/spark-streaming-mqtt_2.10-1.5.2.jar","file":"/tmp/mqtt/MqttPython.py"}' -H "Content-Type: application/json" localhost:8998/batches
Here MqttPython.py is my python code and “spark-streaming-mqtt_2.10-1.5.2.jar” is external jar which I need to submit.
Below is the error for your kind reference:
“ Can not deserialize instance of scala.collection.immutable.List out of VALUE_STRING”
I am trying to submit livy command through my Centos system which has spark 1.5 installed.
Could you please help me in resolving the issue.
Regards
Arpit Goel
Lead Analytics Engineer
___________________________________________________________________________________________________________________________
www.neewee.in |Mob: +9886788718 | Phn: +918042024519 |email: arpit...@neewee.in
Confidentiality Notice:
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain confidential or privileged information. Any review, re-transmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify the sender at Neewee Analytics or con...@neewee.in immediately and destroy all copies of this message and any attachments.
