How to retrieve results remotely

[Vivek Suvarna]

I'm trying to execute a simple pi.py file on the remote cluster and want to print the results back on the console.

get batches command only returns id, state and log.

How do I get the result back from the remote cluster execution.

my get command is as follows

statement_url = http://<hadoop cluster edge node>:port/batches/<sessionid>/log

I'm using the python requests module as below

r = requests.get(statement_url, headers=headers)

the response json only gives me id, log and state.

If I execute the same command on the cluster directly it prints out the result on the console.

My use case is that I need to execute jobs on the remote cluster and return results back

[Vivek Suvarna]

Apologies, forgot to mention that I'm trying to run this via Livy server.