Genie and Spark

[kkuma...@gmail.com]

Genie job completed successfully even though the  job never ran on EMR, when we looked at genie error log it has a 

“/opt/tomcat/webapps/genie-jobs/ecb3e445-6fe1-4f9e-a522-ef9b21fff4b3/jars/bin/spark-class: line 83: [: too many arguments”

we are using spark version 2.0.0, Genie version 2.2.3

can you please let me know if anyone faced this issue?

[Marco Primi]

Hello,

Genie marks the job successful or not depending on the exit code returned by the command invoked.

Commands and applications (such as Spark) are not part of Genie, they are installed and maintained by someone in your organization.

I’d recommend checking with them, it seems likely that the script currently provided terminates with exit code 0 even if an error is encountered. 

Also recommended, upgrade to Genie 3!

Cheers,

M

--
You received this message because you are subscribed to the Google Groups "genie" group.
To unsubscribe from this group and stop receiving emails from it, send an email to genieoss+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.