mrjob v0.6.7 released!
15 views
Skip to first unread message
Dave Marin
Jan 16, 2019, 7:02:10 PM1/16/19
to mr...@googlegroups.com
mrjob v0.6.7 is out!
This release adds the `mrjob spark-submit` subcommand, which works just like regular spark-submit exept that it gives access to mrjob runners and features. For example, even someone who’s not interested in writing MRJobs could use `mrjob spark-submit -r emr ...` to launch their Spark job on EMR.
mrjob now supports --setup scripts in Spark (at least on Hadoop/YARN).
Fixed a few bugs:
- os.SIGKILL ImportError on Windows
- idle termination scripts were broken on 2.x AMIs
- create_bucket() didn’t work in us-east-1
For the full list of changes, see https://mrjob.readthedocs.io/en/stable/whats-new.html#v0-6-7
-Dave
This release adds the `mrjob spark-submit` subcommand, which works just like regular spark-submit exept that it gives access to mrjob runners and features. For example, even someone who’s not interested in writing MRJobs could use `mrjob spark-submit -r emr ...` to launch their Spark job on EMR.
mrjob now supports --setup scripts in Spark (at least on Hadoop/YARN).
Fixed a few bugs:
- os.SIGKILL ImportError on Windows
- idle termination scripts were broken on 2.x AMIs
- create_bucket() didn’t work in us-east-1
For the full list of changes, see https://mrjob.readthedocs.io/en/stable/whats-new.html#v0-6-7
-Dave
Reply all
Reply to author
Forward
0 new messages