mrjob v0.6.6 released!

7 views
Skip to first unread message

Dave Marin

unread,
Nov 5, 2018, 8:06:15 PM11/5/18
to mr...@googlegroups.com
mrjob v0.6.6 is released. This is mostly a series of small changes that make your life easier:

- you can safely use booleans as jobconf values in mrjob.conf
- faster probable cause of error on EMR if you have SSH set up
- you can use --spark-args=‘...’ instead of --spark-arg with one argument at a time
- same with --hadoop-args
- you can use -D instead of --jobconf
- --local-tmp-dir option can override your config, or just use the default temp dir (--local-tmp-dir ‘')

mrjob can also now run JarSteps that don’t understand Hadoop generic options (e.g. -D, -libjar)

For the full list of changes, see:

https://mrjob.readthedocs.io/en/latest/whats-new.html#v0-6-6

-Dave

Reply all
Reply to author
Forward
0 new messages