mrjob v0.6.9 released!
3 views
Skip to first unread message
Dave Marin
May 29, 2019, 11:00:19 PM5/29/19
to mr...@googlegroups.com
This is mostly a bugfix release.
Fixes a bug introduced in v0.6.8 that could break uploading archives/directories
Improved the way the Spark runner and sim runners simulate Hadoop. Specifically, added an emulate_map_input_file option to the Spark runner that allows it to run MRJobs that use code like:
jobconf_from_env(‘mapreduce.map.input.file’)
to read the name of input files.
`mrjob spark-submit` better emulates spark-submit; you no longer have to use -- to pass switches to the Spark script.
add_passthrough_option() and add_file_option() (which are deprecated) now better emulate optparse by allowing the *type* argument to be a type
Finally, this version drops support for Python 3.4
-Dave
Fixes a bug introduced in v0.6.8 that could break uploading archives/directories
Improved the way the Spark runner and sim runners simulate Hadoop. Specifically, added an emulate_map_input_file option to the Spark runner that allows it to run MRJobs that use code like:
jobconf_from_env(‘mapreduce.map.input.file’)
to read the name of input files.
`mrjob spark-submit` better emulates spark-submit; you no longer have to use -- to pass switches to the Spark script.
add_passthrough_option() and add_file_option() (which are deprecated) now better emulate optparse by allowing the *type* argument to be a type
Finally, this version drops support for Python 3.4
-Dave
Reply all
Reply to author
Forward
0 new messages