mrjob v0.6.10 is out!
6 views
Skip to first unread message
Dave Marin
Jul 22, 2019, 8:37:39 PM7/22/19
to mr...@googlegroups.com
mrjob v0.6.10 is released! Some important changes:
- PyPy-aware. If you launch a MRJob in PyPy, python_bin will default to `pypy` or `pypy3` as appropriate
- The Spark harness now works on EMR, allowing you to run “classic” MRJobs on EMR using Spark rather than Hadoop streaming
- see https://mrjob.readthedocs.io/en/stable/guides/spark.html#mrjobs-on-spark-on-emr
- When using the Spark harness, you can turn off the internal protocol with --skip-internal-protocol, making the job run faster with (usually) the same semantics
- The default instance type on EMR is now m5.xlarge
For more details, see https://mrjob.readthedocs.io/en/stable/whats-new.html#v0-6-10
-Dave
- PyPy-aware. If you launch a MRJob in PyPy, python_bin will default to `pypy` or `pypy3` as appropriate
- The Spark harness now works on EMR, allowing you to run “classic” MRJobs on EMR using Spark rather than Hadoop streaming
- see https://mrjob.readthedocs.io/en/stable/guides/spark.html#mrjobs-on-spark-on-emr
- When using the Spark harness, you can turn off the internal protocol with --skip-internal-protocol, making the job run faster with (usually) the same semantics
- The default instance type on EMR is now m5.xlarge
For more details, see https://mrjob.readthedocs.io/en/stable/whats-new.html#v0-6-10
-Dave
Reply all
Reply to author
Forward
0 new messages