mrjob

mrjob v0.7.4 is out! mrjob now supports Docker on EMR. Using this feature can be as simple as

Can't find any trace of this in the docs. Can I run a Python script with mrjob on my laptop, and

mrjob v0.7.3 is out! This release makes cluster pooling use API calls much more efficiently, reducing

"Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code

Hi, Did you find the solution? I think I have the same problem with google dataproc Thanks! On

mrjob v0.7.2 is out! If you are using Spark, mrjob now emulates archives (a YARN-only feature of

I hit the same underlying issue with trying to write strings to sys.stderr, though in our case it was

Ah perfect, that looks like exactly what I need :) Thanks! On Mon, Mar 9, 2020 at 5:09 PM Dave Marin

mrjob v0.7.1 has been release. * v0.7.1 fixes the bug to set default value of VisibleToAllUsers to

You know, you may be right, local and inline mode might be over-zealous about not sorting output. I

mrjob v0.7.0 has just been released! Unlike the transition from v0.5.0 to v0.6.0, v0.7.0 isn't

I gave up and switched to PySpark and I am so glad I did. On Tuesday, October 29, 2019 at 7:22:21 PM

This is a quick bugfix release, and probably the last one in the v0.6.x series. v0.6.12 fixes a bug

mrjob v0.6.11 is out! The major change in this version is that if a Spark job fails, mrjob can now

I log into Master node to submit job like this: python my_mrjob.py -r hadoop ... Job runs OK, I can

If its an MR job that you submitted to AWS's EMR cluster (--runer=emr) from your laptop's

mrjob v0.6.10 is released! Some important changes: - PyPy-aware. If you launch a MRJob in PyPy,

This is mostly a bugfix release. Fixes a bug introduced in v0.6.8 that could break uploading archives

Dear all, I have a question about SORT_VALUES. I have two datasets, one with countries and one with

mrjob v0.6.8 provides full support for Spark. You can now launch Spark code with any runner (except

mrjob v0.6.7 is out! This release adds the `mrjob spark-submit` subcommand, which works just like

how to pass input file in args in make runner mrjob instead of command line options

mrjob v0.6.6 is released. This is mostly a series of small changes that make your life easier: - you

This release has a number of small changes, including: - more robust idle cluster self-termination (

mrjob v0.6.4 is up on PyPI. This release adds the DIRS attribute to MRJobs, which allows you to

This release is mostly a backport of some features in v0.6.x: * dropped support for Python 2.6 and

Hi! Is there an input protocol for parquet? Tried org.apache.parquet.hadoop.ParquetInputFormat and a

mrjob can now read arbitrary file formats (eg image files) using mapper_raw(). mrjob also now has

Thanks for your feedback Dave. My problem is that I needed to wait until the results of step 1

Good-day I have an assignment wherein I have to do matrix multiplication. I am given two files in the