Announcing ghadoop-0.32.0 and version 1.2.2 of the GCS Connector for Hadoop

[Hadoop on Google Cloud Platform Team]

Greetings Hadoop on Google Cloud Platform users!

We’ve just released an update to the the ghadoop deployment script and some minor bug fixes to the Google Cloud Storage connector.  

For your convenience, ghadoop now bundles the Hive and Pig sample app from the Google Cloud Solutions team. Now you can deploy a Hive and Pig enabled cluster by typing the following command:

./ghadoop deploy extensions/querytools/querytools_env.sh

The querytools are set up to be used by user “hdpuser”, so after SSH’ing into the namenode, you should run:

sudo sudo -i -u hdpuser

You can download the new ghadoop-0.32.0.tar.gz or ghadoop-0.32.0.zip directly, or visit the developer documentation where the download links now point to the latest version.

The following change logs summarize the updates in this release, and can also be found in CHANGES.txt inside ghadoop-0.32.0.[tar.gz|zip] and gcs-connector-1.2.2.jar, respectively.

ghadoop-0.32.0: CHANGES.txt

0.32.0 - 2014-02-12

 1. ghadoop now always includes ghadoop_env.sh; only the overrides file needs
    to be specified, e.g. ghadoop deploy single_node_env.sh.
 2. Files in COMMAND_GROUPS are now relative to the directory in which ghadoop
    resides, rather than having to be inside libexec/. Absolute paths are
    also supported now.
 3. Added UPLOAD_FILES to ghadoop_env.sh which ghadoop will use to upload
    a list of relative or absolute file paths to every VM before starting
    execution of COMMAND_STEPS.
 4. Include full Hive and Pig sampleapp from Cloud Solutions with ghadoop;
    added extensions/querytools/querytools_env.sh to auto-install Hive and
    Pig as part of deployment. Usage:
        ./ghadoop deploy extensions/querytools/querytools_env.sh

gcs-connector-1.2.2: CHANGES.txt

1.2.2 - 2014-02-12

 1. Fixed a bug where filenames with '+' were unreadable due to premature

    URL-decoding.

 2. Modified a check to allow fs.gs.io.buffersize.write to be a non-multiple

    of 8MB, just printing out a warning instead of check-failing.

 3. Added some debug-level logging of exceptions before throwing in cases

    where Hadoop tends to swallows the exception along with its useful info.

Please send any questions or comments to gcp-hadoo...@google.com

Thanks, and happy computing!