Announcing ghadoop-0.31.1 and version 1.2.1 of the GCS Connector for Hadoop
Hadoop on Google Cloud Platform Team
Greetings Hadoop on Google Cloud Platform users!
We’ve just released some minor bug fixes and updates to the Google Cloud Storage connector and the ‘ghadoop’ deployment script which improve compatibility with Hadoop applications and which make ghadoop deployment easier to use and debug. You can download the new ghadoop-0.31.1.tar.gz or ghadoop-0.31.1.zip directly, or visit the developer documentation where the download links now point to the latest version.
The following change logs summarize the updates in this release, and can also be found in CHANGES.txt inside ghadoop-0.31.1.[tar.gz|zip] and gcs-connector-1.2.1.jar, respectively.
ghadoop-0.31.1: CHANGES.txt
0.31.1 - 2014-01-23
1. Added CHANGES.txt for release notes.
2. Switched from /hadoop/temp to /hadoop/tmp.
3. Added support for running ghadoop as root; will display additional
confirmation prompt before starting.
4. run_gcutil_cmd() now displays the full copy/paste-able gcutil command.
5. Now, only the public key of the master-generated ssh keypair is copied
into GCS during setup and onto the datanodes. This fixes the occasional
failed deployment due to GCS list-consistency, and is cleaner anyways.
The ssh keypair is now more descriptively named: 'hadoop_master_id_rsa'.
6. Added check for sshability from master to workers in start_hadoop.sh.
7. Cleaned up internal gcutil commands, added printout of full command
to ssh into the namenode at the end of the deployment.
8. Removed indirect config references from *-site.xml.
9. Stopped explicitly setting mapred.*.dir.
gcs-connector-1.2.1: CHANGES.txt
1.2.1 - 2014-01-23
1. Added CHANGES.txt for release notes.
2. Fixed a bug where accidental URI decoding makes it impossible to use
pre-escaped filenames, e.g. foo%3Abar.
3. Fixed a bug where an IOException was thrown when trying to read a
zero-byte file.
Please send any questions or comments to gcp-hadoo...@google.com
Thanks, and happy computing!