Announcing new versions of bdutil and the Hadoop connectors

[Hadoop on Google Cloud Platform Team]

Greetings, users of Hadoop on Google Cloud Platform!

We're pleased to announce new versions of bdutil, the GCS connector for Hadoop, the BigQuery connector for Hadoop and the Datastore connector for Hadoop with bugfixes and minor improvements. Download bdutil-0.34.2.tar.gz or bdutil-0.34.2.zip now to try it out, or visit the developer documentation where the download links now point to the latest version.

GCS Connector Binaries

You can download the connector jarfile directly, gcs-connector-1.2.6-hadoop1.jar for use with Hadoop 1 (and other versions of the same series, like 0.20.205.0), and gcs-connector-1.2.6-hadoop2.jar for use with Hadoop 2 or allow bdutil to perform the installation and configuration for you without having to deal directly with jarfiles.

GCS Connector Open Source

The GCS connector can now be found in source form in our Github repository, GoogleCloudPlatform/bigdata-interop.  

As always, please send any questions or comments to gcp-hadoo...@google.com

All the best,

Your Google Team

gcs-connector-1.2.6: CHANGES.txt

1.2.6 - 2014-06-05

 1. When running in hadoop 0.23+ (hadoop 2+), listStatus will now throw a

    FileNotFoundException when a non-existent path is passed in.

 2. The GCS connector now uses the v1 JSON API when accessing Google

    Cloud Storage.

 3. The GoogleHadoopFileSystem now treats the parent of the root path as if

    it is the root path. This behavior mimics the POSIX behavior of "/.."

    being the same as "/".

 4. When creating a new file, a zero-length marker file will be created

    before the FSDataOutputStream is returned in create(). This allows for

    early detection of overwrite conflicts that may occur and prevents

    certain race conditions that could be encountered when relying on

    a single exists() check.

 5. The dependencies on cglib and asm were removed from the GCS connector

    and the classes for these are no longer included in the JAR.

bdutil-0.34.2: CHANGES.txt

0.34.2 - 2014-06-05

 1. When using Hadoop 2 / YARN, and the default filesystem is set to 'gs', YARN

    log aggregation will be enabled and YARN application logs, including

    map-reduce task logs will be persisted to gs://<CONFIGBUCKET>/yarn-logs/.

datastore-connector-0.14.5: CHANGES.txt

0.14.5 - 2014-06-05

 1. Misc updates in library dependencies.

bigquery-connector-0.4.2: CHANGES.txt

0.4.2 - 2014-06-05

 1. Misc updates in library dependencies.