S3 underfs which supports PROXY

[Haochen Zhang]

Hi All,

I've create a pull request to support the proxy-based access to the s3 underfs.

It makes it possible for tachyon to use some s3-compatible services as the underfs.

This patch also fix a potential bug: no MD5 specified while creating the directory over s3, which may cause an exception while running 'bin/tachyon runTests'

Following is the stack of the bug:

2015-10-19 13:41:24,945 ERROR  (S3UnderFileSystem.java:mkdirsInternal) - Failed to create directory: /data/default_tests_files

org.jets3t.service.S3ServiceException: Mismatch between MD5 hash of uploaded data () and ETag returned by S3 (d41d8cd98f00b204e9800998ecf8427e) for object key: data/default_tests_files_$folder$

at org.jets3t.service.S3Service.putObject(S3Service.java:2123)

at tachyon.underfs.s3.S3UnderFileSystem.mkdirsInternal(S3UnderFileSystem.java:504)

at tachyon.underfs.s3.S3UnderFileSystem.mkdirs(S3UnderFileSystem.java:256)

at tachyon.client.file.FileOutStream.<init>(FileOutStream.java:94)

at tachyon.examples.BasicOperations.writeFile(BasicOperations.java:109)

at tachyon.examples.BasicOperations.call(BasicOperations.java:80)

at tachyon.examples.BasicOperations.call(BasicOperations.java:43)

at tachyon.examples.Utils.runExample(Utils.java:103)

at tachyon.examples.BasicOperations.main(BasicOperations.java:145)

2015-10-19 13:41:24,946 ERROR  (Utils.java:runExample) - Exception running test: tachyon.examples.BasicOperations@24b1d79b

java.io.IOException: Failed to create /data/default_tests_files

at tachyon.client.file.FileOutStream.<init>(FileOutStream.java:95)

at tachyon.examples.BasicOperations.writeFile(BasicOperations.java:109)

at tachyon.examples.BasicOperations.call(BasicOperations.java:80)

at tachyon.examples.BasicOperations.call(BasicOperations.java:43)

at tachyon.examples.Utils.runExample(Utils.java:103)

at tachyon.examples.BasicOperations.main(BasicOperations.java:145)

Failed the test!