s3n - Unhandled internal error. Any conflicts between Genie v1 and HDFS 2.7.1+, Pig 15.1+?

[Joe]

I'm seeing an error in Pig when I attempt to use either s3n or s3a to read/write data from S3.

 

grunt> REGISTER s3n://korrelate/hadoop/lib/piggybank-0.12.0.jar
2016-02-23 20:19:48,705 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-02-23 20:19:49,351 [main] INFO  org.apache.hadoop.fs.s3native.NativeS3FileSystem - Opening 's3n://korrelate/hadoop/lib/piggybank-0.12.0.jar' for reading
2016-02-23 20:19:49,974 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
Details at logfile: /mnt/tomcat/genie-jobs/7d59a710-bb07-4a8e-8865-fb9305d44bcc/pig_1456258720521.log

Relevant bits from the log file:

[hadoop@genie-qa-corndog-internal 7d59a710-bb07-4a8e-8865-fb9305d44bcc]$ tail -n 200 /mnt/tomcat/genie-jobs/7d59a710-bb07-4a8e-8865-fb9305d44bcc/pig_1456258720521.log
Pig Stack Trace
---------------
ERROR 2997: Encountered IOException. No such file or directory 's3n://korrelate/hadoop/lib/piggybank-0.12.0.jar`'
​
java.io.FileNotFoundException: No such file or directory 's3n://korrelate/hadoop/lib/piggybank-0.12.0.jar`'
at org.apache.hadoop.fs.s3native.NativeS3FileSystem.getFileStatus(NativeS3FileSystem.java:498)
at org.apache.pig.impl.io.FileLocalizer.fetchFilesInternal(FileLocalizer.java:818)
at org.apache.pig.impl.io.FileLocalizer.fetchFile(FileLocalizer.java:756)
at org.apache.pig.PigServer.registerCode(PigServer.java:593)
at org.apache.pig.tools.grunt.GruntParser.processRegister(GruntParser.java:471)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:550)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)
at org.apache.pig.Main.run(Main.java:565)
at org.apache.pig.Main.main(Main.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
================================================================================
Pig Stack Trace
---------------
ERROR 2998: Unhandled internal error. org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
​
java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
at org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(Native Method)
at org.apache.hadoop.util.NativeCrc32.calculateChunkedSumsByteArray(NativeCrc32.java:86)
at org.apache.hadoop.util.DataChecksum.calculateChunkedSums(DataChecksum.java:430)
at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:202)
at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:124)
at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:110)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:87)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:59)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:119)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:366)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:338)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:2030)
at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1999)
at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1975)
at org.apache.pig.impl.io.FileLocalizer.fetchFilesInternal(FileLocalizer.java:840)
at org.apache.pig.impl.io.FileLocalizer.fetchFiles(FileLocalizer.java:767)
at org.apache.pig.PigServer.registerJar(PigServer.java:558)
at org.apache.pig.tools.grunt.GruntParser.processRegister(GruntParser.java:468)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:550)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)
at org.apache.pig.Main.run(Main.java:565)
at org.apache.pig.Main.main(Main.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
================================================================================

Versions:  

• Genie v1, forked here.
• HDFS 2.7.1
• Pig  15.1

Note:  We fully plan to upgrade to Genie v2.  But currently this is a bug/blocker for us bumping our cluster versions.  If it can't be fixed or won't fix on Genie v1, that is relevant feedback and we can prioritize the Genie v2 upgrade.  But due to the significant API changes in Genie v2, the team would prefer I patch this issue and we move forward with Genie v2 on a more luxurious timeline.