Failing Tests for Kite Data Core Module

7 views

Skip to first unread message

aguynamedryan

unread,

Nov 18, 2015, 7:29:32 PM11/18/15

to CDK Development

Hello,

I've been working on a pull request for Kite and it looks like all the tests pass under Travis-CI, but when I run the tests on my Cloudera Quickstart VM, I get some failed tests for the "Kite Data Core Module" test suite (see below). I'm pretty new to Cloudera, Hadoop, Kite, etc. so I'm not sure where to look for the source of these failures.

Has anyone else seen these failures? Am I running the tests on the wrong VM or something?

Any help would be appreciated,

Ryan

--- SNIP ---

Tests run: 2, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 3.358 sec <<< FAILURE!

testAvroSyncDFS(org.kitesdk.data.spi.filesystem.TestAvroAppender) Time elapsed: 1.86 sec <<< ERROR!

org.kitesdk.data.DatasetIOException: Unable to create reader path:hdfs://localhost:42189/tmp/test.avro

at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:102)

at org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97)

at org.kitesdk.data.spi.filesystem.FileSystemDatasetReader.initialize(FileSystemDatasetReader.java:68)

at org.kitesdk.data.spi.filesystem.TestAvroAppender.count(TestAvroAppender.java:132)

at org.kitesdk.data.spi.filesystem.TestAvroAppender.testAvroSyncDFS(TestAvroAppender.java:78)

Running org.kitesdk.data.spi.filesystem.TestAvroWriter

Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.029 sec

Running org.kitesdk.data.spi.filesystem.TestCSVAppender

Tests run: 2, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 1.604 sec <<< FAILURE!

testCSVSyncDFS(org.kitesdk.data.spi.filesystem.TestCSVAppender) Time elapsed: 0.476 sec <<< ERROR!

org.kitesdk.data.DatasetIOException: Could not read record

at org.apache.hadoop.util.DataChecksum.verifyChunkedSums(DataChecksum.java:325)

at org.apache.hadoop.hdfs.RemoteBlockReader2.readNextPacket(RemoteBlockReader2.java:216)

at org.apache.hadoop.hdfs.RemoteBlockReader2.read(RemoteBlockReader2.java:146)

at org.apache.hadoop.hdfs.DFSInputStream$ByteArrayStrategy.doRead(DFSInputStream.java:693)

at org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:749)

at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:806)

at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:847)

at java.io.DataInputStream.read(DataInputStream.java:149)

at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)

at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)

at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)

at java.io.InputStreamReader.read(InputStreamReader.java:184)

at java.io.BufferedReader.fill(BufferedReader.java:154)

at java.io.BufferedReader.readLine(BufferedReader.java:317)

at java.io.BufferedReader.readLine(BufferedReader.java:382)

at au.com.bytecode.opencsv.CSVReader.getNextLine(CSVReader.java:266)

at au.com.bytecode.opencsv.CSVReader.readNext(CSVReader.java:233)

at org.kitesdk.data.spi.filesystem.CSVFileReader.advance(CSVFileReader.java:174)

at org.kitesdk.data.spi.filesystem.CSVFileReader.initialize(CSVFileReader.java:139)

at org.kitesdk.data.spi.filesystem.TestCSVAppender.count(TestCSVAppender.java:136)

at org.kitesdk.data.spi.filesystem.TestCSVAppender.testCSVSyncDFS(TestCSVAppender.java:81)