I have a customize MessageDecoder.
Message format: 2 fields. The first field is timestamp. As following:
yyMMddHHmmss<table>aaaaaaa
the code like this:
public class StringMessageDecoder extends MessageDecoder<Message, String> {
....................
try {
String[] tokens = payloadString.split("\t");
SimpleDateFormat sdf = new SimpleDateFormat("yyMMddHHmmss");
Date dt = sdf.parse(tokens[0]);
timestamp = dt.getTime();
} catch (ParseException e) {
log.error("Unable to parse time", e);
timestamp = System.currentTimeMillis();
}
return new CamusWrapper<String>(payloadString, timestamp);
}
Then, run the camus for houly partition.
for example, The result is:
all of the records with time(150709040000) were pulled to '2015/07/09/03' partition instead of '2015/07/09/04' partition.
some records with time(150709040001) were pulled to '2015/07/09/03' partition instead of '2015/07/09/04', and others were correctly pulled to '2015/07/09/04' partition.
all of the records with time(150709050000) were pulled to '2015/07/09/04' partition instead of '2015/07/09/05' partition.
others records with time(04:00:02~04:59:59) were correctly pulled to '2015/07/09/04' partition.
there seems to be 1 second deviation...If anyone know why?
Thanks.