[Scalding] Reading Timestamp from Parquet File
65 views
Skip to first unread message
Dheeraj Vc
Jan 20, 2017, 1:14:08 AM1/20/17
to cascading-user
Hi I am trying to read the timestamp field from A parquet file using a case class.
The parquet file is from an impala table.
Does anyone know what the field type should be ?
If I use Long , then it always reads it as None.
The parquet file is from an impala table.
Does anyone know what the field type should be ?
If I use Long , then it always reads it as None.
Message has been deleted
Dheeraj Vc
Jan 20, 2017, 2:00:13 AM1/20/17
to cascading-user
Correction - I get the error
Caused by: org.apache.parquet.io.ParquetDecodingException: The requested schema is not compatible with the file schema. incompatible types: optional int64 datetime != optional int96 datetime
Dheeraj Vc
Feb 6, 2017, 4:36:50 PM2/6/17
to cascading-user
ANyone elese facing this issue, I got this resolved by using the parquet-avro source
Reply all
Reply to author
Forward
0 new messages