TypedPipe to matrix

14 views
Skip to first unread message

Nikhil J Joshi

unread,
Nov 24, 2016, 4:19:12 AM11/24/16
to Scalding Development
Hi,

I have a library (XGBoost) that is optimized for matrix manipulation and hence I need to port my input (read from avro) `TypedPipe` to a matrix. I am able to map each avro record to an array of floats. But I don't know what to write as the fields in `toMatrix` arguments List

I am doing something like

private val inputData = TypedPipe.from(PackedAvroSource[SomeAvroRecord](args("inputData")))
.map(avroToFloatArray)  
.toMatrix(....

Any help would be great.
Thanks,
Nikhil
Reply all
Reply to author
Forward
0 new messages