Label as vector in BigDL 2.1.0

[Phuong LE-HONG]

Hi all,

I'm developing a sequence model which takes as input Seq[String] and produces output labels as Seq[Double] or Vector. I'd like to use the Keras-like API of BigDL on a DataFrame for this purpose.

However, it seems that the current implementation supports scalar (Float or Double) labels only if we use one "label" column. It does not support vector type as label, as the code excerpt shows. 

I'm thinking about creating an array of label columns, each storing an element of the label sequence in order to use the SeqToMultipleTensors utility. But it is not a neat solution at all. 

Is there a better way to do this?

Thanks,

Phuong

===

https://github.com/intel-analytics/BigDL/blob/v2.1.0/scala/dllib/src/main/scala/com/intel/analytics/bigdl/dllib/keras/models/Topology.scala

val preprocessing = if (labelCols.size == 1) {
  FeatureLabelPreprocessing(featurePreprocessing, ScalarToTensor())
  .asInstanceOf[Preprocessing[(Any, Option[Any]), Sample[T]]]
} else {
  FeatureLabelsPreprocessing(featurePreprocessing, SeqToMultipleTensors(labelSizes))
  .asInstanceOf[Preprocessing[(Any, Option[Any]), Sample[T]]]
}

===

[Phuong LE-HONG]

Well, after digging into source code packages, I find that the NNEstimator is the best way to go. This is able to let user specify label shape. 

Phuong

[Jason Dai]

Yes - the preferred approach for DLlib is to use Keras-like API to build the model, and use NNFrame (https://bigdl.readthedocs.io/en/latest/doc/DLlib/Overview/nnframes.html) to build the ML pipeline.

Thanks,

-Jason