Collection of collections: anyone have an example (UDT)

[Hugo Ferreira]

Hello,

Can anyone point me to an example where a column is of a type collection of collections. More specifiably I nee a Map[String, Map[String,String] ].

I have been able to create the column family and UDT type via CQL.
I am also able to insert stuff using CQL. See the statements at the end of
the message. I seem to be able to access the data via:

row.getMap[String, UDTValue]("dummies")

where row is a CassandraRow. In the insertion I have:

val xdummies : Map[String, UDTValue] = ndummies.map{ case (k,v) => (k,UDTValue.UDTValueConverter.convert(v)) }

where:

'ndummies' is of the type Map[String, Map[String, String]]

But I get an error:

com.datastax.spark.connector.types.TypeConversionException: Cannot convert object Map(A-038/11 -> x_73_1, A-037/11 -> x_73_2, A-040/11 -> x_73_3) of type class scala.collection.immutable.Map$Map3 to com.datastax.spark.connector.UDTValue.
[info] at com.datastax.spark.connector.types.TypeConverter$$anonfun$convert$1.apply(TypeConverter.scala:42)
[info] at com.datastax.spark.connector.types.TypeConverter$$anonfun$convert$1.apply(TypeConverter.scala:40)
[info] at com.datastax.spark.connector.UDTValue$UDTValueConverter$$anonfun$convertPF$1.applyOrElse(UDTValue.scala:29)
[info] at com.datastax.spark.connector.types.TypeConverter$class.convert(TypeConverter.scala:40)
[info] at com.datastax.spark.connector.UDTValue$UDTValueConverter$.com$datastax$spark$connector$types$NullableTypeConverter$$super$convert(UDTValue.scala:27)

Appreciate any help.

TIA,
HF


val q2a = s"""CREATE TYPE ${sc.keyspace}.${sc.column_family_models}_dummy (vars MAP<TEXT, TEXT>)"""
val q2 = s"""CREATE TABLE IF NOT EXISTS ${sc.keyspace}.${sc.column_family_models}
($iID UUID, type VARCHAR, cpu BIGINT,
min DOUBLE, max DOUBLE, sum DOUBLE, count BIGINT, sum_squared_residuals DOUBLE,
params MAP<TEXT,DOUBLE>,
$iDUMMIES MAP<TEXT, FROZEN< ${sc.column_family_models}_dummy> >,
$iSCALER BLOB,
${sc.independent_vars.mkString(",")} ,
${sc.dependent_vars.mkString(",")},
PRIMARY KEY (id, type) )"""

val istmt = s"""INSERT INTO ${sc.keyspace}.${sc.column_family_models}
(id, type, count, dummies, $colst)
VALUES ($id, 'regression', 0, {'x_73' : { vars : {'a' : 'x_73_1' }} }, $colsv)"""

[Piotr Kołaczkowski]

Use UDTValue.fromMap.

PIOTR KOŁACZKOWSKI

Lead Software Engineer, DSE Analytics | pkol...@datastax.com

To unsubscribe from this group and stop receiving emails from it, send an email to spark-connector-...@lists.datastax.com.

[Hugo Ferreira]

Hi Piotr,

Had already tried that but unfortunately it did not work.
It complained that the keys of the map were not valid column names (or something to the effect).

I have finally "solved" the problem using simple embedded types (no UDTs
because I found a report that these were working for the Java driver).
So for anyone that might need this, the following changes are required:

1. CQL create with a column of
... dummies MAP<TEXT, FROZEN<MAP<TEXT, TEXT> >>,
2. Getting the data from the CassandraRow
row.getMap[String, Map[String, String]]("dummies")
3. Saving data
val ndummies: Map[String, Map[String, String]]
val updates = Map(
.... ("dummies" -> ndummies),....)
CassandraRow.fromMap(updates)
and then save the CassandraRow via the RDD interface

Thanks for the help Piotr.

HTS,
HF

On Tuesday, 31 March 2015 09:03:14 UTC+1, Piotr Kołaczkowski wrote:
> Use UDTValue.fromMap.

>
>
>
>
>
>
>
>
>
> PIOTR KOŁACZKOWSKI
> Lead Software Engineer, DSE Analytics | pkol...@datastax.com
>
>
>
>