UDT Mapping does not user TypConverters

[Jonathan Nash]

I've run into an issue attempting to find a driver that will work for C# with ScyllaDb. When creating UDTs, it appears that the custom type converters are not used. This prevents UDTs that have custom types from being mapped correctly by the mapper. Is there something I am missing?

Full code snippet below:

=====================

/*
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <RootNamespace>CassandraDriverSample</RootNamespace>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="CassandraCSharpDriver" Version="3.19.3" />
  </ItemGroup>

</Project>
*/

using Cassandra;
using Cassandra.Data.Linq;
using Cassandra.Mapping;
using Cassandra.Mapping.TypeConversion;

MappingConfiguration.Global.ConvertTypesUsing(new DateOnlyTypeConverter());

MappingConfiguration.Global.Define(
    new Map<DbTimestamp>()
        .Column(t => t.Id)
        .Column(t => t.Date, mp => mp.WithDbType<LocalDate>())
        .Column(t => t.MetaData, mp => mp.WithDbType<DbTimestampMetaData>().AsFrozen())
        .PartitionKey(t => t.Id)
    );

MappingConfiguration.Global.Define(
    new Map<DbTimestampMetaData>()
        .Column(t => t.Id)
        .Column(t => t.Date, mp => mp.WithDbType<LocalDate>())
    );

var cluster = Cluster.Builder()
    .AddContactPoint("127.0.0.1")
    .WithPort(9042)
    .WithDefaultKeyspace("fail_v4")
    .Build();

var session = cluster.ConnectAndCreateDefaultKeyspaceIfNotExists();

const string timestampMetaUdtName = "timestamp_meta_udt";

await session.ExecuteAsync(new SimpleStatement($"create type if not exists {timestampMetaUdtName} (id text, date date)"));

await session.UserDefinedTypes.DefineAsync(
    UdtMap.For<DbTimestampMetaData>(udtName: timestampMetaUdtName)
        .Map(udt => udt.Id, "id")
        .Map(udt => udt.Date, "date")
);

var table = new Table<DbTimestamp>(session);
await table.CreateIfNotExistsAsync();

var mapper = new Mapper(session);

var timestamp = DbTimestamp.Now();
await mapper.InsertAsync(timestamp);

foreach (var timestampEntry in await mapper.FetchAsync<DbTimestamp>())
{
    Console.WriteLine(timestampEntry);
}

public record class DbTimestamp
{
    public string Id { get; set; }
    public DateOnly Date { get; set; } // works here as part of a table with a converter
    public DbTimestampMetaData MetaData { get; set; }

    public static DbTimestamp Now() => new()
    {
        Id = Guid.NewGuid().ToString(),
        Date = DateOnly.FromDateTime(DateTime.Now),
        MetaData = DbTimestampMetaData.Current()
    };
}

public record class DbTimestampMetaData
{
    public string Id { get; set; }
    public DateOnly Date { get; set; } // fails here. can't find a converter.

    public static DbTimestampMetaData Current() => new()
    {
        Id = Guid.NewGuid().ToString(),
        Date = DateOnly.FromDateTime(DateTime.Now),
    };
}

public class DateOnlyTypeConverter : TypeConverter
{
    public static LocalDate ConvertCSharpToDbType(DateOnly value)
    {
        return new LocalDate(value.Year, value.Month, value.Day);
    }

    public static DateOnly ConvertDbTypeToCSharp(LocalDate value)
    {
        return new DateOnly(value.Year, value.Month, value.Day);
    }

    protected override Func<TDatabase, TPoco> GetUserDefinedFromDbConverter<TDatabase, TPoco>()
    {
        if (typeof(TDatabase) == typeof(LocalDate) && typeof(TPoco) == typeof(DateOnly))
        {
            return databaseValue => (TPoco)(object)ConvertDbTypeToCSharp((LocalDate)(object)databaseValue);
        }

        return null!;
    }

    protected override Func<TPoco, TDatabase> GetUserDefinedToDbConverter<TPoco, TDatabase>()
    {
        if (typeof(TPoco) == typeof(DateOnly) && typeof(TDatabase) == typeof(LocalDate))
        {
            return pocoValue => (TDatabase)(object)ConvertCSharpToDbType((DateOnly)(object)pocoValue);
        }

        return null!;
    }
}

=====================

The actual UDT and table creations seem to be on point.

=====================

cqlsh:fail_v4> describe fail_v4;

CREATE KEYSPACE fail_v4 WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'}  AND durable_writes = true;

CREATE TYPE fail_v4.timestamp_meta_udt (
    id text,
    date date
);

CREATE TABLE fail_v4.dbtimestamp (
    id text PRIMARY KEY,
    date date,
    metadata frozen<timestamp_meta_udt>
) WITH bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
    AND comment = ''
    AND compaction = {'class': 'SizeTieredCompactionStrategy'}
    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.0
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99.0PERCENTILE';

=====================

[Joao Reis]

Hi,

Thanks for the detailed report with a complete reproducible example. I was able to reproduce this locally.

The default UDTSerializer uses the static TypeConverter property of the UdtMap class which is initialized to the default type converter and there is no way to provide a custom type converter there. Your best shot is to try and provide a custom serializer for UDTs that bypasses the UdtMap's type converter (I think it has to be a subclass of UdtSerializer by the way). Maybe a custom UdtMap subclass + custom UdtSerializer subclass will do the trick but I've never actually tried doing this so I'm not sure if you're going to run into other problems.

I've created 2 new JIRA issues related to this:

• CSHARP-1002
• CSHARP-1001

It's unlikely we will prioritize these 2 issues in the immediate future but we're always open to contributions from the community.

Best regards,

	João Reis   w. www.datastax.com

--
You received this message because you are subscribed to the Google Groups "DataStax C# Driver for Apache Cassandra User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to csharp-driver-u...@lists.datastax.com.
To view this discussion on the web visit https://groups.google.com/a/lists.datastax.com/d/msgid/csharp-driver-user/b6f2ab1c-24e7-4791-9c96-eea9a6e025e2n%40lists.datastax.com.

[Jonathan Nash]

That is very disheartening. Working with this driver has been a painstaking task and has become a pretty big roadblock on migrating from Mongo. Being able to store just a single nested UDT seems like it would be a core requirement for basic functionality. 

I appreciate opening the issues and being honest on the priority. 

You received this message because you are subscribed to a topic in the Google Groups "DataStax C# Driver for Apache Cassandra User Mailing List" group.
To unsubscribe from this topic, visit https://groups.google.com/a/lists.datastax.com/d/topic/csharp-driver-user/E6GwYO1bAdA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to csharp-driver-u...@lists.datastax.com.
To view this discussion on the web visit https://groups.google.com/a/lists.datastax.com/d/msgid/csharp-driver-user/CAExBwLZDuXt1JFEYqEEJvZXhrWT608Kt5m9ag-38TNViqy3Ejg%40mail.gmail.com.

[Joao Reis]

The issue is not caused by nested UDTs, it's caused by using UDTs with .NET types that the driver doesn't natively support. If you use LocalDate instead of DateOnly then it will work fine.

	João Reis   w. www.datastax.com

To view this discussion on the web visit https://groups.google.com/a/lists.datastax.com/d/msgid/csharp-driver-user/CALzUG3nQKFpdgRavexvs8aBUS9zUyM7dPQyU1D_nTiQh75nW2A%40mail.gmail.com.

[Jonathan Nash]

Sure, but then our models that are used in a wide variety of other apps and services now are dependent upon Cassandra types. That is not how things should work. UDTs should be mappable to a converter and respected during the serialization process. 

To view this discussion on the web visit https://groups.google.com/a/lists.datastax.com/d/msgid/csharp-driver-user/CAExBwLbQm79SzR9xM6KDqr%3D3%2BvzZTnpkLfRqhoAyuZZ89EL-rw%40mail.gmail.com.

[Patrick]

Ideally you would not share the database type outside of the data layer.

To view this discussion on the web visit https://groups.google.com/a/lists.datastax.com/d/msgid/csharp-driver-user/CALzUG3nnsTB%2Bq%3DiafhUH0ywv%3DVzoHzbq3m7wMgD6-LJgEpLkRQ%40mail.gmail.com.