Plugging in a custom JsonConverter

[Dawid Ciecierski]

We're trying to serialize some message and would like to customize the way it is sent over the bus. We set MassTransit to UseJsonSerializer and decorated the class in question with JsonConverterAttribute like so:

[JsonConverter(typeof(TestConverter))]
public class SomeMessageType
{
    public string Id { get; set; }
    public string Name { get; set; }
}

public class TestConverter : JsonConverter
{
    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        var message = (SomeMessageType)value;
        writer.WriteStartObject();
        writer.WritePropertyName("Id");
        writer.WriteValue(message.Id);
        writer.WriteEndObject();
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        return serializer.Deserialize(reader, objectType);
    }

    public override bool CanConvert(Type objectType)
    {
        return true;
    }
}

However, our TestConverter is not used by MassTransit, even thou when the same thing is serialized by MVC (using Json.Net also) milliseconds later, the attribute is corretly found and TestConverter is used for serialization.

Is there any specific reason as to why MT might not allow what we're trying to achieve?

Regards,
Dawid Ciecierski

[Dru Sellers]

No idea. Can you give us a failing unit test?

-d

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To post to this group, send email to masstrans...@googlegroups.com.
To unsubscribe from this group, send email to masstransit-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/masstransit-discuss/-/v09yne2pdPcJ.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Travis Smith]

It's possible because of the way we set the Converters, that the
serializer won't use attributes on a type. I would have to dig into
the Json.NET source to see if that's the case or not. I don't think
about it much since JSON serialization normally just works :)

https://github.com/MassTransit/MassTransit/blob/master/src/MassTransit/Serialization/JsonMessageSerializer.cs

-Travis

[Chris Patterson]

Probably because we internalize Newtonsoft in the MT assembly. With the next release, it's likely that all of the ilmerge code will be gone and NuGet packages used instead - but that's a big if due to version issues with JSON.NET (it breaks constantly with each release).

[Dawid Ciecierski]

ILmerge is indeed the right culprit. For a while I thought it might be that MT references an old version of Json.net which is bugged when it comes to converters; and so I built MT's latest from the development branch against Json.net 4.5.8 to see if that would improve things. My code, however, behaved the same (incorrect) way.

I then build latest against Json.net 4.5.8 without internalizing Newtonsoft.Json.dll (and Stact.dll), and voila, it works! My converter is called and is a happy bunny.

From my naïve point of view, referencing Json.net off nuget should not be a major pain. I for one did not have to change a thing when swapping 4.0.5 for 4.5.8; plus, with nuget you should be able to specify dependencies more precisely. But until you decide to rely on Nuget I will be rolling my own versions of MassTransit every release :-)

Thanks for your help!
Dawid

>> masstransit-discuss+unsub...@googlegroups.com.

>> To view this discussion on the web visit
>> https://groups.google.com/d/msg/masstransit-discuss/-/v09yne2pdPcJ.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "masstransit-discuss" group.
> To post to this group, send email to masstrans...@googlegroups.com.
> To unsubscribe from this group, send email to

> masstransit-discuss+unsub...@googlegroups.com.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To post to this group, send email to masstrans...@googlegroups.com.

To unsubscribe from this group, send email to masstransit-discuss+unsub...@googlegroups.com.

[Dru Sellers]

Using JSON.Net as a non-ilmerged .dll is a huge PITA because of how JSON.Net does versioning combined with how .Net loads dlls. The API is usually just fine. The reason is JSON.net is so friggin' cool that everyone uses it. Let's say you have another OSS project that uses it like FubuMVC. If you try to get the two current released versions of these projects to run together you might end up with a bad time as MT wants JSON.Net 4.0.5 and FubuMVC might want 4.5.8. Due to the way .Net assembly loading works your are going to get an assembly load exception, this then forces you to do things like Assembly Redirects :(

So while it is technically very easy to use Nuget, in practice JSON.net much like log4net can prove to be a sticky wicket.

@Chris, maybe we need serializer dlls? Ugh.

-d

To unsubscribe from this group, send email to masstransit-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/masstransit-discuss/-/c06kjj5JFb0J.

[Travis Smith]

Or maybe we just need "use binding redirects" loud everywhere? What a pain. Maybe we could just build and release a non-signed josn.net version?

-Travis

[Dawid Ciecierski]

I don't neccessarily see assembly redirects as Evil™. Json.net also tries to limit any mayhem that could ensue upon every update by specifying assembly version to be "4.5.0.0" for all 4.5.* releases. Just my two cents, I'm sure you have a much broader view on that than I do!

Coming back to the main subject of this thread - it turns out I was prematurely happy. What I got working was property converters, which are fine once I severed the close MT-Json.net relationship by disabling ILmerge.

What still does not fully work upon is JsonConverterAttribute applied to the message class, like so:

[JsonConverter(typeof(TestClassConverter))] // does not work

public class SomeMessageType
{
    public string Id { get; set; }

    [JsonConverter(typeof(TestPropertyConverter))] // works

    public string Name { get; set; }
}

The class converter is picked up when serialization takes place, but is not when deserializing the message on the other end. After a quick code skim it seems that JsonMessageTypeConverter could be the culprit. In its bool TryConvert<T>(out T message) method it does the following:

obj = FastActivator<T>.Create();
UsingReader(jsonReader => _serializer.Populate(jsonReader, obj));

This effectively bypasses Json.net mechanisms for object creation and only uses them for populating the properties of the object in question. (This seems to be the reason why converters work for properties.) As a dumb workaround I tried to do the following:

obj = null;
UsingReader(jsonReader => { obj = _serializer.Deserialize(jsonReader, typeof(T)); });

It worked the first time, but on the second run I got a stack overflow :-] So... any suggestions on whether it might be possible with current MT workflow or should I stop trying to nail that IMO elegant solution?

-Dawid

-d

>> masstransit-discuss+unsubscribe...@googlegroups.com.

>> To view this discussion on the web visit
>> https://groups.google.com/d/msg/masstransit-discuss/-/v09yne2pdPcJ.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "masstransit-discuss" group.

> To post to this group, send email to masstrans...@googlegroups.com.

> To unsubscribe from this group, send email to

> masstransit-discuss+unsubscribe...@googlegroups.com.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To post to this group, send email to masstrans...@googlegroups.com.

To unsubscribe from this group, send email to masstransit-discuss+unsubscribe...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To post to this group, send email to masstrans...@googlegroups.com.
To unsubscribe from this group, send email to masstransit-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/masstransit-discuss/-/c06kjj5JFb0J.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To post to this group, send email to masstrans...@googlegroups.com.

To unsubscribe from this group, send email to masstransit-discuss+unsub...@googlegroups.com.

[Dru Sellers]

Let's start w/ a few tests against the JSON serializer that demonstrates the error. 

- if you do that in a pull request, + your fix in a separate commit - we can start to put together something that might actually work for ya. :)

-d

To unsubscribe from this group, send email to masstransit-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/masstransit-discuss/-/mlc9nkgkVgwJ.

[Dawid Ciecierski]

Done, pull request just submitted. Have a look and let me know if tests were added in the right places, it's my first time with MassTransit codebase other than stepping through code.

The "fix" I mentioned is more of a shot-in-the-dark. Don't know MT's architecture well enough to judge if Json.net deserializer should simply be plugged in instead of FastActivator (which I'm sure you had good reason to come up with in the first place). And if so, for what scenarios and at what stage in MT's message pipeline.

Will be more than happy to do some more checking if pointed in the right direction thou.

Regards,
Dawid

-d

-d

To unsubscribe from this group, send email to masstransit-discuss+unsubscribe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/masstransit-discuss/-/c06kjj5JFb0J.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To post to this group, send email to masstrans...@googlegroups.com.
To unsubscribe from this group, send email to masstransit-discuss+unsubscribe...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To post to this group, send email to masstrans...@googlegroups.com.
To unsubscribe from this group, send email to masstransit-discuss+unsub...@googlegroups.com.

[Dawid Ciecierski]

Came across the converter issue for the second time in just a few days :-) This time we tried to make our domain "message-bus-ingorant", and have MT automatically apply a particular deserialization strategy (ie. SomeJsonConverter) via the Converters property of serializer settings.

We moved settings to a static property on JsonMessageSerializer (so that it can be shared between tread-static [de]serializers), then exposed those via callbacks on ServiceBusConfigurator like so:

public static T UseJsonSerializer<T>(this T configurator,
    Action<JsonSerializerSettings> serializerConfigurator = null,
    Action<JsonSerializerSettings> deserializerConfigurator = null)
    where T : EndpointFactoryConfigurator
{
    configurator.SetDefaultSerializer<JsonMessageSerializer>();
    if (serializerConfigurator != null)
        serializerConfigurator(JsonMessageSerializer.SerializerSettings);
    if (deserializerConfigurator != null)
        deserializerConfigurator(JsonMessageSerializer.DeserializerSettings);

    return configurator;
}

(Admittedly a little hacky but just testing the waters.) We then added our custom JsonConverter to the Converters collection on Deserializer. It does get added, but for the same reasons as before, the converter is not called. The Envelope's Message property is an object with no type information attached to it directly, and gets deserialized without type information. A little later JsonMessageTypeConverter picks up and simply populates the required properties.

I had a thought that perhaps we could make use of the $type property Json.NET can attach to serialized messages? But I guess that $type information emmision would have to be turned on just for the Message property in order not to blow message sizes too much... Not sure if such custom serialization of Envelope can be done neatly.

Any thoughts on this? Or should we just stop right there before this gets too crazy?

Regards,
Dawid