Should Serilog use JSON.NET as the json engine.

[Brad Phelan]

I've just started using serilog and have set up my logging the following way.

        public static Serilog.LoggerConfiguration DefaultConfiguration = new LoggerConfiguration()

            .WriteTo

            .Console(formatProvider: CultureInfo.InvariantCulture, outputTemplate: DefaultConsoleTemplate)

            .WriteTo.Sink(new FileSink(Logfile, new JsonFormatter(), null));

Now in the sink file I notice that a log entry is not really JSON. Each property is destructured or stringified and stored as a string within the properties structure of the log entry which means when de-serializing the log for analysis

So contrary to your homepage example I find the storage is not

{ "SensorInput": { "Latitude": 25, "Longitude": 134 },
  "Elapsed": 34 }

but

{ "SensorInput": "{ \"Latitude\": 25, \"Longitude\": 134 }",
  "Elapsed": 34 }

Why is destructured data stored as a stringified JSON  and not as JSON?

I notice that this is a design decision. LogEventPropertyValue is designed to convert a property to a string rather than output to a JSON writer.

I'm using the JSON.Net library for general JSON serialization and they have the class JsonWriter

http://james.newtonking.com/json/help/index.html?topic=html/T_Newtonsoft_Json_JsonWriter.htm

If Serilog was based on JSON.NET then you could redefine LogEventPropertyValue simply  as

public abstract class LogEventPropertyValue : IFormattable

{

    public abstract void Render(JsonWriter output);

}

and then the log entry could be JSON all the way down. As well you could take advantage of the superior JSON.Net serialization capability. I annotate all my serializable classes using [DataContract] and [DataMember] attributes which serilog doesn't recognise. I've worked around this by creating my own destructuring policy

    public class JSonNetDestructuringPolicy : IDestructuringPolicy

    {

        public class JsonNetLogEventPropertyValue : LogEventPropertyValue

        {

            private readonly object _Data;

            public JsonNetLogEventPropertyValue(object data)

            {

                _Data = data;

            }

            public override void Render

                (TextWriter output,

                 string format = null,

                 IFormatProvider formatProvider = null)

            {

                            // ToJson is an extension method to serialize to string

                            // using the NewtonSoft JSON.Net library 

                output.Write(_Data.ToJson(Json.NonIndentedSettings));

            }

        }

        public bool TryDestructure

            (object value,

             ILogEventPropertyValueFactory propertyValueFactory,

             out LogEventPropertyValue result)

        {

            result = new JsonNetLogEventPropertyValue(value);

            return true;

        }

    }

but because I have to render to a string (TextWriter) I get the effect that I get individual blobs of stringified json in the log.

Am I missing something here or doing something wrong?

Thanks for the library. The idea of structured logging was just what I was looking for.

Regards  

Brad

[nblum...@nblumhardt.com]

Hi Brad,

Thanks for the ideas! There's probably some interesting ground to cover around [DataContract] and so-on that we haven't explored yet.

Regarding string v.s. object storage of properties however, you'll notice that the example uses an @ symbol to prefix the SensorInput property name:

var sensorInput = new { Latitude = 25, Longitude = 134 };
var elapsedMs= 34;

log.Information("Processed {@SensorInput} in {Elapsed:000} ms.", sensorInput, elapsedMs);

This is the flag that tells Serilog to serialize the sensorInput object, instead of converting it to a string as your example shows.

We don't do this by default because in general the ToString() representation of an object is pretty "safe", while serializing something can result in a lot of data storage if large object graphs are passed through unintentionally.

If you're already using @ on your property, would you be able to post a complete code snippet so we can figure out why you're seeing this?

All the best,
Nick

[nblum...@nblumhardt.com]

Just to add a couple more notes on why things are the way they are, because you raise a couple of other interesting points.

First, on using JSON.NET for serialization - it's a great library, definitely my go-to. For the core of a logging library though we don't want to take a dependency on it, because so many libraries use JSON.NET and therefore we'd be a frequent source of dependency conflicts. Our use of JSON is fairly pedestrian too - write-only - which is much easier to do ourselves than reading would be.

Related point, making rendering work with a JsonWriter-type output rather than TextWriter - we're staying agnostic of JSON as the rendering format because in some situations other representations (e.g. XML) work better. May not be a huge priority right now, but we're in it for the long haul so we want to be ready for whatever structured storage representations get thrown at us :) We could still use JSON as an intermediate format, but since in its idiomatic form it is lossy it's not great in that role either.

Thanks again for the post!