REQUEST: Allow Raven.Client to use MY version of Newtonsoft

[Frank Radocaj]

Ok, I know this is controversial, but i'm forever battling Raven.Imports.Newtonsoft against my own referenced version of Newtonsoft in my projects.

What this means is that when I query Raven, I have to "unpack" the result from the query into my own object, and then move to the next task. e.g.:

What I have to do currently in an ASP.Net MVC website:

1. Create a POCO class that is the shape of my desired "view model" result from my MVC action <== Massive pain in the arse

2. Inside my MVC action: Query Raven using docSession.Query<MyPoco>(...)

3. Return result

What I WANT to be able to do:

1. Just ONCE in my own project: Configure Raven.Client (note: NOT the server) to use Newtonsoft version of my choice (e.g. 6.0)

2. Inside my MVC action: Query Raven using docSession.Query<dynamic>(...) <=== Note the use of dynamic!

3. Return result

 This is a big friction point in development, and Raven would be sweet as if it could do this.

Please don't with respond:

- What's wrong with POCOs? The point is I don't wanna create POCOs that are just for sending out of MVC actions;

- Just use Raven.Imports.Newtonsoft in your own project. WRONG. Lots of libs require MUCH newer versions of Newtonsoft, so this won't work.

Frank

[Tom Allard]

It's not (always) necessary to make a projection class for each controller operation. I only create projection classes if there is a huge gap between the amount of data in the document stored in the database and the data that should be returned.

So, most of the time, you can just use the main entity to query, and use anonymous types to return the view model data to the client. For example (This is ASP.NET Web Api): 

 

        [Route("find")]

        [HttpGet]

        public IHttpActionResult FindPublic(string name)

        {

            if (string.IsNullOrWhiteSpace(name) || name.Length <= 3)

                return Ok(new String[0]);

            var entities =

                Session.Query<Entity, Entities_DynamicGroup>().Where(x => x.IsClient).Where(x => x.Name.StartsWith(name)).Take(5).As<Entity>().ToList();

            return Ok(entities.Select(x => new { x.Name, Office = x.DefaultOffice, x.Id }).ToList());

        }

Entity is a class with many more properties, but the view only needs Name & Office.

-Tom

[Chris Marisic]

Unlikely to be possible.

Perhaps with ASP.NET vNext and the termination of assemblies.

[Kijana Woodard]

I guess I'm unclear as to what you want to achieve and how json.net fits in.

You want to query a document, but use json.net to shape the output of the document?

You want to put json.net attributes on the document to manipulate the output?

IIRC, you can do that by fully qualifying the attributes with "your version of newtonsoft". I thought I've seen people using "raven newtonsoft" to shape what gets saved in raven _and_ use "whatever newtonsoft" to shape what mvc serializes. I think I remember two different attributes [one from each lib version] on a single property. Either fully qualify the namespace or create an alias.

Fwiw, I've tried that path and didn't like it.

To me, it's more work up front and less discoverable by maintenance devs than declaring a class to shape the output. You can then use OfType or write a transformer to coerce the document into the right shape, if hand mapping is unappealing.

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Frank Radocaj]

The size of the project i'm working on means we've ended up using transformers for most queries that end up being re-used in multiple places. I agree with your approach for one-off queries. 

The problem is that when you use transformers, the version of Newtonsoft i've referenced in my web project (e.g. 6.0) knows nothing about Raven's version of Newtonsoft. Let's say I have a user:

class User {
    string Id { get; set; }
    string Name { get; set; }
}

So, when my 6.0 version serialises the *results* of the transformer query I get this JSON:

{
    {
        Key: "Id",
        Value: "users/348576"
    },
    {
        Key: "Name",
        Value: "Susie"
    }
}

Instead of:

{
    Id: "users/348576"
    Name: "Susie"
}

Its Raven internal implementation bubbling to the surface and causing unexpected issues.

Frank

[Frank Radocaj]

See response to Tom Allard.

I've never tried putting the attributes of *both* Newtonsofts on my docs. I guess I could try that, but I don't see how that is going to work. They are effectively two different assemblies that know nothing of each other. Hence, my Newtonsoft 6.0 serialises this:

class User {
 string Id { get; set; }
 string Name { get; set; }
}

Into this crazy JSON representation:

{
  {
     Key: "Id",
     Value: "users/348576"
  },
  {
     Key: "Name",
     Value: "Susie"
  }
}

> To me, it's more work up front and less discoverable by maintenance devs than declaring a class to shape the output.

This is not the case for me. The vast majority of the time, my queries involve indexes, and subsequently, transformers. So, my dynamically anonymous-typed "shaped output" is nicely wrapped in a transformer class that explicitly describes intent. So, my solution skips the useless step of creating superfluous view model shaped classes. Let the transformer massage the data straight out into the shape I require.

Frank

[Frank Radocaj]

They're terminating assemblies in vNext?

I'm assuming you mean "assembly neutral types". Maybe, but it still requires participation from the library author I presume?

Frank

[Kijana Woodard]

"The vast majority of the time, my queries involve indexes, and subsequently, transformers. "

Oh. I assumed you were dealing with "document classes" that were annotated with json.net attributes. If you already have a transformer result class, yes, it would be annoying to copy that to "something else". In mose cases, I would expect the transformer result class to be the "view model".

Why does that not work? How is Raven's json.net interfering with mvc serializing the class. The transformer result class is a poco defined in your code. For the raven server, it's not a c# class at all.

I think something else is going on here. Can you show some code? A repro?

What does the transformer class look like [including usings]?

Are you modifying serialization for MVC? If so, what does that look like? [I generally modify the settings for datetime serialization.]

Fwiw, I've been inundated with web api projects lately, but I can't imagine why mvc would work differently in this regard.

[Frank Radocaj]

Oh. I assumed you were dealing with "document classes" that were annotated with json.net attributes. 

Sorry - I didn't clarify in my original question that this was output from transformers.

In mose cases, I would expect the transformer result class to be the "view model".

Yes, so the issue is that I don't event want to create a view model class for the result transformer.  I want to return an anonymously typed object (as dynamic) straight back out from my web api:

return session.Query<Order>()
 .TransformWith<OrderStatisticsTransformer, dynamic>()
 .Where(x => x.CustomerId == "customers/1")
 .ToList();

And here in lies the problem. In my web api project (which uses NancyFx instead of ASP.Net Web API, but that's beside the point), I use Newtonsoft v6.0 as my JSON serialiser. Newtonsoft 6.0 treats my Raven.Imports.Newtonsoft dynamic anonymous type (created inside the transformer) as a dictionary and outputs:

{
  {
     Key: "Id",
     Value: "users/348576"
  },
  {
     Key: "Name",
     Value: "Susie"
  }
}

Frank

[Oren Eini (Ayende Rahien)]

How are you querying RavenDB?

This is the raw HTTP result here, no?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Oren Eini (Ayende Rahien)]

Note that this has nothing to do with JSON.Net, the transformers code path knows how to work with such shapes, that is all.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Oren Eini (Ayende Rahien)]

That isn't actually what happens.

Now I know what is going on.

Basically, you asked for dynamic. When you do that, what we actually return back is a RavenJObject item.

This isn't serialized properly by JSON 6.0 that isn't surprising, we have a converter for that in our code.

Basically, you need to register a JsonConverter in your JSON 6.0 configuration,and it would work.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Chris Marisic]

No, they are eliminating assemblies, vNext is an interpreted language. (technically assemblies will exist in memory after they are compiled JIT by rosyln)

http://www.hanselman.com/blog/content/binary/Windows-Live-Writer/Introducing-ASP.NET-vNext_14215/image_15.png

There is only a single DLL in the bin folder, no other assembiles.

http://www.hanselman.com/blog/IntroducingASPNETVNext.aspx

[Kijana Woodard]

IIRC, you can build assemblies for deployment.

By default, they won't write to disk during development. But I think you can override that behavior. Not sure why you would want to.

[Chris Marisic]

The only thing I could think of is people who shipped boxed software that want to ship encrypted/obfuscated code

[Kijana Woodard]

Oh. I can see having assemblies instead of source code for qa/prod. 

I don't know why you would care on your dev machine.

[Michael Carter]

As Oren stats, you need to create a JsonConverter.

Here's what I use:

    using System;
    using Raven.Imports.Newtonsoft.Json;
    using Raven.Json.Linq;

    public class RavenJObjectConverter : JsonConverter
    {
        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            var json = ((RavenJObject)value).ToString(serializer.Formatting);
            writer.WriteRawValue(json);
        }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            if (reader.TokenType == JsonToken.Null)
            {
                return null;
            }

            // Load JObject from stream
            var jObject = RavenJObject.Load(reader);

            return jObject;
        }

        public override bool CanConvert(Type objectType)
        {
            return objectType == typeof(RavenJObject);
        }
    }

I then subclass the JsonResult to inject the JsonConverter

    public class MyJsonResult : JsonResult
    {
        public override void ExecuteResult(ControllerContext context)
        {
            if (context == null)
            {
                throw new ArgumentNullException("context");
            }
            var response = context.HttpContext.Response;
            if (!string.IsNullOrEmpty(ContentType))
            {
                response.ContentType = ContentType;
            }
            else
            {
                response.ContentType = "application/json";
            }
            if (ContentEncoding != null)
            {
                response.ContentEncoding = ContentEncoding;
            }
            if (Data != null)
            {
                var json = JsonConvert.SerializeObject(Data,
                    new Raven.Imports.Newtonsoft.Json.Converters.IsoDateTimeConverter(),
                    new RavenJObjectConverter());
                response.Write(json);
            }
        }
    }

What's nice is that my RavenJObjectConverter also supports deserializing Json to RavenJObjects. 

I have some models where I need custom properties. I then embed a RavenJObject in my class like this:

    public class Job
    {
        public string Id { get; set; }
        public string Status { get; set; }
        //... other properties ...

        public RavenJObject Json { get; set; }
 }

I can then deserialize that like this;

var job = JsonConvert.DeserializeObject<Job>(json, new RavenJObjectConverter());

Hope this helps!

Kiliman

[Chris Marisic]

Assemblies are slower than source code, that's why they're ditching them.

[Oren Eini (Ayende Rahien)]

No, they aren't

[Kijana Woodard]

I think they are ditching them for dev because memory is faster than disk.

[Federico Lois]

In Roslyn they have the entire semantic tree already in memory. Emiting the assemblies in-memory is far faster than hit the disk because you already are accessing the last mile "compiling wise". So IMHO it makes sense for development to just go with it. So IMHO it is not that they are ditching the assemblies, they are just emitting them in a faster storage. I dont know the internals of Roslyn that well but I presume they may be able to just "patch" a method and go on (a la smalltalk). 

Having said that, assemblies will still be present, just not be seen as physical artifacts if they dont need to (runtime, core assemblies, etc) enable side-by-side deployment. For deployment, user assemblies will still exist, noone in their right mind should deploy via source (even if possible).

Just my 2 cents.

 

[Oren Eini (Ayende Rahien)]

Unless they made significant changes to the actual .NET platform (unlikely), they are basically recompiling the whole assembly.

Since in ASP.Net, a lot of the time assemblies are small, that is very small cost.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Chris Marisic]

On Friday, October 31, 2014 6:25:50 PM UTC-4, Federico Lois wrote:

, noone in their right mind should deploy via source (even if possible).

my 2 cents.

I completely disagree. Source only is the way to go. Assemblies don't offer any kinds of protections.  Even the best of the .NET obsfuctators/protectors  are trivial for skilled crackers to bypass

[Kijana Woodard]

It's not about hackers per se. It's about having verified assemblies that move to each environment. FWIW, when I setup a build, it gets built _once_. The same assemblies are deployed in each environment. I don't understand the idea of "we're done qa testing, now lets cut the release build". Undercuts everything you've just done.

You could do the same with source obviously. It's just more moving parts. It's also more [too] tempting for someone to edit the source on the server "just this once for this emergency". PHP, here we come.

In a disciplined environment, I could probably be convinced to deploy source. It would make for smaller diffs for the robocopy. But I don't see that becoming the industry standard anytime soon.

--

[Chris Marisic]

This concern is all mitigated by not allowing access to production servers. Especially using methodologies like commit to deploy, no one is allowed to touch the servers because any change would be overwritten anyway.

This argument is fundamentally no different than do we deploy this 1 DLL we changed or do we deploy every single file we didn't change AND the one we did?

I also can't say I've never updated a view in production in the past, but after commit to deploy i never once did. Commit to deploy is one of the things you can do to instantly create discipline. Anyone who wants to take shortcuts the last thing they want is to have to do stuff again because it was overwritten.

[Matt Warren]

You can do incremental-parsing in Roslyn, but Compilation is re-done everytime, see https://roslyn.codeplex.com/discussions/569318#post1315835

[Frank Radocaj]

> Basically, you need to register a JsonConverter in your JSON 6.0 configuration,and it would work.

Yep, I understand that i'm getting back a RavenJObject.

But your solution is ridiculously unworkable. I would need to remap every single type returned from Raven in my JsonConverter. My project would become 50% just remapping code. Forget it.

Frank

[Frank Radocaj]

Sorry, have been busy and didn't get a chance to read the responses. 

Michael, this is a great solution! 

One question - does your RavenJObjectConverter subclass from Raven's imported JsonConverter, or some other referenced version of Json.Net?

Thanks for explaining your approach, i'm definitely going to use it.

Frank

[Oren Eini (Ayende Rahien)]

No, you would need the  handle ravenjobject and ravenjarray, nothing else