Please add support for IETF "application/json-patch" media type

[Gary Weaver]

Mentioned this in the another thread, but thought would be cleaner to start a new request thread for discussion about this spec:
http://tools.ietf.org/html/draft-ietf-appsawg-json-patch-03

[Aaron Patterson]

I'm not sure Rails needs to do anything specific to support this mime
type. We *may* need to update a list of mime type mappings or
something, but I'm not sure.

Maybe someone can investigate?

--
Aaron Patterson
http://tenderlovemaking.com/

[Prem Sichanugrist]

I'd assume that this is something that's new, and totally different from the way we implement REST resource endpoints. So, I'd assume that we don't need that for now.

By the way, these 'please add support for xxx' request usually just go to trash faster than you thought. I'd suggest to make a topic asking whether the core team interest in this kind of feature or not, then you implement it and send in a pull request. Also, for something that's totally new like this, it might even worth to make a gem for it first, then I'm pretty sure the core team will pull it in if they found out that it's useful.

Thanks,

- Prem

[Steve Klabnik]

I'm interested in supporting this in some fashion, but like Aaron
said, I don't think that it actually needs anything special from Rails
itself. It may expose bugs in our MIME implementation, though, but
other than that, I'm not sure what Rails would actually need.

[Gary Weaver]

On Wednesday, September 5, 2012 7:36:58 PM UTC-4, Prem Sichanugrist wrote:

I'd assume that this is something that's new, and totally different from the way we implement REST resource endpoints. So, I'd assume that we don't need that for now.

PATCH was added to the methods accepted already, so even if PATCH-ing isn't RESTful, someone built the diving board to this pool, water in it or not.
 

By the way, these 'please add support for xxx' request usually just go to trash faster than you thought. I'd suggest to make a topic asking whether the core team interest in this kind of feature or not, then you implement it and send in a pull request.

That was the intent, and I did here where Nick Sutterer expressed interest in talking with Steve Klabnik about it: https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-core/UcJfYEzK3MY
 

Also, for something that's totally new like this, it might even worth to make a gem for it first, then I'm pretty sure the core team will pull it in if they found out that it's useful.

Ok, thanks! Aaron Patterson just posted the start of one in another thread: https://github.com/tenderlove/hana
 
So, here is the argument for further consideration of this request, even if it is just for advice or opinions about how supporting JSON patch in Rails would be something the core team agrees with:

Rails contains ActiveRecord, and ActiveRecord supports updates via deserialization of JSON into models. Whether or not PATCH is RESTful or not, PATCH was added as a supported Rails HTTP method. This may not have made sense, because PATCH is not necessarily RESTful, even though you could patch in a RESTful way. Implementing support for JSON patching in Rails could mean that specific columns in specific records could be updated in the DB. This is a massive change to really take advantage of it. Instead of just sending request params with data for updates from forms, you'd want to send JSON, but to support graceful degradation, those forms should by default use the old Rails behavior to just post request params and the form would be overriden and completely changed to use Javascript for JSON patch behavior in such a way that the client knows if the field was changed from the original value passed to it (not the current value in the DB) to determine whether the field should be included in the patch. That is not just a patching gem, that is a Rails overhaul, imo. And it makes things complex, but if it isn't supported, then Rails may become less preferable to something that makes it easy to produce clients that do PATCH since they could claim better data integrity.

I'm probably way off here, though. I understand that things don't get done without people doing them, but there would be so many changes to Rails to implement this that it isn't just writing a gem to handle it.

Thanks,
Gary

[Steve Klabnik]

Im at Windy City Rails today, but expect a long-ish response in a few hours. 

[Prem Sichanugrist]

I think HTTP PATCH is really something that goes well with REST and rails, as those are used for updating a single resource the same way as PUT does. However, "application/json-patch" introduce multiple-resources update, which isn't really compatible with the way we implement REST today.

- Prem

On Thursday, September 6, 2012 at 9:45 AM, Steve Klabnik wrote:

Im at Windy City Rails today, but expect a long-ish response in a few hours. 

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group.
To post to this group, send email to rubyonra...@googlegroups.com.
To unsubscribe from this group, send email to rubyonrails-co...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en.

[Steve Klabnik]

Okay. First thing I want to say: It's important to separate "What
Rails does" from REST. Rails implements something that is mostly
RESTful but also not in many places, and that's fine, but you can't
always directly compare the two.

That said...

> PATCH was added to the methods accepted already, so even if PATCH-ing isn't
> RESTful, someone built the diving board to this pool, water in it or not.

PATCH is totally RESTful. But you need to follow the RFC, which states
that you send a diff media type.

We also do not send a full representation for PUT either, and assume
that POST is always create, and override HTTP verbs with a form
parameter. So Rails is already not RESTful here.

> So, here is the argument for further consideration of this request,

Well, here's the thing: it's not as if we have some sort of tracker
where features go in our out. Someone still needs to implement it. If
nobody in core feels like implementing it, then it's up to somebody
else. And that's if core is okay with the feature going in.

I think you can summarize the feeling so far as "not opposed, but not
sure what 'support' actually means or looks like." So let's talk more
about the details of this theoretical application.

> even if
> it is just for advice or opinions about how supporting JSON patch in Rails
> would be something the core team agrees with:

Right. Should have gotten here first. ;)

> This
> is a massive change to really take advantage of it.

What would ActiveRecord need? It already has methods to upload just
certain columns.

> Instead of just sending
> request params with data for updates from forms, you'd want to send JSON,

Whoah, HTML doesn't work this way. It sends
application/www-url-form-encoded from forms, this isn't gonna change.

This means that HTML can't (in general) take advantage of PATCH. It
can't take advantage of PUT or DELETE either. Such is life.


Okay, that didn't end up as long as I thought it'd be.

[Gary Weaver]

On Thursday, September 6, 2012 12:43:36 PM UTC-4, Steve Klabnik wrote:

What would ActiveRecord need? It already has methods to upload just
certain columns.

Sorry- my fault.

Let's say the client using JSON service defined on the controller is a Javascript framework such as AngularJS or Ember, and let's assume they would add support for sending patches, then a patch request would get routed to the controller's update method which would then call:

    @my_model.update_attributes(params[:my_model])

And that wouldn't map up to the right attributes, because it isn't meant to take a patch.

So, instead, let's say that one of those frameworks only sends JSON containing the id and only attributes that had been modified or nullified.

How would they easily indicate changes in associations, like the removal of an object from a collection that would need to destroy a tie model or set the foreign key column on a child model to null rather than destroying it?

How would you indicate other changes in associated models via JSON without accepts_nested_attributes_for without JSON patch implementation?

It would be an edge case, but the default generation of a scaffold supports json format, right? Doesn't that indicate that it should support all of the features like accepts_nested_attributes_for that can be set on a model?
 

> Instead of just sending
> request params with data for updates from forms, you'd want to send JSON,

Whoah, HTML doesn't work this way. It sends
application/www-url-form-encoded from forms, this isn't gonna change.

True that would be the case for HTML forms, which would be the case if the browser didn't support javascript and the page gracefully degraded to use HTML forms. However, Javascript could eliminate the form altogether and send JSON with a different mime type (for the JSON patch) in the update form perhaps.
 

This means that HTML can't (in general) take advantage of PATCH. It
can't take advantage of PUT or DELETE either. Such is life.

Understood, but that is not what I was saying. I am saying that Rails would take advantage of a Javascript framework that would interact with Rails without forms if possible, and fallback to forms (no patch) if needed. At least, that would be the best case for graceful degradation.

[Steve Klabnik]

> And that wouldn't map up to the right attributes, because it isn't meant to
> take a patch.

Ah, sure. So now we're getting to something. What would you propose in
this area?

> It would be an edge case, but the default generation of a scaffold supports
> json format, right? Doesn't that indicate that it should support all of the
> features like accepts_nested_attributes_for that can be set on a model?

Sure, but only displaying, not receiving. So I'm not sure how Rails
would help here.

> True that would be the case for HTML forms, which would be the case if the
> browser didn't support javascript and the page gracefully degraded to use
> HTML forms. However, Javascript could eliminate the form altogether and send
> JSON with a different mime type (for the JSON patch) in the update form
> perhaps.

This would be a super crazy change that basically everyone on the core
team would -1, for sure.

> Understood, but that is not what I was saying. I am saying that Rails would
> take advantage of a Javascript framework that would interact with Rails
> without forms if possible, and fallback to forms (no patch) if needed. At
> least, that would be the best case for graceful degradation.

Yeah, even disregarding the rest of us, I know for a fact that David
would -1000000000 this idea.

[Gary Weaver]

On Thursday, September 6, 2012 3:11:37 PM UTC-4, Steve Klabnik wrote:

> And that wouldn't map up to the right attributes, because it isn't meant to
> take a patch.

Ah, sure. So now we're getting to something. What would you propose in
this area?

Good question. I can probably put in less than 2 cents.

Looks like ActiveSerializer is the current Rails 4 way for serializing/deserializing but in this case a patch can apply to more than just a model. And having to specify attributes in ActiveSerializer, strong_parameters, mass assignment security, ParamsParser, etc. isn't DRY as mentioned earlier.

I guess that the deserialization of the incoming JSON would happen in a way that was fully decoupled both from the model and controller, but would allow a single place to define both what is serialized/deserialized and what is persisted, similar to Roar's representers. In a perfect world, you could also possibly define these things in the model to keep common things together, and define in the association what it is allow to do in that association, and its association, etc. Later when that got unmanageable for more complex schemas, the representer-ish type of abstraction of what can be serialized could be closer to a 1:1 with the controller if you had non-RESTful controllers that were making patch updates to various models, if you had a UI that needed to do that.

I don't know though. Maybe will have a better idea in the coming months, but I would guess you or someone else will have better ideas than I would during that time.

> It would be an edge case, but the default generation of a scaffold supports
> json format, right? Doesn't that indicate that it should support all of the
> features like accepts_nested_attributes_for that can be set on a model?

Sure, but only displaying, not receiving. So I'm not sure how Rails
would help here.

accepts_nested_attributes_for is for receiving, even if it is somewhat of a red-headed stepchild from the way I've heard people talk about it.

> True that would be the case for HTML forms, which would be the case if the
> browser didn't support javascript and the page gracefully degraded to use
> HTML forms. However, Javascript could eliminate the form altogether and send
> JSON with a different mime type (for the JSON patch) in the update form
> perhaps.

This would be a super crazy change that basically everyone on the core
team would -1, for sure.

I agree that it sounds crazy. I should probably not even have mentioned it. Let me step away from the edge of the top of the building where I was shouting at you and David to push me me off.

Think about those that develop with Rails that want to use Javascript frameworks for a lot of the UI, but use Rails for its asset pipeline, available gems for HTML-based CRUD management, layouts/views/etc. for things they want to do in HTML.

If everyone using Rails were HTML focused only (which has not been the case for the last few to several years), it would be one thing. But, people out there have been smoking that Javascript pipe like it is going out of style.

Then you argue that you don't care and they can use Node or Sinatra or JEE6 or whatever.

What if Rails allowed you to create an HTML view like normal OR an Ember view OR some other future framework because it became UI implementation agnostic? A few major versions of Rails away and maybe the UI implementation could be modularized, managed separately from Rails, and included as gems.

Then you argue that it isn't what Rails is, or that it allows it already.

I'm all for DRY and simplicity, and making things intuitive and not complex.

I'm really just wondering if those that want to use Rails to serve up their Javascript framework UI have made the wrong decision?

A JSON patch discussion isn't make or break. But, statements like being -1'd by the team or being -1000000000'd by DHH seem to show that maybe I should just assume that there will be a big middle finger shoved in the direction of anyone that thinks of Rails as a viable way to serve UI implemented in some Javascript framework? Just curious.

Thanks.

[dburry]

Rails strikes me as a practical framework.  Not only does it solve everyday problems that people have, but the best way to start a conversation seems to be to demonstrate, with actual working code (possibly gemmed or forked as necessary), why something is needed, rather than just discussing theory.  I find many other frameworks aggravating when they are so bloated with solving theoretical problems that it just so turned out that nobody really needed after all, whereas Rails is a breath of fresh air when it comes to this practicality.

People do use javascript frameworks with Rails, I know because I and my co-workers use Angular and Ember both with it.  And we like it a lot, and their use is becoming more and more extensive with each project.  Could Rails in general fall off in use due to them? well, maybe someday... but in practical terms, there's still too much unique-to-each-app server side stuff that needs to happen for me to envision that happening any time soon.

There's that word again, practicality.  This is the opposite of theory.  Gary, if you change your approach to a more practical demonstration than just theory, I think you'll find better receptivity.  This means: you, write a simultaneous-multiuser-updating demo site that needs/uses json patch, fork or gem whatever framework you need to make it happen.  Your work and/or ideas will surely get folded into their respective frameworks (like rails and angular) once people see the practical benefit through working code.

Since I've used such javascript frameworks with rails, I can kind of see the point of json patch in theory, but I'm not fully convinced until I try it.  And I don't have the time to write such a demo site today to experiment and see.  Let he with the most enthusiasm for it be the first to do so (and if nobody does, then I guess it wasn't needed enough.. yet).

Dave

[Steve Klabnik]

Yehuda has previously stated that he will do everything possible to make Rails and Ember work well together. I also have a big interest in making this happen. I think that regardless of my personal feelings on the matter, people will want to build them, and I think Rails should provide them with a good experience. I know that David has said that he feels SPAs are a Bad Idea, and that they're a fad.

Take from that what you will.

[Xavier Noria]

Just wanted to say it has little sense IMO to say Rails is RESTful of not. Individual applications may or may not be RESTful. Rails gives you the means to do whatever you want. In particular it gives you the means of writing RESTful applications easily and has some golden path you may follow.

You can write RESTful applications using bash and CGI, it is all up to the programmer.

[Gary Weaver]

On Thursday, September 6, 2012 5:21:04 PM UTC-4, dburry wrote:

There's that word again, practicality.  This is the opposite of theory.  Gary, if you change your approach to a more practical demonstration than just theory, I think you'll find better receptivity.  This means: you, write a simultaneous-multiuser-updating demo site that needs/uses json patch, fork or gem whatever framework you need to make it happen.  Your work and/or ideas will surely get folded into their respective frameworks (like rails and angular) once people see the practical benefit through working code.

Dave, it isn't pretty, but I have been working on making writing JSON APIs easier to implement.

https://github.com/garysweaver/restful_json

It was a project born out of a spike, and it needs a lot of work, and it could be completely the wrong direction. So far the biggest challenge for me has been just returning and deserializing/persisting the JSON in a way that made it easier on the guys developing the Angular code. The legacy system/schema uses STI, MTI, generic record types, entity-refs, non-rails-standard table and column names with Java based services and a decently complex system running in tandem. Our goal has been to implement new features that could be quickly added in Rails and AngularJS. The other developers on my team have been asking for the ability to easily have restful_json update only attributes and allow patches. There was a misunderstanding in the beginning that the new Rails support for the PATCH method was implementing it for JSON, but I learned that wasn't the case. I considered implementing it on my own, at the very least for what we needed. I've contributed a handful of gems to rubygems over the last month mostly related to ActiveRecord, and had briefly started to patch Rails to add custom timestamps, but ended up just doing it as a gem. I saw the IETF media type spec yesterday and before I could blink, Aaron Patterson had already written hana yesterday and shared that in response to my query. I had thought maybe more would be interested.

So, my intent was not to start up a question on theory. I had assumed that one of the goals of this list was people thinking about the future, and I didn't want to spin my wheels on something that would go against the flow. I appreciate your response and advice.
 

Let he with the most enthusiasm for it be the first to do so (and if nobody does, then I guess it wasn't needed enough.. yet).

I guess that was Aaron. ;)

[Steve Klabnik]

Have you seen Jose's active model serializers? It's way better than the default.

--

You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group.

To view this discussion on the web visit https://groups.google.com/d/msg/rubyonrails-core/-/zkvt-OdMJ1wJ.

[Gary Weaver]

On Thursday, September 6, 2012 7:15:50 PM UTC-4, Steve Klabnik wrote:

Have you seen Jose's active model serializers? It's way better than the default.

Yeah, it does look nice: https://github.com/josevalim/active_model_serializers

Note when I said ActiveSerializer above and meant ActiveModel::Serializer.

And strong_parameters is nice (and being integrated into Rails 4): https://github.com/rails/strong_parameters

But, it would be nice to do something more like what Nick Sutterer is doing with Roar representers to have both deserialization of incoming JSON and what is persisted vs. having to specify attributes in strong_representers and in the ActiveModel::Serializer, and mass assignment security, and Nick noted ParamsParser also; having to specify the attribute that many times is not DRY. But we talked about that in the other thread, so sorry for bringing it up again.

Thanks again for your time and any advice. Mostly just want Rails to allow very DRY but very flexible RESTful (and non RESTful) JSON APIs (incoming and outgoing) that allow the power of patching, so if anyone is interested in that, maybe they can help me or I can help them, assuming I'm able to make headway beyond what's currently in https://github.com/garysweaver/restful_json which is going to continue to evolve, I hope, until I find a better way to keep things DRY. Since I'm reusing (abusing) the attr_accessible whitelist/mass assignment security to handle what is shown, I even wrote https://github.com/garysweaver/classmeta to allow stuff like:

  has_many :riders, class_name: Person.meta(:std, attrs: [:name, :seat_preference]).name

Though I'm really not a big fan of abusing ActiveRecord so much just to try to have DRY code.

[Gary Weaver]

On Friday, September 7, 2012 9:09:38 AM UTC-4, Gary Weaver wrote: 

But, it would be nice to do something more like what Nick Sutterer is doing with Roar representers

I should have provided some links:
https://github.com/apotonick/roar
https://github.com/apotonick/roar-rails

And in case anyone missed Aaron's work on the JSON patch spec implementation:
https://github.com/tenderlove/hana

Ok. Now done turning this thread into link spam.