AMS, proposed action plan

[Adrian Mugnolo]

Hi everyone,

In this short writeup I'll try to summarize some concerns and ideas, trying to draft an action plan for the Active Model Serializers gem.

In my opinion, the project has lately been suffering from lack of conceptual integrity, lack of direction and overall confusion. Unless we do something concrete about it, users will eventually feel being left behind and switch to some other solution.

To me, these are the most worrying issues with the project as of today:

1. The project name can be confusing for newcomers. There's already a [similarly named module](http://api.rubyonrails.org/classes/ActiveModel/Serializers.html) in Rails. What's the point in having a gem, with a separate team and distribution but, with a matching name?

2. AMS tries to be everything to everyone. Instead, it should aim to be the best at something. While enforcing some conventions, it still allows ad hoc formats to be created. That brings complexity and bugs. With the recent progress in the JSON API specification and other related work (more on this below), we should probably follow and embrace conventions as much as possible.

3. The number of open issues and pull requests keeps growing, lacking clear project objetives. More than once I've tried to either accept or reject cases, feeling unease with both positions. Missing a project curator and clear direction translates to 71 open issues and 33 pending pull requests as of today.

4. There is related ongoing work that we shouldn't ignore but, integrate with. Most notably: the JSON API specification (by @steveklabnik) and, an alternative JSON API resources library (by @dgeb and @lgebhardt, still private). There has also been recent progress in JSON generation and caching gems.

An action plan for AMS could begin with:

1. Freeze any pending work on new features for both the 0.8 and 0.9 branches and, support current users with timely bugfixes and new releases.

2. Accept or reject and tag all pending issues and pull requests.

3. Change the project name from Active Model Serializers to "AMS" for any upcoming releases under a new 0.10 branch. Make module names reflect this change.

4. Make a clear statement about the project objectives.

5. Embrace JSON API, making sure it's fully specification compliant.

6. Work on and resolve all pending open issues and pull requests.

7. Collaborate with any related projects, either help with or benefit from.

8. Ensure these policy changes get reflected in the README file.

Hope this serves as a starting point to start curing the project.

Thanks for your time!

Regards

[Steve Klabnik]

Hey Adrian!

Thanks for doing this. To add some comments:

> What's the point in having a gem, with a separate team and distribution but, with a matching name?

This is historical. Our AMS was an improvement to that AMS, until DHH
reverted it from Rails master. So we pulled it out in the hopes of
getting it back in someday.

I would love an AMS that just produced JSON API. The only reason it
doesn't is because it's older than JSON API is. But that was always
the intention.

Issues and PRs are piling up, because we thought the 0.9 work would go
fast, and then it just kinda stalled. You're right that this sucks.

I like your action plan.

[Mike Kelly]

On Thu, May 8, 2014 at 10:21 PM, Steve Klabnik <st...@steveklabnik.com> wrote:
> Hey Adrian!
>
> Thanks for doing this. To add some comments:
>
>> What's the point in having a gem, with a separate team and distribution but, with a matching name?
>
> This is historical. Our AMS was an improvement to that AMS, until DHH
> reverted it from Rails master. So we pulled it out in the hopes of
> getting it back in someday.
>
> I would love an AMS that just produced JSON API. The only reason it
> doesn't is because it's older than JSON API is. But that was always
> the intention.

The intention of AMS was to exclusively support the generation of a
format that didn't even exist when it was created?

I would love an AMS that wasn't coupled to a specific format, with an
architecture that supported pluggable renderers. Given JSON API is
very early stages, it's probably not in the best interest of the
project to put all its eggs in that one basket.

Cheers,
M

[Mike Kelly]

If you do decide to make it JSON API centric, please change the name
to something less generic.

--
Mike

http://twitter.com/mikekelly85
http://github.com/mikekelly
http://linkedin.com/in/mikekelly123

[Steve Klabnik]

The purpose of AMS was to bring convention over configuration to JSON
generation. JSON API is a codification of something very similar, a
set of conventions for how your JSON should look.

> Given JSON API is
> very early stages,

We have multiple production deployments, a 1.0 draft that's due in a
day or two, etc.

I am not _opposed_ to an AMS that generates tons of stuff. But if it's
not, then I have a bias on which format is chosen.

[Mike Kelly]

On Fri, May 9, 2014 at 12:39 AM, Steve Klabnik <st...@steveklabnik.com> wrote:
> The purpose of AMS was to bring convention over configuration to JSON
> generation. JSON API is a codification of something very similar, a
> set of conventions for how your JSON should look.

Sure, the same could be said of any JSON based format. I was wondering
about your comment that JSON API featured in the original intentions
of AMS?

>> Given JSON API is
>> very early stages,
>
> We have multiple production deployments, a 1.0 draft that's due in a
> day or two, etc.

Exactly. That sounds pretty early stage to me.

As you know, I've been working on a JSON based media type that has
been around for over 3 years, does something very similar, is more
widely used, has had more feedback incorporated, and more libraries
available, etc. I would still call this early stage, so I would
definitely not push for a "serialization" library to adopt it
exclusively.

Cheers,
M

[Mike Kelly]

In other words, given Adrian's point about confusion over the name, it
seems like you should just rename the project to something like "JSON
API Serializers"

[Adrian Mugnolo]

Steve, Mike,

Thanks for your replies.

Please excuse me for any historical mistakes in my first post. I'm new
to the project and might have gotten a few details wrong. Still, I
think the purpose of the message and overall ideas remain valid.

Mike,

Might have expressed myself wrong but I didn't mean to leave out HAL
nor any other formats. Personally, I'd love to see an AMS that emits
multiple outputs but, it would be great to first see it support one
format really well.

The purpose of adopting the acronym people already use as the name for
the project is twofold: communicate a project change of focus while
retaining history and, make sure it doesn't get confused with its
homonymous in the activemodel gem.

Back to the name itself, say you were starting a library like
activerecord with only a SQLite adapter, would you name it
activerecord_sqlite? I wouldn't, unless I wanted to rule out any
further adapters in the future.

Thanks once again.

Regards

[Steven Harman]

To Adrian’s original point #4, there are several hypermedia-focused Internet Media Types emerging - JSON API, HAL, and Siren being three examples. There is the desire to be able to generate arbitrarily (and ad-hoc) structured JSON, as AMS and Rails in general have always done — someday we’ll refer to this as an “accident of history.” These other media types and efforts should not be ignored.

The desire to support a single media type going forward would allow us to focus, but also seems quite limiting as we’d be abandoning any users who’ve chosen a different media type, and all of those who are still dealing with ad-hoc JSON. From a design perspective, I like what Oat (https://github.com/ismasan/oat) is doing with their JSON adapters.

They provide a DSL for defining the properties, links, and meta-data. They then use an “adapter” (which I might call a “serialization strategy”) to transform the defined data into the desired media type. One could imagine a strategy which even allowed for more ad-hoc generation.

I’ve not spent any serious time digging into, nor using, Oat, but it seems a real contender to AMS. Maybe we could work with them, or at least take some inspiration from their work, to find a way forward?

- Steven Harman

--
You received this message because you are subscribed to the Google Groups "rails-api-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rails-api-cor...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Steven Harman]

Aaand by "point #4," I meant "points #2 and #4." Mea culpa!

To unsubscribe from this group and stop receiving emails from it, send an email to rails-api-core+unsubscribe@googlegroups.com.

[Arne Brasseur]

Hi everyone,

Adrian, thank you for taking some much needed stewardship. I hope it can be a turning point to help AMS to be what many wanted it to be.

We set out building a new API 6 months ago, and initially chose AMS and JSON-API.  I can say the experience was very frustrating. It seemed impossible to get bug fix PRs merged, which gave us very little confidence in the project.

Eventually I wrote Yaks Serializers to suit our needs. (https://github.com/plexus/yaks). It looks a lot like the AMS DSL but supports multiple formats out of the box, including JSON API and HAL, and is completely independent of Rails. We're using it in production to serve HAL to an Ember app.

So maybe there are some ideas to take from there as well. I would be happy to bounce some ideas around.

Cheers,
Arne

To unsubscribe from this group and stop receiving emails from it, send an email to rails-api-cor...@googlegroups.com.

[Adrian Mugnolo]

Steven,

All good points. Again, I'm not suggesting that HAL or Siren should be left out.

Regarding existing users, I would keep the 0.8 and 0.9 branches around with the features they already have (but nothing added). Then, on a new branch, I would start the cleanup work focused on JSON API for now. So, no users would be left behind. Incompatible changes would need an upgrade.

As for ad hoc formats, I think AMS should build on top of higher level abstractions and well-known, documented formats. For custom schemas, there're several other JSON libraries available.

I am familiar with the work in both Oak and Yaks and would closely look and follow their codebases. Still, AMS is quite popular and a dependency for other gems such as rails-api and ember-rails. There's a need to support these existing users.

Arne,

Thanks for your reply and thanks for the work in both Yaks and AMS. I hear your pain with AMS not incorporating your proposed changes. I've already tried to explain how hard is today for someone to either accept or reject contributions to the project. This is what we should fix first.

Regards

To unsubscribe from this group and stop receiving emails from it, send an email to rails-api-cor...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Arne Brasseur]

Adrian, I agree it's important to support AMS, this is where the mindshare and the community is, and it's a piece of the Rails API puzzle.

It's also often mentioned in the same breath as JSON API, even though it's not actually compliant with the spec. Mike, I agree that HAL is already a more established format. I also think there's room for an "almost hypermedia" format like JSON API. Because of the people involved it makes sense for AMS to be one of its implementations.

To get a bit more concrete, I think questions that need answering include

1) does AMS keep support for the formats it currently emits (IIRC there are at least three different types of output based on flags)

2) will it go to an abstract representation plus format adapters rather than serializing in one pass? This is what yaks does, and from the sound of it oak as well. It would probably be a little less performant, but I think it's the only sane approach. It would also make supporting HAL or other formats straightforward.

Answering these questions should make it easier to decide what is and what isn't in scope.

Arne

[Adrian Mugnolo]

Arne,

Agree with your thoughts. Answering your questions but, just speaking for myself:

1. It should on the existing branches. Any incompatible changes should go in a separate new branch.

2. In the long term, it should. Even though AMS presents a rather low level DSL it is opinionated about the output format.

Regards

[Dwayne Macgowan]

Hi!

I'm developing several webservices with Rails with my team. I would love to contribute to AMS. What's the repositorie for "our" AMS?

---
Dwayne

[Steve Klabnik]

http://github.com/rails-api/active_model_serializers

[Phuong Nguyen]

So is that something to be executed soon or we'll wait until a hero coming from space to help with?

[Steve Klabnik]

I asked Adrian about this two weeks ago, and he said something was
coming soon. Adrian?