Making Active Resource act like Active Record...

[taryneast]

Hi there,

I've been using Active Resource pretty heavily over the last four
months and have found, like many, that it's not as much like Active
Record as I'd really like. So I've been improving a plugin called
Hyperactive Resource which does just that. (http://github.com/
taryneast/hyperactiveresource/tree/master)

I've got it to a state where it's fairly useful, and want to start
moving some of my stuff back into Active Resource itself. I spoke with
Yehuda at Rails Underground and he said that this would be useful -
especially the validations and callbacks.

My fork of rails is here:
http://github.com/taryneast/rails/tree/master

I've got a lot of stuff I'd like to do to pull in my work from Hyres.
Some of it may change the way that Active Resource behaves and I'd
like to chat to you guys about whether it's a) wanted b) the best way
to go about it.


As an example to begin with. I added Validations by using the
ActiveModel::Validations module (BTB - is that a module or a
component?)

I also added ,as yehuda called it, a couple of simple smoke-tests to
make sure they work. They do - but with a caveat:
validates_presence_of expects that an attribute will return 'nil' if
it's not set... and doesn't recognise a MethodMissing as meaning "this
attribute is not set yet".

ARes, as it currently stands, does not provide a MethodMissing for
attributes unless they are present on the object because we currently
don't know what "column-equivalents" we're likely to get back from the
remote system.

That's fine for somebody grabbing remote objects from a system where
they don't know the structure of the objects they're getting back...
but in my experience so far - that's reasonably rare.
Most of the systems that access a remote API (at least that I've
worked on) run against a known API with known attributes... IMHO there
should be no reason why we couldn't provide a set of known
attributes/"columns" for ARes to do something useful with.

This is how HyRes works. You can manually hand it a set of columns
that can then be used to feed MethodMissing and nicely spit out a nil
if the attribute is not currently set on the given object.

We *could* override validates_presence_of to not explode with an
Exception in this case - but that feels ugly to me... and somehow a
little dangerous. IMO the HyRes method makes more sense. The known
columns do not interfere with any pre-existing functionality which
allows any values that come back from the remote system to be set as
attributes - which means we just get an added benefit without any
disruption to existing code.

This helps us with the validates_presence_of situation, because if
we're running validations on it - it'll be on a column that we know
about. But it also paves the way for a whole bunch of other
functionality as well (eg attr_accessible/protected and some funky
stuff once you start getting into Associations - but more on that
later).

So my question is - are columns ok? and is anybody interested?
If so - how do I go about getting this stuff back into Rails?

Cheers,
Taryn

[Jeremy Kemper]

Awesome, Taryn! I suggest pairing up with Josh Peek, who's current
working on Active Model for Google Summer of Code, to start merging
your work in ASAP.

Best,
jeremy

[Allen]

I'm very excited about this. Thanks for all the work Taryn!

[Chris]

My memory may be failing me, but doesn't ARes support (or plan to
support) some lightweight standard (!= SOAP, != XMLRPC) for remote
introspection of resources? Seems like that would be the means of
getting column data.

-Chris

On Aug 9, 6:18 pm, taryneast <taryne...@gmail.com> wrote:

[Mike Gunderloy]

Mike Burrows [http://positiveincline.com/] has been working along
those lines. Don't know whether it would make sense to try to
integrate his work into core or not.

Mike

[Luca Guidi]

I don't know if this may help, but a couple of years ago I wrote a plugin in order to unify AR and ARes behaviors in one class: http://github.com/jodosha/acts-as-resource/tree/master

class Carrot

  acts_as_resource

  self.site = "http://api.example.com"

  belongs_to :bunny

  valdates_presence_of :color

end

carrot = Carrot.find(23) # local find (database)

carrot = Carrot.find(23, :remote => true) # remote find (rest service)

Carrot.create(:color => "orange") # local save

Carrot.create(:color => "orange", :remote => true) # local save

# etc.. etc..

This is basically an hack, it lazily wraps two classes: an AR instance and an ARes one, and executes commands according to the :remote flag.

Cheers,

Luca
--
lucaguidi.com
twitter.com/jodosha

[Adam Milligan]

Perhaps I'm alone in my opinion, but I don't see the value of this.
ActiveRecord is an ORM that interfaces with the database;
ActiveResource is a convenience for manipulating a RESTful API that
exposes resources as XML. My validations and callbacks and the like
exist in the app server as the canonical behavior of my domain
objects. I don't want clients of my API who use ActiveResource
duplicating the validations in their code; that's just unnecessary
duplication of behavior. If I add or remove a validation then
suddenly all those clients are wrong. If validation fails now then
the client receives a 400 response with an XML representation of the
errors; no duplication needed.

Now, there's nothing to stop clients from adding their own logic on
their end, just as a form on a web page may include JS to validate
inputs before POSTing to the server. But, I don't see that as a
responsibility of ActiveResource any more than Rails form helpers
should include validation options that write JS.

Even if you control both ends of the equation you still end up
violating DRY.

[taryneast]

Hi Chris,

AFAICS it only does so after the object comes back from the remote
system. Currently it just grabs what it can out of the returned XML
(or, now json) and whatever it has is counted as an attribute.

If there's another way planned I'd love to see it.
Querying the remote system, for example, certainly has its benefits.

Still - I think there wouldn't be any issue with letting ARes know
what columns are *supposed* to be there on a remote system we may or
may not trust. I think I pointed out - in some way it can be used a
little like attr_accessible. On our own system we make sure that th
set of columns are all that's sent during create/update because we use
this for our user and don't want to send the unencrypted version of
the password over the wire ;)

Cheers,
Taryn

[taryneast]

Nice.

I've also been working on something like that - for a resource that
has some local fields. Mine was just a "quick get it working" until I
could get the HyRes stuff working, so I'm guessing mine is far more
ugly. So I'd love to have a look at what you've got there :)

Taryn

[Michael Koziarski]

> If there's another way planned I'd love to see it.
> Querying the remote system, for example, certainly has its benefits.

Previously we've discussed requesting:

/posts/new.xml

Which could return an 'empty object' which would tell you all the
columns and their relative defaults. I'm not sure if anything came of
this but it could be worth looking into.

Another option would be to just declare the attributes and their types
in your ARes model declarations. I know everyone likes the idea of
self-learning-self-aware XML webservices magic, but 99 times out of
100 you know the attributes of the object you're building a service
against.

Either way, nice work!

--
Cheers

Koz

[taryneast]

Hi Adam.

On Aug 10, 9:09 pm, Adam Milligan <amilli...@pivotallabs.com> wrote:
> Perhaps I'm alone in my opinion, but I don't see the value of this.
> ActiveRecord is an ORM that interfaces with the database;
> ActiveResource is a convenience for manipulating a RESTful API that
> exposes resources as XML.  My validations and callbacks and the like
> exist in the app server as the canonical behavior of my domain
> objects.  I don't want clients of my API who use ActiveResource
> duplicating the validations in their code; that's just unnecessary
> duplication of behavior.  If I add or remove a validation then
> suddenly all those clients are wrong.  If validation fails now then
> the client receives a 400 response with an XML representation of the
> errors; no duplication needed.

I think you may be confusing being an API producer with an API
consumer. You are producing an API - ARes consumes it... a person may
or may not even own the API being consumed.


We have a big project that involves having *all* of our data on a
remote database and accessible via a newly-crafted RESTful API.
We do, in fact, have the luxury of adding validations on the remote
side of the equation - though many may not, which makes it good to
have them available.
You are also assuming that the API-system is written in Rails. It's
written in Cold Fusion and the validations simply aren't as good in
CF.

In fact - as we built th HyRes project up we realised that a lot of
assumptions that have gone into ARes are based around the remote
system also being Rails. Decoupling that assumption has been important
as I worked on HyRes.


One of the big reasons why *we* want Validations on AREs is that
relying on the remote system's validations means we have to have a
network hit for every simple check to make sure the user has filled
out the name-field when signing up :P
Adding client-side validations is a simple thing to do to reduce
network traffic and lag.

> Now, there's nothing to stop clients from adding their own logic on
> their end, just as a form on a web page may include JS to validate
> inputs before POSTing to the server.  But, I don't see that as a
> responsibility of ActiveResource any more than Rails form helpers
> should include validation options that write JS.

but there's no reason we can't do it in ARes. The validations that are
in Rails are just so darned ice to use - and it means we have
immediate access to all the error-method helpers. So your comment is
noted - but I disagree... in fact I've always had a niggling though
that writing JS error-helpers might be a good idea too (even if just
as a plugin) ;)

> Even if you control both ends of the equation you still end up
> violating DRY.

Yep - that happens sometimes when you're increasing throughput. Sad
but true.
But IMO it's better to do it by reusing a big code library (eg
Validations) tan rewriting validations themselves by hand in JS.

Cheers,
Taryn

[taryneast]

Yay - sounds good. So I'll just email him?
Taryn

On Aug 10, 7:53 am, Jeremy Kemper <jer...@bitsweat.net> wrote:

[taryneast]

On Aug 11, 8:05 am, Michael Koziarski <mich...@koziarski.com> wrote:
> > If there's another way planned I'd love to see it.
> > Querying the remote system, for example, certainly has its benefits.
>
> Previously we've discussed requesting:
>
> /posts/new.xml
>
> Which could return an 'empty object' which would tell you all the
> columns and their relative defaults.  I'm not sure if anything came of
> this but it could be worth looking into.

Interesting idea.
This would probably add a network hit every time you want to show the
new-page, which seems a bit like overkill.
Still - there's no reason why we can't have both options
available. :)

> Another option would be to just declare the attributes and their types
> in your ARes model declarations.  I know everyone likes the idea of
> self-learning-self-aware XML webservices magic, but 99 times out of
> 100 you know the attributes of the object you're building a service
> against.

Yup - this was exactly the idea. In the HyperactiveResource plugin
we've been using - this is done simply with the 'columns' accessor...
which is then used internally just as 'columns' is used in Active
Record... which makes it neat.
The naming of this accessor can be anything... "fields" perhaps... but
using "columns" just means that any plugins that currently work on
AR's columns can Just Work with ARes... which is nice ;)

> Either way, nice work!

thanks :)

Taryn

[taryneast]

Further to this... have been in touch with Josh and have made my first
patches which have been accepted.
So, thanks for all the help so far ;)

Taryn

[Jeremy Kemper]

On Thu, Aug 20, 2009 at 3:30 AM, taryneast<tary...@gmail.com> wrote:
>
>
> On Aug 11, 8:05 am, Michael Koziarski <mich...@koziarski.com> wrote:
>> > If there's another way planned I'd love to see it.
>> > Querying the remote system, for example, certainly has its benefits.
>>
>> Previously we've discussed requesting:
>>
>> /posts/new.xml
>>
>> Which could return an 'empty object' which would tell you all the
>> columns and their relative defaults.  I'm not sure if anything came of
>> this but it could be worth looking into.
>
> Interesting idea.
> This would probably add a network hit every time you want to show the
> new-page, which seems a bit like overkill.
> Still - there's no reason why we can't  have both options
> available. :)

Hit the /new resource once and cache, like introspecting db columns.

>> Another option would be to just declare the attributes and their types
>> in your ARes model declarations.  I know everyone likes the idea of
>> self-learning-self-aware XML webservices magic, but 99 times out of
>> 100 you know the attributes of the object you're building a service
>> against.
>
> Yup - this was exactly the idea. In the HyperactiveResource plugin
> we've been using - this is done simply with the 'columns' accessor...
> which is then used internally just as 'columns' is used in Active
> Record... which makes it neat.
> The naming of this accessor can be anything... "fields" perhaps... but
> using "columns" just means that any plugins that currently work on
> AR's columns can Just Work with ARes... which is nice ;)

Declaring attributes in a common Active Model style would rock.

>> Either way, nice work!
>
> thanks :)

Seconded! :)

Best,
jeremy