Memcache support

[Jason J. W. Williams]

Hi Guys,

There's a pull request to integrate memcache support into Paisley's mainline code. Since it's a pretty major architecture change I thought I'd bring it up here for discussion.

I'd argue against putting it into paisley proper, especially the main module/codepath. It adds complication that's not related to CouchDB support. Also, what happens when someone wants to add Redis support? That would add even more complication.

I'd argue one of two paths:

1.) Establish a caching base-class that would establish a caching API interface. This would be sub-classed for memcache, Redis, whatever you desire support. When initializing a CouchDB object, you would pass in the caching instance of your choosing.

2.) Write the memcache support as a direct subclass of Paisley in a separate module. If you want memcache support you use this object instead of the CouchDB one.

I lean towards #1, as it would be cleaner and encourage other folks to contribute new caching modules. I think it would truly enhance the project.

-J

[Jason J. W. Williams]

This is the pull request the discussion is happening on: https://github.com/smcq/paisley/pull/14

[Sean McQuillan]

Seeing if I can't make github notifications hit the mailing list.

Cheers,

Sean

--
You received this message because you are subscribed to the Google Groups "Paisley" group.
To post to this group, send email to paisley-d...@googlegroups.com.
To unsubscribe from this group, send email to paisley-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/paisley-developers?hl=en.

--
Sean McQuillan

415.990.0854

[Thomas Vander Stichele]

On Tue, 2011-03-22 at 11:58 -0600, Jason J. W. Williams wrote:
> Hi Guys,
>
>
> There's a pull request to integrate memcache support into Paisley's
> mainline code. Since it's a pretty major architecture change I thought
> I'd bring it up here for discussion.
>
>
> I'd argue against putting it into paisley proper, especially the main
> module/codepath. It adds complication that's not related to CouchDB
> support. Also, what happens when someone wants to add Redis support?
> That would add even more complication.
>
>
> I'd argue one of two paths:
>
>
> 1.) Establish a caching base-class that would establish a caching API
> interface. This would be sub-classed for memcache, Redis, whatever you
> desire support. When initializing a CouchDB object, you would pass in
> the caching instance of your choosing.
>

1) Is what I did in one of the branches of my spiderweb of paisley
forks.

Look at couchdb.py here:
https://thomas.apestaart.org/thomas/trac/browser/src/dad/trunk/dadcouch/python/dadcouch/extern/paisley/couchdb.py

(bottom, map and mapped)

and a very simple (but for me already very useful) memory caching
subclass:

https://thomas.apestaart.org/thomas/trac/browser/src/dad/trunk/dadcouch/python/dadcouch/common/cachedb.py

The reason I split it off was because I was unsure if caching was in the
scope of Paisley. I think it is, and apparently more people think so,
so I think it makes sense to add the necessary primitives to the base
class.

However, as you might be able to see in that code, it is built on the
object-views work that David started, and that probably needs discussing
before. Note the objFactory which is a class that transforms view
result lines to actual objects.

I will not pretend to know enough about redis/memcache (beyond what they
do or how they vaguely work) to be able to evaluate if the minimal API I
added is right or enough for those use cases - so why not ask those
people to discuss it on this list so we can agree on a common caching
API ?

If we do a caching API, I think paisley should also show how to
implement it, and in that case this simple memory caching implementation
makes perfect sense to add IMO.

Thomas

[Sean McQuillan]

Hey Thomas,

We're having an active discussion on github here: https://github.com/smcq/paisley/pull/14

Sorry not used to having a mailing list, will try to be better about cross posting.

Cheers,

Sean

--
You received this message because you are subscribed to the Google Groups "Paisley" group.
To post to this group, send email to paisley-d...@googlegroups.com.
To unsubscribe from this group, send email to paisley-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/paisley-developers?hl=en.

--
Sean McQuillan

415.990.0854

[Jason Williams]

> However, as you might be able to see in that code, it is built on the
> object-views work that David started, and that probably needs discussing
> before.  Note the objFactory which is a class that transforms view
> result lines to actual objects.

Is the point of the object views to wrap the entire view result in a
class with accessor methods rather than the dictionary/list that gets
returned now?

I think the KV caching API we're discussing is simple enough that it
can be used with or without the object mapper. Keeps the intelligence
of the object mapper and how to leverage the cache in the mapper.

-J

[Thomas Vander Stichele]

On Wed, 2011-03-23 at 10:33 -0700, Jason Williams wrote:
> > However, as you might be able to see in that code, it is built on the
> > object-views work that David started, and that probably needs discussing
> > before. Note the objFactory which is a class that transforms view
> > result lines to actual objects.
>
> Is the point of the object views to wrap the entire view result in a
> class with accessor methods rather than the dictionary/list that gets
> returned now?
>

it's an object mapping layer. Instead of getting a dict, you get a full
object. It uses the mapping code from couchdb-python

Example:
https://thomas.apestaart.org/thomas/trac/browser/src/dad/trunk/dadcouch/python/dadcouch/model/couch.py

I guess it's similar to an ORM, though I haven't written ORM code
myself.

> I think the KV caching API we're discussing is simple enough that it
> can be used with or without the object mapper. Keeps the intelligence
> of the object mapper and how to leverage the cache in the mapper.

yeah, it might be. for example, if objFactory == None then store the
value directly.

T

>
> -J
>

--
All my old friends
they don't know me now
--
GStreamer - bringing multimedia to your desktop
http://gstreamer.freedesktop.org/

[Jason J. W. Williams]

it's an object mapping layer.  Instead of getting a dict, you get a full

object.  It uses the mapping code from couchdb-python

The object mapper layer in couchdb-python has always frustrated me. Gets in the way when you want to do introspection. I've always liked that Paisley had straightforward dict return. Staying closer to the JSON you get out of Couch always seems to make the output easier to understand and compare to the raw output from Couch. Generally speaking I'm not an ORM fan though.

-J

[Thomas Vander Stichele]

I guess different people will have different preferences. I haven't run
into what you call introspection problems but maybe I'm
misunderstanding. I just picked off from david reid's branch where he
had started doing this.

As long as paisley doesn't force you to use it (so you can do your
preference) I think it's fine, no ?

T

>
>
> -J

> --
> You received this message because you are subscribed to the Google
> Groups "Paisley" group.
> To post to this group, send email to
> paisley-d...@googlegroups.com.

> To unsubscribe from this group, send email to paisley-developers
> +unsub...@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/paisley-developers?hl=en.

--
Cos I feel like a fake when I feel
any feeling
And I wouldn't wanna happen to you
Cos I know that you mean it
--
URGent, best radio on the net - 24/7 !
http://urgent.fm/

[Jason J. W. Williams]

I guess different people will have different preferences.  I haven't run
into what you call introspection problems but maybe I'm
misunderstanding.  I just picked off from david reid's branch where he
had started doing this.

As long as paisley doesn't force you to use it (so you can do your
preference) I think it's fine, no ?

I don't have a problem with it as long you're not forced to use it. I'm only against it if it breaks backwards compatibility with existing apps.

-J