Yesod and CouchDB

[Jean-Marc Notin]

Hi,

I started playing with Yesod a few days ago, and I really enjoyed it.

Now, I want to use it for my web application, but with a CouchDB
database. And I would like to do it the right way.
Unfortunately, i can see no Persistent/CouchDB interface, and I could
not find any example use of CouchDB with Yesod...

Does someone have any experience about Yesod and CouchDB ?

Is there any reason for persistent-couchdb not to exist ? Is there
strong incompatibilities between CouchDB and Persistent ?

Thanks for any help !

--
Jean-Marc Notin

[Greg Weber]

Let us know if you have any (monad) issues using the haskell-couchdb bindings without the use of Persistent. I think someone has used couchdb with Yesod already.

From what little I know on the subject, couchdb seems like it could be a good match for Persistent in the same way that MongoDB is. Someone went through the initial motions of creating a backend, but didn't get to the point of writing the couchdb queries: https://github.com/tbh/persistent-couchdb

[John Lenz]

On 11/07/2011 12:17 PM, Greg Weber wrote:
> Let us know if you have any (monad) issues using the haskell-couchdb
> bindings without the use of Persistent. I think someone has used couchdb
> with Yesod already.
>
> From what little I know on the subject, couchdb seems like it could be
> a good match for Persistent in the same way that MongoDB is. Someone
> went through the initial motions of creating a backend, but didn't get
> to the point of writing the couchdb queries:
> https://github.com/tbh/persistent-couchdb
>

Actually, I think couchdb is a pretty poor match for the current
Persistent API.

1) In couchdb, each object has a revision id and this must be passed in
when overwriting an object. Persistent API has no concept of object
revisions. You could assume a field called "rev" held the revision, but
then the code using the Persistent API would need to be couchdb aware,
breaking the whole point of using Persistent.

2) Multiple revisions of an object can co-exist in the db, and the
application must deal with multiple revisions. This means that you
can't just code to Persistent API, you must have some couchdb specific
code to handle multiple revisions of the same object. And normally, the
code to deal with multiple revisions is tied deeply into the use of the
database so it would be less elegent to have some code using Persistent
API with some couchdb specific.

3) Couchdb strongly discourages creating temporary views, but this would
be the only feasible way to implement the Persistent API since
Persistent allows arbitrary queries. Also, a lot of the power of
couchdb comes from the fact you can write your own map/reduce (or
map/fold) functions and are not just limited to simple indexed queries.
Custom views are really hard to integrate into the Persistent API
since the objects returned from a view could have completely different
fields than the objects you are querying. Persistent API assumes that
if you query a table full of customers, you get customers back. But
with couchdb, you could query a table of customers and get back any list
of json objects.

All in all, while it would be possible to implement a couchdb Persistent
backend, if you wanted to take advantage of any of the features of
couchdb which make couchdb useful over just sqlite you would need custom
code.

Thus, I just skipped Persistent completely, and used my
couchdb-enumerator hackage package. Also, I don't think it is very
useful to even attempt to make a couchdb backend for Persistent.

http://hackage.haskell.org/package/couchdb-enumerator

Directly using couchdb-enumerator works fine since most parts of yesod
do not require Persistent.

John

[Greg Weber]

Glad to hear from someone that actually knows something about couchDB :)

On Tue, Nov 8, 2011 at 3:36 PM, John Lenz <jle...@uic.edu> wrote:

On 11/07/2011 12:17 PM, Greg Weber wrote:

Let us know if you have any (monad) issues using the haskell-couchdb
bindings without the use of Persistent. I think someone has used couchdb
with Yesod already.

 From what little I know on the subject, couchdb seems like it could be
a good match for Persistent in the same way that MongoDB is. Someone
went through the initial motions of creating a backend, but didn't get
to the point of writing the couchdb queries:
https://github.com/tbh/persistent-couchdb

Actually, I think couchdb is a pretty poor match for the current Persistent API.

1) In couchdb, each object has a revision id and this must be passed in when overwriting an object.  Persistent API has no concept of object revisions.  You could assume a field called "rev" held the revision, but then the code using the Persistent API would need to be couchdb aware, breaking the whole point of using Persistent.

We could just require the user to have a rev field in their record. 

 

2) Multiple revisions of an object can co-exist in the db, and the application must deal with multiple revisions.  This means that you can't just code to Persistent API, you must have some couchdb specific code to handle multiple revisions of the same object.  And normally, the code to deal with multiple revisions is tied deeply into the use of the database so it would be less elegent to have some code using Persistent API with some couchdb specific.

3) Couchdb strongly discourages creating temporary views, but this would be the only feasible way to implement the Persistent API since Persistent allows arbitrary queries.  Also, a lot of the power of couchdb comes from the fact you can write your own map/reduce (or map/fold) functions and are not just limited to simple indexed queries.

This situation is a little similar to indexing. MongoDB won't even allow you to do a sort on a field without an index. In the original Persistent that required annotations specifying what operations were going on we had a thought that Persistent might automatically setup indexes for you, but no work has gone into that.

But this definitely isn't as simple as indexing and would need more thought on how to generate permanent views.

 

Custom views are really hard to integrate into the Persistent API since the objects returned from a view could have completely different fields than the objects you are querying.  Persistent API assumes that if you query a table full of customers, you get customers back.  But with couchdb, you could query a table of customers and get back any list of json objects.

the persistent-hsqlppp package is demonstrating a technique for running custom queries in which the results have arbitrary schemas.

 

All in all, while it would be possible to implement a couchdb Persistent backend, if you wanted to take advantage of any of the features of couchdb which make couchdb useful over just sqlite you would need custom code.

Thus, I just skipped Persistent completely, and used my couchdb-enumerator hackage package.  Also, I don't think it is very useful to even attempt to make a couchdb backend for Persistent.

http://hackage.haskell.org/package/couchdb-enumerator

Directly using couchdb-enumerator works fine since most parts of yesod do not require Persistent.

Your package looks great. And perhaps couchDB wouldn't care for the persistent query interface. But I think a lot of benefit could be derived from adding something on top that does automatic serialization/deserialization. Then the next goal would be to ensure the queries are correct. I am actually hoping we can come up with something like this for generating custom MongoDB queries, which is a very similar concept.

 

John

[Greg Weber]

There is now a user contributed couchdb implementation for Persistent
that needs to be re-based against the upcoming Persistent release:
https://github.com/yesodweb/persistent/pull/36

Cheers,
Greg