Good choices for a NoSQL database with Clojure?

[Base]

hi All -

Any recommendations on a NoSQL database to use with clojure? I am
experimenting if it will fit my project better than a SQL db and have
no real experience with them.

Strong clojure support is obviously important for this. The only one
I know of is MongoDb...

Thoughts??

Thanks

Base

[Timothy Baldridge]

> Any recommendations on a NoSQL database to use with clojure? I am
> experimenting if it will fit my project better than a SQL db and have
> no real experience with them.

I highly recommend couchdb via the clutch clojure lib. It's fast, and
works well. CouchDB is very easy to setup and learn. And it has a
decent front-end. As a plus side it's even written in a functional
language (Erlang).

More info here:

http://www.ibm.com/developerworks/java/library/j-couchdb-clojure/index.html?ca=drs-

Timothy

[Sean Allen]

That depends on what you are storing, how you want to access it etc.
Couch and Mongo and Riak etc etc each solve different problems.
So, what is your problem? Figure that out. Then pick the one that is
aimed at solving that problem.

[Vivek Khurana]

What problem are you trying to solve ? I am using mongodb with
clojure with no issues..

regards
Vivek

--
The hidden harmony is better than the obvious!!

[Michael Ossareh]

On Sat, Mar 26, 2011 at 09:56, Base <basse...@gmail.com> wrote:

hi All -

Any recommendations on a NoSQL database to use with clojure?  I am
experimenting if it will fit my project better than a SQL db and have
no real experience with them.

I've replaced my rdbms with Riak (www.basho.com).

 

Strong clojure support is obviously important for this.  The only one
I know of is MongoDb...

It doesn't have the best clojure interface into it unfortunately, so if you're looking for 0-60 in the shortest time choose something else. It is a medium term plan of mine to build a clojure implementation. 

 

Thoughts??

Were I making the decisions again, i'd definitely look into Redis. Not least because every page of their documentation has a live redis connection on it, so you can play with the function which you're reading about: http://redis.io/commands/expire

 

Thanks

Base

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

[Timothy Washington]

I briefly tried out FleetDB, which is a DB with a clojure, s-expression data model. It works very well for what it does. But there were things I needed that it didn't supply: 

• A) adding / searching within sub-directories
• B) inability to search within a document that's more than 1 level deep 

I'm currently trying out MongoDB with congomongo on a test project. It mostly works very well, although I haven't tested it out under load. The main gap I'm finding though, is that 

• A) congomongo doesn't handle (from what I can see) Mongo's DB references as described here 

HTH

Tim Washington 

[Michael Ossareh]

On Sat, Mar 26, 2011 at 12:15, Michael Ossareh <oss...@gmail.com> wrote:

On Sat, Mar 26, 2011 at 09:56, Base <basse...@gmail.com> wrote:

hi All -

Any recommendations on a NoSQL database to use with clojure?  I am
experimenting if it will fit my project better than a SQL db and have
no real experience with them.

I've replaced my rdbms with Riak (www.basho.com).

 

Strong clojure support is obviously important for this.  The only one
I know of is MongoDb...

It doesn't have the best clojure interface into it unfortunately, so if you're looking for 0-60 in the shortest time choose something else. It is a medium term plan of mine to build a clojure implementation. 

.... a clojure implementation of the interface into riak.

[Base]

Thanks All! I guess I didnt even realize that they were so different
(though that certainly makes sense).

Basically I want to design a system that will have maybe 30M
(eventually) complex graphs of data that need to be searchable through
either pattern matching or unification or ?...
The pattern s could conceivably become very complicated.

My thought was that this would need to scale out horizontally and
allow for a map-reduce like process to burn through these searches
(though at this point i am still planning this part out and currently
have my data in an RDBMS (using H2 in development - great little
database).

This is very new territory for me, as i am very much used to working
with traditional databases so I kind of dont know where to start...

On Mar 26, 2:55 pm, Michael Ossareh <ossa...@gmail.com> wrote:
> On Sat, Mar 26, 2011 at 12:15, Michael Ossareh <ossa...@gmail.com> wrote:

[Michael Ossareh]

On Sat, Mar 26, 2011 at 13:12, Base <basse...@gmail.com> wrote:

Thanks All!  I guess I didnt even realize that they were so different
(though that certainly makes sense).

Basically I want to design a system that will have maybe 30M
(eventually) complex graphs of data that need to be searchable through
either pattern matching or unification or ?...
The pattern s could conceivably become very complicated.

My thought was that this would need to scale out horizontally and
allow for a map-reduce like process to burn through these searches
(though at this point i am still planning this part out and currently
have my data in an RDBMS (using H2 in development - great little
database).

That helps, so you need:

 - key:value store

 - support for graph structures.

 - mapreduce

My experience is with Riak, and none of the others, which can certainly help you here (in the same order as above):

 - You store your objects,

 - You can define links between objects,

 - You can walk these links during map reduce jobs

This is very new territory for me, as i am very much used to working
with traditional databases so I kind of dont know where to start...

http://mmcgrana.github.com/2010/08/riak-clojure.html does a decent job of giving you a low down on riak with clojure. I've accentuated the api to permit storing links against your objects: https://github.com/ossareh/clj-riak - this hasn't been pulled into Mark's main branch yet. A high level run through (tested):

(ns riak-test

  (:require [clj-riak.client :as riak]

            [clojure.contrib.json :as json]))

(def rc (riak/init {:host "127.0.0.1" :port 8087}))

(defn fetch [type key]

  (try

    (let [bucket (name type)

          val (riak/get rc bucket key)

             

          links (:links val)

          val (->> val

                     :value

                     String.

                     json/read-json)]

      (with-meta val {:links links}))

    (catch Exception e nil)))

(defn put [bucket key data]

  (let [bucket (name bucket)

         links (or (:links (meta data)) '())

        data {:value (.getBytes (json/json-str data))

                :content-type "application/json"

                :links links}]

    (riak/put rc bucket key data)))

(defn link [obj to-type to-id type]

  (let [links (:links (meta obj))

        links (conj links {:bucket (name to-type)

                           :key to-id

                           :tag (name type)})]

    (with-meta obj {:links links})))

;; now you can store objects with links. 

(put :test "foo" {:some "data" :about {:your "domain"}})

(put :test "bar" (link {:more "data"}

                             :test "foo" :example))

;; The following is from the repl

riak-test> (fetch :test "foo")

{:some "data", :about {:your "domain"}}

riak-test> (fetch :test "bar")

{:more "data"}

riak-test> (:links (meta (fetch :test "bar")))

({:bucket "test", :key "foo", :tag "example"})

Now you'd have two independent objects in riak, which during map reduce could be used in a graph like manner. In this case the "bar" object points to the "foo" object.

A point worth making is that Riak is a dynamo based kv store. Which, most notably to a RDBMS user, means you don't have the concept of updating in place. That is, if I wanted to change the bar object to have a key of :name with a value of "bob" you have to do this:

riak-test> (put :test "bar" (assoc (fetch :test "bar") :name "bob"))

nil

riak-test> (fetch :test "bar")

{:name "bob", :more "data"}

i.e.

fetch bar,

assoc bar :name bob

put result.

HTH

[Sean Allen]

On Sat, Mar 26, 2011 at 4:12 PM, Base <basse...@gmail.com> wrote:
> Thanks All!  I guess I didnt even realize that they were so different
> (though that certainly makes sense).
>
> Basically I want to design a system that will have maybe 30M
> (eventually) complex graphs of data that need to be searchable through
> either pattern matching or unification or ?...
> The pattern s could conceivably become very complicated.
>
> My thought was that this would need to scale out horizontally and
> allow for a map-reduce like process to burn through these searches
> (though at this point i am still planning this part out and currently
> have my data in an RDBMS (using H2 in development - great little
> database).
>
> This is very new territory for me, as i am very much used to working
> with traditional databases so I kind of dont know where to start...
>

Couchdb and other document oriented stores def sound like
they wouldnt be what you are looking for.

I havent done anything really like what you are doing so I don't know
what would be right, I have done some work with Couchdb
and know it wouldnt be the best fit for what you are doing.

[Wilson MacGyver]

Since you want a graph db with search, and you want to use it with clojure which means at least a decent java API.

I recommend you check out
http://neo4j.org/

It's a graph db with search ability, and has a very good java API.

[Benny Tsai]

A Clojure wrapper for neo4j was recently posted:
https://github.com/wagjo/borneo

There's also Jiraph, another embedded graph database for Clojure:
https://github.com/ninjudd/jiraph

[Base]

Thank you all so much (and michael....you rock!!)

I will give Riak a shot.

Thanks again.