persisting defrecords / persisting keywords values

[Cédric Pineau]

 Hi all,

 I'm testing embedded fleet (0.3.1) for a web app mock but face two problems.
 Here is a sample code of what I'm trying to do

(ns fleet-test
  (:require (fleetdb [embedded :as fleet])))

(def datastore
  (fleet/init-persistent "/tmp/fleet-db.log"))

(defrecord User [id email passwd fname lname gender])
(fleet/query datastore ["insert" "users" [ (User. 1 "cedric.pineau@" "" "Cédric" "Pineau" :male) ]]))
; 1) -> fleetdb.FleetDBException: Malformed query: attr not a string ':id'

(fleet/query datastore ["insert" "users" [ {"id" 1, "email" "c...@straw-votes.com", "passwd" "", "fname" "Cédric", "lname" "Pineau", "gender" :male} ]])
; 2) -> fleetdb.FleetDBException: Malformed query: value not of a recognized type ':male'


So :

1) I "think" (I'm also learning clojure and not very sure of my chocies :-)) I want to use defrecords to model my app entities.
But it seems that fleet only allow String keys.
Question : what would be the nice tiny trick to transform my defrecords in maps with string keys as fleet expect ? I mean how would you write an on-the-fly transformation to do that back and forth ?

2) Why do I get my keyword value refused here ?
Documentation says "Records can have arbitrary attribute names and values that are numbers, keywords, strings, booleans, or nil. The only requirement is that each record have a unique id."

3) Fleet's nice anyway. Thanks for that !


--
Cédric

[Mark McGranaghan]

Hi Cédric,

Both of your problems stem from the inability to use keywords for keys
or values in FleetDB. This is the intended behavior, but as you
pointed out the documentation was wrong. I've since fixed the website.

In the case of defrecords you will need to covert the record to a map
with string keys before inserting into FleetDB. Likewise the keyword
value :male will need to be converted to "male" or "m" or similar.

Hope this helps,
- Mark

2011/1/7 Cédric Pineau <cedric...@gmail.com>:

[Cédric Pineau]

2011/1/10 Mark McGranaghan <mmcg...@gmail.com>

Hi Cédric,

Both of your problems stem from the inability to use keywords for keys
or values in FleetDB. This is the intended behavior, but as you
pointed out the documentation was wrong. I've since fixed the website.

 Oh ok, I tought I had misunderstood the way to use it (as most of the doc relate to distant usage via clojure-client)

In the case of defrecords you will need to covert the record to a map
with string keys before inserting into FleetDB. Likewise the keyword
value :male will need to be converted to "male" or "m" or similar.

 That's what I did (not generic yet, and I guess, not very elegant either ;-)):

(defn to-fleet [m]
  (apply conj (map (fn [[k v]] {(apply str (rest (str k))) (if (keyword? v) (str v) v)}) m)))
(defn from-fleet [m]
  (if (nil? m) nil (apply conj (map (fn [[k v]] {(keyword k) (if (and (string? v) (= \: (first v))) (keyword (apply str (rest v))) v)}) m))))
(defn user-from-fleet [fleet-user]
  (if (nil? fleet-user) nil (merge (util/empty-record sv.core.User) (from-fleet fleet-user))))
(defn persist-user [x]
   (fleet/query datastore ["insert" "users" [ (to-fleet x) ]]))
(defn find-user-by-email [email]
  (user-from-fleet (first (fleet/query datastore ["select", "users", {"where" ["=", "email", email]}]))))

Hope this helps,
- Mark

 It does :-)

 Now, what I'm looking for, is a straight way to store/index/query records in memory with persistent log just as fleet has (which reminds me of (java) prevayler but far less intrusive !)
 Is there a reason for this limitation in fleet other than being friendly with python/ruby/etc and do you think it's worth going deeper into fleet and forking it in a clojure only specific db or would you recommend another approach/project you may know ?

 Thanks,

--
Cédric

[Mark McGranaghan]

2011/1/10 Cédric Pineau <cedric...@gmail.com>:

Right, it is important to me that FleetDB be accessible from all
dynamic languages, and the greatest common denominator among them in
terms of data representation is JSON. In particular, not all languages
have a keyword-like type.

In terms of getting a more Clojure-specific interface, I think you're
best bet is to continue the approach you started to explore above,
which is to use a thin wrapper around the raw FleetDB interface (.i.e.
the interface FleetDB Clojure client exposes) to support more strongly
typed objects and more datatypes.

I can't think of a similar system of the top of my head that does
exactly what you're looking for out of the box, but do let us know
here if you find something like that!

- Mark

>  Thanks,
>
> --
> Cédric
>
>

[Cédric Pineau]

2011/1/10 Mark McGranaghan <mmcg...@gmail.com>

Right, it is important to me that FleetDB be accessible from all
dynamic languages, and the greatest common denominator among them in
terms of data representation is JSON. In particular, not all languages
have a keyword-like type.

 Do you use it on real project (maybe at Heroku ?) ?
 

In terms of getting a more Clojure-specific interface, I think you're
best bet is to continue the approach you started to explore above,
which is to use a thin wrapper around the raw FleetDB interface (.i.e.
the interface FleetDB Clojure client exposes) to support more strongly
typed objects and more datatypes.

 Ok

 And, as for the strings/keyword keys convertion, I just find a similar bit of code in ring, a project you may know about ;-) :
(defn wrap-keyword-params
  "Middleware that converts the string-keyed :params map to one with keyword
  keys" ...

--
Cédric