Ohm
12 views
Skip to first unread message
Erlenstar
Nov 27, 2009, 1:30:35 PM11/27/09
to nodejs
Anyone out there looking at replicating something like Ohm's object-
hash mapping (http://ohm.keyvalue.org/) onto Node's Redis module?
That would be a killer combo, but I'm not sure my ruby skills are
sharp enough to pull off a port.
hash mapping (http://ohm.keyvalue.org/) onto Node's Redis module?
That would be a killer combo, but I'm not sure my ruby skills are
sharp enough to pull off a port.
Brian Hammond
Nov 27, 2009, 11:26:27 PM11/27/09
to nodejs
Yes. I have been working on "Nohm" for a few days.
Marak Squires
Nov 28, 2009, 12:09:20 AM11/28/09
to nod...@googlegroups.com
Awesome! I was requesting this a few days ago in the IRC.
--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
Michel Martens
Nov 28, 2009, 4:26:38 PM11/28/09
to nod...@googlegroups.com
On Sat, Nov 28, 2009 at 1:26 AM, Brian Hammond <br...@fictorial.com> wrote:
> Yes. I have been working on "Nohm" for a few days.
>
I may help you with that. Is it in a public repo?
> Yes. I have been working on "Nohm" for a few days.
>
--
Michel
Greg Ritter
Nov 29, 2009, 12:39:01 AM11/29/09
to nod...@googlegroups.com
I'd be interested in using and helping with this as well. I wrote a
node/redis based wiki, and was thinking about abstracting the redis
logic out of the wiki into something more reusable. I wasn't aware of
Ohm, but it looks like exactly what I want.
Greg Ritter - Codespin Inc. - (780) 716-GREG
node/redis based wiki, and was thinking about abstracting the redis
logic out of the wiki into something more reusable. I wasn't aware of
Ohm, but it looks like exactly what I want.
> --
>
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.
> To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
>
>
>
--
>
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.
> To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
>
>
>
Greg Ritter - Codespin Inc. - (780) 716-GREG
Erlenstar
Nov 29, 2009, 8:03:31 PM11/29/09
to nodejs
> Yes. I have been working on "Nohm" for a few days.
Brian Hammond
Nov 29, 2009, 10:25:59 PM11/29/09
to nodejs
Hey guys, I am on vacation at the moment. I will have nohm up on
github by the end of the week.
github by the end of the week.
Jeff Zerger
Dec 10, 2009, 11:33:59 PM12/10/09
to nodejs
Howdy Brian,
Any update on the Nohm work? I've got some free time this week and
may dive into my own port if you've throttled back on yours.
Any update on the Nohm work? I've got some free time this week and
may dive into my own port if you've throttled back on yours.
Brian Hammond
Dec 11, 2009, 9:40:36 AM12/11/09
to nodejs
I started working on NOM (cf. "NOHM"), and had a chat with soveran
(author of Ohm for Ruby) about something interesting.
In Ohm, the level of abstraction is (at least in my understanding)
raised up a level from individual keys and values to the level of
"objects" which are aggregations of keys and values in Redis. The
attractive part of this concept to me was that it should offer what I
call "object-level atomicity"; either all pending changes to an object
are committed or none are depending on whether the user-provided
object-level validation passes/fails.
Unfortunately, that's not how Ohm works. In Ohm, changes to scalar
object properties are buffered while list/set operations are immediate-
mode. That is, when one changes a :string property to "foobar" that
change does not immediately result in a call to SET in Redis. When
one appends a value to a :list property, that _does_ immediately
result in a call to RPUSH in Redis. When one calls save on an object
the scalar properties are updated in one operation via MSET (or in
multiple operations via SET if you have an old version of Redis).
Thus, should your object pre-save validation fail, your lists and sets
are not rolled back. You are neither here nor there.
For NOM, I went the route of buffering _all_ object changes client
side to provide object-level atomicity. Thus, appending an item to
list does not immediately commit that operation in Redis. Instead,
the Redis-equivalent operation is performed client side on an Array,
and the corresponding Redis command needed to realize the change is
buffered in the same order. This strategy allows a pre-save
validation routine to validate what an object *will* look like when
saved to Redis. If what the object will look like is invalid to the
user (pre-save validation fails), none of the pending changes to that
object are committed. This provides object-level atomicity.
This does imply something. NOM is essentially re-implementing Redis
semantics on supported value types. E.g. LREM removes the first or
last N elements of a list. This has been re-implemented on the client
side in the method List.prototype.removeFirstN, etc. What I stopped
at was zset or sorted sets. These are implemented in skip lists in
Redis. Skip lists are not difficult to implement, but I started to
question my strategy.
Why? Well it implies that the user first pulls down the full object
to the client side first, makes local modifications, then pushes the
buffered commands back to Redis to realize the (valid) changes. This
means that it will not be appropriate for "large" objects (e.g.
objects that contain a list property whose value has 100K elements)
due to the bandwidth overhead.
There are a number paths to follow that I can see.
1. Drop the concept of validations. Instead, just route object
properties to the equivalent key, value, and value type in Redis.
This implies that no Redis commands need be buffered client side.
2. Continue with the full object atomicity by finishing skip lists and
mirroring Redis value type semantics on the client, checking that the
local state is valid, and if so, sending/replaying buffered Redis
commands needed to make the changes remotely in Redis.
3. Do not re-implement lists, sets, and zsets (and HASH soon
thereafter) on the client side in Javascript (too late for lists and
sets). Instead, simply buffer the commands to perform the same
operations in Redis. We're already on this path. The difference
would be that the client wouldn't really have the ability to inspect
the full collection for validation. But, how often do you do this
anyway? Never, right? You tend to perhaps do an LTRIM after a RPUSH
but that's probably it. I bet no one says "oh, I have to have the
whole list because it can only be saved if the stddev is less than 1.5
across all elements" or similar. Thus, perhaps it does not make much
sense to mirror the Redis value type semantics locally but instead
just buffer the commands. When the rest of the object is validated,
send the buffered commands. Thus, you get object-level atomicity and
avoid the overhead of mirroring what Redis does.
4. Store the JSON representation of objects in string values in Redis
instead of mapping object properties to Redis keys, values, and value
types (list, set, etc.). Why do this? One, NOM could provide
equivalent data types on the client side for Redis data types to
start. Since the point would be to provide object level atomicity, we
wouldn't be doing the work twice, and wouldn't have to ensure that the
semantics of Redis are mirrored client side. Also note that in the
near future Redis will support paging (ala virtual memory) which will
only happen for large-ish values. Storing objects as JSON would work
better with this scheme than individual properties as lists/sets/
strings/numbers since the whole-object-as-JSON would be larger. That
being said, you don't always want to fetch the whole object to make a
change (I'm looking at you Riak). So there's a tradeoff there too.
Number 4 would no longer be an object mapper but really an object
storage "engine". I hereby call "dibs" on NOM and NOSE (Node Object
Storage Engine) until the dust settles :)
I'm leaning towards #3. It provides object-level atomicity and
doesn't require a significant maintenance overhead to match Redis'
value type semantics.
Thanks,
Brian
(author of Ohm for Ruby) about something interesting.
In Ohm, the level of abstraction is (at least in my understanding)
raised up a level from individual keys and values to the level of
"objects" which are aggregations of keys and values in Redis. The
attractive part of this concept to me was that it should offer what I
call "object-level atomicity"; either all pending changes to an object
are committed or none are depending on whether the user-provided
object-level validation passes/fails.
Unfortunately, that's not how Ohm works. In Ohm, changes to scalar
object properties are buffered while list/set operations are immediate-
mode. That is, when one changes a :string property to "foobar" that
change does not immediately result in a call to SET in Redis. When
one appends a value to a :list property, that _does_ immediately
result in a call to RPUSH in Redis. When one calls save on an object
the scalar properties are updated in one operation via MSET (or in
multiple operations via SET if you have an old version of Redis).
Thus, should your object pre-save validation fail, your lists and sets
are not rolled back. You are neither here nor there.
For NOM, I went the route of buffering _all_ object changes client
side to provide object-level atomicity. Thus, appending an item to
list does not immediately commit that operation in Redis. Instead,
the Redis-equivalent operation is performed client side on an Array,
and the corresponding Redis command needed to realize the change is
buffered in the same order. This strategy allows a pre-save
validation routine to validate what an object *will* look like when
saved to Redis. If what the object will look like is invalid to the
user (pre-save validation fails), none of the pending changes to that
object are committed. This provides object-level atomicity.
This does imply something. NOM is essentially re-implementing Redis
semantics on supported value types. E.g. LREM removes the first or
last N elements of a list. This has been re-implemented on the client
side in the method List.prototype.removeFirstN, etc. What I stopped
at was zset or sorted sets. These are implemented in skip lists in
Redis. Skip lists are not difficult to implement, but I started to
question my strategy.
Why? Well it implies that the user first pulls down the full object
to the client side first, makes local modifications, then pushes the
buffered commands back to Redis to realize the (valid) changes. This
means that it will not be appropriate for "large" objects (e.g.
objects that contain a list property whose value has 100K elements)
due to the bandwidth overhead.
There are a number paths to follow that I can see.
1. Drop the concept of validations. Instead, just route object
properties to the equivalent key, value, and value type in Redis.
This implies that no Redis commands need be buffered client side.
2. Continue with the full object atomicity by finishing skip lists and
mirroring Redis value type semantics on the client, checking that the
local state is valid, and if so, sending/replaying buffered Redis
commands needed to make the changes remotely in Redis.
3. Do not re-implement lists, sets, and zsets (and HASH soon
thereafter) on the client side in Javascript (too late for lists and
sets). Instead, simply buffer the commands to perform the same
operations in Redis. We're already on this path. The difference
would be that the client wouldn't really have the ability to inspect
the full collection for validation. But, how often do you do this
anyway? Never, right? You tend to perhaps do an LTRIM after a RPUSH
but that's probably it. I bet no one says "oh, I have to have the
whole list because it can only be saved if the stddev is less than 1.5
across all elements" or similar. Thus, perhaps it does not make much
sense to mirror the Redis value type semantics locally but instead
just buffer the commands. When the rest of the object is validated,
send the buffered commands. Thus, you get object-level atomicity and
avoid the overhead of mirroring what Redis does.
4. Store the JSON representation of objects in string values in Redis
instead of mapping object properties to Redis keys, values, and value
types (list, set, etc.). Why do this? One, NOM could provide
equivalent data types on the client side for Redis data types to
start. Since the point would be to provide object level atomicity, we
wouldn't be doing the work twice, and wouldn't have to ensure that the
semantics of Redis are mirrored client side. Also note that in the
near future Redis will support paging (ala virtual memory) which will
only happen for large-ish values. Storing objects as JSON would work
better with this scheme than individual properties as lists/sets/
strings/numbers since the whole-object-as-JSON would be larger. That
being said, you don't always want to fetch the whole object to make a
change (I'm looking at you Riak). So there's a tradeoff there too.
Number 4 would no longer be an object mapper but really an object
storage "engine". I hereby call "dibs" on NOM and NOSE (Node Object
Storage Engine) until the dust settles :)
I'm leaning towards #3. It provides object-level atomicity and
doesn't require a significant maintenance overhead to match Redis'
value type semantics.
Thanks,
Brian
weepy
Jan 18, 2010, 6:56:23 AM1/18/10
to nodejs
#3 seems to be the obvious choice ?
> > Any update on theNohmwork? I've got some free time this week and
coffeemate
Jun 16, 2010, 10:17:24 AM6/16/10
to nod...@googlegroups.com
Hi all,
I love all the stuff you contribute guys.
Anyway,
I'm interested in implementing "ohm" (ruby) like simple framework for
node dealing with "Redis" using fictorial's "node-redis-client" . I
found the topic below discussed before. Any progress on this? I would
like to participate if any or i'll go for a one from scratch. At least
a personal one for my personal startup project. Any comment also for
this topic is very much appreciated.
Thanks so.
kadir pekel
http://twitter.com/kadirpekel
Kadir Pekel
Jun 16, 2010, 11:08:46 AM6/16/10
to nod...@googlegroups.com
Oh sorry for the duplicate topic, posted this one accidentally. Please discard this message and follow the post titled "any ohm like library?"
Sorry for inconvenience, thanks.
--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
--
Kadir Pekel
Software Developer
Bilyoner.com - Maslak / Sisli / İSTANBUL
Reply all
Reply to author
Forward
0 new messages