Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Converting a relation model to a document model
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  20 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Mircea Chirea  
View profile  
 More options Aug 11 2012, 4:23 am
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Sat, 11 Aug 2012 01:23:48 -0700 (PDT)
Local: Sat, Aug 11 2012 4:23 am
Subject: Converting a relation model to a document model

Hello everyone!

I have been working on a website in my spare time for the past few months
and I have reached a point where I don't think SQL databases can cut it. I
originally picked that because it's what I had any experience with and
Entity Framework made it very easy to use, but now queries have become
rather hard to do and worse when you need to take into account the time and
effort it takes to get EF to generate a *good* query. My model is rather
complex and I decided a month ago to migrate to MongoDB but their LINQ
provider is not great and I had issues getting the model right for Mongo
anyway. Since I've heard about Raven I've been impressed, especially by the
integrated nature of Lucene, so I decided to try again, but since I am a
newbie, I find this task to be daunting. Take a look for yourself:
http://imgur.com/9355f

The two biggest issues I have at the moment are getting a specific
comparison, including topics with specs, votes, tags, comments with replies
and votes. I have about 20 queries, for this alone, loading everything (as
well as doing some checks first); even with memcached it only goes so far.
It is also a pain to extend, migrations being hard to do without losing
data and migrating down is a royal pain.

I have identified the entities (or aggregate roots... all these terms) to
be: User, Badge, Level, Group, Topic, Tag, Comment, Comparison. Everything
else simply has a is-owned-by relationship, even though the Synonyms table
is shared with Tags and Topics, they don't use the same rows anywhere. An
Edit is just a different version of a topic's content, in a particular
point in time; I don't think I can use Raven's versions features, as I need
to be able to have edits require approval (new users for example);
currently edits don't save tags, specs and synonyms, as it would be a *royal
pain* to refactor the code and create the migrations necessary (thanks
EF!). I have separated most of the database activity into command objects,
so 95% of the rest of the code (still a few bits to move) is oblivious to
the underlying database, which also made memcached relatively easy to add.

So, my question is, how should I rewrite the model into one suitable for
RavenDB?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile  
 More options Aug 11 2012, 1:08 pm
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Sat, 11 Aug 2012 20:08:24 +0300
Local: Sat, Aug 11 2012 1:08 pm
Subject: Re: [RavenDB] Converting a relation model to a document model

You already have root aggregates, so start with those:  User, Badge, Level,
Group, Topic, Tag, Comment, Comparison
Then follow naturally from there. Once you have the aggregates, you are
pretty much done.

On Sat, Aug 11, 2012 at 11:23 AM, Mircea Chirea <chirea.mir...@gmail.com>wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mircea Chirea  
View profile  
 More options Aug 11 2012, 2:24 pm
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Sat, 11 Aug 2012 11:24:16 -0700 (PDT)
Local: Sat, Aug 11 2012 2:24 pm
Subject: Re: [RavenDB] Converting a relation model to a document model

Right. The thing is I am not sure how to set up any relationships between
these aggregates. I could just add an property ID a la SQL, but RavenDB
doesn't seem to have joins or anything similar AFAIK, so I would need to
issue another query; this starts to add up just as quickly as on SQL, if
not worse.

I also have a special case:

Comments and votes are not per-topic, they are per-topic in a single
comparison. So X has 4 comments and Y has 2, in the X vs Y, but not in X vs
Z or Y vs Z. How would I store that? Currently I have set the relations to
be Comparison -> Comments and Comparison - Votes, with properties like
CommentsForFirst in code, but this is pretty horrible and I have to always
switch on whether the topic id is the first or the second one in a specific
comparison, to load comments and votes. A mess.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile  
 More options Aug 12 2012, 6:15 am
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Sun, 12 Aug 2012 13:15:48 +0300
Local: Sun, Aug 12 2012 6:15 am
Subject: Re: [RavenDB] Converting a relation model to a document model

inline

On Sat, Aug 11, 2012 at 9:24 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:

> Right. The thing is I am not sure how to set up any relationships between
> these aggregates. I could just add an property ID a la SQL, but RavenDB
> doesn't seem to have joins or anything similar AFAIK, so I would need to
> issue another query; this starts to add up just as quickly as on SQL, if
> not worse.

You hold the id, and then you include it when needed.
No additional query required.

> I also have a special case:

> Comments and votes are not per-topic, they are per-topic in a single
> comparison. So X has 4 comments and Y has 2, in the X vs Y, but not in X vs
> Z or Y vs Z. How would I store that?

You put it in the topic, comments and votes are part of it.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mircea Chirea  
View profile  
 More options Aug 12 2012, 6:27 am
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Sun, 12 Aug 2012 03:27:55 -0700 (PDT)
Local: Sun, Aug 12 2012 6:27 am
Subject: Re: [RavenDB] Converting a relation model to a document model

I can't put the comments and votes in the topic, as... uh let's take some
examples: VW Passat vs Audi A4. The Passat has 50 votes and 20 comments,
the Audi has 30 votes and 10 comments. But if you compare Passat to say BMW
335d, Passat doesn't have those votes and comments from Passat vs Audi.
That's what I mean, comments and votes are local inside a comparison and
each side has votes/comments there. How do I store those?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile  
 More options Aug 12 2012, 6:34 am
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Sun, 12 Aug 2012 13:34:11 +0300
Local: Sun, Aug 12 2012 6:34 am
Subject: Re: [RavenDB] Converting a relation model to a document model

Okay, then why not put them directly on the comparison?

On Sun, Aug 12, 2012 at 1:27 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mircea Chirea  
View profile  
 More options Aug 12 2012, 6:35 am
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Sun, 12 Aug 2012 03:35:39 -0700 (PDT)
Local: Sun, Aug 12 2012 6:35 am
Subject: Re: Converting a relation model to a document model

Well how would I store them? Where exactly to put them? Surely there is a
better way than the hack I did on EF: CommentsForFirst, CommentsForSecond,
etc.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile  
 More options Aug 12 2012, 6:43 am
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Sun, 12 Aug 2012 13:43:53 +0300
Local: Sun, Aug 12 2012 6:43 am
Subject: Re: [RavenDB] Re: Converting a relation model to a document model

What do you want to have there?

On Sun, Aug 12, 2012 at 1:35 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mircea Chirea  
View profile  
 More options Aug 12 2012, 6:52 am
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Sun, 12 Aug 2012 03:52:30 -0700 (PDT)
Local: Sun, Aug 12 2012 6:52 am
Subject: Re: [RavenDB] Re: Converting a relation model to a document model

An array or list of votes (embedded) and a list of comment ids.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile  
 More options Aug 12 2012, 6:53 am
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Sun, 12 Aug 2012 13:53:48 +0300
Local: Sun, Aug 12 2012 6:53 am
Subject: Re: [RavenDB] Re: Converting a relation model to a document model

And what prevents you from doing just that?

On Sun, Aug 12, 2012 at 1:52 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mircea Chirea  
View profile  
 More options Aug 12 2012, 7:03 am
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Sun, 12 Aug 2012 04:03:49 -0700 (PDT)
Local: Sun, Aug 12 2012 7:03 am
Subject: Re: Converting a relation model to a document model

Well how do I store them in the Comparison document?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile  
 More options Aug 12 2012, 7:37 am
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Sun, 12 Aug 2012 14:37:52 +0300
Local: Sun, Aug 12 2012 7:37 am
Subject: Re: [RavenDB] Converting a relation model to a document model

Show us the class structure as you would like it to be


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mircea Chirea  
View profile  
 More options Aug 12 2012, 1:33 pm
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Sun, 12 Aug 2012 10:33:16 -0700 (PDT)
Local: Sun, Aug 12 2012 1:33 pm
Subject: Re: [RavenDB] Converting a relation model to a document model

I've spent some time reading your blog, the docs and some SO posts and I
learned about denormalized references and a few tricks. I've converted my
current EF entities to Raven as best as I could:

EF:  http://codepaste.net/sh3kke
Raven:  http://codepaste.net/cnb9ib

There are some differences:

   - Removed some navigation properties, like User.CommentVotes and
   User.TopicVotes, which no longer make sense.
   - Removed the hacks around EF not supporting enums (excluding the beta
   5.0 release that needs .NET 4.5)
   - Removed data annotations, since most are EF specific; can Raven handle
   data validation or do I have to do it in my application?
   - Removed constructors and use plain arrays, for readability.
   - Left the BuildModel static functions as they might help you better
   understand the relations between entities.
   - Moved the "Computed Properties" from topic into auto properties, which
   would logically mirror the last edit (thus removing the need for a query).
   - Created Ref classes to deal with references, storing a little more
   data than just the Id.

Please look at both and check if it is a sane conversion and more
importantly, makes sense as a Raven model. I'd also appreciate criticism on
my EF entities, as I will continue using it for a while until I can fully
port the application, so a few improvements wouldn't hurt. I know that the
topic edits do not handle specifications and tags; this is a flaw in the
current design, I've not fixed it in the Raven model to make the
differences easier to see, but ideally the edit will also store specs and
tags, having those mirrored in the topic document itself for ease of
accessibility.

For Comparison, I wrote a custom reference class "Computed Properties"
which stores the topic ID and the respective comments and votes - there
would be the CommentsForFirst and CommentsForSecond in the EF model. Is
that a good thing to do? I might also make the Comparison class be
something like Pair<Topic> and implement IEnumerable, so I could more
easily use it in the code; how does Raven handle that? I'm worried it might
treat each Comparison as a JSON array, which would be bad.

Lastly, Comment has a circular reference; a comment can be a reply to
another, so I need to track that. What is the best way to do that? I
currently use a FK key to the same table and a ReplyTo column; if that is
NULL, the comment is not a reply, if it has a value it's the ID of the
ascensor comment, so it's basically a tree: a comment can have many replies
and can be a reply to a single comment.

Quite a lot of questions, but what are you gonna do :| Being used to SQL
and being taught that SQL is the swiff army knife make it hard to think
straight how to model objects and their relations in a non-relational
database (heck, it's hard as hell to do it in a relational - seems to me
"relational" is not the proper term for SQL databases, really).

Thank you very much for your effort so far Oren Eini, I've yet to see
someone so dedicated to helping users of their products. I have subscribed
to your blog and have a crapload of good posts bookmarked to read later.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mircea Chirea  
View profile  
 More options Aug 12 2012, 2:30 pm
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Sun, 12 Aug 2012 11:30:39 -0700 (PDT)
Local: Sun, Aug 12 2012 2:30 pm
Subject: Re: Converting a relation model to a document model

Nevermind the part about making Comparison a pair. I think I'll just store
the topics in a single array, like public EmbeddedTopic[] Topics { get;
set; }, as that does what I want just fine.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile  
 More options Aug 13 2012, 4:23 am
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Mon, 13 Aug 2012 11:23:34 +0300
Local: Mon, Aug 13 2012 4:23 am
Subject: Re: [RavenDB] Converting a relation model to a document model

inline

On Sun, Aug 12, 2012 at 8:33 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:

> I've spent some time reading your blog, the docs and some SO posts and I
> learned about denormalized references and a few tricks. I've converted my
> current EF entities to Raven as best as I could:

> EF:  http://codepaste.net/sh3kke
> Raven:  http://codepaste.net/cnb9ib

> There are some differences:

>    - Removed some navigation properties, like User.CommentVotes and
>    User.TopicVotes, which no longer make sense.

> I really don't like Comment.Replies

Is there a reason each comment is a separate document?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mircea Chirea  
View profile  
 More options Aug 13 2012, 5:12 am
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Mon, 13 Aug 2012 02:12:17 -0700 (PDT)
Local: Mon, Aug 13 2012 5:12 am
Subject: Re: [RavenDB] Converting a relation model to a document model

Yes, they are loaded async. I also need their ID to delete them, see them
on the user's page, notify users or replies, that sort of thing.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mircea Chirea  
View profile  
 More options Aug 14 2012, 7:40 am
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Tue, 14 Aug 2012 04:40:00 -0700 (PDT)
Subject: Re: Converting a relation model to a document model

I've been doing some research on Oren Eini's blog, but I haven't found
answers to my questions, especially the circular reference one with
comments. I am not sure if my model fits well into RavenDB's design either,
though it seems it might work.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Kijana Woodard  
View profile  
 More options Aug 14 2012, 5:37 pm
From: Kijana Woodard <kijana.wood...@gmail.com>
Date: Tue, 14 Aug 2012 16:37:41 -0500
Local: Tues, Aug 14 2012 5:37 pm
Subject: Re: [RavenDB] Converting a relation model to a document model

@Oren - I'm not crazy about comments with replies or comments as separate
docs either.

However, here are the use cases I'm facing:
Comments can be replied to creating a "mini conversation" within the
comments section of a post.
Comments can be marked as spam by the system.
Comments can be marked as spam/inappropriate by users; if enough users do
this, the comment is hidden/removed.
Comments can be redacted (removing offensive words).
Users earn points for comments.

My initial approach was this:

I'm inserting the comments as individual docs and using an index to
retrieve them as a single doc by post id.

https://github.com/kijanawoodard/Commenter/blob/master/Commenter.Docu...

https://github.com/kijanawoodard/Commenter/blob/master/Comments.Web/I...

I'll sort them so that the replies are inline either on the server or in
javascript.

Assumptions:
Most comments are of similar to "Yes, this is cool!!!", in other words
short.
Most posts will have less than 500 comments.
We will use cache headers / akamai to limit the number of requests to raven.

Anyone have any....comments? :-)

On Mon, Aug 13, 2012 at 3:23 AM, Oren Eini (Ayende Rahien) <


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mircea Chirea  
View profile  
 More options Aug 14 2012, 5:41 pm
From: Mircea Chirea <chirea.mir...@gmail.com>
Date: Tue, 14 Aug 2012 14:41:36 -0700 (PDT)
Local: Tues, Aug 14 2012 5:41 pm
Subject: Re: Converting a relation model to a document model

Those assumptions don't hold in my usage scenario; comments are going to
have a limit of 500-1000 characters and posts could have well over 500
comments too (but that's something to worry about in the future).

But yeah, those are my reasons as having comments separate from posts.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Kijana Woodard  
View profile  
 More options Aug 14 2012, 5:46 pm
From: Kijana Woodard <kijana.wood...@gmail.com>
Date: Tue, 14 Aug 2012 16:46:36 -0500
Local: Tues, Aug 14 2012 5:46 pm
Subject: Re: [RavenDB] Re: Converting a relation model to a document model

@Mircea - sorry, I hijacked your post. I should have started a new thread.
Those assumptions were for my situation, not yours.

On Tue, Aug 14, 2012 at 4:41 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »