I need a way to load a user's replies, including information about the post they're on. In the SQL world I'd have several joins which would bring in all the information, but here I'm a stumped. I am not sure how to do the query without causing a bazillion of queries to load the posts, which would be a significant amount if the user posts a stupid comment. I suspect a Map/Reduce (probably multi-map) index would need to be created to process the replies and simply query them by user ID. This is the basic idea http://codepaste.net/mp8x1p
I also need to get a count of how many *unread* replies there are for a particular user. That is a much simpler query: http://codepaste.net/m8fxp9 (and just do a Count() of the results).
Each comment doesn't store its replies directly, it stores a Ref class (which at the moment only has the Id property), as I need every comment to be addressed separately by an ID. The same applies for posts (cmp.First is a reference class).
Note: I have no idea if those queries would even run on RavenDB; I suspect they will, albeit not terribly efficient, I made them up to get the idea across.
There is no relation between having a unique id for a reply and having a
separate document.
How many replies do you expect to have?
Why are you splitting them apart like this?
And is there a reason you can't just query for the reply based on the user
id?
On Mon, Aug 20, 2012 at 1:29 AM, Mircea Chirea <chirea.mir...@gmail.com>wrote:
> I need a way to load a user's replies, including information about the
> post they're on. In the SQL world I'd have several joins which would bring
> in all the information, but here I'm a stumped. I am not sure how to do the
> query without causing a bazillion of queries to load the posts, which would
> be a significant amount if the user posts a stupid comment. I suspect a
> Map/Reduce (probably multi-map) index would need to be created to process
> the replies and simply query them by user ID. This is the basic idea
> http://codepaste.net/mp8x1p
> I also need to get a count of how many *unread* replies there are for a
> particular user. That is a much simpler query: http://codepaste.net/m8fxp9 (and
> just do a Count() of the results).
> Each comment doesn't store its replies directly, it stores a Ref class
> (which at the moment only has the Id property), as I need every comment to
> be addressed separately by an ID. The same applies for posts (cmp.First is
> a reference class).
> Note: I have no idea if those queries would even run on RavenDB; I suspect
> they will, albeit not terribly efficient, I made them up to get the idea
> across.
Well how do I get an unique ID while having a separate document? I kinda need a way to load all comments, regardless of reply or not, and sort them on various properties, as well as search within them. I expect to have an at most 50 replies to any single comment. I am splitting them apart because I need to address comments independently.
I cannot query replies directly based on user ID, because I need the replies too all a user's comments. So I have users/5, he has five comments, each with a few replies. I need all of those replies.
Well, look at how we do it for RaccoonBlog, we have a numeric id that is
unique within the scope of a single document.
So the comment have an id that is valid within the doc, and the full
"comment id" include the doc id as well.
What are your aggregates, and what are the ops you need to support?
On Mon, Aug 20, 2012 at 4:03 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:
> Well how do I get an unique ID while having a separate document? I kinda
> need a way to load all comments, regardless of reply or not, and sort them
> on various properties, as well as search within them.
> I expect to have an at most 50 replies to any single comment.
> I am splitting them apart because I need to address comments independently.
> I cannot query replies directly based on user ID, because I need the
> replies too all a user's comments. So I have users/5, he has five comments,
> each with a few replies. I need all of those replies.
I kinda need all comments to have unique ids, whether they're a reply to another or not. All of them. The JS code and default page sorting depends on this and it would be a lot of work to rewrite those.
My aggregates are User, Comment, Comparison. Each comparison has two topics; each topic within a comparison has comments; A user has several comments and needs to be notified of new replies; I want to show the user on where the reply was posted, so a join with Comparison (the topic's name is denormalized).
So I need to get all comments which are a reply to user/1234`s comments (ReplyTo is not null), including the comparison they're posted on (ids and names of both topics), as well as the information of the user who wrote the reply (this is denormalized in the User.Ref class, so not an issue).
On Monday, August 20, 2012 4:08:11 PM UTC+3, Oren Eini wrote:
> Well, look at how we do it for RaccoonBlog, we have a numeric id that is > unique within the scope of a single document. > So the comment have an id that is valid within the doc, and the full > "comment id" include the doc id as well.
> What are your aggregates, and what are the ops you need to support?
> On Mon, Aug 20, 2012 at 4:03 PM, Mircea Chirea <chirea...@gmail.com<javascript:> > > wrote:
>> Well how do I get an unique ID while having a separate document? I kinda >> need a way to load all comments, regardless of reply or not, and sort them >> on various properties, as well as search within them. >> I expect to have an at most 50 replies to any single comment. >> I am splitting them apart because I need to address comments >> independently.
>> I cannot query replies directly based on user ID, because I need the >> replies too all a user's comments. So I have users/5, he has five comments, >> each with a few replies. I need all of those replies.
On Mon, Aug 20, 2012 at 5:37 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:
> I kinda need all comments to have unique ids, whether they're a reply to
> another or not. All of them. The JS code and default page sorting depends
> on this and it would be a lot of work to rewrite those.
As I said, you don't need to have a separate doc for each reply to give
them unique ids.
> My aggregates are User, Comment, Comparison. Each comparison has two
> topics; each topic within a comparison has comments; A user has several
> comments and needs to be notified of new replies; I want to show the user
> on where the reply was posted, so a join with Comparison (the topic's name
> is denormalized).
> So I need to get all comments which are a reply to user/1234`s comments
> (ReplyTo is not null), including the comparison they're posted on (ids and
> names of both topics), as well as the information of the user who wrote the
> reply (this is denormalized in the User.Ref class, so not an issue).
Model comments like this:
public class Comment
{
public Comment[] Replies;
}
Otherwise, you are going to have to do two queries to do this, and you
probably don't want to go there.
> On Monday, August 20, 2012 4:08:11 PM UTC+3, Oren Eini wrote:
>> Well, look at how we do it for RaccoonBlog, we have a numeric id that is
>> unique within the scope of a single document.
>> So the comment have an id that is valid within the doc, and the full
>> "comment id" include the doc id as well.
>> What are your aggregates, and what are the ops you need to support?
>> On Mon, Aug 20, 2012 at 4:03 PM, Mircea Chirea <chirea...@gmail.com>wrote:
>>> Well how do I get an unique ID while having a separate document? I kinda
>>> need a way to load all comments, regardless of reply or not, and sort them
>>> on various properties, as well as search within them.
>>> I expect to have an at most 50 replies to any single comment.
>>> I am splitting them apart because I need to address comments
>>> independently.
>>> I cannot query replies directly based on user ID, because I need the
>>> replies too all a user's comments. So I have users/5, he has five comments,
>>> each with a few replies. I need all of those replies.
How is Raccoon avoiding race conditions on
comments.GenerateNewCommentId<http://sourcecodereader.apphb.com/>
()?
Are BackgroundTasks processed single threaded?
On Mon, Aug 20, 2012 at 2:49 PM, Oren Eini (Ayende Rahien) <
> On Mon, Aug 20, 2012 at 5:37 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:
>> I kinda need all comments to have unique ids, whether they're a reply to
>> another or not. All of them. The JS code and default page sorting depends
>> on this and it would be a lot of work to rewrite those.
> As I said, you don't need to have a separate doc for each reply to give
> them unique ids.
>> My aggregates are User, Comment, Comparison. Each comparison has two
>> topics; each topic within a comparison has comments; A user has several
>> comments and needs to be notified of new replies; I want to show the user
>> on where the reply was posted, so a join with Comparison (the topic's name
>> is denormalized).
>> So I need to get all comments which are a reply to user/1234`s comments
>> (ReplyTo is not null), including the comparison they're posted on (ids and
>> names of both topics), as well as the information of the user who wrote the
>> reply (this is denormalized in the User.Ref class, so not an issue).
> Model comments like this:
> public class Comment
> {
> public Comment[] Replies;
> }
> Otherwise, you are going to have to do two queries to do this, and you
> probably don't want to go there.
>> On Monday, August 20, 2012 4:08:11 PM UTC+3, Oren Eini wrote:
>>> Well, look at how we do it for RaccoonBlog, we have a numeric id that is
>>> unique within the scope of a single document.
>>> So the comment have an id that is valid within the doc, and the full
>>> "comment id" include the doc id as well.
>>> What are your aggregates, and what are the ops you need to support?
>>> On Mon, Aug 20, 2012 at 4:03 PM, Mircea Chirea <chirea...@gmail.com>wrote:
>>>> Well how do I get an unique ID while having a separate document? I
>>>> kinda need a way to load all comments, regardless of reply or not, and sort
>>>> them on various properties, as well as search within them.
>>>> I expect to have an at most 50 replies to any single comment.
>>>> I am splitting them apart because I need to address comments
>>>> independently.
>>>> I cannot query replies directly based on user ID, because I need the
>>>> replies too all a user's comments. So I have users/5, he has five comments,
>>>> each with a few replies. I need all of those replies.
> On Mon, Aug 20, 2012 at 5:37 PM, Mircea Chirea <chirea...@gmail.com<javascript:> > > wrote:
>> I kinda need all comments to have unique ids, whether they're a reply to >> another or not. All of them. The JS code and default page sorting depends >> on this and it would be a lot of work to rewrite those.
> As I said, you don't need to have a separate doc for each reply to give > them unique ids.
I can't find exactly how to have a globally unique id for each reply; globally unique as no other document has the same ID, so not unique only with one comment. That was I can easily load all of them, reply or not and make sure the rest of the code is oblivious of their status. Unless of course you have a better idea; since I'm rewriting almost all database access code I can rewrite it in any way.
> My aggregates are User, Comment, Comparison. Each comparison has two >> topics; each topic within a comparison has comments; A user has several >> comments and needs to be notified of new replies; I want to show the user >> on where the reply was posted, so a join with Comparison (the topic's name >> is denormalized).
>> So I need to get all comments which are a reply to user/1234`s comments >> (ReplyTo is not null), including the comparison they're posted on (ids and >> names of both topics), as well as the information of the user who wrote the >> reply (this is denormalized in the User.Ref class, so not an issue).
> Model comments like this:
> public class Comment > {
> public Comment[] Replies; > }
> Otherwise, you are going to have to do two queries to do this, and you > probably don't want to go there.
Right. That wouldn't be a problem assuming the ID issue has an easy fix.
The thing is, how do I query all comments? I wan thinking of this index, but `Hierarchy` is not found:
Map = comments => from c in comments let all = Hierarchy(c, "Replies").Union(new[] { c }) from x in all select x;
On Monday, August 20, 2012 4:08:11 PM UTC+3, Oren Eini wrote:
>>> Well, look at how we do it for RaccoonBlog, we have a numeric id that is >>> unique within the scope of a single document. >>> So the comment have an id that is valid within the doc, and the full >>> "comment id" include the doc id as well.
>>> What are your aggregates, and what are the ops you need to support?
>>> On Mon, Aug 20, 2012 at 4:03 PM, Mircea Chirea <chirea...@gmail.com>wrote:
>>>> Well how do I get an unique ID while having a separate document? I >>>> kinda need a way to load all comments, regardless of reply or not, and sort >>>> them on various properties, as well as search within them. >>>> I expect to have an at most 50 replies to any single comment. >>>> I am splitting them apart because I need to address comments >>>> independently.
>>>> I cannot query replies directly based on user ID, because I need the >>>> replies too all a user's comments. So I have users/5, he has five comments, >>>> each with a few replies. I need all of those replies.
> How is Raccoon avoiding race conditions on comments.GenerateNewCommentId<http://sourcecodereader.apphb.com/>
> ()?
> Are BackgroundTasks processed single threaded?
> On Mon, Aug 20, 2012 at 2:49 PM, Oren Eini (Ayende Rahien) <
> aye...@ayende.com> wrote:
>> inline
>> On Mon, Aug 20, 2012 at 5:37 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:
>>> I kinda need all comments to have unique ids, whether they're a reply to
>>> another or not. All of them. The JS code and default page sorting depends
>>> on this and it would be a lot of work to rewrite those.
>> As I said, you don't need to have a separate doc for each reply to give
>> them unique ids.
>>> My aggregates are User, Comment, Comparison. Each comparison has two
>>> topics; each topic within a comparison has comments; A user has several
>>> comments and needs to be notified of new replies; I want to show the user
>>> on where the reply was posted, so a join with Comparison (the topic's name
>>> is denormalized).
>>> So I need to get all comments which are a reply to user/1234`s comments
>>> (ReplyTo is not null), including the comparison they're posted on (ids and
>>> names of both topics), as well as the information of the user who wrote the
>>> reply (this is denormalized in the User.Ref class, so not an issue).
>> Model comments like this:
>> public class Comment
>> {
>> public Comment[] Replies;
>> }
>> Otherwise, you are going to have to do two queries to do this, and you
>> probably don't want to go there.
>>> On Monday, August 20, 2012 4:08:11 PM UTC+3, Oren Eini wrote:
>>>> Well, look at how we do it for RaccoonBlog, we have a numeric id that
>>>> is unique within the scope of a single document.
>>>> So the comment have an id that is valid within the doc, and the full
>>>> "comment id" include the doc id as well.
>>>> What are your aggregates, and what are the ops you need to support?
>>>> On Mon, Aug 20, 2012 at 4:03 PM, Mircea Chirea <chirea...@gmail.com>wrote:
>>>>> Well how do I get an unique ID while having a separate document? I
>>>>> kinda need a way to load all comments, regardless of reply or not, and sort
>>>>> them on various properties, as well as search within them.
>>>>> I expect to have an at most 50 replies to any single comment.
>>>>> I am splitting them apart because I need to address comments
>>>>> independently.
>>>>> I cannot query replies directly based on user ID, because I need the
>>>>> replies too all a user's comments. So I have users/5, he has five comments,
>>>>> each with a few replies. I need all of those replies.
> On Mon, Aug 20, 2012 at 5:37 PM, Mircea Chirea <chirea...@gmail.com>wrote:
>>> I kinda need all comments to have unique ids, whether they're a reply to
>>> another or not. All of them. The JS code and default page sorting depends
>>> on this and it would be a lot of work to rewrite those.
>> As I said, you don't need to have a separate doc for each reply to give
>> them unique ids.
> I can't find exactly how to have a globally unique id for each reply;
> globally unique as no other document has the same ID, so not unique only
> with one comment. That was I can easily load all of them, reply or not and
> make sure the rest of the code is oblivious of their status. Unless of
> course you have a better idea; since I'm rewriting almost all database
> access code I can rewrite it in any way.
a) GUID
b) Use a numeric unique to the scope of a document, append it the doc id,
done.
>> My aggregates are User, Comment, Comparison. Each comparison has two
>>> topics; each topic within a comparison has comments; A user has several
>>> comments and needs to be notified of new replies; I want to show the user
>>> on where the reply was posted, so a join with Comparison (the topic's name
>>> is denormalized).
>>> So I need to get all comments which are a reply to user/1234`s comments
>>> (ReplyTo is not null), including the comparison they're posted on (ids and
>>> names of both topics), as well as the information of the user who wrote the
>>> reply (this is denormalized in the User.Ref class, so not an issue).
>> Model comments like this:
>> public class Comment
>> {
>> public Comment[] Replies;
>> }
>> Otherwise, you are going to have to do two queries to do this, and you
>> probably don't want to go there.
> Right. That wouldn't be a problem assuming the ID issue has an easy fix.
> The thing is, how do I query all comments? I wan thinking of this index,
> but `Hierarchy` is not found:
> Map = comments => from c in comments
> let all = Hierarchy(c, "Replies").Union(new[] { c })
> from x in all
> select x;
> On Monday, August 20, 2012 4:08:11 PM UTC+3, Oren Eini wrote:
>>>> Well, look at how we do it for RaccoonBlog, we have a numeric id that
>>>> is unique within the scope of a single document.
>>>> So the comment have an id that is valid within the doc, and the full
>>>> "comment id" include the doc id as well.
>>>> What are your aggregates, and what are the ops you need to support?
>>>> On Mon, Aug 20, 2012 at 4:03 PM, Mircea Chirea <chirea...@gmail.com>wrote:
>>>>> Well how do I get an unique ID while having a separate document? I
>>>>> kinda need a way to load all comments, regardless of reply or not, and sort
>>>>> them on various properties, as well as search within them.
>>>>> I expect to have an at most 50 replies to any single comment.
>>>>> I am splitting them apart because I need to address comments
>>>>> independently.
>>>>> I cannot query replies directly based on user ID, because I need the
>>>>> replies too all a user's comments. So I have users/5, he has five comments,
>>>>> each with a few replies. I need all of those replies.
from c in comments from all in Recurse(c, x => x.Replies) from x in all select x;
Would Recurse also include the source? In that query meaning the `c` in `from c in comments`? Or do I need to add it to the Recurse result? I couldn't find documentation for the function.
Yes, that is good.
But I am not sure about the index, it seems to imply that a Comparison
contains all comments and replies in a single doc.
Just to verify, we expect there to be a small (< 1,000) number of those,
right?
On Tue, Aug 21, 2012 at 2:16 AM, Mircea Chirea <chirea.mir...@gmail.com>wrote:
> from c in comments
> from all in Recurse(c, x => x.Replies)
> from x in all
> select x;
> Would Recurse also include the source? In that query meaning the `c` in
> `from c in comments`?
> Or do I need to add it to the Recurse result? I couldn't find
> documentation for the function.
Yes, a comparison contains all comments inside a single doc. Most won't have more than 100 comments, some will have quite a bit but still < 1000, I expect only a handful to have lots of comments but I don't think a few big docs would be a problem.
On Tuesday, August 21, 2012 10:15:53 AM UTC+3, Oren Eini wrote:
> Yes, that is good. > But I am not sure about the index, it seems to imply that a Comparison > contains all comments and replies in a single doc. > Just to verify, we expect there to be a small (< 1,000) number of those, > right?
> On Tue, Aug 21, 2012 at 2:16 AM, Mircea Chirea <chirea...@gmail.com<javascript:> > > wrote:
>> Alright, so that becomes:
>> from c in comments >> from all in Recurse(c, x => x.Replies) >> from x in all >> select x;
>> Would Recurse also include the source? In that query meaning the `c` in >> `from c in comments`? >> Or do I need to add it to the Recurse result? I couldn't find >> documentation for the function.
>> How is Raccoon avoiding race conditions on comments.GenerateNewCommentId<http://sourcecodereader.apphb.com/>
>> ()?
>> Are BackgroundTasks processed single threaded?
>> On Mon, Aug 20, 2012 at 2:49 PM, Oren Eini (Ayende Rahien) <
>> aye...@ayende.com> wrote:
>>> inline
>>> On Mon, Aug 20, 2012 at 5:37 PM, Mircea Chirea <chirea.mir...@gmail.com>wrote:
>>>> I kinda need all comments to have unique ids, whether they're a reply
>>>> to another or not. All of them. The JS code and default page sorting
>>>> depends on this and it would be a lot of work to rewrite those.
>>> As I said, you don't need to have a separate doc for each reply to give
>>> them unique ids.
>>>> My aggregates are User, Comment, Comparison. Each comparison has two
>>>> topics; each topic within a comparison has comments; A user has several
>>>> comments and needs to be notified of new replies; I want to show the user
>>>> on where the reply was posted, so a join with Comparison (the topic's name
>>>> is denormalized).
>>>> So I need to get all comments which are a reply to user/1234`s comments
>>>> (ReplyTo is not null), including the comparison they're posted on (ids and
>>>> names of both topics), as well as the information of the user who wrote the
>>>> reply (this is denormalized in the User.Ref class, so not an issue).
>>> Model comments like this:
>>> public class Comment
>>> {
>>> public Comment[] Replies;
>>> }
>>> Otherwise, you are going to have to do two queries to do this, and you
>>> probably don't want to go there.
>>>> On Monday, August 20, 2012 4:08:11 PM UTC+3, Oren Eini wrote:
>>>>> Well, look at how we do it for RaccoonBlog, we have a numeric id that
>>>>> is unique within the scope of a single document.
>>>>> So the comment have an id that is valid within the doc, and the full
>>>>> "comment id" include the doc id as well.
>>>>> What are your aggregates, and what are the ops you need to support?
>>>>> On Mon, Aug 20, 2012 at 4:03 PM, Mircea Chirea <chirea...@gmail.com>wrote:
>>>>>> Well how do I get an unique ID while having a separate document? I
>>>>>> kinda need a way to load all comments, regardless of reply or not, and sort
>>>>>> them on various properties, as well as search within them.
>>>>>> I expect to have an at most 50 replies to any single comment.
>>>>>> I am splitting them apart because I need to address comments
>>>>>> independently.
>>>>>> I cannot query replies directly based on user ID, because I need the
>>>>>> replies too all a user's comments. So I have users/5, he has five comments,
>>>>>> each with a few replies. I need all of those replies.
