Having trouble designing how to store data for "who viewed this" functionality
Gavin
use mongo for this).
Scenario:
Need to keep track of who viewed something. Should be unique and
ordered by when the user viewed something. Every time they view, they
should go back to the front of the list.
Solution Ideas:
More traditional sql approach (suggested by coworker, works, but I
don't really like the design):
Insert one row(document) at a time with a timestamp.
If a previous user views again, they get get an updated timestamp.
Every insert of new row, we find the first 10 rows, look at the oldest
timestamp, and delete anything older than that.
Alternative
Have one document for a user with a key for who viewed them.
1) $pull user from document.whoViewed
2) $push user to document.whoViewed
3) if document.whoViewed.size() > 10, then $pop a user
I'm just not sure how to do step 3 properly.
Any help would be appreciated. I'm using the perl driver and mongo
1.6.
Gavin
Scott Hernandez
approach. What is wrong with that? It is a very flat collection, but
that isn't nec. bad.
You could also keep an array of views embedded in each user but that
will not perform well if they are viewing lots of new things,
constantly.
How many things can they view as compared to the number of users? How
often will they be viewing something new versus updating their last
viewed time?
> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>
Gavin
Sorry, the design is who checked out my profile.
User a clicks on user b's profile
User b can see the last 10 users who has viewed their profile.
If user a looks again then tney move back to the front of the list.
The simple 3 column (viewer; viewie, timestamp) works, but from the
docs I thought keeping the data together in one document was good
practice. But I'm afraid that an embedded array can grow too large
without a simple way to trim it.
Scott Hernandez
array feature is added (http://jira.mongodb.org/browse/SERVER-991).
You can manage this yourself for now by periodically culling the array
to 10 items. You would only need to process the users who have
recently been viewed.
It still might be best to just keep that flat collection if you want
historical information, other that the latest 10.
Gavin
top) so I don't think that'll be an option.
By flat you mean 1 row/document per set of data, so user a, user b,
timestamp type thing?
Scott Hernandez
> That doesn't look like it'll be done until 1.9 (says the info at the
> top) so I don't think that'll be an option.
No, but you can do yourself, and replace it with the feature when it
comes out (in a few months).
>
> By flat you mean 1 row/document per set of data, so user a, user b,
> timestamp type thing?
Yes, you can either do an update w/upsert, or just insert a new one
each view. An index will help return the latest 10, quickly.
Keith Branton
Scott Hernandez
> Even with http://jira.mongodb.org/browse/SERVER-991 you would still need a
> modifier that doesn't exist. A way of pushing to an array while maintaining
> set semantics - so your new item always appears at the end of the set
> regardless if it was in there previously.
set != ordered/sorted.
You can $pull + $push, since you want to replace the item. That will
allow you to maintain set-like behavior, with newest at the bottom.
> Since I am also very interested in implementing this use case in my own web
> site, I have raised a jira requesting $pushToSet which would do
> that: http://jira.mongodb.org/browse/SERVER-2362. Votes will help it get
> scheduled sooner :)
This seems pretty complicated. A lot this should be handled with
virtual collections and the ability to $addToSet/$push/$pushAll to the
top, or bottom (current behavior).
> If you follow Scott's advice and do this yourself for now then it would
> probably be best to have a single array of visitors in one document per
> profile instead of document per visit - much less to change later.
Yes, I was suggesting you keep the last 10 visitor list in the user/profile.
> The flat option Scott suggested would probably be faster for inserts, and so
> that may be the only one you can make work in the interim, depending on your
> traffic.
This will produce *much* more data, and should be cheaper on insert
but at the cost of more expensive indexes.
Keith Branton
On Sat, Jan 15, 2011 at 11:06 AM, Keith Branton <ke...@branton.co.uk> wrote:set != ordered/sorted.
> Even with http://jira.mongodb.org/browse/SERVER-991 you would still need a
> modifier that doesn't exist. A way of pushing to an array while maintaining
> set semantics - so your new item always appears at the end of the set
> regardless if it was in there previously.
You can $pull + $push, since you want to replace the item. That will
allow you to maintain set-like behavior, with newest at the bottom.
This seems pretty complicated. A lot this should be handled with
> Since I am also very interested in implementing this use case in my own web
> site, I have raised a jira requesting $pushToSet which would do
> that: http://jira.mongodb.org/browse/SERVER-2362. Votes will help it get
> scheduled sooner :)
virtual collections and the ability to $addToSet/$push/$pushAll to the
top, or bottom (current behavior).
Scott Hernandez
> @Scott
>
> On Sat, Jan 15, 2011 at 1:01 PM, Scott Hernandez <scotthe...@gmail.com>
> wrote:
>>
>> On Sat, Jan 15, 2011 at 11:06 AM, Keith Branton <ke...@branton.co.uk>
>> wrote:
>> > Even with http://jira.mongodb.org/browse/SERVER-991 you would still need
>> > a
>> > modifier that doesn't exist. A way of pushing to an array while
>> > maintaining
>> > set semantics - so your new item always appears at the end of the set
>> > regardless if it was in there previously.
>>
>> set != ordered/sorted.
>
> I think you're nit-picking here. I don't consider LinkedHashSet in java to
> be breaking set semantics, yet it maintains insertion order. Wikipedia
> defines set as "a collection of distinct objects" - good enough for me. By
> "maintaining set semantics" I meant "ensuring no duplicates". Anyway, these
> are implemented as arrays in mogo which certainly are ordered.
Yep, I was just pointing out that sets don't *need* to be ordered. I
wasn't trying to nit-pick, but just make it clear that that wasn't an
assumption my answers included.
>>
>> You can $pull + $push, since you want to replace the item. That will
>> allow you to maintain set-like behavior, with newest at the bottom.
>
> No I can't. Not in one isolated operation - I get a "Field name duplication
> not allowed with modifiers" error when I try this in the shell. At that's
> what I get in 1.7.4.
Yes, there is an outstanding bug: http://jira.mongodb.org/browse/SERVER-1050
>> > Since I am also very interested in implementing this use case in my own
>> > web
>> > site, I have raised a jira requesting $pushToSet which would do
>> > that: http://jira.mongodb.org/browse/SERVER-2362. Votes will help it get
>> > scheduled sooner :)
>>
>> This seems pretty complicated. A lot this should be handled with
>> virtual collections and the ability to $addToSet/$push/$pushAll to the
>> top, or bottom (current behavior).
>>
>
> Should? perhaps, yet these operations are not specified
> on http://jira.mongodb.org/browse/SERVER-142. In fact pretty much nothing is
> specified beyond db.foo.$bar.find(). I have to be honest I don't even know
> what that means. It is not scheduled, despite having a huge number of votes
Yeah, it does seem to pointed to like a magic bullet but without any
specification. I feel like the lack or implementation isn't because it
isn't well understood or conceptualized but because it will change a
lot of how things work internally.
> (which suggests it lacks specification) and depends on another unscheduled
> task. I'm not holding my breath for that golden hammer until I understand
> what it really is!
As well you shouldn't; unless you amazing lungs.
> No offence intended, but you seem to have missed the main point - with add
> to set new items are not added (or moved) if they already exist. With
No, I get it but my point is that ordering doesn't matter for meeting
a set requirement. Why should there be a restriction that they stay in
the same ordinal position? That my be a side effect of the
implementation, but it isn't a requirement.
> $pushToSet the idea is to treat the array like both a set (like $addToSet
> does) and a stack (like $push does) - the new items will always appear at
> the end. If the array already includes one of the new items then it would be
> removed first. Nothing you are suggesting would achieve this.
If you could $pull + $push (since the $addToSet doesn't mean much at
that point) would that not achieve your goal?
> If it seems complicated then perhaps my explanations are not as clear as
> they should be, as they are very simple variations of existing features imo.
> If you can point out where the complexity is I'd be happy to try again!
I'll add some comments to the issue :)
> Regards,
> Keith.
Keith Branton
Yep, I was just pointing out that sets don't *need* to be ordered. I
wasn't trying to nit-pick, but just make it clear that that wasn't an
assumption my answers included.
Yes, there is an outstanding bug: http://jira.mongodb.org/browse/SERVER-1050
> $pushToSet the idea is to treat the array like both a set (like $addToSetIf you could $pull + $push (since the $addToSet doesn't mean much at
> does) and a stack (like $push does) - the new items will always appear at
> the end. If the array already includes one of the new items then it would be
> removed first. Nothing you are suggesting would achieve this.
that point) would that not achieve your goal?