You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
Happy New Year Meteorites!
Right now I'm working on getting oplog to support as many queries as
possible. In preparation, I've given minimongo a thorough
refactoring, based on testing lots of weird MongoDB query edge cases.
That's because the oplog feature uses minimongo's selector logic on
the server in security-critical codepaths.
minimongo has pretty good test coverage, so I feel pretty confident
about these improvements. My belief is that they don't affect most
apps and make some others slightly more MongoDB-compatible, and that
they don't significantly affect performance. But maybe I missed
something. I'd love to get feedback from our users as to how these
changes affect your apps!
Just run meteor with "meteor --release mongo-refactor" and let me know
if you notice any differences.
(This is based on devel, which does not contain the 'shark'
template-engine changes.)
--dave
Tim Heckel
unread,
Jan 6, 2014, 1:52:23 PM1/6/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
David, I pulled down devel just recently and am running off of that...also, I had our mongo provider spin up a new replica set for the sake of testing the oplog tailing. I mentioned to Avi that we use a number of $-based queries...as those get added, can I assume that the FACTS observe-drivers-polling will go down, and observe-drivers-oplog will go up? I will run the app through its paces and let you know, but I have confirmed oplog tailing is working when I direct my local MONGO_URL and MONGO_OPLOG_URL to our testing databases in the cloud via the facts package. I sent you an email with a screenshot...
In the meantime, I'll use devel against our app for a bit here and let you know if I see any strange minimongo exceptions. Thanks!
David Glasser
unread,
Jan 6, 2014, 1:56:39 PM1/6/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
Note that I haven't actually switched the $ queries over to oplog
observe driver yet, though I'm going to do that very soon. Merely
rewrote the underlying logic in minimongo itself (which will mostly
affect, or hopefully not affect, your *client app* right now).
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
Right...that I understood...and so far so good. The client side queries all seem to be working well for me using the latest devel...hope this helps
David Glasser
unread,
Jan 8, 2014, 2:13:37 AM1/8/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
Since this seems good so far, I've put out another release that enable
oplog for most $ operators!
Try your app with `meteor --release oplog-with-operators`! (In
production, you still need to configure the oplog environment
variable.)
oplog is still disabled for skip/limit queries, or for queries with
$where or $near.
Tim Heckel
unread,
Jan 8, 2014, 4:54:39 PM1/8/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
David, I am going to test this shortly -- thanks for these improvements, that sounds great!
I have a question -- a bit off this topic. We are having some issues with our node servers cumulatively spiking since moving to 0.7.1 from 0.6.4.1....we have a sharded db, but have NOT specified MONGO_OPLOG_URL -- can you think of any difference adding the disable-oplog package would make if we haven't even specified an oplog url? I know sharding is not supported for oplog-tailing. I'm just trying to eliminate this as a possible issue for our prod env and the slowdown of the web servers. Does my question make sense?
Tim Heckel
unread,
Jan 9, 2014, 10:02:33 AM1/9/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
David, I just figured out our issue -- and it is NOT due to any issues with the mongo driver in 0.7.0.1 at all -- we had a problem with our main nginx configuration that was routing traffic with invalid sticky sessions to one of our web servers and causing havoc on that box. I did roll back to 0.6.6 instead of 0.6.4.1 because of a compatibility issue between iron-router and 0.6.4.1, but plan on going back to 0.7.0.1 now shortly. We will continue testing oplog tailing on our newly created replica set. Thanks again for your help in troubleshooting.
David Glasser
unread,
Jan 9, 2014, 1:14:09 PM1/9/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
Great.
0.7.0.* is expected to be pretty compatible with 0.6.6.* apps, btw
(eg, we didn't think any changes were backwards-incompatible enough to
warrant release notes on 'meteor update').
> --
> You received this message because you are subscribed to the Google Groups
> "meteor-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
Hi Dave,
Just ran a couple tests of our app using the oplog-with-operations release and everything looks good so far.
Noticed `observeChanges` now requires `_id` to be returned (ie. can't do `_id: 0` in `fields:{}`) so that would be nice to mention in the changelog.
Nice job!
David Glasser
unread,
Jan 11, 2014, 5:00:21 PM1/11/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
Yeah, that was an explicit change which will show up in our
pre-release History.md pass. (I haven't been so good lately about
making History.md changes as I go; should try harder since I know how
important changelogs are!)
Tim Heckel
unread,
Jan 13, 2014, 4:38:55 PM1/13/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
David, I've finally had a chance to now run again against our oplog db....in the facts package, I no longer see any observe-drivers-polling, and I only see observe-drivers-oplog (there are 54) ... can I assume that we are now only using the oplog for difs? If so, woo-hoo ! We need to start doing some perf benchmarking really soon...
I will test our app with this setup and let you know
David Glasser
unread,
Jan 13, 2014, 4:53:22 PM1/13/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
Yup, that's what that means.
> "meteor-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
Yup, that means you're using oplog entirely. Very cool!
Are your running multiple meteor app processes? Depending on your data access pattern, oplog may or may not help on one CPU. Where it really shines is running on multiple CPUs at once.
Very curious to hear what you find =)
-- Nick
Tim Heckel
unread,
Jan 14, 2014, 6:39:48 PM1/14/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
I will hopefully have a chance to do some real testing very soon! I'm really excited to try this. And yes, we've had up to 12 (!) 8 core machines running our meteor app (each core we have configured listening on a different port with nginx) to keep up with the load. We've also recently (i think) been running up against connection pooling issues, and hope that the oplog tailing will help alleviate that as well...my biggest concern with the switch is our db performance, because we'll be going from a sharded to a non-sharded. Do you think it's beneficial then to change our readPreference to always be the secondary and leave the primary for the writes?
David Glasser
unread,
Jan 14, 2014, 7:19:24 PM1/14/14
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Sign in to report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to meteo...@googlegroups.com, meteo...@googlegroups.com
If that's consistent with the level of consistency your app requires.