Does Mongo 3 offers a priority feature for queries?

[yongzh...@gmail.com]

Hi all,

Recently I work on performance profiling analysis of Mongodb relative operations in our system. I figured out one major reason that the user experiences a long review page load time is that Mongod work on building collections' indexes for bunch of keys back-end. We would like to provide customers better product experience (quicker response & shorter query latency). I just wonder if Mongo 3.0 offers a priority feature for queries. That is, Mongod periodically scans the waiting tasks/queries and try to respond those ones with high priorities. Actually I think this should be a popular requirements since for most of systems there are always some clients/queries being more important than other clients/queries.

I searched this forum and the most relative one I found is as follow. Any relative development since then? Thanks a lot.

https://groups.google.com/forum/#!searchin/mongodb-user/priority$20query/mongodb-user/CcqyJLZRKPo/grCE8FCjex8J


Best Regards,
-Yongzhi

[Asya Kamsky]

There is no such feature currently. However, shouldn't you be doing
your index builds off-line (i.e. not on a running live system)? I'm
assuming you're talking about adding an index, not regular inserts and
updates updating indexes, as that's part of the basic write work).

Asya

> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user"
> group.
>
> For other MongoDB technical support options, see:
> http://www.mongodb.org/about/support/.
> ---
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mongodb-user...@googlegroups.com.
> To post to this group, send email to mongod...@googlegroups.com.
> Visit this group at http://groups.google.com/group/mongodb-user.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mongodb-user/3bd8ec05-3d9c-4cbb-a5e7-704cb35af299%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[yongzh...@gmail.com]

Hello Asya,

Building indexes is a part of the analysis and the customers will need indexes right away when they review the results after the analysis complete.

My point is that normally customers have better patience in waiting for analysis execution than waiting for loading review page since the analysis may take a while but they want to see the data soon after they click the button. For example, in my test case, when 5 analyses are running concurrently, the building indexes may take 5*9=45 seconds. When mongod is NOT busy, it take ~10 seconds to load the review page. However, when mongod is busy with building indexes, the customer has to wait for one minute to see the results. The customer may think the system is slow so s/he has to wait for a long time. If mongo provides priority feature for queries and respond this kind of query prior to the index building , the customer may have much better experience and it takes 10 more second for the analysis execution. This extra time is nothing, compared to original 10-30 minute execution time. Thanks a lot.

Best Regards,
-Yongzhi

[Stephen Steneker]

On Friday, 26 June 2015 09:11:43 UTC+10, yongzhi.chen wrote:

Building indexes is a part of the analysis and the customers will need indexes right away when they review the results after the analysis complete.

Hi,

You can also build indexes in the background to avoid the default blocking behaviour of foreground index builds:
  http://docs.mongodb.org/manual/tutorial/build-indexes-in-the-background/

Background index builds will allow read & write operations to continue during the initial index builds.

Note: if you have a replica set deployment you should upgrade to MongoDB 2.6 or newer for support of background index builds on secondaries:

 http://docs.mongodb.org/manual/release-notes/2.6/#index-build-enhancements

Regards,

Stephen

[yongzh...@gmail.com]

Hello Stephen,

Thank you so much for pointing out this option. I just conducted a test to verify and it seemed that this may solve my issue.

My understanding is that:
(1) The cost or disadvantage of background indexes is longer building time and larger index storage size;
(2) Those background indexes would be created by the end of the execution and they are functionally identical to those index build in the foreground.

Is my understanding correct? Thanks a lot.

Best Regards,
-Yongzhi