Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Is there any way to make this query faster?
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  14 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Tareq Perez  
View profile  
 More options Jun 22 2012, 9:28 am
From: Tareq Perez <tare...@gmail.com>
Date: Fri, 22 Jun 2012 06:28:07 -0700 (PDT)
Local: Fri, Jun 22 2012 9:28 am
Subject: Is there any way to make this query faster?

I need to retrieve the number of records from a Machine of a certain Type
during the date of 7 am and 7 am the next day (24 hour period)

This is the structure of my document:

//Date is in Ticks
 public Long Date { get; set; }
        public string Process { get; set; }
        public string Type { get; set; }
        public int Index { get; set; }
        public string Machine { get; set; }
        public string Plant { get; set; }
        public string Body { get; set; }
public int Hour { get; set; }
public string key{ get; set;}

The query I'm currently using is

session.Query<Warning>("GetWarningCountByTypeAndMachine").Statistics(out
stats)
.Where(x => x.Machine.Equals(MachineName,
StringComparison.OrdinalIgnoreCase) && x.Type.Equals(WarningType,
StringComparison.OrdinalIgnoreCase) && x.Date >= date_begin.Ticks && x.Date
<= date_end.Ticks).ToList();

where GetWarningCountByTypeAndMachine is just:

from doc in docs
select new { Machine = doc.Machine, Type = doc.Type, Date = doc.Date }

Is there any way to make a reduction or just any faster way to query this?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Kijana Woodard  
View profile  
 More options Jun 22 2012, 10:36 am
From: Kijana Woodard <kijana.wood...@gmail.com>
Date: Fri, 22 Jun 2012 09:36:14 -0500
Local: Fri, Jun 22 2012 10:36 am
Subject: Re: [RavenDB] Is there any way to make this query faster?

How fast is fast?

How many docs?

You could try preprocessing/reducing the ticks into a "report date". So
"6/22/2012" would be 7am 6/22/2012 - 6:59:59 6/23/2012. You could also just
compress that to an long like round ticks back to the previous 7am ticks.
Make sense?

Whatever you do, benchmark to make sure you're getting gains.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Chris Marisic  
View profile  
 More options Jun 22 2012, 10:48 am
From: Chris Marisic <ch...@marisic.com>
Date: Fri, 22 Jun 2012 07:48:59 -0700 (PDT)
Local: Fri, Jun 22 2012 10:48 am
Subject: Re: [RavenDB] Is there any way to make this query faster?

I agree with Oren, creating a key that you're actually going to reduce on
seems to be a much more optimal solution.

I'd probably add a property to the model to reduce over, something like

public string ReportingDateGroup { return real date do something to create
key that creates 1 value for the entire day you wish to cover; }


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Kijana Woodard  
View profile  
 More options Jun 22 2012, 11:17 am
From: Kijana Woodard <kijana.wood...@gmail.com>
Date: Fri, 22 Jun 2012 10:17:10 -0500
Local: Fri, Jun 22 2012 11:17 am
Subject: Re: [RavenDB] Is there any way to make this query faster?

Thanks for that compliment. ;-)


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Matt Warren  
View profile  
 More options Jun 22 2012, 12:34 pm
From: Matt Warren <mattd...@gmail.com>
Date: Fri, 22 Jun 2012 09:34:59 -0700 (PDT)
Local: Fri, Jun 22 2012 12:34 pm
Subject: Re: [RavenDB] Is there any way to make this query faster?

Yeah that's a good idea, large range queries can be slow,

So this :
     x.Date >= date_begin.Ticks && x.Date <= date_end.Ticks

will be slower than this:
     x.ReportHour == 7


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile  
 More options Jun 22 2012, 1:50 pm
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Fri, 22 Jun 2012 20:50:17 +0300
Local: Fri, Jun 22 2012 1:50 pm
Subject: Re: [RavenDB] Is there any way to make this query faster?

Note that in 1.2 we optimized this pattern:   x.Date >= date_begin.Ticks &&
x.Date <= date_end.Ticks
So it would be faster, but doing an exact match is still faster.

Normalizing the values is probably the easiest way to get faster
performance.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Tareq Perez  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 1:35 pm
From: Tareq Perez <tare...@gmail.com>
Date: Mon, 25 Jun 2012 10:35:56 -0700 (PDT)
Local: Mon, Jun 25 2012 1:35 pm
Subject: Re: [RavenDB] Is there any way to make this query faster?

> thanks for the tips. But is there a map reduction I can make for this?

The map function is the same as my original post, but is there some way to
reduce this using the date.Ticks ranges?

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 1:59 pm
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Mon, 25 Jun 2012 20:59:10 +0300
Local: Mon, Jun 25 2012 1:59 pm
Subject: Re: [RavenDB] Is there any way to make this query faster?

I am not sure that I am following, can you explain in more details?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Tareq Perez  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 2:06 pm
From: Tareq Perez <tare...@gmail.com>
Date: Mon, 25 Jun 2012 11:06:48 -0700 (PDT)
Local: Mon, Jun 25 2012 2:06 pm
Subject: Re: [RavenDB] Is there any way to make this query faster?

I'm trying to get a list of documents who's date falls in range with a
specified date.

I have a Machine, Type, and begin/end Date.

I was wondering if there was a map reduction function I could write to
speed up the query for finding the documents that fit the date range
criteria.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 2:15 pm
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Mon, 25 Jun 2012 21:15:11 +0300
Local: Mon, Jun 25 2012 2:15 pm
Subject: Re: [RavenDB] Is there any way to make this query faster?

You want to get the total count for machine / type for a range?
What sort of range? Day/ month?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Tareq Perez  
View profile  
 More options Jun 25 2012, 2:20 pm
From: Tareq Perez <tare...@gmail.com>
Date: Mon, 25 Jun 2012 11:20:25 -0700 (PDT)
Local: Mon, Jun 25 2012 2:20 pm
Subject: Re: [RavenDB] Is there any way to make this query faster?

Nope, I want a List of each document grouped by Machine that fall within
the range of begin and end date that are of the same Type. And the
difference between my begin and end date is 1 day. In my original post you
can see that I stored the date in Ticks, so some converting is necessary.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Itamar Syn-Hershko  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 3:00 pm
From: Itamar Syn-Hershko <ita...@hibernatingrhinos.com>
Date: Mon, 25 Jun 2012 22:00:29 +0300
Local: Mon, Jun 25 2012 3:00 pm
Subject: Re: [RavenDB] Is there any way to make this query faster?

Save all that 24-period info in one document, and give them IDs like
infos/machine-name/2012-06-25, and then you can get them very efficiently
directly from the document store using StartsWith


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
nightwatch  
View profile  
 More options Jun 26 2012, 5:33 am
From: nightwatch <rafal.gwizd...@gmail.com>
Date: Tue, 26 Jun 2012 02:33:54 -0700 (PDT)
Local: Tues, Jun 26 2012 5:33 am
Subject: Re: [RavenDB] Is there any way to make this query faster?

are you saying that a simple  'select' query is too slow and has to be
replaced with some trickery?
How 'slow' it is now and how fast does it need to be? If search criteria
are always the same then maybe caching would be the solution?

rg


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Oren Eini (Ayende Rahien)  
View profile   Translate to Translated (View Original)
 More options Jun 26 2012, 6:18 am
From: "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
Date: Tue, 26 Jun 2012 13:18:37 +0300
Local: Tues, Jun 26 2012 6:18 am
Subject: Re: [RavenDB] Is there any way to make this query faster?

It is NOT a simple select.
You want a group by on a dynamic range. This is specifically something that
RavenDB doesn't do well.

On Tue, Jun 26, 2012 at 12:33 PM, nightwatch <rafal.gwizd...@gmail.com>wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »