I have a table where I'm trying to get the average time delta for each minute of the past X minutes. I have the SQL written and working but I'm unsure how to translate it into Sequel ruby; it may not even be possible but I thought I'd ask the list.
Here's my table definition and the SQL I am running against it:
class Order < Sequel::Model(:orders) set_schema do primary_key :id, :auto_increment => true varchar :market, :size => 12 decimal :created_at, :elements => [15,5] decimal :ack, :elements => [15,5] end end
select market, minute(from_unixtime(ack)) AS minute, round(avg((ack - created_at) * 1000), 2) AS ack_average from orders where 'ack' > (UNIX_TIMESTAMP(now()) - (15 * 60) and 'market' = 'ICE' group by minute(from_unixtime(ack));
As you may infer, the ack and created_at columns are large floating point numbers (seconds plus microseconds) representing time relative to the UNIX epoch. Is it possible to write this in Sequel and have it generate the correct SQL? Here's an attempt...
I need to do math (subtraction) on two columns in the result set and average that value. I also need access to mysql's +minute+ function and +unix_timestamp+ function. I looked through the code but got lost pretty quickly when trying to figure out how to monkey patch this in.
Am I best off just executing the SQL above and saving it into a dataset for ruby manipulation?
> Am I best off just executing the SQL above and saving it into a > dataset for ruby manipulation?
Right now yes. You can do that easily:
sql = "select market, minute(from_unixtime(ack)) AS minute, round(avg((ack - created_at) * 1000), 2) AS ack_average from orders where 'ack' > (UNIX_TIMESTAMP(now()) - (15 * 60) and 'market' = 'ICE' group by minute(from_unixtime(ack));"
dataset = DB[sql] dataset.each {|r| p r}
I've played with it a bit and right now, field quoting in the mysql adapter is broken, so you can't use any functions or formulas in #select, #group etc. Ultimately our goal should look like this:
market = 'ICE' ack_stamp = 15.minutes.ago Order.query do select do :market, :minute[:from_unixtime[:ack]].AS :minute :round[:avg[(:ack - :created_at) * 1000], 2].AS :ack_average end where do :ack > ack_stamp :market == market end group_by :minute[:from_unixtime[:ack]] end
With the help of ParseTree I think it's possible to do that, and also to fix field quoting. Can you please open an issue for this?
> > Am I best off just executing the SQL above and saving it into a > > dataset for ruby manipulation?
> Right now yes. You can do that easily:
> sql = "select market, minute(from_unixtime(ack)) AS minute, > round(avg((ack - > created_at) * 1000), 2) AS ack_average from orders where 'ack' > > (UNIX_TIMESTAMP(now()) - (15 * 60) and 'market' = 'ICE' group by > minute(from_unixtime(ack));"
> dataset = DB[sql] > dataset.each {|r| p r}
> I've played with it a bit and right now, field quoting in the mysql > adapter is broken, so you can't use any functions or formulas in > #select, #group etc. Ultimately our goal should look like this:
> market = 'ICE' > ack_stamp = 15.minutes.ago > Order.query do > select do > :market, > :minute[:from_unixtime[:ack]].AS :minute > :round[:avg[(:ack - :created_at) * 1000], 2].AS :ack_average > end > where do > :ack > ack_stamp > :market == market > end > group_by :minute[:from_unixtime[:ack]] > end
... the following presentation (discussed on the Nitro mail list) and the code beneath it illustrate a neat implementation of just this sort of thing. It seems very elegant and it would simplify the above syntax, so may be worth looking into?
> ... the following presentation (discussed on the Nitro mail list) and > the code beneath it illustrate a neat implementation of just this sort > of thing. It seems very elegant and it would simplify the above > syntax, so may be worth looking into?
Sequel is already much more advanced than that. Have a look at the way you can specify filters in Sequel. My plan is to take the the great filtering capability that we have and extend it so you can specify Ruby expressions for any SQL clause and have it automagically translated to SQL.
On Nov 6, 2007 7:50 PM, Sharon Rosner <cico...@gmail.com> wrote:
> > ... the following presentation (discussed on the Nitro mail list) and > > the code beneath it illustrate a neat implementation of just this sort > > of thing. It seems very elegant and it would simplify the above > > syntax, so may be worth looking into?
> Sequel is already much more advanced than that. Have a look at the way > you can specify filters in Sequel.
Apologies, I'm not yet familiar with the filters. I was just superficially comparing the code inside the query block with Peter's, in particular I thought:
from :c => Customer, :b => Bank, :a => Account select b.name where ((c.firstname == first1) | (c.firstname == first2)), etc.
appeared simpler than having blocks and .AS, but no doubt there is a limit to his approach I'm unaware of.
> My plan is to take the the great > filtering capability that we have and extend it so you can specify > Ruby expressions for any SQL clause and have it automagically > translated to SQL.
> I've played with it a bit and right now, field quoting in the mysql > adapter is broken, so you can't use any functions or formulas in > #select, #group etc. Ultimately our goal should look like this:
> market = 'ICE' > ack_stamp = 15.minutes.ago > Order.query do > select do > :market, > :minute[:from_unixtime[:ack]].AS :minute > :round[:avg[(:ack - :created_at) * 1000], 2].AS :ack_average > end > where do > :ack > ack_stamp > :market == market > end > group_by :minute[:from_unixtime[:ack]] > end
Ooh, that's pretty!
> With the help of ParseTree I think it's possible to do that, and also > to fix field quoting. Can you please open an issue for this?
Just opened it. Listed as Defect #88.
I'll take a stab at providing some of this functionality but it means learning ParseTree and probably ruby2ruby which might be a bit beyond my ruby capabilities right now. Don't hold your breath waiting for my patch... ;-)
Another food for thought would be to have the Order.query cache the
generated SQL query so the second time the block is run, it doesn't
have to generate the SQL code all over again.
A potential problem is the fixation of dynamically generated values
such as the "market" or "ack_stamp" variables in the example code.
I'm not sure how far you can go with ParseTree, but you could search
for the presence of dynamic elements (variables, methods...) in the
block passed to Order.query and turn on caching only if it only
contains static elements (constants, integers, symbols...).