How to aggregate values by month

5,606 views
Skip to first unread message

Rogério Carrasqueira

unread,
Oct 28, 2010, 9:31:48 AM10/28/10
to django...@googlegroups.com
Hello!

I'm having an issue to make complex queries in django. My problem is, I have a model where I have the sales and I need to make a report showing the sales amount per month, by the way I made this query:

init_date = datetime.date(datetime.now()-timedelta(days=365))
ends_date = datetime.date(datetime.now())
sales = Sale.objects.filter(date_created__range=(init_date,ends_date)).values(date_created__month).aggregate(total_sales=Sum('total_value'))

At the first line I get the today's date past one year
after this I got the today date

at sales I'm trying to between a range get the sales amount grouped by month, but unfortunatelly I was unhappy on this, because this error appeared:
global name 'date_created__month' is not defined

At date_created is the field where I store the information about when the sale was done., the __moth was a tentative to group by this by month.

So, my question: how to do that thing without using a raw sql query and not touching on database independence?

Thanks so much!

Rogério Carrasqueira

---
e-mail: rogerio.ca...@gmail.com
skype: rgcarrasqueira
MSN: rcarra...@hotmail.com
ICQ: 50525616
Tel.: (11) 7805-0074

Franklin Einspruch

unread,
Oct 28, 2010, 9:36:05 AM10/28/10
to django...@googlegroups.com
It may not be a complete answer, but you should know about {% regroup
%} just in case:

http://docs.djangoproject.com/en/dev/ref/templates/builtins/?from=olddocs#regroup

- Franklin

--
Art, writing, journal: http://einspruch.com
Comics: http://themoonfellonme.com


2010/10/28 Rogério Carrasqueira <rogerio.ca...@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>

Rogério Carrasqueira

unread,
Oct 28, 2010, 9:41:21 AM10/28/10
to django...@googlegroups.com
Hi Franklin!

Thanks for you answer. Unfortunatelly I need to output my results on a JSON file. Do you have any other approach?

Cheers,


Rogério Carrasqueira

---
e-mail: rogerio.ca...@gmail.com
skype: rgcarrasqueira
MSN: rcarra...@hotmail.com
ICQ: 50525616
Tel.: (11) 7805-0074



2010/10/28 Franklin Einspruch <franklin....@gmail.com>

Rogério Carrasqueira

unread,
Oct 28, 2010, 10:01:28 AM10/28/10
to django...@googlegroups.com
Hello Everybody!

I tried this way also:

sales = Sale.objects.extra(select={'
month': 'month(date_created)'}).filter(date_created__range=(init_date,ends_date)).values('month').aggregate(total_sales=Sum('total_value'))

but returned {}

Any ideias?

Cheers,

Rogério Carrasqueira

---
e-mail: rogerio.ca...@gmail.com
skype: rgcarrasqueira
MSN: rcarra...@hotmail.com
ICQ: 50525616
Tel.: (11) 7805-0074



Scott Gould

unread,
Oct 28, 2010, 10:28:30 AM10/28/10
to Django users
Personally I hate writing raw SQL so I would probably try something
like this (untested):

sales = Sale.objects.filter(date_created__range=(init_date,ends_date))
.values(date_ created__month)
.aggregate(total_sales=Sum('total_value'))

sales_by_month = [(x.year, x.month, [y for y in sales if y.year ==
x.year and y.month == x.month])
for x in set([z.year, z.month for z in sales])]

Should give you a list of (year, month, subqueryset) tuples. Of
course, there might be performance issues with nested list
comprehensions like that depending on the size of the initial
queryset, but at least it's a one-line, SQL-free solution!

On Oct 28, 9:31 am, Rogério Carrasqueira
<rogerio.carrasque...@gmail.com> wrote:
> Hello!
>
> I'm having an issue to make complex queries in django. My problem is, I have
> a model where I have the sales and I need to make a report showing the sales
> amount per month, by the way I made this query:
>
> init_date = datetime.date(datetime.now()-timedelta(days=365))
> ends_date = datetime.date(datetime.now())
> sales =
> Sale.objects.filter(date_created__range=(init_date,ends_date)).values(date_ created__month).aggregate(total_sales=Sum('total_value'))
>
> At the first line I get the today's date past one year
> after this I got the today date
>
> at sales I'm trying to between a range get the sales amount grouped by
> month, but unfortunatelly I was unhappy on this, because this error
> appeared:
>
> global name 'date_created__month' is not defined
>
> At date_created is the field where I store the information about when the
> sale was done., the __moth was a tentative to group by this by month.
>
> So, my question: how to do that thing without using a raw sql query and not
> touching on database independence?
>
> Thanks so much!
>
> Rogério Carrasqueira
>
> ---
> e-mail: rogerio.carrasque...@gmail.com
> skype: rgcarrasqueira
> MSN: rcarrasque...@hotmail.com

Casey S. Greene

unread,
Oct 28, 2010, 10:35:02 AM10/28/10
to django...@googlegroups.com
Hi Rog�rio,

You can output to json using templates if this is the only thing holding
you back from a working solution using regroup. You can use
render_to_string with a json template which you can then return with an
HttpResponse. It doesn't take advantage of the serialization library
but it does work.

-- Casey

On 10/28/2010 09:41 AM, Rog�rio Carrasqueira wrote:
> Hi Franklin!
>
> Thanks for you answer. Unfortunatelly I need to output my results on a
> JSON file. Do you have any other approach?
>
> Cheers,
>

> Rog�rio Carrasqueira
>
> ---
> e-mail: rogerio.ca...@gmail.com
> <mailto:rogerio.ca...@gmail.com>
> skype: rgcarrasqueira
> MSN: rcarra...@hotmail.com <mailto:rcarra...@hotmail.com>


> ICQ: 50525616
> Tel.: (11) 7805-0074
>
>
>
> 2010/10/28 Franklin Einspruch <franklin....@gmail.com

> <mailto:franklin....@gmail.com>>


>
> It may not be a complete answer, but you should know about {% regroup
> %} just in case:
>
> http://docs.djangoproject.com/en/dev/ref/templates/builtins/?from=olddocs#regroup
>
> - Franklin
>
> --
> Art, writing, journal: http://einspruch.com
> Comics: http://themoonfellonme.com
>
>
>
>

> 2010/10/28 Rog�rio Carrasqueira <rogerio.ca...@gmail.com
> <mailto:rogerio.ca...@gmail.com>>:


> > Hello!
> >
> > I'm having an issue to make complex queries in django. My problem
> is, I have
> > a model where I have the sales and I need to make a report
> showing the sales
> > amount per month, by the way I made this query:
> >
> > init_date = datetime.date(datetime.now()-timedelta(days=365))
> > ends_date = datetime.date(datetime.now())
> > sales =
> >
> Sale.objects.filter(date_created__range=(init_date,ends_date)).values(date_created__month).aggregate(total_sales=Sum('total_value'))
> >
> > At the first line I get the today's date past one year
> > after this I got the today date
> >
> > at sales I'm trying to between a range get the sales amount
> grouped by
> > month, but unfortunatelly I was unhappy on this, because this error
> > appeared:
> >
> > global name 'date_created__month' is not defined
> >
> > At date_created is the field where I store the information about
> when the
> > sale was done., the __moth was a tentative to group by this by month.
> >
> > So, my question: how to do that thing without using a raw sql
> query and not
> > touching on database independence?
> >
> > Thanks so much!
> >

> > Rog�rio Carrasqueira
> >
> > ---
> > e-mail: rogerio.ca...@gmail.com
> <mailto:rogerio.ca...@gmail.com>
> > skype: rgcarrasqueira
> > MSN: rcarra...@hotmail.com <mailto:rcarra...@hotmail.com>


> > ICQ: 50525616
> > Tel.: (11) 7805-0074
> >
> > --
> > You received this message because you are subscribed to the
> Google Groups
> > "Django users" group.
> > To post to this group, send email to

> django...@googlegroups.com <mailto:django...@googlegroups.com>.


> > To unsubscribe from this group, send email to
> > django-users...@googlegroups.com

> <mailto:django-users%2Bunsu...@googlegroups.com>.


> > For more options, visit this group at
> > http://groups.google.com/group/django-users?hl=en.
> >
>
> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To post to this group, send email to django...@googlegroups.com

> <mailto:django...@googlegroups.com>.


> To unsubscribe from this group, send email to
> django-users...@googlegroups.com

> <mailto:django-users%2Bunsu...@googlegroups.com>.

Javier Guerra Giraldez

unread,
Oct 28, 2010, 1:56:55 PM10/28/10
to django...@googlegroups.com
2010/10/28 Rogério Carrasqueira <rogerio.ca...@gmail.com>:

> Thanks for you answer. Unfortunatelly I need to output my results on a JSON
> file. Do you have any other approach?

if the {% regroup %} is what you need, you should know that it's an
application of itertools.groupby()

--
Javier

Mikhail Korobov

unread,
Oct 28, 2010, 5:54:09 PM10/28/10
to Django users
Hi Rogério,

You can give http://bitbucket.org/kmike/django-qsstats-magic/src a
try.
It currently have efficient aggregate lookups (1 query for the whole
time series) only for mysql but it'll be great if someone contribute
efficient lookups for other databases :)

On 28 окт, 19:31, Rogério Carrasqueira
<rogerio.carrasque...@gmail.com> wrote:
> Hello!
>
> I'm having an issue to make complex queries in django. My problem is, I have
> a model where I have the sales and I need to make a report showing the sales
> amount per month, by the way I made this query:
>
> init_date = datetime.date(datetime.now()-timedelta(days=365))
> ends_date = datetime.date(datetime.now())
> sales =
> Sale.objects.filter(date_created__range=(init_date,ends_date)).values(date_ created__month).aggregate(total_sales=Sum('total_value'))
>
> At the first line I get the today's date past one year
> after this I got the today date
>
> at sales I'm trying to between a range get the sales amount grouped by
> month, but unfortunatelly I was unhappy on this, because this error
> appeared:
>
> global name 'date_created__month' is not defined
>
> At date_created is the field where I store the information about when the
> sale was done., the __moth was a tentative to group by this by month.
>
> So, my question: how to do that thing without using a raw sql query and not
> touching on database independence?
>
> Thanks so much!
>
> Rogério Carrasqueira
>
> ---
> e-mail: rogerio.carrasque...@gmail.com
> skype: rgcarrasqueira
> MSN: rcarrasque...@hotmail.com

sebastien piquemal

unread,
Oct 29, 2010, 5:33:50 AM10/29/10
to Django users
Hi !

You could also give django-cube a try :
http://code.google.com/p/django-cube/
Unlike Mikhail's app, the aggregates are not efficient (because no
optimization is made, I am working on this), but this is more than
enough if you have a reasonable amount of data (less than millions of
rows !!!).
Use the following code, and you should have what you need :

from cube.models import Cube

class SalesCube(Cube):

month = Dimension('date_created__absmonth',
queryset=Sale.objects.filter(date_created__range=(init_date,ends_date)))

@staticmethod
def aggregation(queryset):
return queryset.count()

The advantage is that if you want to add more dimensions (type of sale/
place/month, etc ...), you can do it very easily.
Hope that helps, and don't hesitate to ask me if you can't have it
working (documentation is not very good yet).

On Oct 29, 12:54 am, Mikhail Korobov <kmik...@googlemail.com> wrote:
> Hi Rogério,
>
> You can givehttp://bitbucket.org/kmike/django-qsstats-magic/srca

Rogério Carrasqueira

unread,
Nov 3, 2010, 9:41:27 AM11/3/10
to django...@googlegroups.com
Hi Scott
Thanks for you help, unfortunately on trying

sales = Sale.objects.filter(date_
created__range=(init_date,ends_date))

   .values(date_ created__month)
   .aggregate(total_sales=Sum('total_value'))

This error appeared,
global name 'date_created__month' is not defined


Do you have another approach?

Regards,


Rogério Carrasqueira

---
e-mail: rogerio.ca...@gmail.com
skype: rgcarrasqueira
MSN: rcarra...@hotmail.com
ICQ: 50525616
Tel.: (11) 7805-0074



2010/10/28 Scott Gould <zinc...@gmail.com>

Rogério Carrasqueira

unread,
Nov 3, 2010, 2:44:16 PM11/3/10
to django...@googlegroups.com
Hi Mikhail!

Can you give some clue on how to use your plugin considering my scenario?

Thanks


Rogério Carrasqueira

---
e-mail: rogerio.ca...@gmail.com
skype: rgcarrasqueira
MSN: rcarra...@hotmail.com
ICQ: 50525616
Tel.: (11) 7805-0074



2010/10/28 Mikhail Korobov <kmi...@googlemail.com>

Rogério Carrasqueira

unread,
Nov 4, 2010, 8:34:17 AM11/4/10
to django...@googlegroups.com
Hi Sebastien!

Thanks for you reply. I'm a newbie on Django and I must confess unfortunately I don't know everything yet ;-). So I saw that you made a snippet regarding about the use of Django Cube. So, where do I put this snippet: at my views.py? Or should I do another class at my models.py?

Thanks so much!

Regards,


Rogério Carrasqueira

---
e-mail: rogerio.ca...@gmail.com
skype: rgcarrasqueira
MSN: rcarra...@hotmail.com
ICQ: 50525616
Tel.: (11) 7805-0074



2010/10/29 sebastien piquemal <seb...@gmail.com>

Rogério Carrasqueira

unread,
Nov 4, 2010, 1:48:23 PM11/4/10
to django...@googlegroups.com
Hello Folks!

I've got the solution, putting here for future searchs:

sales = Sale.objects.extra(select={'month':'month(date_created)','year':'year(date_created)'}).values('year','month').annotate(total_month=Sum('total_value'), average_month=Avg('total_value'))

This query works only using MySQL, to use with PGSQL you need to know to work with EXTRACT clauses.

Cheers


Rogério Carrasqueira

---
e-mail: rogerio.ca...@gmail.com
skype: rgcarrasqueira
MSN: rcarra...@hotmail.com
ICQ: 50525616
Tel.: (11) 7805-0074



Mikhail Korobov

unread,
Nov 4, 2010, 7:01:08 PM11/4/10
to Django users
Hi Rogério!

With django-qsstats-magic it would be something like this:

stats = QuerySetStats(Sale.objects.all(), 'date_created')
totals = stats.time_series(start, end, 'months',
aggregate=Sum('total_value'))
averages = stats.time_series(start, end, 'months',
aggregate=Avg('total_value'))

Great you've found the solution without any plugins!


On 3 ноя, 23:44, Rogério Carrasqueira <rogerio.carrasque...@gmail.com>
wrote:
> Hi Mikhail!
>
> Can you give some clue on how to use your plugin considering my scenario?
>
> Thanks
>
> Rogério Carrasqueira
>
> ---
> e-mail: rogerio.carrasque...@gmail.com
> skype: rgcarrasqueira
> MSN: rcarrasque...@hotmail.com
> ICQ: 50525616
> Tel.: (11) 7805-0074
>
> 2010/10/28 Mikhail Korobov <kmik...@googlemail.com>
>
>
>
> > Hi Rogério,
>
> > You can givehttp://bitbucket.org/kmike/django-qsstats-magic/srca
> > django-users...@googlegroups.com<django-users%2Bunsubscribe@google groups.com>
> > .

David Zhou

unread,
Nov 4, 2010, 7:10:19 PM11/4/10
to django...@googlegroups.com
FWIW, on Postgres DBs, I've done the following:

qs = FooModel.objects.filter(date__gte=start_date, date__lt=end_date).extra(select={'datetrunc': "date_trunc('month', date)"}).values('datetrunc').annotate(total=Sum("value"))

date_trunc in postgres also accepts "day" and "week" truncations.

-- dz


2010/11/4 Rogério Carrasqueira <rogerio.ca...@gmail.com>

derek

unread,
Nov 5, 2010, 9:18:23 AM11/5/10
to Django users
Maybe slightly "off topic" but does anyone have robust strategies for
readily switching between different DB-specific SQL code which is
(presumably) embedded in one's app/project? (As shown in the two
examples presented here)

On Nov 5, 1:10 am, David Zhou <da...@nodnod.net> wrote:
> FWIW, on Postgres DBs, I've done the following:
>
> qs = FooModel.objects.filter(date__gte=start_date,
> date__lt=end_date).extra(select={'datetrunc': "date_trunc('month',
> date)"}).values('datetrunc').annotate(total=Sum("value"))
>
> date_trunc in postgres also accepts "day" and "week" truncations.
>
> -- dz
>
> 2010/11/4 Rogério Carrasqueira <rogerio.carrasque...@gmail.com>
>
> > Hello Folks!
>
> > I've got the solution, putting here for future searchs:
>
> > sales =
> > Sale.objects.extra(select={'month':'month(date_created)','year':'year(date_created)'}).values('year','month').annotate(total_month=Sum('total_value'),
> > average_month=Avg('total_value'))
>
> > This query works only using MySQL, to use with PGSQL you need to know to
> > work with EXTRACT clauses.
>
> > Cheers
>
> > Rogério Carrasqueira
>
> > ---
> > e-mail: rogerio.carrasque...@gmail.com
> > skype: rgcarrasqueira
> > MSN: rcarrasque...@hotmail.com
> > ICQ: 50525616
> > Tel.: (11) 7805-0074
>
> > Em 4 de novembro de 2010 10:34, Rogério Carrasqueira <
> > rogerio.carrasque...@gmail.com> escreveu:
>
> > Hi Sebastien!
>
> >> Thanks for you reply. I'm a newbie on Django and I must confess
> >> unfortunately I don't know everything yet ;-). So I saw that you made a
> >> snippet regarding about the use of Django Cube. So, where do I put this
> >> snippet: at my views.py? Or should I do another class at my models.py?
>
> >> Thanks so much!
>
> >> Regards,
>
> >> Rogério Carrasqueira
>
> >> ---
> >>> django-users...@googlegroups.com<django-users%2Bunsu...@googlegroups.com>
> >>> .
> >>> For more options, visit this group at
> >>>http://groups.google.com/group/django-users?hl=en.
>
> >  --
> > You received this message because you are subscribed to the Google Groups
> > "Django users" group.
> > To post to this group, send email to django...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-users...@googlegroups.com<django-users%2Bunsu...@googlegroups.com>
> > .
Reply all
Reply to author
Forward
0 new messages