[Django] #16335: Cannot iterate defaultdict in template

Django

unread,

Jun 24, 2011, 11:44:17 AM6/24/11

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template
-------------------------------+---------------------------------
Reporter: jacob.ninja.dev@… | Owner: nobody
Type: Bug | Status: new
Milestone: | Component: Template system
Version: 1.3 | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Easy pickings: 1
UI/UX: 0 |
-------------------------------+---------------------------------
I am unable to iterate a defaultdict using a for loop in a template.

Code to reproduce issue:

{{{
dictionary = defaultdict(list)
dictionary['foo'].append('bar')

{% for key, value in dictionary.items %}
{# Never iterates #}
{% endfor %}
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/16335>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Jun 24, 2011, 8:36:36 PM6/24/11

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0

Comment:

Right, because dictionary key access takes precedence in template variable
lookups, so {{{ {% ... dictionary.items %} }}} adds an empty list named
'items' to the dictionary. Not sure there's a good workaround here.

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:1>

Django

unread,

Jun 25, 2011, 2:50:52 AM6/25/11

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Triage Stage: Accepted | Needs tests: 0
Needs documentation: 0 | Easy pickings: 1

Patch needs improvement: 0 |
UI/UX: 0 |
-------------------------------------+-------------------------------------

Changes (by aaugustin):

* stage: Unreviewed => Accepted

Comment:

Indeed, it boils down to the fact that the template language uses the same
syntax for dictionary and attribute lookups. The resolution order is
documented here:
https://docs.djangoproject.com/en/1.3/topics/templates/#variables. It's
probably a bad idea to change it.

We could add a note there to warn about objects that accept a dictionary
lookup with any name, suggesting to convert them to `dict` before passing
them to the view, in your example:
{{{
context['dictionary'] = dict(dictionary)
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:2>

Django

unread,

Sep 9, 2011, 3:19:15 PM9/9/11

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

-------------------------------------+-------------------------------------
Reporter: | Owner: bluejeansummer
jacob.ninja.dev@… | Status: assigned

Type: Bug | Component: Template system
Milestone: | Severity: Normal
Version: 1.3 | Keywords:
Resolution: | Has patch: 0

Triage Stage: Accepted | Needs tests: 1
Needs documentation: 1 | Easy pickings: 1

Patch needs improvement: 0 |
UI/UX: 0 |
-------------------------------------+-------------------------------------

Changes (by bluejeansummer):

* status: new => assigned
* needs_docs: 0 => 1
* cc: bluejeansummer (added)
* owner: nobody => bluejeansummer
* needs_tests: 0 => 1

Comment:

How much would break if we were to do the defaultdict -> dict conversion
automatically?

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:3>

Django

unread,

Sep 9, 2011, 5:23:24 PM9/9/11

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Resolution: | Has patch: 1
Triage Stage: Accepted | Needs tests: 0

Needs documentation: 1 | Easy pickings: 1

Patch needs improvement: 1 |

UI/UX: 0 |
-------------------------------------+-------------------------------------
Changes (by bluejeansummer):

* needs_better_patch: 0 => 1
* has_patch: 0 => 1
* needs_tests: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:4>

Django

unread,

Sep 10, 2011, 4:30:36 PM9/10/11

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Needs documentation: 0 | Easy pickings: 1

Patch needs improvement: 1 |
UI/UX: 0 |
-------------------------------------+-------------------------------------
Changes (by bluejeansummer):

* needs_docs: 1 => 0

Comment:

After some consideration, perhaps performing the conversion is not such a
good idea. I've added an example to the docs that explains why the current
behavior happens.

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:5>

Django

unread,

Sep 21, 2011, 10:11:37 AM9/21/11

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Changes (by akvadrako):

* cc: akvadrako (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:6>

Django

unread,

Feb 4, 2012, 1:15:59 PM2/4/12

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

-------------------------------------+-------------------------------------
Reporter: jacob.ninja.dev@… | Owner:
Type: Bug | bluejeansummer
Component: Documentation | Status: assigned

Severity: Normal | Version: 1.3
Keywords: | Resolution:

Has patch: 1 | Triage Stage: Ready for
Needs tests: 0 | checkin

Easy pickings: 1 | Needs documentation: 0

| Patch needs improvement: 1
| UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by oinopion):

* component: Template system => Documentation
* stage: Accepted => Ready for checkin

Comment:

Patch with docs explains issue nicely.

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:7>

Django

unread,

Feb 4, 2012, 1:19:05 PM2/4/12

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Comment (by anonymous):

Patch looks great, however just to be super clear, I think it should
include a bit like, "So while this looks like it should do
``defaultdict.iteritems()`` in fact it's doing
``defaultdict["iteritems"]`` which returns the default value".

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:8>

Django

unread,

Mar 4, 2012, 10:36:52 PM3/4/12

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Comment (by SmileyChris):

Hrm, I wonder if the `_resolve_lookup` method could be made smarter with
something like the following for the dictionary lookup section:
{{{
try:
if bit not in current:
raise KeyError
except TypeError:
pass
current = current[bit]
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:9>

Django

unread,

May 10, 2012, 4:35:28 PM5/10/12

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

-------------------------------------+-------------------------------------
Reporter: jacob.ninja.dev@… | Owner:
Type: Bug | bluejeansummer

Component: Documentation | Status: closed
Severity: Normal | Version: 1.3
Keywords: | Resolution: fixed

Changes (by Aymeric Augustin):

* status: assigned => closed
* resolution: => fixed

Comment:

Fixed #16335 -- Clarified an unintuitive behavior.

The DTL will perform dict lookup before method lookup, which yields
an unexpected result for defaultdicts.
Changeset: d171b3cc0b32374fd5891254b04693e9ec8ed946

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:10>

Django

unread,

May 10, 2012, 4:48:55 PM5/10/12

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Comment (by Aymeric Augustin):

Fixed #16335 -- Clarified an unintuitive behavior.

The DTL will perform dict lookup before method lookup, which yields
an unexpected result for defaultdicts.
Changeset: d171b3cc0b32374fd5891254b04693e9ec8ed946

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:11>

Django

unread,

May 11, 2012, 2:26:13 AM5/11/12

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Comment (by Aymeric Augustin):

Fixed #16335 -- Clarified an unintuitive behavior.

The DTL will perform dict lookup before method lookup, which yields
an unexpected result for defaultdicts.
Changeset: d171b3cc0b32374fd5891254b04693e9ec8ed946

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:12>

Django

unread,

Aug 26, 2013, 4:04:05 AM8/26/13

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Comment (by anonymous):

Having this case:

{{{
my_dict = collections.defaultdict(lambda: collections.defaultdict(list))
dictionary['foo']['foo1'].append(obj)
}}}

and doing

{{{
return Response({'my_dict ': dict(my_dict)},
template_name='_internal_template.html')
}}}

I'm able to loop the first for only

{{{
{% for groups in my_dict.itervalues %}
groups = {{ groups }} <br><br> //It prints: groups = defaultdict(<type
'list'>, {1: [<Obj: Obj 1 by me>]})
{% for cols in groups.itervalues %}
cols = {{ cols }} <br><br>
{% for maps in cols.itervalues %}
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:13>

Django

unread,

Feb 26, 2014, 6:59:13 AM2/26/14

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Changes (by krommail@…):

* resolution: fixed =>

Comment:

Django 1.6, Pyhton 3.3.3. Even in the simpliest case:
{% for key, values in dict.items %}
{{ key }}
{% endfor %}
there is nothing in "key".

While
{% for key in dict %}
{{ key }}
{% endfor %}

works well.
Also preliminary conversion of a defaultdict to dict helps.

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:14>

Django

unread,

Jun 3, 2014, 3:17:12 PM6/3/14

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Comment (by ericpulvino@…):

I am also seeing this for Django 1.6.1 python 2.7.6. Conversion to
standard dict does not help in my case. Otherwise same symptoms as the
post from krommail.

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:15>

Django

unread,

Jun 3, 2014, 3:31:12 PM6/3/14

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

Comment (by ericpulvino@…):

After upgrading to 1.6.5 still with python 2.7.6 the workaround
(converting to standard dictionary prior to handing dict to the template)
functions.

As a Django newbie I have sunk 5 hours on this silly issue. I need a break
today.

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:16>

Django

unread,

Jun 19, 2014, 10:32:25 AM6/19/14

to django-...@googlegroups.com

#16335: Cannot iterate defaultdict in template

-------------------------------------+-------------------------------------
Reporter: jacob.ninja.dev@… | Owner:
Type: Bug | bluejeansummer

Component: Documentation | Status: new

Changes (by anonymous):

* status: closed => new

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:17>

Django

unread,

Jun 19, 2014, 2:33:41 PM6/19/14

to django-...@googlegroups.com

#16335: Document why you cannot iterate defaultdict in templates

-------------------------------------+-------------------------------------
Reporter: jacob.ninja.dev@… | Owner:
Type: Bug | bluejeansummer

Component: Documentation | Status: closed
Severity: Normal | Version: 1.3
Keywords: | Resolution: fixed

Changes (by timo):

* status: new => closed
* resolution: => fixed

Comment:

The documentation that was added as part of the ticket
[d171b3cc0b32374fd5891254b04693e9ec8ed946] explains why it doesn't work.

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:18>

Django

unread,

Jul 30, 2014, 5:10:23 AM7/30/14

to django-...@googlegroups.com

#16335: Document why you cannot iterate defaultdict in templates
-------------------------------------+-------------------------------------
Reporter: jacob.ninja.dev@… | Owner:
Type: Bug | bluejeansummer
Component: Documentation | Status: closed
Severity: Normal | Version: 1.3
Keywords: | Resolution: fixed
Has patch: 1 | Triage Stage: Ready for
Needs tests: 0 | checkin
Easy pickings: 1 | Needs documentation: 0
| Patch needs improvement: 1
| UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by diego.gaustein@…):

I am actually getting a MemoryError (which sometimes end in segfault) when
trying to iterate the values in a defaultdict

In views.py:
{{{
totals = defaultdict(Decimal)
add_stuff_to_it(totals)
}}}

In the template:
{{{
{% for t in totals.values %}
{{ t }}
{% endfor %}
}}}

Results in
{{{
$ python manage.py runserver 0.0.0.0:8000 --settings=xxx.settings_debug
Validating models...

0 errors found
July 30, 2014 - 09:57:05
Django version 1.6.5, using settings 'xxx.settings_debug'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
Traceback (most recent call last):
File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
self.result = application(self.environ, self.start_response)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/contrib/staticfiles/handlers.py", line 67, in __call__
return self.application(environ, start_response)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/core/handlers/wsgi.py", line 206, in __call__
response = self.get_response(request)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/core/handlers/base.py", line 194, in get_response
response = self.handle_uncaught_exception(request, resolver,
sys.exc_info())
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/core/handlers/base.py", line 229, in
handle_uncaught_exception
return debug.technical_500_response(request, *exc_info)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/views/debug.py", line 69, in technical_500_response
html = reporter.get_traceback_html()
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/views/debug.py", line 324, in get_traceback_html
return t.render(c)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/base.py", line 140, in render
return self._render(context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/base.py", line 134, in _render
return self.nodelist.render(context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/base.py", line 840, in render
bit = self.render_node(node, context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/debug.py", line 78, in render_node
return node.render(context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/defaulttags.py", line 305, in render
return nodelist.render(context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/base.py", line 840, in render
bit = self.render_node(node, context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/debug.py", line 78, in render_node
return node.render(context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/defaulttags.py", line 36, in render
output = self.nodelist.render(context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/base.py", line 840, in render
bit = self.render_node(node, context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/debug.py", line 78, in render_node
return node.render(context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/defaulttags.py", line 212, in render
return nodelist.render(context)
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/template/base.py", line 844, in render
return mark_safe(''.join(bits))
File "/home/vagrant/venv/local/lib/python2.7/site-
packages/django/utils/safestring.py", line 116, in mark_safe
return SafeText(s)
MemoryError
[30/Jul/2014 09:57:16] "GET /statistics/order_book_and_wip/ HTTP/1.1" 500
59
}}}

Strangely, iterating over the keys seems to work fine.
Setting the default_factory to None as detailed
[http://stackoverflow.com/a/12842716 here] makes it work as expected. It
would seem like a cheap and sensible workaround to apply that to
defaultdicts. Or otherwise at least raise an exception so users can know
what's going on.

--
Ticket URL: <https://code.djangoproject.com/ticket/16335#comment:19>

Reply all

Reply to author

Forward