[Django] #23112: Field.get_choices tries to index an iterable
Django
----------------------------------------------+--------------------
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: master
Severity: Release blocker | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+--------------------
Issue #23098 was fixed with this commit:
https://github.com/django/django/commit/2f73b527dda6683868fac2791f7f07ccb01ea0d9
Which adds the following line:
{{{
named_groups = self.choices and isinstance(self.choices[0][1], (list,
tuple))
}}}
According to the docs -
https://docs.djangoproject.com/en/dev/ref/models/fields/#choices -
Field.choices can be any iterable of choices. The above line introduces
the requirement that the choices object to be subscriptable, and breaks
both my own code and a 3rd party library I'm using (django_countries).
--
Ticket URL: <https://code.djangoproject.com/ticket/23112>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage:
Keywords: | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by magopian):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
Also, we can have a list of choices made of both named groups and "normal"
choices, so we might no "see" that the blank choice is defined in the list
of choices (it would be odd to not have it at the beginning of the list,
but i don't think it's required, at least from the previous code and the
docs).
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:1>
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by timo):
* version: master => 1.7-rc-2
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:2>
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by apollo13):
Replying to [comment:1 magopian]:
> Also, we can have a list of choices made of both named groups and
"normal" choices, so we might no "see" that the blank choice is defined in
the list of choices (it would be odd to not have it at the beginning of
the list, but i don't think it's required, at least from the previous code
and the docs).
That didn't work either previously, so we are somewhat safe in that
regard… The original iterable argument seems valid, we can just call list
on it I guess.
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:3>
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* cc: apollo13 (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:4>
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* cc: areski@… (added)
Comment:
I could reproduce the issue using an iterator, here a PR that fix the
issue at my end
https://github.com/django/django/pull/2990
Calling list as @apollo13 just mentioned. Thanks
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:5>
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by areski):
* has_patch: 0 => 1
Comment:
The PR now contains tests: https://github.com/django/django/pull/2990
I noticed that with Iterators, we cannot use `_get_FIELD_display`
properly, now this problem might not be blocking.
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:6>
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by zsiciarz):
I got bitten by this bug too (talked about it yesterday with Baptiste).
Here's a traceback from my project which used to work on RC1, breaks on
RC2.
{{{
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/core/management/__init__.py", line 385, in
execute_from_command_line
utility.execute()
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/core/management/__init__.py", line 354, in execute
django.setup()
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/__init__.py", line 21, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/apps/registry.py", line 115, in populate
app_config.ready()
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/contrib/admin/apps.py", line 22, in ready
self.module.autodiscover()
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/contrib/admin/__init__.py", line 23, in autodiscover
autodiscover_modules('admin', register_to=site)
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/utils/module_loading.py", line 74, in autodiscover_modules
import_module('%s.%s' % (app_config.name, module_to_search))
File
"/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/importlib/__init__.py",
line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1584, in _gcd_import
File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
File "<frozen importlib._bootstrap>", line 1532, in
_find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 584, in _check_name_wrapper
File "<frozen importlib._bootstrap>", line 1022, in load_module
File "<frozen importlib._bootstrap>", line 1003, in load_module
File "<frozen importlib._bootstrap>", line 560, in
module_for_loader_wrapper
File "<frozen importlib._bootstrap>", line 868, in _load_module
File "<frozen importlib._bootstrap>", line 313, in
_call_with_frames_removed
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/allauth/socialaccount/admin.py", line 12, in <module>
class SocialAppForm(forms.ModelForm):
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/forms/models.py", line 284, in __new__
opts.help_texts, opts.error_messages)
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/forms/models.py", line 210, in fields_for_model
formfield = f.formfield(**kwargs)
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/db/models/fields/__init__.py", line 1063, in formfield
return super(CharField, self).formfield(**defaults)
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/db/models/fields/__init__.py", line 822, in formfield
defaults['choices'] = self.get_choices(include_blank=include_blank)
File "/home/zbigniewsiciarz/v/bazawiedzy/lib/python3.3/site-
packages/django/db/models/fields/__init__.py", line 733, in get_choices
named_groups = self.choices and isinstance(self.choices[0][1], (list,
tuple))
TypeError: 'itertools._tee' object is not subscriptable
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:7>
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by areski):
Did you try with this patch https://github.com/django/django/pull/2990?
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:8>
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* cc: pdewacht@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:9>
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by zsiciarz):
Yes, wrapping self.choices with a list() call fixed the problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:10>
Django
Reporter: bernie_sumption | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by areski):
Update of the PR thanks to the feedback from @Apollo13
https://github.com/django/django/pull/2990
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:11>
Django
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: assigned
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* owner: nobody => areski
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:12>
Django
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: assigned
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by areski):
Following @timgraham advice, the commented tests has been removed from the
PR
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:13>
Django
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: assigned
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* cc: lemuelf@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:14>
Django
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: assigned
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* cc: cmawebsite@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:15>
Django
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: assigned
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by CollinAnderson):
As noticed in #23119, the patch doesn't work if the iterable is empty
(empty iterable is fine on 1.6)
{{{
field = models.CharField(choices=(x for x in []), blank=True,
max_length=1)
}}}
raises `IndexError: list index out of range` with the pull request. No
problem on 1.6
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:16>
Django
attribute '__getitem__'
-------------------------------------+-------------------------------------
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: assigned
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:17>
Django
attribute '__getitem__'
-------------------------------------+-------------------------------------
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: assigned
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by areski):
Thanks @CollinAnderson check the pull request, I just added a fix for
empty iterators and also some regression tests
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:18>
Django
attribute '__getitem__'
-------------------------------------+-------------------------------------
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: assigned
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* cc: Jcuotpc@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:19>
Django
attribute '__getitem__'
-------------------------------------+-------------------------------------
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: assigned
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by jcuotpc):
* cc: Jcuotpc@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:20>
Django
attribute '__getitem__'
-------------------------------------+-------------------------------------
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: assigned
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by CollinAnderson):
The pull request works for me. I closed #23119 as duplicate.
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:21>
Django
attribute '__getitem__'
-------------------------------------+-------------------------------------
Reporter: bernie_sumption | Owner: areski
Component: Database layer | Version: 1.7-rc-2
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"97a38de230371c0b6ad8a86abba8425186c147c7"]:
{{{
#!CommitTicketReference repository=""
revision="97a38de230371c0b6ad8a86abba8425186c147c7"
Fixed #23112 -- Field.get_choices tries to index an iterable
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:22>
Django
attribute '__getitem__'
-------------------------------------+-------------------------------------
Reporter: bernie_sumption | Owner: areski
Type: Bug | Status: closed
Component: Database layer | Version: 1.7-rc-2
(models, ORM) | Resolution: fixed
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by Florian Apolloner <florian@…>):
In [changeset:"e22ad1c3250a59d046d73bf34ae39c0c40e03c98"]:
{{{
#!CommitTicketReference repository=""
revision="e22ad1c3250a59d046d73bf34ae39c0c40e03c98"
[1.7.x] Fixed #23112 -- Field.get_choices tries to index an iterable
Backport of 97a38de230371c0b6ad8a86abba8425186c147c7 from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23112#comment:23>