[Django] #35395: Python 3.12 filter bug with Django 4.2.11

[Django]

#35395: Python 3.12 filter bug with Django 4.2.11
-------------------------------------------+------------------------
Reporter: Tim Richardson | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 4.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------------+------------------------
I have a template with a fragment like this:


<span
class="fs-6">Supplier: {{
po_metadata|get_item:dict_item.PONumber|get_item:"supplier" }} PO Date: {{
po_metadata|get_item:dict_item.PONumber|get_item:"order_date" |
slice:":10" }} </span>


When using python 3.13.2, I get an exception in
django/template/defaultfilters.py slice_filter()

This code does throw an exception with 3.11

The exception is a Key Error and it happens because instead a list being
passed to the filter, a dictionary is. An empty dict in my case.

In python 3.11, the breakpoints in slice_filter() are not hit. Somehow
the preceeding filters are behaving differently in case of po_metadata
being an empty dict.
--
Ticket URL: <https://code.djangoproject.com/ticket/35395>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35395: Python 3.12 filter bug with Django 4.2.11

---------------------------------+--------------------------------------

Reporter: Tim Richardson | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 4.2

Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

---------------------------------+--------------------------------------
Description changed by Tim Richardson:

Old description:

> I have a template with a fragment like this:
>

> <span
> class="fs-6">Supplier: {{
> po_metadata|get_item:dict_item.PONumber|get_item:"supplier" }} PO Date:
> {{ po_metadata|get_item:dict_item.PONumber|get_item:"order_date" |
> slice:":10" }} </span>
>

> When using python 3.13.2, I get an exception in
> django/template/defaultfilters.py slice_filter()
>
> This code does throw an exception with 3.11
>
> The exception is a Key Error and it happens because instead a list being
> passed to the filter, a dictionary is. An empty dict in my case.
>
> In python 3.11, the breakpoints in slice_filter() are not hit. Somehow
> the preceeding filters are behaving differently in case of po_metadata
> being an empty dict.

New description:

I have a template with a fragment like this:


<span
class="fs-6">Supplier: {{
po_metadata|get_item:dict_item.PONumber|get_item:"supplier" }} PO Date: {{
po_metadata|get_item:dict_item.PONumber|get_item:"order_date" |
slice:":10" }} </span>


When using python 3.13.2, I get an exception in
django/template/defaultfilters.py slice_filter()

This code does not throw an exception with 3.11

The exception is a Key Error and it happens because instead a list being
passed to the filter, a dictionary is. An empty dict in my case.

In python 3.11, the breakpoints in slice_filter() are not hit. Somehow
the preceeding filters are behaving differently in case of po_metadata
being an empty dict.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/35395#comment:1>

[Django]

#35395: Python 3.12 filter bug with Django 4.2.11
---------------------------------+--------------------------------------
Reporter: Tim Richardson | Owner: nobody

Type: Bug | Status: closed

Component: Template system | Version: 4.2

Severity: Normal | Resolution: needsinfo

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------

Changes (by Tim Graham):

* resolution: => needsinfo
* status: new => closed

Old description:

> I have a template with a fragment like this:
>

> <span
> class="fs-6">Supplier: {{
> po_metadata|get_item:dict_item.PONumber|get_item:"supplier" }} PO Date:
> {{ po_metadata|get_item:dict_item.PONumber|get_item:"order_date" |
> slice:":10" }} </span>
>

> When using python 3.13.2, I get an exception in
> django/template/defaultfilters.py slice_filter()
>

> This code does not throw an exception with 3.11
>
> The exception is a Key Error and it happens because instead a list being
> passed to the filter, a dictionary is. An empty dict in my case.
>
> In python 3.11, the breakpoints in slice_filter() are not hit. Somehow
> the preceeding filters are behaving differently in case of po_metadata
> being an empty dict.

New description:

I have a template with a fragment like this:


`<span class="fs-6">Supplier: {{
po_metadata|get_item:dict_item.PONumber|get_item:"supplier" }} PO Date: {{
po_metadata|get_item:dict_item.PONumber|get_item:"order_date" |
slice:":10" }} </span>`


When using python 3.13.2, I get an exception in
django/template/defaultfilters.py slice_filter()

This code does not throw an exception with 3.11

The exception is a Key Error and it happens because instead a list being
passed to the filter, a dictionary is. An empty dict in my case.

In python 3.11, the breakpoints in slice_filter() are not hit. Somehow
the preceeding filters are behaving differently in case of po_metadata
being an empty dict.

--

Comment:

Hi Tim, You haven't provided complete steps to reproduce the issue as your
snippet involves custom template filters. The issue may be in those
filters. You should debug the problem and confirm that Django is at fault.
If so, reopen the issue with an explanation. See
TicketClosingReasons/UseSupportChannels if you need help.
--
Ticket URL: <https://code.djangoproject.com/ticket/35395#comment:2>

[Django]

#35395: Python 3.12 filter bug with Django 4.2.11
---------------------------------+--------------------------------------
Reporter: Tim Richardson | Owner: nobody
Type: Bug | Status: closed
Component: Template system | Version: 4.2
Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------

Comment (by Tim Richardson):

Fair enough.

The code below is the slice_filter from the 4.2.11 source.


If you run this code in python 3.11,
the filter returns
{}

(an empty dict)

If you run it in python 3.12, it raises an exception.

{{{
Traceback (most recent call last):
File "/app/./test_filter.py", line 21, in <module>
r = slice_filter({},":5")
^^^^^^^^^^^^^^^^^^^^^
File "/app/./test_filter.py", line 13, in slice_filter
return value[slice(*bits)]
~~~~~^^^^^^^^^^^^^^
}}}
KeyError: slice(None, 5, None)



{{{
def slice_filter(value, arg):
"""
Return a slice of the list using the same syntax as Python's list
slicing.
"""
try:
bits = []
for x in str(arg).split(":"):
if not x:
bits.append(None)
else:
bits.append(int(x))
return value[slice(*bits)]
except (ValueError, TypeError) as e:
return value # Fail silently.




if __name__ == "__main__":
r = slice_filter({},":5")
print(f"{r=}")
}}}


I will try to work out why, but this at least describes the problem I
think.
--
Ticket URL: <https://code.djangoproject.com/ticket/35395#comment:3>

[Django]

#35395: Python 3.12 filter bug with Django 4.2.11
---------------------------------+--------------------------------------
Reporter: Tim Richardson | Owner: nobody
Type: Bug | Status: closed
Component: Template system | Version: 4.2
Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------
Comment (by Tim Richardson):

the slice() function is being interpreted as key in 3.12 but not in prior
python versions.

the calculation of

bits

is identical.

Python 3.12 docs say that slice objects are now hashable, so it gets
treated as a key I suppose.
I wonder how it worked in prior python versions.

Including KeyError in the caught exceptions restores consistent behaviour.

{{{
def slice_filter(value, arg):
"""
Return a slice of the list using the same syntax as Python's list
slicing.
"""
try:
bits = []
for x in str(arg).split(":"):
if not x:
bits.append(None)
else:
bits.append(int(x))
return value[slice(*bits)]

except (ValueError, TypeError, KeyError):

return value # Fail silently.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/35395#comment:4>