[Django] #36098: TypeError: object of type 'IPv6Address' has no len() when running tests with GeoDjango

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia | Owner: Natalia Bidart
Bidart |
Type: Bug | Status: assigned
Component: GIS | Version: 5.1
Severity: Release | Keywords:
blocker |
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Following the security release for 5.1.5, 5.0.11 and 4.2.18, there seems
to be an issue with the GeoDjango tests:
{{{
Traceback (most recent call last):
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/core/validators.py", line 305, in
validate_ipv4_address
ipaddress.IPv4Address(value)
File "/usr/lib/python3.10/ipaddress.py", line 1305, in __init__
self._ip = self._ip_int_from_string(addr_str)
File "/usr/lib/python3.10/ipaddress.py", line 1192, in
_ip_int_from_string
raise AddressValueError("Expected 4 octets in %r" % ip_str)
ipaddress.AddressValueError: Expected 4 octets in '::ffff:27d:a0d8'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/core/validators.py", line 325, in
validate_ipv46_address
validate_ipv4_address(value)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/core/validators.py", line 307, in
validate_ipv4_address
raise ValidationError(
django.core.exceptions.ValidationError: ['Enter a valid IPv4 address.']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/tests/gis_tests/test_geoip2.py", line 131, in
test_country
self.assertEqual(g.country(query), self.expected_country)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/contrib/gis/geoip2.py", line 207, in country
response = self._query(query, require_city=False)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/contrib/gis/geoip2.py", line 157, in _query
validate_ipv46_address(query)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/core/validators.py", line 328, in
validate_ipv46_address
validate_ipv6_address(value)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/core/validators.py", line 315, in
validate_ipv6_address
if not is_valid_ipv6_address(value):
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/utils/ipv6.py", line 59, in
is_valid_ipv6_address
_ipv6_address_from_str(ip_str)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/utils/ipv6.py", line 10, in
_ipv6_address_from_str
if len(ip_str) > max_length:
TypeError: object of type 'IPv6Address' has no len()

======================================================================
ERROR [0.003s]: test_country_using_city_database
(gis_tests.test_geoip2.GeoIP2Test) (query=IPv6Address('::ffff:27d:a0d8'))
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/core/validators.py", line 305, in
validate_ipv4_address
ipaddress.IPv4Address(value)
File "/usr/lib/python3.10/ipaddress.py", line 1305, in __init__
self._ip = self._ip_int_from_string(addr_str)
File "/usr/lib/python3.10/ipaddress.py", line 1192, in
_ip_int_from_string
raise AddressValueError("Expected 4 octets in %r" % ip_str)
ipaddress.AddressValueError: Expected 4 octets in '::ffff:27d:a0d8'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/core/validators.py", line 325, in
validate_ipv46_address
validate_ipv4_address(value)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/core/validators.py", line 307, in
validate_ipv4_address
raise ValidationError(
django.core.exceptions.ValidationError: ['Enter a valid IPv4 address.']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/tests/gis_tests/test_geoip2.py", line 145, in
test_country_using_city_database
self.assertEqual(g.country(query), self.expected_country)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/contrib/gis/geoip2.py", line 207, in country
response = self._query(query, require_city=False)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/contrib/gis/geoip2.py", line 157, in _query
validate_ipv46_address(query)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/core/validators.py", line 328, in
validate_ipv46_address
validate_ipv6_address(value)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/core/validators.py", line 315, in
validate_ipv6_address
if not is_valid_ipv6_address(value):
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/utils/ipv6.py", line 59, in
is_valid_ipv6_address
_ipv6_address_from_str(ip_str)
File "/home/jenkins/workspace/pull-requests-
focal/database/mysql_gis/label/focal-
pr/python/python3.10/django/utils/ipv6.py", line 10, in
_ipv6_address_from_str
if len(ip_str) > max_length:
TypeError: object of type 'IPv6Address' has no len()
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36098>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------

Reporter: Natalia Bidart | Owner: Natalia

| Bidart
Type: Bug | Status: assigned
Component: GIS | Version: 5.1

Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* stage: Unreviewed => Accepted

Comment:

Sorry, I've notice the ticket when I had a patch ready. Feel-free to close
it, [https://github.com/django/django/pull/19047 PR].

We have two issues here:
- first `GeoIP2._query()` unnecessarily calls `validate_ipv46_address` for
always valid `query`,
- second `validate_ipv46_address()` and `validate_ipv6_address()` crash on
`ipaddress.IPv6Address` instances.

I know that calling `validate_ipv46_address()`/`validate_ipv6_address()`
is pointless, but it worked for years so I'd fix it.
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:1>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Natalia
| Bidart
Type: Bug | Status: assigned
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Natalia Bidart):

Thank you Mariusz, let's push forward your PR, mine needs tests still and
release note. I will re-assign this to you. I guess I should issue
releases ASAP? Is there a precedent for how to handle cases like this?

We also need to update the docstrings of the two validator helpers because
they clearly state that a string is expected (and this is what caused the
issue in the first place). I agree with you that we can't change that
behavior (i.e. the validators should work with ipv4 and ipv6 instances),
so we need to update the docstrings. I will add further comments in the
PR.
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:2>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------

Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak

Type: Bug | Status: assigned
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* has_patch: 0 => 1
* owner: Natalia Bidart => Mariusz Felisiak


Old description:

> TypeError: object of type 'IPv6Address' has no len()
>

> TypeError: object of type 'IPv6Address' has no len()
> }}}

New description:

TypeError: object of type 'IPv6Address' has no len()
}}}

--
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:3>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak
Type: Bug | Status: assigned
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Natalia Bidart):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:4>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak
Type: Bug | Status: assigned
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak):

> I guess I should issue releases ASAP? Is there a precedent for how to
handle cases like this?

It depends how serious a regression is. It's not very urgent, but
releasing a hot-fix wouldn't hurt. The last time I remember was in 2021,
when we issued 9 releases in two week :)

- https://www.djangoproject.com/weblog/2021/may/04/security-releases/
- https://www.djangoproject.com/weblog/2021/may/06/security-releases/
- https://www.djangoproject.com/weblog/2021/may/13/bugfix-releases/
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:5>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak
Type: Bug | Status: assigned
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* cc: Sarah Boyce (added)

Comment:

Replying to [comment:5 Mariusz Felisiak]:

> > I guess I should issue releases ASAP? Is there a precedent for how to
handle cases like this?
>
> It depends how serious a regression is. It's not very urgent, but
releasing a hot-fix wouldn't hurt. The last time I remember was in 2021,
when we issued 9 releases in two week :)
>
> - https://www.djangoproject.com/weblog/2021/may/04/security-releases/
> - https://www.djangoproject.com/weblog/2021/may/06/security-releases/
> - https://www.djangoproject.com/weblog/2021/may/13/bugfix-releases/

Thank you for the pointers, I will consider doing releases tomorrow
morning, before Sarah cuts the new stable branch (will add Sarah as cc).
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:6>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak
Type: Bug | Status: assigned
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Sarah Boyce):

> Thank you for the pointers, I will consider doing releases tomorrow
morning, before Sarah cuts the new stable branch (will add Sarah as cc).

I don't think the cutting of the 5.2 branch and an urgency as to when we
release. I think it's preferable to merge into main (and do the back-
ports) before the cut, but whether there is a release or not doesn't make
a difference to me (I don't think).
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:7>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak

Type: Bug | Status: closed
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by nessita <124304+nessita@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"b3c5830769d8a5dbf2f974da7116fe503c9454d9" b3c5830]:
{{{#!CommitTicketReference repository=""
revision="b3c5830769d8a5dbf2f974da7116fe503c9454d9"
Fixed #36098 -- Fixed validate_ipv6_address()/validate_ipv46_address()
crash for non-string values.

Regression in ca2be7724e1244a4cb723de40a070f873c6e94bf.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:8>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak
Type: Bug | Status: closed
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Natalia <124304+nessita@…>):

In [changeset:"c81669cb54f964c6e726c7f130211babe93e6991" c81669cb]:
{{{#!CommitTicketReference repository=""
revision="c81669cb54f964c6e726c7f130211babe93e6991"
[5.1.x] Fixed #36098 -- Fixed

validate_ipv6_address()/validate_ipv46_address() crash for non-string
values.

Regression in ca2be7724e1244a4cb723de40a070f873c6e94bf.

Backport of b3c5830769d8a5dbf2f974da7116fe503c9454d9 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:9>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak
Type: Bug | Status: closed
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Natalia <124304+nessita@…>):

In [changeset:"21dfd30d69b44f82d7361d4941a0987268bac55e" 21dfd30]:
{{{#!CommitTicketReference repository=""
revision="21dfd30d69b44f82d7361d4941a0987268bac55e"
[5.0.x] Fixed #36098 -- Fixed

validate_ipv6_address()/validate_ipv46_address() crash for non-string
values.

Regression in ca2be7724e1244a4cb723de40a070f873c6e94bf.

Backport of b3c5830769d8a5dbf2f974da7116fe503c9454d9 from main.
}}}
--

Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:10>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak
Type: Bug | Status: closed
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Natalia <124304+nessita@…>):

In [changeset:"043dfadbcebc752d5ab2e9ff08c7e48044d20dc2" 043dfad]:
{{{#!CommitTicketReference repository=""
revision="043dfadbcebc752d5ab2e9ff08c7e48044d20dc2"
[4.2.x] Fixed #36098 -- Fixed

validate_ipv6_address()/validate_ipv46_address() crash for non-string
values.

Regression in ca2be7724e1244a4cb723de40a070f873c6e94bf.

Backport of b3c5830769d8a5dbf2f974da7116fe503c9454d9 from main.
}}}
--

Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:11>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak
Type: Bug | Status: closed
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by GitHub <noreply@…>):

In [changeset:"57b0229421ad64fea6ba50a8628e6aedce017152" 57b0229]:
{{{#!CommitTicketReference repository=""
revision="57b0229421ad64fea6ba50a8628e6aedce017152"
[4.2.x] Refs #36098 -- Fixed validate_ipv4_address() crash for non-string
values.

Regression in ca2be7724e1244a4cb723de40a070f873c6e94bf.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:12>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak
Type: Bug | Status: closed
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by sysedit):

Hello,
the fix for the CVE introduces the following change:


{{{
from django.db.models import GenericIPAddressField
GenericIPAddressField().run_validators('fe80::f59c:37a5:cab1:8bd9%ethernet_32770')
}}}

This would be ok with Django 5.0.10 but would raise ValidationError:
['Enter a valid IPv4 or IPv6 address.'] in Django 5.0.11 as the string is
40 char long and not 39.

As this ticket only fixes non-str handling, the exception will still be
raised, correct ?

Should the max-length be raised to something larger, like 65 ?
https://superuser.com/questions/381022/how-many-characters-can-an-ip-
address-be seems to indicate that this would be the common value to handle
IPv6 with scope zone properly ?
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:13>

[Django]

#36098: TypeError: object of type 'IPv6Address' has no len() when running tests
with GeoDjango
-------------------------------------+-------------------------------------
Reporter: Natalia Bidart | Owner: Mariusz
| Felisiak
Type: Bug | Status: closed
Component: GIS | Version: 5.1
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Natalia Bidart):

Hello sysedit!

Replying to [comment:13 sysedit]:

> in Django 5.0.11 as the string is 40 char long and not 39. As this
ticket only fixes non-str handling, the exception will still be raised,
correct ?

Yes, this is correct.

> Should the max-length be raised to something larger, like 65 ?
https://superuser.com/questions/381022/how-many-characters-can-an-ip-
address-be seems to indicate that this would be the common value to handle
IPv6 with scope zone properly ?

You can customize the `max_length` yourself in your forms, see a wider
discussion in [https://forum.djangoproject.com/t/length-of-ipv6-address-
string/37953 this forum post], which I believe would solve the
`ValidationError` issue for your project.
--
Ticket URL: <https://code.djangoproject.com/ticket/36098#comment:14>