[Django] #32874: URLValidator is not validating schema relative URLs
Django
------------------------------------------+------------------------
Reporter: maciejstromich | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 3.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------+------------------------
Based on rfc1808#section-2.4.3 a valid schema relative URL is the one
starting with `//`
URLValidator is not validating such urls.
```
>>> from django.core.validators import URLValidator
>>> value = '//example.com'
>>> URLValidator().__call__(value)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/opt/venv/lib/python3.8/site-packages/django/core/validators.py",
line 110, in __call__
raise ValidationError(self.message, code=self.code, params={'value':
value})
django.core.exceptions.ValidationError: ['Enter a valid URL.']
```
--
Ticket URL: <https://code.djangoproject.com/ticket/32874>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Maciej Strömich | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 3.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Old description:
> Based on rfc1808#section-2.4.3 a valid schema relative URL is the one
> starting with `//`
>
> URLValidator is not validating such urls.
>
> ```
> >>> from django.core.validators import URLValidator
> >>> value = '//example.com'
> >>> URLValidator().__call__(value)
> Traceback (most recent call last):
> File "<console>", line 1, in <module>
> File "/opt/venv/lib/python3.8/site-packages/django/core/validators.py",
> line 110, in __call__
> raise ValidationError(self.message, code=self.code, params={'value':
> value})
> django.core.exceptions.ValidationError: ['Enter a valid URL.']
> ```
New description:
Based on rfc1808#section-2.4.3 a valid schema relative URL is the one
starting with `//`
URLValidator is not validating such urls.
{{{
>>> from django.core.validators import URLValidator
>>> value = '//example.com'
>>> URLValidator().__call__(value)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/opt/venv/lib/python3.8/site-packages/django/core/validators.py",
line 110, in __call__
raise ValidationError(self.message, code=self.code, params={'value':
value})
django.core.exceptions.ValidationError: ['Enter a valid URL.']
}}}
--
Comment (by Maciej Strömich):
formatting
--
Ticket URL: <https://code.djangoproject.com/ticket/32874#comment:1>
Django
Reporter: Maciej Strömich | Owner: overtoncmo
Type: Bug | Status: assigned
Component: Uncategorized | Version: 3.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by overtoncmo):
* owner: nobody => overtoncmo
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/32874#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Maciej Strömich | Owner: Charlie
| Overton
Type: New feature | Status: assigned
Component: Core (Other) | Version: 3.2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* type: Bug => New feature
* component: Uncategorized => Core (Other)
Comment:
The
[https://github.com/django/django/blob/bbb3965826c91406b94b819af7315ea1d39ec217/django/core/validators.py#L94
default list of accepted schemes] is `['http', 'https', 'ftp', 'ftps']`,
so I wouldn't call this a bug. I guess the question is whether or not
there should be a way to allow schema relative URLs (assuming adding `//`
to schemes don't already work, I haven't tried it).
--
Ticket URL: <https://code.djangoproject.com/ticket/32874#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Maciej Strömich | Owner: Charlie
| Overton
Type: New feature | Status: assigned
Component: Core (Other) | Version: 3.2
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Carlton Gibson):
* stage: Unreviewed => Accepted
Comment:
Yes, I guess it's worth looking into whether we can add support for
reasonable complexity. Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/32874#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Maciej Strömich | Owner: Charlie
| Overton
Type: New feature | Status: assigned
Component: Core (Other) | Version: 3.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Maciej Strömich):
The issue with scheme relative urls lies in this code block
https://github.com/django/django/blob/stable/3.2.x/django/core/validators.py#L108-L110
in which scheme is split from url based on `://` which in the absolute url
makes total sense.
> The default list of accepted schemes is ['http', 'https', 'ftp',
'ftps'], so I wouldn't call this a bug. I guess the question is whether or
not there should be a way to allow schema relative URLs (assuming adding
// to schemes don't already work, I haven't tried it).
We were supporting schema relative urls with update to mentioned list in a
way like this {{{["", "http", "https"]}}} and it works in Django 2.2 well
if you're allowing to define the url in your forms like `://example.com`
instead of `//example.com`.
Between Django2.2 and Django 3.2
https://github.com/django/django/commit/b41d38ae26b1da9519a6cd765bc2f2ce7d355007
was introduced which changed the behaviour of `urlsplit()` call.
When upgrading to Django 3.2 URLValidator started to throw TypeError
exceptions because the default return value of
`urlsplit('://example.com).hostname` is `None` and it wasn't even getting
to the `ValidationError` line below.
In Django 2.2 the code was checking for `netloc` instead of `hostname`
which even if the netloc is not correctly found would return an empty
string `''` and the `len(urlsplit(..` code would pass because `''` is less
than 253 characters.
--
Ticket URL: <https://code.djangoproject.com/ticket/32874#comment:5>
Django
-------------------------------------+-------------------------------------
| Szatmáry
Type: New feature | Status: assigned
Component: Core (Other) | Version: 3.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Zoltán Szatmáry):
* owner: Charlie Overton => Zoltán Szatmáry
* has_patch: 0 => 1
Comment:
I've just made a PR for that.
[https://code.djangoproject.com/ticket/32874]
--
Ticket URL: <https://code.djangoproject.com/ticket/32874#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Maciej Strömich | Owner: Zoltán
| Szatmáry
Component: Core (Other) | Version: 3.2
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Carlton Gibson):
* status: assigned => closed
* resolution: => wontfix
Comment:
After review by the Django Security Team, we're going to close this as
wontfix.
Protocol relative URLs are something of a legacy from times before HTTPS
was the norm, and their use now is generally discouraged. (e.g.
[https://webhint.io/docs/user-guide/hints/hint-no-protocol-relative-urls/
modern linters will flag them].)
What's more URLs are used in many non-web contexts, where the lack of a
scheme is not valid.
As such it's not something we want to add to URLValidator.
As with other similar cases, if users want to accept such URLs in their
application, a custom validator is the way forward.
--
Ticket URL: <https://code.djangoproject.com/ticket/32874#comment:7>