[Django] #31819: Django + Django REST Framework generates API errors with swagger
Django
-------------------------------------+-------------------------------------
Reporter: drwonky | Owner: nobody
Type: Bug | Status: new
Component: Core | Version: 3.0
(Other) | Keywords: swagger regex
Severity: Normal | javascrip rest openapi
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
The Django Validators use the syntax \Z to designate an end of line,
similar to the more conventional $ marker.
This swagger ticket gives context to the following: https://github.com
/swagger-api/swagger-editor/issues/1601
Javascript uses the $ as the end anchor, this has slightly different
semantics to the intended \Z anchor in Python.
The problem is that swagger chokes on this syntax, and looking at the code
it comes directly from the Python regexes being exported to the openapi
spec.
I propose 2 possible solutions:
1. Change the Django regexes to use the $ anchor, this may have unintended
behavior. The \Z regex is not valid as an anchor in Javascript.
2. Run all of the python regexes through a simple replace, changing \Z for
$
I'm open to any other suggestions, but as I see it now, the yaml produced
from Django regexes is not compliant with Javascript and javascript
validators by extension.
I can manually edit yaml files, but I'd like to have a dynamic API
endpoint for schema definitions.
--
Ticket URL: <https://code.djangoproject.com/ticket/31819>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: drwonky | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: swagger regex | Triage Stage:
javascrip rest openapi | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: drwonky (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/31819#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: drwonky | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: swagger regex | Triage Stage:
javascrip rest openapi | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by drwonky):
Here's a comparison of the regex APIs for Python and Javascript:
Python:
\Z
Matches only at the end of the string.
$
Matches the end of the string or just before the newline at the end of
the string, and in MULTILINE mode also matches before a newline. foo
matches both ‘foo’ and ‘foobar’, while the regular expression foo$ matches
only ‘foo’. More interestingly, searching for foo.$ in 'foo1\nfoo2\n'
matches ‘foo2’ normally, but ‘foo1’ in MULTILINE mode; searching for a
single $ in 'foo\n' will find two (empty) matches: one just before the
newline, and one at the end of the string.
Javascript:
$
Matches the end of input. If the multiline flag is set to true, also
matches immediately before a line break character. For example, /t$/ does
not match the "t" in "eater", but does match it in "eat".
PHP:
\Z
end of subject or newline at end (independent of multiline mode)
A dollar character ($) is an assertion which is TRUE only if the current
matching point is at the end of the subject string, or immediately before
a newline character that is the last character in the string (by default).
Dollar ($) need not be the last character of the pattern if a number of
alternatives are involved, but it should be the last item in any branch in
which it appears. Dollar has no special meaning in a character class.
The meaning of dollar can be changed so that it matches only at the very
end of the string, by setting the PCRE_DOLLAR_ENDONLY option at compile or
matching time. This does not affect the \Z assertion.
--
Ticket URL: <https://code.djangoproject.com/ticket/31819#comment:2>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Core (Other) | Version: 3.0
Keywords: swagger regex | Triage Stage:
javascrip rest openapi | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => invalid
Comment:
Hi Perry. This isn't in scope for Django: differences between language
implementations of regex isn't something we're going to address.
The particular issue with DRF OpenAPI schema generation should (at least
in part) be addressed by [https://github.com/encode/django-rest-
framework/pull/7389/ this PR, which was merged into DRF]. That will be
part of DRF 3.12, which is pending, but has been delayed by (let us just
call it) 2020, and will be released some time soon. In the meantime you
can install the latest master using
[https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support pip's
VCS support].
If you find there are further changes needed, please open a new issue or
preferably a PR on the DRF repo: it is appropriate there that we'll do
what we can to map between the required formats when generating the
OpenAPI schema.
Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/31819#comment:3>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Core (Other) | Version: 3.0
Severity: Normal | Resolution: invalid
Keywords: swagger regex | Triage Stage:
javascrip rest openapi | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Perry Harrington):
Hi, thank you for your reply.
I looked at the commit and it changes \Z to \z, from my research and
reading \z is not a valid anchor in several languages.
I think that $ is a more suitable replacement based on behavior, since \Z
matched before a newline like $ in most languages.
Like you said, not Django's problem, DRF's problem.
Let's see if I get any traction on that front.
--
Ticket URL: <https://code.djangoproject.com/ticket/31819#comment:4>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Core (Other) | Version: 3.0
Severity: Normal | Resolution: invalid
Keywords: swagger regex | Triage Stage:
javascrip rest openapi | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Perry Harrington):
Quick update because an edit doesn't trigger an email and you won't have
context in email:
I see you are the one who merged that pull request, would you care to
elaborate on why \z was accepted over $?
For me it's a simple code hack, but I would really like for it to be
upstream.
--
Ticket URL: <https://code.djangoproject.com/ticket/31819#comment:5>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Core (Other) | Version: 3.0
Severity: Normal | Resolution: invalid
Keywords: swagger regex | Triage Stage:
javascrip rest openapi | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Carlton Gibson):
It's best effort — the contributor suggested that fix (targeting
JavaScript I think.)
I'm sceptical that there's any perfect solution here. If you can do
better, please do suggest a PR with reasoning on DRF.
--
Ticket URL: <https://code.djangoproject.com/ticket/31819#comment:6>