Re: [Django] #6681: Problem with restructuredtext in markup module
Django
--------------------------------+------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: reopened
Component: contrib.markup | Version: master
Severity: Normal | Resolution:
Keywords: markup bug | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+------------------------------------
Changes (by quinode):
* cc: dguardiola@… (added)
Comment:
Replying to [comment:11 resplin]:
> I confirmed that the problem still exists in Django 1.1, and forsberg's
workaround solves it.
Same bug, but with django 1.3.1 , I commented the line mentioned in
#comment:10 to get rid of it
I just used a blog app using restructured text and contrib.admindocs.
Does that mean I'll always have to patch django if I want to use this code
?
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:20>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
--------------------------------+------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: reopened
Component: contrib.markup | Version: master
Severity: Normal | Resolution:
Keywords: markup bug | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
--------------------------------+------------------------------------
* needs_better_patch: 0 => 1
Comment:
@quinode: No, but we need a viable patch, first. So far, there's a
suggestion as to a workaround, but nobody has answered @justinlilly's
question regarding what the purpose of the line was in the first place.
I'm not going to commit any patch that removes a line unless someone can
verify that the line doesn't have a purpose.
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:21>
Django
--------------------------------+------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: reopened
Component: contrib.markup | Version: master
Severity: Normal | Resolution:
Keywords: markup bug | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
--------------------------------+------------------------------------
Comment (by jgsogo):
I've just written a patch for this bug. I would wan't you to check it out:
https://github.com/django/django/pull/368
When **cmsreference** role fails (everywhere but in django admindocs) it
tries to exec docutils' default role.
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:22>
Django
--------------------------------+------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: reopened
Component: contrib.markup | Version: master
Severity: Normal | Resolution:
Keywords: markup bug | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
--------------------------------+------------------------------------
Comment (by poeml):
jgsogo's patch works for me. Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:23>
Django
--------------------------------+------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: reopened
Component: contrib.markup | Version: master
Severity: Normal | Resolution:
Keywords: markup bug | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
--------------------------------+------------------------------------
Comment (by anonymous):
Had the same issue, would like to see this fixed in core. This seems to be
the relevant commit:
$ git show 7106c68e
commit 7106c68e598e0f2ad2036249122222f1adb3b2d1
Author: Adrian Holovaty <adr...@holovaty.com>
Date: Wed Dec 7 05:11:19 2005 +0000
Fixed #985 -- Fixed admin docs behavior if docutils isn't installed.
Thanks, Tim Keating
git-svn-id: http://code.djangoproject.com/svn/django/trunk@1563
bcc190cf-cafb-0310-a4f2-bffc1f526a37
diff --git a/django/contrib/admin/utils.py b/django/contrib/admin/utils.py
index 513067a..80a8f2a 100644
--- a/django/contrib/admin/utils.py
+++ b/django/contrib/admin/utils.py
@@ -90,8 +90,10 @@ def default_reference_role(name, rawtext, text, lineno,
inliner, options={}, con
context = inliner.document.settings.default_reference_context
node = docutils.nodes.reference(rawtext, text, refuri=(ROLES[context]
% (inliner.document.settings.link_base, text)), **options)
return [node], []
-docutils.parsers.rst.roles.register_canonical_role('cmsreference',
default_reference_role)
-docutils.parsers.rst.roles.DEFAULT_INTERPRETED_ROLE = 'cmsreference'
-for (name, urlbase) in ROLES.items():
- create_reference_role(name, urlbase)
+if docutils_is_available:
+ docutils.parsers.rst.roles.register_canonical_role('cmsreference',
default_reference_role)
+ docutils.parsers.rst.roles.DEFAULT_INTERPRETED_ROLE = 'cmsreference'
+
+ for (name, urlbase) in ROLES.items():
+ create_reference_role(name, urlbase)
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:24>
Django
--------------------------------+------------------------------------
Reporter: mtredinnick | Owner: nobody
Component: contrib.markup | Version: master
Severity: Normal | Resolution: invalid
Keywords: markup bug | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
--------------------------------+------------------------------------
* status: reopened => closed
* resolution: => invalid
Comment:
`django.contrib.markup` is deprecated and was recently removed in master.
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:25>
Django
--------------------------------+------------------------------------
Reporter: mtredinnick | Owner: nobody
Component: contrib.markup | Version: master
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
--------------------------------+------------------------------------
* status: closed => new
* resolution: invalid =>
Comment:
This is not a problem with `django.contrib.markup`. This problem still
exists for anyone who wants use `admindocs` and also implement their own
`restructuredtext` filter (or use docutils to render rst for any other
purpose).
Django should not assign a *default* interpreted role that *only* works
with `django.contrib.admindocs.util.parse_rst()`.
The ideal fix would be for Django to not set a default to begin with, but
I do not know of a way to set a default inside `parse_rst()` only. If
Django *must* set a default, then the default should be as compatible with
user and 3rd party code as the original default. A refined version of
https://github.com/django/django/pull/368 should do the trick.
FYI this code appears to have been here from the *very* beginning, when
Django was a private framework where it likely didn't need to worry about
people wanting to use docutils to render reStructuredText outside of
Django's narrow focus.
https://github.com/django/django/commit/ed114e15106192b22ebb78ef5bf5bce72b419d13
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:26>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 1
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by mrmachine):
* cc: real.human@… (added)
* keywords: markup bug => restructuredtext docutils
* has_patch: 0 => 1
* component: contrib.markup => contrib.admindocs
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:27>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: new
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution:
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by mrmachine):
* needs_better_patch: 1 => 0
Comment:
New PR opened.
https://github.com/django/django/pull/1277
This just sets the default inside `parse_rst()` and resets it before
returning. This appears to be the only function that sets and passes the
settings overrides that are required by the "cmsreference" role function.
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:28>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: new
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution:
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by mrmachine):
As apollo13 mentioned, my approach of setting and unsetting the default
inside `parse_rst()` is not safe, I have updated my PR with a refined
version of the approach taken in PR368.
https://github.com/django/django/pull/1277
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:29>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: new
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution:
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by mrmachine):
Finally figured out how to set the default just for the block of
reStructuredText being published in `parse_rst()`. We don't need to set a
global default at all. PR updated.
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:30>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: new
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution:
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by timo):
* cc: timograham@… (added)
* needs_tests: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:31>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution: fixed
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 1 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"bcd4c3f27d5bcf701337ac7c74c042d90792bdc3"]:
{{{
#!CommitTicketReference repository=""
revision="bcd4c3f27d5bcf701337ac7c74c042d90792bdc3"
Fixed #6681 -- Don't break docutils when rendering reStructuredText.
Don't set a global default interpreted role function for reStructuredText.
Instead, use the `default-role` directive to change the default only
within
the `parse_rst()` function.
Thanks Malcolm Tredinnick for the report.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:32>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution:
docutils | Needs documentation: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by timo):
* status: closed => new
* has_patch: 1 => 0
* resolution: fixed =>
* needs_tests: 1 => 0
Comment:
This causes docutils to throw an error when running the full test suite
(or just the `webdesign` + `admin_views` tests. I'm not sure the best way
to fix it.
{{{
$ ./runtests.py django.contrib.webdesign admin_views
<tag:lorem>:2: (ERROR/3) Error in "default-role" directive:
no content permitted.
.. default-role:: cmsreference
* ``{% lorem %}`` will output the common "lorem ipsum" paragraph
* ``{% lorem 3 p %}`` will output the common "lorem ipsum" paragraph
and two random paragraphs each wrapped in HTML ``<p>`` tags
* ``{% lorem 2 w random %}`` will output two random latin words
}}}
The "Example:" bit in the docstring for
django.contrib.webdesign.templatetags.webdesign.lorem is parsed as
metadata so the bullet list is then interpreted as a new block and
unexpectedly indented under the `.. default-role` directive. Possibly we
can fix this particular docstring to avoid the problem, but I'm wondering
if other users will encounter the problem with their own docstrings.
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:33>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: new
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution:
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by bmispelon):
Would dedenting the text work in the general case?
It fixes the current issue and seems like the appropriate thing to do (I'm
not very familiar with the surrounding code however so I might be
completely wrong).
{{{#!diff
diff --git a/django/contrib/admindocs/utils.py
b/django/contrib/admindocs/utils.py
index f836253..a669531 100644
--- a/django/contrib/admindocs/utils.py
+++ b/django/contrib/admindocs/utils.py
@@ -1,6 +1,7 @@
"Misc. utility functions/classes for admin documentation generator."
import re
+from textwrap import dedent
from email.parser import HeaderParser
from email.errors import HeaderParseError
@@ -76,7 +77,7 @@ def parse_rst(text, default_reference_context,
thing_being_parsed=None):
.. default-role::
"""
- parts = docutils.core.publish_parts(source % text,
+ parts = docutils.core.publish_parts(source % dedent(text),
source_path=thing_being_parsed, destination_path=None,
writer_name='html', settings_overrides=overrides)
return mark_safe(parts['fragment'])
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:34>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: new
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution:
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* cc: bmispelon@… (added)
Comment:
(sorry for the noise, I forgot to add myself to the Cc list)
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:35>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: new
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution:
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by timo):
Yes, it seems it would fix the issue although not yield the same result as
before the first patch was committed. Originally the `<ul>` would appear
in a `<blockquote>` since it's indented. That's not necessary or really
desired in this case - I'm not sure if there's a use case that would
benefit from preserving indentation.
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:36>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: new
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution:
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* cc: dguardiola@… (removed)
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:37>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Type: Bug | Status: new
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution:
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"975415a8cef38cf2a49c3ce8234c15ef97a69b5f"]:
{{{
#!CommitTicketReference repository=""
revision="975415a8cef38cf2a49c3ce8234c15ef97a69b5f"
Fixed a webdesign template tag docstring to prevent parsing as metadata.
Previously admindocs would throw an error when processing it:
"Error in "default-role" directive: no content permitted."
refs #6681
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:38>
Django
reStructuredText
-------------------------------------+-------------------------------------
Reporter: mtredinnick | Owner: nobody
Component: contrib.admindocs | Version: master
Severity: Normal | Resolution: fixed
Keywords: restructuredtext | Triage Stage: Accepted
docutils | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
Comment:
I decided to fix the webdesign tag docstring. I don't expect normalized
docstrings to start with indentation -- clearly the intent here wasn't to
parse the "Example:" section as metadata.
--
Ticket URL: <https://code.djangoproject.com/ticket/6681#comment:39>