[Django] #18922: Proliferation of dev docs on search engines confuses newbies
Django
--------------------------------------------+--------------------
Reporter: dloewenherz | Owner: nobody
Type: Bug | Status: new
Component: Djangoproject.com Web site | Version: 1.4
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 1
Easy pickings: 0 | UI/UX: 0
--------------------------------------------+--------------------
I brought this issue up earlier today at DjangoCon, but the basic issue
can be summarized hence:
1. User searches to find info on a specific feature, gets directed to dev
documentation.
2. Said feature (on the development version) is backwards incompatible
with previous versions of Django.
3. User does not know better, assumes Django has a bug.
4. Invalid bug is filed in trac.
Hopefully, removing the dev documentation pages from search engines will
help solve
this issue, since those who want to read the dev docs can just click
through to
them by using the version navigation at the bottom of the docs pages.
I think there is a small subset of actual Django users who run their
applications
on trunk. I don't have data to back that up but I can't imagine it's a
lot.
I talked with Alex earlier today about how to deal with this issue--he
also suggested
redirecting users from dev -> 1.4 (or whatever the latest version is). I
initially
thought it was a good idea, but I thought about it and realized you would
have to at
least add some sort of referrer check. I then thought whether it would be
ok to do
something like
{{{
def conditional_documentation_redirect(request):
if not
request.META['REFERER'].startswith("http://docs.djangoproject.com"):
return HttpResponseRedirect #... and so on
}}}
This felt wrong to me because if that sort of check were in place, users
would
no longer be able to permalink to dev docs.
PR @ https://github.com/django/djangoproject.com/pull/43
--
Ticket URL: <https://code.djangoproject.com/ticket/18922>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: Djangoproject.com | Status: new
Web site | Version: 1.4
Severity: Normal | Resolution:
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by dloewenherz):
* owner: nobody => dloewenherz
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:1>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Web site | Version: 1.4
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => wontfix
Comment:
Thanks for the suggestion.
I understand the idea, but I'm not ready to wipe the dev docs from
Google's index. People should be able to find information about an
upcoming feature by googling its name.
Besides, we take care to mention in which version new feature are added.
And closing invalid tickets is cheap. The most troublesome change was the
new `{% url %]`, and the flow of tickets eventually stopped.
'''tl;dr''' The cost of the solution seems too high to me compared to the
magnitude of the problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:2>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: Djangoproject.com | Status: closed
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by dloewenherz):
This issue wasn't opened with a specific solution in mind--it's just
illuminating that this is a problem. I get if the solution I presented is
a bit too drastic, but I think closing this ticket ignores the real issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:3>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Web site | Version: 1.4
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by aaugustin):
* status: closed => reopened
* resolution: wontfix =>
* stage: Unreviewed => Accepted
Old description:
New description:
I brought this issue up earlier today at DjangoCon, but the basic issue
can be summarized hence:
1. User searches to find info on a specific feature, gets directed to dev
documentation.
2. Said feature (on the development version) is backwards incompatible
with previous versions of Django.
3. User does not know better, assumes Django has a bug.
4. Invalid bug is filed in trac.
EDIT: rejected proposal removed, see comment 4.
--
Comment:
Yes, we can keep this ticket to track other ideas to resolve this problem.
For the record, the original proposal was:
> PR @ https://github.com/django/djangoproject.com/pull/43
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:4>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: Djangoproject.com | Status: reopened
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by ptone):
I think if there is a choice when addressing this - we need to make
special efforts for the tutorials.
Perhaps a special admonition like from #14464
refs: #16954, #19359
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:5>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: Djangoproject.com | Status: reopened
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In
[changeset:"248bc06ed8904fff30c648737a97c9a01628686f/djangoproject.com"]:
{{{
#!CommitTicketReference repository="djangoproject.com"
revision="248bc06ed8904fff30c648737a97c9a01628686f"
Add a sitemap for the docs site.
Current docs have a higher priority than dev docs. Refs #18922.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:6>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: Djangoproject.com | Status: reopened
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In
[changeset:"170944d1f6f554a44e7f6d735a55d13531a9b7cf/djangoproject.com"]:
{{{
#!CommitTicketReference repository="djangoproject.com"
revision="170944d1f6f554a44e7f6d735a55d13531a9b7cf"
Added /stable/ that redirects to the default version.
Refs #18922. Unfortunately this isn't referenced anywhere and I don't
know how search engines could pick this URL.
Refs #19783.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:7>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Severity: Normal | Version: 1.4
Keywords: | Resolution:
Has patch: 1 | Triage Stage: Accepted
Needs tests: 0 | Needs documentation: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by giuliettamasina):
Is it a reasonable request that any version of the documentation other
than dev and the latest release should be hidden from search indexes
entirely?
Tried searching for "django authentication" on Google now, and it shows
dev, 1.5, and 1.4, in that order. I think 1.4 should be hidden to avoid
this kind of thing.
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:9>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: *.djangoproject.com | Status: new
Severity: Normal | Version: 1.4
Keywords: | Resolution:
Has patch: 1 | Triage Stage: Accepted
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
* cc: markus.magnuson@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:10>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: *.djangoproject.com | Status: new
Severity: Normal | Version: 1.4
Keywords: | Resolution:
Has patch: 1 | Triage Stage: Accepted
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by aaugustin):
1.4 is still supported, it shouldn't be hidden from search engines.
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:11>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: *.djangoproject.com | Status: new
Severity: Normal | Version: 1.4
Keywords: | Resolution:
Has patch: 1 | Triage Stage: Accepted
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by giuliettamasina):
Just out of curiosity, how are versions 1.3 and below hidden from search
index?
(They are all in the sitemap, so obviously search engines see them there,
although with low priority.)
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:12>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: *.djangoproject.com | Status: new
Severity: Normal | Version: 1.4
Keywords: | Resolution:
Has patch: 1 | Triage Stage: Accepted
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by aaugustin):
They aren't hidden, and depending on what you're searching Google can
still return various more or less recent versions of the docs.
Our main problem is that the docs for each new release duplicate content
from previous releases, and Google favors the original URL in this case.
Canonicalizing /en/1.5/ (or whatever the current version is) =>
/en/stable/ may help.
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:13>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: *.djangoproject.com | Status: new
Severity: Normal | Version: 1.4
Keywords: | Resolution:
Has patch: 1 | Triage Stage: Accepted
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by giuliettamasina):
Does that mean that /en/stable/ should be included in sitemap.xml with
priority 1, instead of the latest version (i.e. /en/1.5/). Or should both
variants be in there? Or are we talking redirects?
Sorry to hassle you with a lot of questions, but I'm looking to hopefully
improve the situation, because I'm annoyed by the proliferation too :)
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:14>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: *.djangoproject.com | Status: new
Severity: Normal | Version: 1.4
Keywords: | Resolution:
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:15>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Component: *.djangoproject.com | Status: new
Severity: Normal | Version: 1.4
Keywords: | Resolution:
Has patch: 0 | Triage Stage: Accepted
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by aaugustin):
Eric Holscher just pointed me to
http://docs.readthedocs.org/en/latest/faq.html#can-i-make-search-engines-
only-see-one-version-of-my-docs.
We could use that to point search engines to the stable version.
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:16>
Django
Reporter: dloewenherz | Owner:
Type: Bug | dloewenherz
Severity: Normal | Version: 1.4
Keywords: | Resolution: duplicate
Has patch: 0 | Triage Stage: Accepted
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => duplicate
Comment:
Moved the last action item to
https://github.com/django/djangoproject.com/issues/242
--
Ticket URL: <https://code.djangoproject.com/ticket/18922#comment:17>