More options for "Show Source" links?

[Matthias Geier]

Dear list.

Currently, source files are renamed to get the extension ".txt" and
copied to the hard-coded directory "_sources/" within the HTML output
directory.

This is reasonable for the classic case where all source files are
reST files. However, when using different source file types (via
source_suffix and source_parsers), this might not make so much sense
anymore.
Especially if the source files are not really meant to be read by
humans (e.g. Jupyter notebooks), it would be better to keep the
original file extension.
I know this is marked as "wontfix" in
https://github.com/sphinx-doc/sphinx/issues/88, but I think my above
motivation wasn't taken into account at the time (I guess in 2009
source_parsers didn't exist yet).

Apart from the file extension, it would also be useful to choose the
location of the files differently.
It would be nice to have an option to place the source files not in
"_sources/" but directly in the HTML output directory (keeping all
relative sub-directories).
This way, if there are links from a source file to some other local
files (which again, may be the case in Jupyter notebooks), those links
will still work if the source file is viewed within the HTML output
directory (assuming that those other files are copied there, too).

Yet another useful option would be to specify a URL where the original
source file is hosted (e.g. Github/Bitbucket/...).
In this case, the source files don't have to be copied to the HTML
output directory. With the correct configuration options, this would
allow to link to the correct version/branch/tag of the original source
on Github/Bitbucket/...
There is an extension that does a similar thing, but I think this
would be really great to have directly within Sphinx:
https://gist.github.com/mgedmin/6052926
https://pypi.python.org/pypi/sphinxcontrib-srclinks

The suggested changes would be very helpful in the context of my
extension for Jupyter notebooks (http://nbsphinx.rtfd.org/), but I'm
sure they are of more general interest.

The defaults could stay unchanged, so this wouldn't be a breaking change.

I think the same options would also make sense for the :download:
role, probably the settings should even be combined for the two
things?

Was this discussed already?

Would anyone else find this useful?

Would this be a bad idea?

cheers,
Matthias

[Bram Geron]

That looks like a neat extension.

I'm not sold yet on the idea of copying .rst sources over to a .rst
output, as many web servers do rely on file extensions to determine the
Content-Type header. For unknown file types (like .ipynb), web servers
also sometimes automatically set a Content-Disposition header so that
the browser opens a Save dialog. Is that what you're after?

For your use case, wouldn't it be almost as good to just upload the
source/ as well as the build/html/ directories to the web server?

I kind of like the idea of srclinks, but I want to see an example. Is
there a demo somewhere? Also, I don't know how srclinks could easily be
"general enough". Does srclinks always use the original extension?

Cheers, Bram

> --
> You received this message because you are subscribed to the Google Groups
> "sphinx-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sphinx-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Eric Holscher]

I agree that it makes sense to enable folks to change the Show Source links to Show on GitHub or something. We do this already with the Read the Docs theme, and it would be a reasonable thing to backport into Sphinx. +1 on it being a new option, to maintain backwards compat.

Cheers,

Eric

[Matthias Geier]

Hi Bram.

I'll try to eliminate your doubts ...

On Thu, Jan 21, 2016 at 4:17 PM, Bram Geron wrote:
> That looks like a neat extension.
>
> I'm not sold yet on the idea of copying .rst sources over to a .rst
> output, as many web servers do rely on file extensions to determine the
> Content-Type header.

Yes, that's why renaming to ".txt" is definitely a reasonable default
behavior. I'm not suggesting to change this default behavior!

But if you can control the web server (or at least know what it's
doing), you may not need this and you could choose the option to keep
the original file extensions.

> For unknown file types (like .ipynb), web servers
> also sometimes automatically set a Content-Disposition header so that
> the browser opens a Save dialog. Is that what you're after?

Yes, IMHO that's definitely more useful than seeing the JSON content
in the browser.

It would be even more extreme if you would want to use e.g. ODT files
as Sphinx sources (which I'm not planning to do, BTW, but someone else
might).

> For your use case, wouldn't it be almost as good to just upload the
> source/ as well as the build/html/ directories to the web server?

I'm not sure what you mean here.
Do you mean "manually upload"?

I'll describe some use cases below, I hope that'll clarify what I'm
trying to do.

> I kind of like the idea of srclinks, but I want to see an example. Is
> there a demo somewhere? Also, I don't know how srclinks could easily be
> "general enough".

TBH, I've never actually used srclinks, I've only read the description
and it sounded like something that I'd like to have in the Sphinx

core.

> Does srclinks always use the original extension?

I don't know, but I guess so. Otherwise I don't think it would work.

As a basis for discussion, I'd like to list (hopefully more clearly
than before) the options and behaviors that I have in mind for the
suggested change:

There should be a separate option to control if the extensions of
source files should be changed or not.

Another option should control where (within the HTML output directory)
the source files should be copied to. I can think of the following
options:

1) The default option. Unchanged behavior. Source files are copied to
"_sources/" (keeping the relative sub-directories).

2) Source files are not copied at all and no "Show Source" links are
created. This is probably already done with "html_show_sourcelink =
False"?

3) Same as 1), but with the possibility to choose a different
directory other than "_sources/". The for me most important choice
would be the HTML directory itself, without a new subdirectory (but
still keeping the relative sub-directories from the original source
directory).

4) Source files are not copied at all, but instead the "Show Source"
links are created from a user-specified URL-prefix plus the relative
source path (and probably a suffix?).

I think options 1, 3 and 4 also make perfect sense for :download:
links, but for those, the relative sub-directories would also have to
be kept, which is currently not the case!

Now let me illustrate my use cases for options 3 and 4 in the context
of using Jupyter notebooks as Sphinx sources:

Use case for 3:
If the source files (= Jupyter notebooks) and other local files (e.g.
by means of :download: links) are available in the appropriate
sub-directories of the HTML output directory, then the URL can be used
in tools like http://nbviewer.jupyter.org/ or http://mybinder.org/.
This would be a useful option if the original Sphinx source files are
not publicly available but only the HTML output.

This is not necessary if the source files are stored in a publicly
available repository. In that case the next option comes into play:

Use case for 4:
If the source files (= Jupyter notebooks) are hosted in a publicly
available repository, it doesn't really make sense to copy them to the
HTML output directory. It would be much more meaningful to link to the
original repository, because then visitors will know where to get all
the files and they can directly fork the repository and create issues
and all those good things.
The important part here is that the Sphinx-generated docs point to the
correct version/branch/tag of the repository, but that can be done by
choosing the correct URL prefix.

I think I personally would mostly use option 4, but I guess there
might be people who would appreciate option 3.
If you think option 3 is silly, then let's just do option 4!

Also, I'm quite confident that a more general handling of
source/download files would be useful in far more cases than I've
outlined above.

cheers,
Matthias