best way to build multiple documents with sphinx but only install/ship a subset?

[Peter Maydell]

Hi; for the QEMU project we're looking to convert our existing documentation to use Sphinx, and I have a question about how best to structure things, which I haven't been able to find an answer for in the documentation.

For QEMU we (will) have effectively several different manuals, but we don't want to ship them all to end users in "make install". To be concrete, we'll have, say, a "User documentation" manual which we do want to ship, but also a "Developer's guide" which deals with things that people working on QEMU internals want to know. We don't want to install the dev guide on every system, but it should be built in the source tree so that people using it can refer to it.

What's the best way to structure the sphinx docs so that we can do something like:

* building in the source tree builds all the docs (user and dev)

* "make install" takes the subset of those docs that the user cares about and copies them into the install directory, and that subset doesn't have any cross references, links or index entries into documents that aren't installed

?

The Sphinx documentation doesn't really seem to say anything about how to install generated HTML documentation, and mostly the examples assume you have exactly one lot of documentation and it's OK for it all to refer to each other and for the top level to have a toctree that links in everything.

https://github.com/sphinx-doc/sphinx/issues/3132 has a comment from somebody saying they'd like to do something like what we want, but that issue isn't fixed yet -- is there a standard workaround?

We haven't started the conversion yet, so we can be flexible about how to lay things out to work with whatever is best for Sphinx.

Thanks in advance for any suggestions!

-- PMM

[Matthew Woehlke]

On 25/01/2019 07.08, Peter Maydell wrote:
> Hi; for the QEMU project we're looking to convert our existing
> documentation to use Sphinx, and I have a question about how best to
> structure things, which I haven't been able to find an answer for in the
> documentation.
>
> For QEMU we (will) have effectively several different manuals, but we don't
> want to ship them all to end users in "make install". To be concrete, we'll
> have, say, a "User documentation" manual which we do want to ship, but also
> a "Developer's guide" which deals with things that people working on QEMU
> internals want to know. We don't want to install the dev guide on every
> system, but it should be built in the source tree so that people using it
> can refer to it.
>
> What's the best way to structure the sphinx docs so that we can do
> something like:
> * building in the source tree builds all the docs (user and dev)
> * "make install" takes the subset of those docs that the user cares about
> and copies them into the install directory, and that subset doesn't have
> any cross references, links or index entries into documents that aren't
> installed
> ?

Do they need to cross-reference each other?

If not, the easiest solution may be to just treat the two manuals as
entirely separate. That is, separate directories in the repo, separate
build steps, separate conf.py, etc... and then just don't write install
rules for the dev guide.

If the dev guide needs to link to the user guide, you can probably still
accomplish this using intersphinx. Again, same deal, but the build rule
for building the dev guide will depend on the rule for building the user
manual so that the exported links are up to date.

--
Matthew

[Peter Maydell]

On Mon, 28 Jan 2019 at 17:44, Matthew Woehlke <mwoehlk...@gmail.com> wrote:
>
> On 25/01/2019 07.08, Peter Maydell wrote:
> > What's the best way to structure the sphinx docs so that we can do
> > something like:
> > * building in the source tree builds all the docs (user and dev)
> > * "make install" takes the subset of those docs that the user cares about
> > and copies them into the install directory, and that subset doesn't have
> > any cross references, links or index entries into documents that aren't
> > installed
> > ?
>
> Do they need to cross-reference each other?
>
> If not, the easiest solution may be to just treat the two manuals as
> entirely separate. That is, separate directories in the repo, separate
> build steps, separate conf.py, etc... and then just don't write install
> rules for the dev guide.

Thanks; yes, there's no strong need for the different manuals
to cross-reference, so we can just treat them as separate.

(I'm not sure how that would interact with third-party sites like
readthedocs.io, but perhaps we could have a top level conf.py
in docs/ which they would use, but which would be ignored by
our makefile/install process which would just run sphinx-build
separately on docs/devel/conf.py, docs/user/conf.py and so on.
In any case having that kind of third-party site work is definitely
only a secondary "nice to have" feature.)

thanks
-- PMM

[Peter Maydell]

On Tue, 29 Jan 2019 at 10:14, Peter Maydell <peter....@linaro.org> wrote:
> On Mon, 28 Jan 2019 at 17:44, Matthew Woehlke <mwoehlk...@gmail.com> wrote:
> > If not, the easiest solution may be to just treat the two manuals as
> > entirely separate. That is, separate directories in the repo, separate
> > build steps, separate conf.py, etc... and then just don't write install
> > rules for the dev guide.
>
> Thanks; yes, there's no strong need for the different manuals
> to cross-reference, so we can just treat them as separate.

This is working out pretty well. I have noticed though that (with the
theme I've picked) the _static/fonts subdirectory of each manual has
2.7MB of font files which are the same for each manual. Is it possible
to get Sphinx to share the _static/ directories, which I assume are
going to be the same for every manual ?

> (I'm not sure how that would interact with third-party sites like
> readthedocs.io, but perhaps we could have a top level conf.py

> in docs/ which they would use...

I got this bit working quite nicely: each manual's conf.py is just
parent_config = os.path.abspath("../conf.py")
exec(compile(open(parent_config, "rb").read(), parent_config, 'exec'))
# then override/set things for this manual
and then you can build either an individual manual, or if you
point sphinx-build at the top-level conf.py it builds the lot.

thanks
-- PMM