qubes-doc & rtd
72 views
Skip to first unread message
mm
Dec 21, 2021, 2:57:32 PM12/21/21
to qubes-devel
Hello all,
in September there was a thread about migrating the documentation to another platform,
and the silent winner was RTD.
Let me say up front: For me localization, release specific docs, automation & easy and transparent maintenance of the documentation
are one of the most important criteria for this.
#### TRANSIFEX
There is a neat way to handle transifex and sphinx.
Due to a problem, which I am currently trying to solve, the sample translations are due to a transifex trick, downloaded via tx in the root repo and manually added to the qubestranslation repo,
which is not in an optimal state right now.
There is the possibility to have all the translations in one repo as in [13] and this is also the goal.
Problem: The problem is that for common users in RTD once a person has imported a repository there is no way to import it again, and for different translations,
RTD needs different projects, that exist in RTD and are marked as translations for the main one ... As till now I do not know how to tackle this issue.
## PANDOC
And of course there is the option to convert all the markdown documentation files
to restructured text (w pandoc f.x.),
One could convert the md to rst as in
```
pandoc doc.md --from markdown --to rst -s -o doc.rst
```
one can view it with restview (pip install restview)
## RTD problems/TODOs
1. Rendering
There are warnings that have to be adressed such as:
```
/home/user/qubes/qubesos.github.io.testrun/_doc/docs/source/user/advanced-topics/awesomewm.md:22: WARNING: Pygments lexer name 'shell_session' is not known
/home/user/qubes/qubesos.github.io.testrun/_doc/docs/source/project-security/verifying-signatures.md:150: WARNING: 'myst' reference target not found: /support/
/home/user/qubes/qubesos.github.io.testrun/_doc/docs/source/user/troubleshooting/vpn-troubleshooting.md: WARNING: document isn't included in any toctree
...
```
2. Many broken links
3. PDF - the documentation is not being rendered correctly
4. EPUB - the documentation is not being rendered correctly
5. Sphinx uses the Jinja templating engine for its HTML templates. Meaning for files like
https://github.com/maiska/qubes-doc.testrun/blob/49253a1f5a24a9a9f5c1c42e2395ca494a1e4b4b/developer/general/style-guide.md
this is the result: https://qubes-doctestrun.readthedocs.io/de/latest/developer/general/style-guide.html
I have not found a converter from liquid to jinja, so there should be another solution
6. External files: can be just listed in the doc.md and the markdown files can be deleted. But the redirects are to be handled
7. Warning: RTD is somewhat buggy if one pushes immediately one after another several branches for one project that are activated
8. Find out how to handle multiple translations with one repo in RTD
9. Uploading to transifex problem (wip)
in September there was a thread about migrating the documentation to another platform,
and the silent winner was RTD.
Let me say up front: For me localization, release specific docs, automation & easy and transparent maintenance of the documentation
are one of the most important criteria for this.
I took a closer look at RTD wtr
to localisation and this is what I found out:
TL;DR:
####
Here is a preliminary test version of the qubes documentation as in _doc, uploaded to the RTD, using sphinx with markdown support and transifex fake translations:
https://qubes-doctestrun.readthedocs.io/en/latest/README.html
& https://qubesosgithubiotestrun.readthedocs.io/en/latest/doc.html
And a sample wannabe german translation at:
https://qubes-doctestrun.readthedocs.io/de/latest/README.html
& https://qubes-doctestrun.readthedocs.io/de/latest/doc.html
In the bottom menu you can choose a language, specific version (right now only different html themes, pdf, epub)
As there are issues, as explained below, one can for now browse with dedication via specific links such as:
https://qubesosgithubiotestrun.readthedocs.io/en/latest/developer/building/qubes-builder.html
& https://qubes-doctestrun.readthedocs.io/de/latest/developer/building/qubes-builder.html
TL;DR:
####
Here is a preliminary test version of the qubes documentation as in _doc, uploaded to the RTD, using sphinx with markdown support and transifex fake translations:
https://qubes-doctestrun.readthedocs.io/en/latest/README.html
& https://qubesosgithubiotestrun.readthedocs.io/en/latest/doc.html
And a sample wannabe german translation at:
https://qubes-doctestrun.readthedocs.io/de/latest/README.html
& https://qubes-doctestrun.readthedocs.io/de/latest/doc.html
In the bottom menu you can choose a language, specific version (right now only different html themes, pdf, epub)
As there are issues, as explained below, one can for now browse with dedication via specific links such as:
https://qubesosgithubiotestrun.readthedocs.io/en/latest/developer/building/qubes-builder.html
& https://qubes-doctestrun.readthedocs.io/de/latest/developer/building/qubes-builder.html
Of course the test repos from
are behind the current state for ages.
The longer version:
#############
There are two generators that can are used by RTD -- sphinx (reST) or mkdocs (md).
The longer version:
#############
There are two generators that can are used by RTD -- sphinx (reST) or mkdocs (md).
MkDocs is perfect for Markdown,
Sphinx - for restructured text, but can parse markdown, via an
extension
but there are some problems as listed below, which need to be addressed.
but there are some problems as listed below, which need to be addressed.
## MkDocs
MkDocs writes with markdown and is more or less straightforward with qubes current state of the documentation/website.
The problem is that MkDocs does not support localization out of the box,
but through third party plugins such as [1] or [2],
MkDocs writes with markdown and is more or less straightforward with qubes current state of the documentation/website.
The problem is that MkDocs does not support localization out of the box,
but through third party plugins such as [1] or [2],
which more or less defaults to
the current way of localizing the website and thus also _doc.
There are also not many third party plugins for MkDocs.
And most important - localization (in RTD) is only supported for sphinx documentation, which settles it if the docs should be moved to RTD at least for now:
"This feature only applies to Sphinx documentation. We are working to bring it to our other documentation backends." See [3]
## SPHINX:
And most important - localization (in RTD) is only supported for sphinx documentation, which settles it if the docs should be moved to RTD at least for now:
"This feature only applies to Sphinx documentation. We are working to bring it to our other documentation backends." See [3]
## SPHINX:
The default generator used by
RTD, which supports all the features by RTD, including
localization.
### MYST
To use sphinx with markdown I tried MyST
(see [4], [5], [6]. MyST specific configuration can be seen at [7])
### HTML themes
There are some built-in themes ([8]), configuration options ([9] or [10])
Other themes can be found at [11].
Or one could overwrite and customize a theme with [12] or
by following some tutorials ( which should the way to go for Qubes I think )
#### BUILD
is as simple as `sphinx-build -b html _doc/ _build/html`
for latex:
```
sphinx-build -b latex _doc/ _build/latex
cd _build/latex
make # needs latexmk
```
### LOCALIZATION
### MYST
To use sphinx with markdown I tried MyST
(see [4], [5], [6]. MyST specific configuration can be seen at [7])
### HTML themes
There are some built-in themes ([8]), configuration options ([9] or [10])
Other themes can be found at [11].
Or one could overwrite and customize a theme with [12] or
by following some tutorials ( which should the way to go for Qubes I think )
#### BUILD
is as simple as `sphinx-build -b html _doc/ _build/html`
for latex:
```
sphinx-build -b latex _doc/ _build/latex
cd _build/latex
make # needs latexmk
```
### LOCALIZATION
Localization for sphinx is done
via gettext as simple as:
```
sphinx-build -b gettext . _build/gettext
```
for the configuration the following options are good as per documentation:
```
gettext_uuid=True
gettext_compact=False
```
```
sphinx-build -b gettext . _build/gettext
```
for the configuration the following options are good as per documentation:
```
gettext_uuid=True
gettext_compact=False
```
#### TRANSIFEX
There is a neat way to handle transifex and sphinx.
Due to a problem, which I am currently trying to solve, the sample translations are due to a transifex trick, downloaded via tx in the root repo and manually added to the qubestranslation repo,
which is not in an optimal state right now.
There is the possibility to have all the translations in one repo as in [13] and this is also the goal.
Problem: The problem is that for common users in RTD once a person has imported a repository there is no way to import it again, and for different translations,
RTD needs different projects, that exist in RTD and are marked as translations for the main one ... As till now I do not know how to tackle this issue.
## PANDOC
And of course there is the option to convert all the markdown documentation files
to restructured text (w pandoc f.x.),
which is native for Sphinx, and
thus, many of the current rendering issues
should be non-existent.
should be non-existent.
One could convert the md to rst as in
```
pandoc doc.md --from markdown --to rst -s -o doc.rst
```
one can view it with restview (pip install restview)
## RTD problems/TODOs
1. Rendering
There are warnings that have to be adressed such as:
```
/home/user/qubes/qubesos.github.io.testrun/_doc/docs/source/user/advanced-topics/awesomewm.md:22: WARNING: Pygments lexer name 'shell_session' is not known
/home/user/qubes/qubesos.github.io.testrun/_doc/docs/source/project-security/verifying-signatures.md:150: WARNING: 'myst' reference target not found: /support/
/home/user/qubes/qubesos.github.io.testrun/_doc/docs/source/user/troubleshooting/vpn-troubleshooting.md: WARNING: document isn't included in any toctree
...
```
2. Many broken links
3. PDF - the documentation is not being rendered correctly
4. EPUB - the documentation is not being rendered correctly
5. Sphinx uses the Jinja templating engine for its HTML templates. Meaning for files like
https://github.com/maiska/qubes-doc.testrun/blob/49253a1f5a24a9a9f5c1c42e2395ca494a1e4b4b/developer/general/style-guide.md
this is the result: https://qubes-doctestrun.readthedocs.io/de/latest/developer/general/style-guide.html
I have not found a converter from liquid to jinja, so there should be another solution
6. External files: can be just listed in the doc.md and the markdown files can be deleted. But the redirects are to be handled
7. Warning: RTD is somewhat buggy if one pushes immediately one after another several branches for one project that are activated
8. Find out how to handle multiple translations with one repo in RTD
9. Uploading to transifex problem (wip)
10. Analytics (JS in general in
RTD): script as in [14]
with a RTD section regarding this [15]
What are your thoughts on this?
All the best,
m
All the best,
m
Andrew David Wong
Dec 22, 2021, 5:10:30 AM12/22/21
to mm, qubes-devel
On 12/21/21 11:57 AM, mm wrote:
> Hello all,
>
> in September there was a thread about migrating the documentation to
> another platform,
> and the silent winner was RTD.
> Let me say up front: For me localization, release specific docs,
> automation & easy and transparent maintenance of the documentation
> are one of the most important criteria for this.
> I took a closer look at RTD wtr to localisation and this is what I found
> out:
>
> [...]
> Hello all,
>
> in September there was a thread about migrating the documentation to
> another platform,
> and the silent winner was RTD.
> Let me say up front: For me localization, release specific docs,
> automation & easy and transparent maintenance of the documentation
> are one of the most important criteria for this.
> I took a closer look at RTD wtr to localisation and this is what I found
> out:
>
>
> What are your thoughts on this?
>
Thank you very much for investigating this so deeply and sharing your
> What are your thoughts on this?
>
findings! To be honest, a lot of this is over my head, but I'm very much
open to hearing everyone's opinions about the best way to proceed. I'll
support whatever path forward is best for the project.
On a personal note, I've been the designated "website and documentation
maintainer" for a while now, but, for me, the key word is "maintainer."
I'm not a professional developer, and my self-taught skills are very
limited. I'm better suited to editing content and administering an
established system rather than attempting to build and debug new
technical systems.
As we've grown as a project, I've been feeling the pressure as our
website and documentation needs have outstripped my abilities. One of
the main reasons I wanted to look into something like RTD was to find a
more "turnkey" solution to alleviate this pressure [*]. We need new
features that I simply don't have the ability to build, which is a major
limitation of our current custom system.
I would be happy to step aside or share responsibilities with anyone who
has the needed skills and is willing to step in, whether to take the
reins or just to fill certain gaps. After all, as the project gets
larger, it would make sense that we have more specialized roles.
[*] However, it was just an idea that's still being hashed out, not a
plan set in stone! This thread is a welcome and appreciated step forward
in that process.
--
Andrew David Wong
Community Manager
The Qubes OS Project
https://www.qubes-os.org
Manuel Amador (Rudd-O)
Dec 25, 2021, 3:16:39 PM12/25/21
to Andrew David Wong, mm, qubes-devel
Honestly if I were directing the project I would set up a Plone instance, and use its excellent i18n to write docs. Additionally, if wanted, I would set up two way sync between a github repo and the site. Ask me for more details if interested.
mm
Dec 25, 2021, 4:37:09 PM12/25/21
to qubes...@googlegroups.com
On 12/25/21 21:16, Manuel Amador (Rudd-O) wrote:
>
> Honestly if I were directing the project I would set up a Plone instance, and use its excellent i18n to write docs. Additionally, if wanted, I would set up two way sync between a github repo and the site. Ask me for more details if interested.
suggestion.
In my personal opinion Plone as a CMS is much too heavy for the purposes
of the user documentation of Qubes.
I have experience with it, though a decade ago probably, and it is not
as intuitively as one might think.
Perhaps things changed and is a straightforward easy to use idk.
If you would like and have the time, could you outline how would you
proceed with markdown - Plone integration (or if it works w/o issues),
automation of translation/localization wrt to transifex and release
specific docs,
so one could weight the pro and contra of the different approaches till now.
All the best,
m
Manuel Amador (Rudd-O)
Dec 26, 2021, 7:47:01 AM12/26/21
to qubes...@googlegroups.com, mm
On December 25, 2021 10:37:04 PM GMT+01:00, mm <mai...@maiski.net> wrote:
>
>On 12/25/21 21:16, Manuel Amador (Rudd-O) wrote:
>>
>> Honestly if I were directing the project I would set up a Plone instance, and use its excellent i18n to write docs. Additionally, if wanted, I would set up two way sync between a github repo and the site. Ask me for more details if interested.
>
>
>Thank you very much for taking the time to reply and also for your
>suggestion.
>
>In my personal opinion Plone as a CMS is much too heavy for the purposes
>of the user documentation of Qubes.
>
>I have experience with it, though a decade ago probably, and it is not
>as intuitively as one might think.
>
>Perhaps things changed and is a straightforward easy to use idk.
>
>If you would like and have the time, could you outline how would you
>proceed with markdown - Plone integration (or if it works w/o issues),
>
>automation of translation/localization wrt to transifex and release
>specific docs,
E.g. article /en/howtos/vpn would get exported, posted to Transifex (possibly through a Git repo), and then once the German translation is available, /de/howtos/vpn could be posted, linking it to /en/howtos/vpn so the UI can show all translations for the same content item.
The items could then be published by the same pipeline doing the exportimport, or left to be reviewed prior to publishing by people.
mm
Dec 26, 2021, 11:33:20 AM12/26/21
to qubes...@googlegroups.com
Hi again
On 12/26/21 13:46, Manuel Amador (Rudd-O) wrote:
> aps things changed and is a straightforward easy to use idk.
> It's very straightforward to use now.
ok, nice to hear that
https://docs.plone.org/develop/plone/content/importexport.html or
https://docs.plone.org/develop/plone/i18n/internationalisation.html
I see.
Maintenance as you say should not be a problem, so let's stay at that
for now.
I can imagine that release specific docs could be branch based and
configured.
Still my opinion is that it is not light weight enough for the purpose
of the documentation,
would require another set of tools/knowledge/migration from an already
established static html generator to a non static CMS,
could and would introduce security concerns
(see
https://www.cvedetails.com/vulnerability-list/vendor_id-4313/Plone.html
vs
https://www.cvedetails.com/product/56912/Readthedocs-Read-The-Docs.html?vendor_id=20051)
Perhaps others would like to share further views on the overall topic?
All the best,
m
p.s.
imho RDT would still be the way to go
On 12/26/21 13:46, Manuel Amador (Rudd-O) wrote:
> aps things changed and is a straightforward easy to use idk.
> It's very straightforward to use now.
> Plone supports Markdown out of the box.
good
>
>> automation of translation/localization wrt to transifex and release
>> specific docs,
> This would require a bit of code that extracts the strings to translate (using the exportimport framework) from the original language folder, and deploys the translated content (using the exportimport framework) to the respective language folders (linking the translation to the original).
>
> E.g. article /en/howtos/vpn would get exported, posted to Transifex (possibly through a Git repo), and then once the German translation is available, /de/howtos/vpn could be posted, linking it to /en/howtos/vpn so the UI can show all translations for the same content item.
>
> The items could then be published by the same pipeline doing the exportimport, or left to be reviewed prior to publishing by people.
I think you are referring to smth like
>> automation of translation/localization wrt to transifex and release
>> specific docs,
> This would require a bit of code that extracts the strings to translate (using the exportimport framework) from the original language folder, and deploys the translated content (using the exportimport framework) to the respective language folders (linking the translation to the original).
>
> E.g. article /en/howtos/vpn would get exported, posted to Transifex (possibly through a Git repo), and then once the German translation is available, /de/howtos/vpn could be posted, linking it to /en/howtos/vpn so the UI can show all translations for the same content item.
>
> The items could then be published by the same pipeline doing the exportimport, or left to be reviewed prior to publishing by people.
https://docs.plone.org/develop/plone/content/importexport.html or
https://docs.plone.org/develop/plone/i18n/internationalisation.html
I see.
Maintenance as you say should not be a problem, so let's stay at that
for now.
I can imagine that release specific docs could be branch based and
configured.
Still my opinion is that it is not light weight enough for the purpose
of the documentation,
would require another set of tools/knowledge/migration from an already
established static html generator to a non static CMS,
could and would introduce security concerns
(see
https://www.cvedetails.com/vulnerability-list/vendor_id-4313/Plone.html
vs
https://www.cvedetails.com/product/56912/Readthedocs-Read-The-Docs.html?vendor_id=20051)
Perhaps others would like to share further views on the overall topic?
All the best,
m
p.s.
imho RDT would still be the way to go
Marek Marczykowski-Górecki
Sep 2, 2022, 11:31:30 AM9/2/22
to mm, qubes-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Let me try to resurrect this thread :)
First of all, thanks again for all this analysis. I think indeed RTD is
the way to go. We'll have occasion to talk about details on the
upcoming Qubes summit. Personally, I wouldn't exclude an option to
convert documentation to ReST format, especially since we get several
more features then (better internal links tracking, PDF/ePUB generation,
built-in macros for notices, warnings etc, and several more). But of
course this needs to be weighted against other factors like complexity
of the conversion itself, or ease of use for maintainers and
contributors.
See you soon!
> All the best,
>
> m
>
> [1] https://github.com/ultrabug/mkdocs-static-i18n
> [2] https://github.com/mondeja/mkdocs-mdpo-plugin
> [3] https://docs.readthedocs.io/en/stable/localization.html
> [4] https://myst-parser.readthedocs.io/en/latest/
> [5] https://www.sphinx-doc.org/en/master/usage/markdown.html
> [6] https://myst-parser.readthedocs.io/en/latest/sphinx/intro.html
> [7] https://myst-parser.readthedocs.io/en/latest/sphinx/reference.html
> [8] https://www.sphinx-doc.org/en/master/usage/theming.html#builtin-themes
> [9] https://www.sphinx-doc.org/en/master/usage/theming.html
> [10] https://sphinx-rtd-theme.readthedocs.io/en/stable/configuring.html
> [11] https://sphinx-themes.org/#themes
> [12] https://docs.readthedocs.io/en/stable/guides/adding-custom-css.html
> [13] https://github.com/sphinx-doc/sphinx-doc-translations
> [14]
> https://assets.readthedocs.org/static/javascript/readthedocs-analytics.js
> [15] https://docs.readthedocs.io/en/latest/advertising/advertising-details.html#analytics
>
> --
> You received this message because you are subscribed to the Google Groups "qubes-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/f24cb2b0-9465-ef3c-c71f-afb71248ab9f%40maiski.net.
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmMSIcwACgkQ24/THMrX
1yw03Qf/c63FuOGz6zsBwRWMLabRM3JxCA05ILFu76bg3/MI68DtbiRcYEBnCekR
CbvNIxTLq077X4N58HiIlsunhrln+n2eLgsD6H/ljp3ZGmbNdzfZhz2WdbWMaKjK
4dzdxEb6pUfrDqXrNOrxDOxRHBJYLR6mjr2ih211iPVaJccgIny8l8zSkWpTmG6S
JmowbLeaxfNXsv00yODQ9Hv/7hTZj5xd0Kntq17FGrL9BWLFad0fVYxzZ93Dh6NI
g6KgSnwWAvLqUZjQ7BSrjhbXuKUOeIHJ+luA23aqKioWHNcA9SXvtz2bUZyYVNeM
Eve3568YYj12PcZIjbZH+3leDZ+EbA==
=kYM7
-----END PGP SIGNATURE-----
Hash: SHA256
First of all, thanks again for all this analysis. I think indeed RTD is
the way to go. We'll have occasion to talk about details on the
upcoming Qubes summit. Personally, I wouldn't exclude an option to
convert documentation to ReST format, especially since we get several
more features then (better internal links tracking, PDF/ePUB generation,
built-in macros for notices, warnings etc, and several more). But of
course this needs to be weighted against other factors like complexity
of the conversion itself, or ease of use for maintainers and
contributors.
See you soon!
> All the best,
>
> m
>
> [1] https://github.com/ultrabug/mkdocs-static-i18n
> [2] https://github.com/mondeja/mkdocs-mdpo-plugin
> [3] https://docs.readthedocs.io/en/stable/localization.html
> [4] https://myst-parser.readthedocs.io/en/latest/
> [5] https://www.sphinx-doc.org/en/master/usage/markdown.html
> [6] https://myst-parser.readthedocs.io/en/latest/sphinx/intro.html
> [7] https://myst-parser.readthedocs.io/en/latest/sphinx/reference.html
> [8] https://www.sphinx-doc.org/en/master/usage/theming.html#builtin-themes
> [9] https://www.sphinx-doc.org/en/master/usage/theming.html
> [10] https://sphinx-rtd-theme.readthedocs.io/en/stable/configuring.html
> [11] https://sphinx-themes.org/#themes
> [12] https://docs.readthedocs.io/en/stable/guides/adding-custom-css.html
> [13] https://github.com/sphinx-doc/sphinx-doc-translations
> [14]
> https://assets.readthedocs.org/static/javascript/readthedocs-analytics.js
> [15] https://docs.readthedocs.io/en/latest/advertising/advertising-details.html#analytics
>
> You received this message because you are subscribed to the Google Groups "qubes-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/f24cb2b0-9465-ef3c-c71f-afb71248ab9f%40maiski.net.
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmMSIcwACgkQ24/THMrX
1yw03Qf/c63FuOGz6zsBwRWMLabRM3JxCA05ILFu76bg3/MI68DtbiRcYEBnCekR
CbvNIxTLq077X4N58HiIlsunhrln+n2eLgsD6H/ljp3ZGmbNdzfZhz2WdbWMaKjK
4dzdxEb6pUfrDqXrNOrxDOxRHBJYLR6mjr2ih211iPVaJccgIny8l8zSkWpTmG6S
JmowbLeaxfNXsv00yODQ9Hv/7hTZj5xd0Kntq17FGrL9BWLFad0fVYxzZ93Dh6NI
g6KgSnwWAvLqUZjQ7BSrjhbXuKUOeIHJ+luA23aqKioWHNcA9SXvtz2bUZyYVNeM
Eve3568YYj12PcZIjbZH+3leDZ+EbA==
=kYM7
-----END PGP SIGNATURE-----
mm
Sep 4, 2022, 4:05:49 PM9/4/22
to Marek Marczykowski-Górecki, qubes-devel
Hello Marek,
thank you for resurrecting the thread. I wanted to write like a week ago,
but yet, it lasted longer.
I am glad to announce that the initial hurdle for Qubes OS docs
RST-conversion,
localization, hosting on RtD is behind us and you can see a preliminary
result on
https://current-qubes-docrtd.readthedocs.io/en/latest
This is the current status of the branch rtd-test-default-html-theme in
the repo https://github.com/maiska/qubesos.github.io.rtd + submodules
To see the partially translated docs in german you can go
down to the left, click on the arrow right to 'v:latest' and choose the
language.
There are further tweeks to be done, such as epub and pdf, but the big
step is taken (including automated conversion via script and
postprocessing cross-referencing in sphinx, but still needing minor
manually fixes after that, documentation follows) and translation
workflow tested.
I will not bother you with details at that stage but with questions.
The outline of the questions is as follows:
1. Easily fixable dead end urls as found in the official docs
2. Non easily fixable links
3. Website content vs RtD documentation content
4. Hidden documentation files
1. These are the commits that show what can be fixed imho.
https://github.com/maiska/qubes-doc.rtd/commit/0b8529a026da0523355eec55c57c1da2aa110d07
https://github.com/maiska/qubes-doc.rtd/commit/2b05ac2d8418adbc1d4215d3d54195b3ca8e7f58
https://github.com/maiska/qubes-doc.rtd/commit/6772392f3e2e682d953d7e3b8d9f870867de202a
https://github.com/maiska/qubes-doc.rtd/commit/538bac0e0c368cd54e5acb0b5d1ffb77e06e5045
https://github.com/maiska/qubes-doc.rtd/commit/eb4d1dade1697d4b9e725db8871b1c54ec8c61d6
https://github.com/maiska/qubes-doc.rtd/commit/a8884c8e11f91458bb8ea30d6bbea49e25db48b2
Question: Should I just create Pull Requests for the proposed fixes
2. At least one broken for me unfixable link
in
https://github.com/QubesOS/qubes-doc/edit/master/developer/releases/2_0/release-notes.md
l 41
Empty Response or No Access-Control-Allow-Origin
https://wiki.qubes-os.org/query?status=accepted&status=assigned&status=new&status=reopened&type=defect&milestone=Release+2.1+(post+R2)&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority
http://localhost:5656/developer/releases/2_0/release-notes.html
Question: Could you fix that? :)
3. My suggestion is to keep the documentation index and some Markdown
files in the changed qubes os website repo,
the converted RST files in a new one with two submodules: attachment and
_translated
A new sample Qubes RST documentation repository (without any
translations yet) can be found here:
git clone -b qubes-rst-doc https://github.com/maiska/qubes-rst-doc.git
A new sample Qubes Website incorporating RST documenation can be found here:
git clone -b new-master https://github.com/maiska/qubesos.github.io.rtd.git
The following files will be retained at this stage, due to liquid
templating (needs some tweeking if
one wishes to work with sphinx templating) and also these are perhaps
also just part of the official website not the documenation as I see it
at least atm
- https://www.qubes-os.org/hcl/
- https://www.qubes-os.org/downloads/
- https://www.qubes-os.org/security/xsa/
- https://www.qubes-os.org/security/qsb/
- https://www.qubes-os.org/security/canary/
- https://www.qubes-os.org/security/pgp-keys/
Question: Are you OK with this? Or should I leave the question for the talk?
4. Hidden files not yet deleted or referenced in the index are to be
found here (not in the german version of the index rn):
https://current-qubes-docrtd.readthedocs.io/en/latest/#hidden-gems
Question: Should the index be populated with them?
Regarding the old translation markdown workflow - it is still there, and
can be brushed off the dust.
P.S. I just realized that my .vimrc config was broken. Please excuse the
tabs...!
thank you for resurrecting the thread. I wanted to write like a week ago,
but yet, it lasted longer.
I am glad to announce that the initial hurdle for Qubes OS docs
RST-conversion,
localization, hosting on RtD is behind us and you can see a preliminary
result on
https://current-qubes-docrtd.readthedocs.io/en/latest
This is the current status of the branch rtd-test-default-html-theme in
the repo https://github.com/maiska/qubesos.github.io.rtd + submodules
To see the partially translated docs in german you can go
down to the left, click on the arrow right to 'v:latest' and choose the
language.
There are further tweeks to be done, such as epub and pdf, but the big
step is taken (including automated conversion via script and
postprocessing cross-referencing in sphinx, but still needing minor
manually fixes after that, documentation follows) and translation
workflow tested.
I will not bother you with details at that stage but with questions.
The outline of the questions is as follows:
1. Easily fixable dead end urls as found in the official docs
2. Non easily fixable links
3. Website content vs RtD documentation content
4. Hidden documentation files
1. These are the commits that show what can be fixed imho.
https://github.com/maiska/qubes-doc.rtd/commit/0b8529a026da0523355eec55c57c1da2aa110d07
https://github.com/maiska/qubes-doc.rtd/commit/2b05ac2d8418adbc1d4215d3d54195b3ca8e7f58
https://github.com/maiska/qubes-doc.rtd/commit/6772392f3e2e682d953d7e3b8d9f870867de202a
https://github.com/maiska/qubes-doc.rtd/commit/538bac0e0c368cd54e5acb0b5d1ffb77e06e5045
https://github.com/maiska/qubes-doc.rtd/commit/eb4d1dade1697d4b9e725db8871b1c54ec8c61d6
https://github.com/maiska/qubes-doc.rtd/commit/a8884c8e11f91458bb8ea30d6bbea49e25db48b2
Question: Should I just create Pull Requests for the proposed fixes
2. At least one broken for me unfixable link
in
https://github.com/QubesOS/qubes-doc/edit/master/developer/releases/2_0/release-notes.md
l 41
Empty Response or No Access-Control-Allow-Origin
https://wiki.qubes-os.org/query?status=accepted&status=assigned&status=new&status=reopened&type=defect&milestone=Release+2.1+(post+R2)&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority
http://localhost:5656/developer/releases/2_0/release-notes.html
Question: Could you fix that? :)
3. My suggestion is to keep the documentation index and some Markdown
files in the changed qubes os website repo,
the converted RST files in a new one with two submodules: attachment and
_translated
A new sample Qubes RST documentation repository (without any
translations yet) can be found here:
git clone -b qubes-rst-doc https://github.com/maiska/qubes-rst-doc.git
A new sample Qubes Website incorporating RST documenation can be found here:
git clone -b new-master https://github.com/maiska/qubesos.github.io.rtd.git
The following files will be retained at this stage, due to liquid
templating (needs some tweeking if
one wishes to work with sphinx templating) and also these are perhaps
also just part of the official website not the documenation as I see it
at least atm
- https://www.qubes-os.org/hcl/
- https://www.qubes-os.org/downloads/
- https://www.qubes-os.org/security/xsa/
- https://www.qubes-os.org/security/qsb/
- https://www.qubes-os.org/security/canary/
- https://www.qubes-os.org/security/pgp-keys/
Question: Are you OK with this? Or should I leave the question for the talk?
4. Hidden files not yet deleted or referenced in the index are to be
found here (not in the german version of the index rn):
https://current-qubes-docrtd.readthedocs.io/en/latest/#hidden-gems
Question: Should the index be populated with them?
Regarding the old translation markdown workflow - it is still there, and
can be brushed off the dust.
P.S. I just realized that my .vimrc config was broken. Please excuse the
tabs...!
Marek Marczykowski-Górecki
Sep 5, 2022, 7:55:47 AM9/5/22
to mm, qubes-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hash: SHA256
On Sun, Sep 04, 2022 at 10:05:42PM +0200, mm wrote:
> Hello Marek,
>
>
> thank you for resurrecting the thread. I wanted to write like a week ago,
>
> but yet, it lasted longer.
>
> I am glad to announce that the initial hurdle for Qubes OS docs
> RST-conversion,
> localization, hosting on RtD is behind us and you can see a preliminary
> result on
>
> https://current-qubes-docrtd.readthedocs.io/en/latest
>
> This is the current status of the branch rtd-test-default-html-theme in the
> repo https://github.com/maiska/qubesos.github.io.rtd + submodules
This looks very promising!
> Hello Marek,
>
>
> thank you for resurrecting the thread. I wanted to write like a week ago,
>
> but yet, it lasted longer.
>
> I am glad to announce that the initial hurdle for Qubes OS docs
> RST-conversion,
> localization, hosting on RtD is behind us and you can see a preliminary
> result on
>
> https://current-qubes-docrtd.readthedocs.io/en/latest
>
> This is the current status of the branch rtd-test-default-html-theme in the
> repo https://github.com/maiska/qubesos.github.io.rtd + submodules
> To see the partially translated docs in german you can go
> down to the left, click on the arrow right to 'v:latest' and choose the
> language.
>
> There are further tweeks to be done, such as epub and pdf, but the big step
> is taken (including automated conversion via script and postprocessing
> cross-referencing in sphinx, but still needing minor manually fixes after
> that, documentation follows) and translation workflow tested.
>
> I will not bother you with details at that stage but with questions.
>
> The outline of the questions is as follows:
>
> 1. Easily fixable dead end urls as found in the official docs
> 2. Non easily fixable links
> 3. Website content vs RtD documentation content
> 4. Hidden documentation files
>
> 1. These are the commits that show what can be fixed imho.
>
> https://github.com/maiska/qubes-doc.rtd/commit/0b8529a026da0523355eec55c57c1da2aa110d07
> https://github.com/maiska/qubes-doc.rtd/commit/2b05ac2d8418adbc1d4215d3d54195b3ca8e7f58
> https://github.com/maiska/qubes-doc.rtd/commit/6772392f3e2e682d953d7e3b8d9f870867de202a
> https://github.com/maiska/qubes-doc.rtd/commit/538bac0e0c368cd54e5acb0b5d1ffb77e06e5045
> https://github.com/maiska/qubes-doc.rtd/commit/eb4d1dade1697d4b9e725db8871b1c54ec8c61d6
> https://github.com/maiska/qubes-doc.rtd/commit/a8884c8e11f91458bb8ea30d6bbea49e25db48b2
>
> Question: Should I just create Pull Requests for the proposed fixes
it did but we ignored it?
> 2. At least one broken for me unfixable link
>
> in https://github.com/QubesOS/qubes-doc/edit/master/developer/releases/2_0/release-notes.md
> l 41
>
> Empty Response or No Access-Control-Allow-Origin https://wiki.qubes-os.org/query?status=accepted&status=assigned&status=new&status=reopened&type=defect&milestone=Release+2.1+(post+R2)&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority
> http://localhost:5656/developer/releases/2_0/release-notes.html
>
> Question: Could you fix that? :)
(otherwise it's probably possible to write similar github issues filter,
but I wouldn't bother)
> 3. My suggestion is to keep the documentation index and some Markdown files
> in the changed qubes os website repo,
> the converted RST files in a new one with two submodules: attachment and
> _translated
> A new sample Qubes RST documentation repository (without any translations
> yet) can be found here:
>
> git clone -b qubes-rst-doc https://github.com/maiska/qubes-rst-doc.git
>
> A new sample Qubes Website incorporating RST documenation can be found here:
>
> git clone -b new-master https://github.com/maiska/qubesos.github.io.rtd.git
to the new doc. Makes sense to keep links working.
> The following files will be retained at this stage, due to liquid templating
> (needs some tweeking if
> one wishes to work with sphinx templating) and also these are perhaps also
> just part of the official website not the documenation as I see it at least
> atm
>
> - https://www.qubes-os.org/hcl/
> - https://www.qubes-os.org/downloads/
> - https://www.qubes-os.org/security/xsa/
> - https://www.qubes-os.org/security/qsb/
> - https://www.qubes-os.org/security/canary/
> - https://www.qubes-os.org/security/pgp-keys/
data for those pages is already in the main repo (_data/), just not the
.md file...
> Question: Are you OK with this? Or should I leave the question for the talk?
>
> 4. Hidden files not yet deleted or referenced in the index are to be found
> here (not in the german version of the index rn):
>
> https://current-qubes-docrtd.readthedocs.io/en/latest/#hidden-gems
>
> Question: Should the index be populated with them?
main index? Or not linked at all?
Some of those are normally linked from the website footer,
https://www.qubes-os.org/intro/ or similar.
But generally, IMO better have every doc page linked to the main index.
> Regarding the old translation markdown workflow - it is still there, and can
> be brushed off the dust.
>
> P.S. I just realized that my .vimrc config was broken. Please excuse the
> tabs...!
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmMV47oACgkQ24/THMrX
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----
1yw1sgf/fW9F3s/Bp1/NmqH4lBH/9jy/p49/lYs8UTU9qWWw8sJY9w6v8rVZ/N3F
okWW30Pjw8j1DPx6zY9JWdvzYu9jZAABgHOLNrYY9u4O2WD9V0CFUbZNxezqDyGR
rNe8XGdWrWoPDylRW7EYw4BTC6m7RPHK7h3GuyoP0YyTyxDim0+9I4vzgq304uBB
NiUJLCdzRaW2x1S5ulJbgSd6iDYdiz2mGuvkaS/hbqv+OnaMkHLaOJmKaLShWY6g
Z5dU2tDprpx0ohkXIfd9hTvijrDrb981qDP3pjFY2bXoFFrTaHgUn6myXFCKl5tW
F728mHFxzq2LxH1XzCiFeneJ+Qg47Q==
=eLlL
-----END PGP SIGNATURE-----
Andrew David Wong
Sep 7, 2022, 6:34:58 PM9/7/22
to Marek Marczykowski-Górecki, mm, qubes-devel
> I wonder why URL checker hasn't found them... Or maybe it did but we ignored it?
It looks like most of them are broken *external* links, whereas our URL checker only checks for broken *internal* links (which is partially why adding lots of external links increases the maintenance burden on the documentation over time).
Also, some of these are not really broken. For example, `/doc/disk-trim` is a working redirect. Try it: https://www.qubes-os.org/doc/disk-trim
Of course, replacing the redirect with the redirect's target is also fine.
>> 2. At least one broken for me unfixable link
>
>> in https://github.com/QubesOS/qubes-doc/edit/master/developer/releases/2_0/release-notes.md
>> l 41
>
>> Empty Response or No Access-Control-Allow-Origin https://wiki.qubes-os.org/query?status=accepted&status=assigned&status=new&status=reopened&type=defect&milestone=Release+2.1+(post+R2)&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority
>> http://localhost:5656/developer/releases/2_0/release-notes.html
>
>> Question: Could you fix that? :)
>
> Since that's about Qubes R2, I think the link can be simply dropped.
> (otherwise it's probably possible to write similar github issues filter,
> but I wouldn't bother)
>
>> 3. My suggestion is to keep the documentation index and some Markdown files
>> in the changed qubes os website repo,
>> the converted RST files in a new one with two submodules: attachment and
>> _translated
>
> Yes, this sounds like a good plan.
>
>> A new sample Qubes RST documentation repository (without any translations
>> yet) can be found here:
>
>> git clone -b qubes-rst-doc https://github.com/maiska/qubes-rst-doc.git
>
>> A new sample Qubes Website incorporating RST documenation can be found here:
>
>> git clone -b new-master https://github.com/maiska/qubesos.github.io.rtd.git
>
> So, it references _doc submodule that basically is just about redirects
> to the new doc. Makes sense to keep links working.
>
>> The following files will be retained at this stage, due to liquid templating
>> (needs some tweeking if
>> one wishes to work with sphinx templating) and also these are perhaps also
>> just part of the official website not the documenation as I see it at least
>> atm
>
>> - https://www.qubes-os.org/hcl/
>> - https://www.qubes-os.org/downloads/
>> - https://www.qubes-os.org/security/xsa/
>> - https://www.qubes-os.org/security/qsb/
>> - https://www.qubes-os.org/security/canary/
>> - https://www.qubes-os.org/security/pgp-keys/
>
> Sounds fine. IMO those should be moved to the main repo. In fact, actual
> data for those pages is already in the main repo (_data/), just not the
> .md file...
>
Also, it might be a bit weird to have a page (/security/) in qubes-doc, while its "subpages" are in the more general parent repo "above" it. So, organizing based on semantic content vs. source file/data attributes will get you different results.
I'm sure there are many other considerations buried in our archives that I'm forgetting now.
>> Question: Are you OK with this? Or should I leave the question for the talk?
>
>> 4. Hidden files not yet deleted or referenced in the index are to be found
>> here (not in the german version of the index rn):
>
>> https://current-qubes-docrtd.readthedocs.io/en/latest/#hidden-gems
>
>> Question: Should the index be populated with them?
>
> You mean pages that are only linked from some sub-section but not the
> main index? Or not linked at all?
>
> Some of those are normally linked from the website footer,
> https://www.qubes-os.org/intro/ or similar.
> But generally, IMO better have every doc page linked to the main index.
>
Qubes builder -- probably a subpage linked from a main page
Join -- intentionally hidden when active recruiting stopped (to be unhidden if/when it resumes)
Admin API Table -- it's an "embed" on the Admin API page (just a hack to allow opening that huge table by itself in the "fullscreen" layout)
Disposable VM Implementation -- not sure
Qrexec2 -- not sure
Qfileexchgd -- not sure
Code of Conduct -- linked from footer, as Marek said; would be okay to link from index
Intro -- linked from "nav bar" at the very top of the website
Privacy -- intentional footer link, like every other website
Screenshots -- linked from Intro; would be okay to link from index
Statistics -- linked from News page; would be okay to link from index
Video Tours -- linked from Intro; would be okay to link from index
QSB Checklist -- checklist for internal use; probably not of interest to users; would just be clutter in the index
Canary Checklist -- same
Download mirrors -- table embedded on Downloads page
Custom install -- linked from Installation Guide; would be okay to link from index
Install security -- linked from Installation Guide; would be okay to link from index
How to use the hcl -- linked from HCL page; would be okay to link from index
How to reinstall a template -- subpage linked from Templates page
Debian template -- this page is already linked from the index; not sure why it was miscategorized as a hidden gem
Debian upgrade -- in-plage upgrade guide subpage linked from Debian page
Fedora upgrade -- in-plage upgrade guide subpage linked from Fedora page
Fedora -- this page is already linked from the index; not sure why it was miscategorized as a hidden gem
Update Debian Whonix -- looks like this got missed in the contributor overhaul of troubleshooting documentation that occurred a while back; it might just be outdated now
IMO:
- If a page exists purely as a technical workaround (hack), then it makes little sense to list it in the index. Pages that exist purely to serve as embeds in other pages probably fall under this. The content is already presented somewhere in a rational way, and this just creates a confusing duplicate pointer to part of it without its intended context.
- Internal docs like maintainer checklists would probably just be confusing/useless for readers, but I suppose it's okay to list them if you want. (If we don't really use these, then we can just remove them.)
- In a few places, a subpage only exists because people thought that including its content "in-line" in its parent page would make the resulting page "too long." I guess it's okay to list these, but it might help to at least update the titles to make the parent/child relationship clear.
mm
Sep 7, 2022, 9:30:53 PM9/7/22
to Andrew David Wong, Marek Marczykowski-Górecki, qubes-devel
Hello Marek, hello Andrew,
thank you for your input!! Sorry for answering this late!
As a disclaimer: the talk in an ideal world will be packed and last 20
min, so that there would be 10 min for discussion.
OK
>> I wonder why URL checker hasn't found them... Or maybe it did but we ignored it?
> No, we aren't ignoring any. I always immediately fix all the ones I see notifications about.
>
> It looks like most of them are broken *external* links, whereas our URL checker only checks for broken *internal* links (which is partially why adding lots of external links increases the maintenance burden on the documentation over time).
>
> Also, some of these are not really broken. For example, `/doc/disk-trim` is a working redirect. Try it: https://www.qubes-os.org/doc/disk-trim
>
> Of course, replacing the redirect with the redirect's target is also fine.
I think all of my fixes were external ones. Only one is internal, cause
it interferes with my scripting and post-fixing cross-reference in sphinx.
>
>>> 2. At least one broken for me unfixable link
>>> in https://github.com/QubesOS/qubes-doc/edit/master/developer/releases/2_0/release-notes.md
>>> l 41
>>> Empty Response or No Access-Control-Allow-Origin https://wiki.qubes-os.org/query?status=accepted&status=assigned&status=new&status=reopened&type=defect&milestone=Release+2.1+(post+R2)&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority
>>> http://localhost:5656/developer/releases/2_0/release-notes.html
>>> Question: Could you fix that? :)
>> Since that's about Qubes R2, I think the link can be simply dropped.
>> (otherwise it's probably possible to write similar github issues filter,
>> but I wouldn't bother)
OK, I will just submit a PQ without this link.
>>> 3. My suggestion is to keep the documentation index and some Markdown files
>>> in the changed qubes os website repo,
>>> the converted RST files in a new one with two submodules: attachment and
>>> _translated
>> Yes, this sounds like a good plan.
>>
> Sorry, I don't understand. Can anyone explain?
>
>>> A new sample Qubes RST documentation repository (without any translations
>>> yet) can be found here:
>>> git clone -b qubes-rst-doc https://github.com/maiska/qubes-rst-doc.git
> This appears to be empty for me.
Look here: https://github.com/maiska/qubes-rst-doc/tree/qubes-rst-doc
Sorry, I did not posted the url but just the git clone
>
>>> A new sample Qubes Website incorporating RST documenation can be found here:
>>> git clone -b new-master https://github.com/maiska/qubesos.github.io.rtd.git
Here again my mistake. The URL:
https://github.com/maiska/qubesos.github.io.rtd/tree/new-master
and the clone should go recursive if you want to try it out locally
git clone -b new-master --recursive https://github.com/maiska/qubesos.github.io.rtd.git
> This appears to just be a clone of the existing website, with the _doc submodule containing Markdown files.
>
>> So, it references _doc submodule that basically is just about redirects
>> to the new doc. Makes sense to keep links working.
>>
Yes, it contains actually everything from the _doc directory in place,
only without contents and with a redirect to the testing RTD version
from above.
Only some files are retained such as downloads, hcl etc (from below)
Besides the links existing in pages or posts, that link to /doc/foo
content that must be fixed en large if the _doc directory is unpopulated,
other people's articles linking to the official Qubes OS documentation
would be broken, since I do not think that there is a proxy
to handle such requests and redirects? Or there is a simpler way?
> Wouldn't converting all Markdown files to RST result in still using the _doc submodule, just with all the files converted? Maybe I'm not understanding something here.
This could be a possibility yes, perhaps with a sphinx plugin for
redirects. It exists, haven't tried it yet.
Meaning convert qubes-doc, do not create a new doc-rst repo, move
Markdown files like hcl, download etc to the pages submodule due to the
constraints I mentioned below,
as a temp stage or not.
>
>>> The following files will be retained at this stage, due to liquid templating
>>> (needs some tweeking if
>>> one wishes to work with sphinx templating) and also these are perhaps also
>>> just part of the official website not the documenation as I see it at least
>>> atm
>>> - https://www.qubes-os.org/hcl/
>>> - https://www.qubes-os.org/downloads/
>>> - https://www.qubes-os.org/security/xsa/
>>> - https://www.qubes-os.org/security/qsb/
>>> - https://www.qubes-os.org/security/canary/
>>> - https://www.qubes-os.org/security/pgp-keys/
>> Sounds fine. IMO those should be moved to the main repo. In fact, actual
>> data for those pages is already in the main repo (_data/), just not the
>> .md file...
>>
> I think this goes back to the previous discussions we've had over the years about how to organize website vs. doc content, and we landed on organizing it this way for various reasons. I think one of those reasons was to have everything that should be included in offline documentation in qubes-doc. (So, even if the data is in `_data/`, the offline doc generation tool might decide whether to include it based on whether the page referencing that data is in qubes-doc or not.)
>
> Also, it might be a bit weird to have a page (/security/) in qubes-doc, while its "subpages" are in the more general parent repo "above" it. So, organizing based on semantic content vs. source file/data attributes will get you different results.
>
> I'm sure there are many other considerations buried in our archives that I'm forgetting now.
From my part what I see is more work. It dragged itself through the
years enough :) (Of course that is also on me)
As soon as we get the localization workflow going and RTD is the way to
go as discussed previously and is stable,
one can still play with sphinx templating and incorporate the above
files in the RST documentation. I already tried it as a simple example
it functions, but is frustrating, and rn I do not have the nerve to sit
through this.
Perhaps some volunteers will come forward at the summit :)
Why I see them as a part of the website is the following: downloads,
xsa, qsb, canary and pgp-keys are somewhat essential to security of the
project, not truly documentation as I see it atm
HCL is documentation yes, there I just make my life easier
Yep, some of the gems are referenced from the footer for example.
Why I brought them to light in the index for discussion is the weird
feeling getting to qubes os documentation on RTD and not having all the
files at one's click.
As part of the official Qubes OS website it made sense and I did not
really notice.
RTD is different story.
I think if RTD should host them, the reader should be able to find them
from the index somehow, or a subindex whatever that may mean
And it is just a dump, it should be reorganized of course :)
thank you for your input!! Sorry for answering this late!
As a disclaimer: the talk in an ideal world will be packed and last 20
min, so that there would be 10 min for discussion.
>> I wonder why URL checker hasn't found them... Or maybe it did but we ignored it?
> No, we aren't ignoring any. I always immediately fix all the ones I see notifications about.
>
> It looks like most of them are broken *external* links, whereas our URL checker only checks for broken *internal* links (which is partially why adding lots of external links increases the maintenance burden on the documentation over time).
>
> Also, some of these are not really broken. For example, `/doc/disk-trim` is a working redirect. Try it: https://www.qubes-os.org/doc/disk-trim
>
> Of course, replacing the redirect with the redirect's target is also fine.
it interferes with my scripting and post-fixing cross-reference in sphinx.
>
>>> 2. At least one broken for me unfixable link
>>> in https://github.com/QubesOS/qubes-doc/edit/master/developer/releases/2_0/release-notes.md
>>> l 41
>>> Empty Response or No Access-Control-Allow-Origin https://wiki.qubes-os.org/query?status=accepted&status=assigned&status=new&status=reopened&type=defect&milestone=Release+2.1+(post+R2)&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority
>>> http://localhost:5656/developer/releases/2_0/release-notes.html
>>> Question: Could you fix that? :)
>> Since that's about Qubes R2, I think the link can be simply dropped.
>> (otherwise it's probably possible to write similar github issues filter,
>> but I wouldn't bother)
>>> 3. My suggestion is to keep the documentation index and some Markdown files
>>> in the changed qubes os website repo,
>>> the converted RST files in a new one with two submodules: attachment and
>>> _translated
>> Yes, this sounds like a good plan.
>>
> Sorry, I don't understand. Can anyone explain?
>
>>> A new sample Qubes RST documentation repository (without any translations
>>> yet) can be found here:
>>> git clone -b qubes-rst-doc https://github.com/maiska/qubes-rst-doc.git
> This appears to be empty for me.
Sorry, I did not posted the url but just the git clone
>
>>> A new sample Qubes Website incorporating RST documenation can be found here:
>>> git clone -b new-master https://github.com/maiska/qubesos.github.io.rtd.git
https://github.com/maiska/qubesos.github.io.rtd/tree/new-master
and the clone should go recursive if you want to try it out locally
git clone -b new-master --recursive https://github.com/maiska/qubesos.github.io.rtd.git
> This appears to just be a clone of the existing website, with the _doc submodule containing Markdown files.
>
>> So, it references _doc submodule that basically is just about redirects
>> to the new doc. Makes sense to keep links working.
>>
only without contents and with a redirect to the testing RTD version
from above.
Only some files are retained such as downloads, hcl etc (from below)
Besides the links existing in pages or posts, that link to /doc/foo
content that must be fixed en large if the _doc directory is unpopulated,
other people's articles linking to the official Qubes OS documentation
would be broken, since I do not think that there is a proxy
to handle such requests and redirects? Or there is a simpler way?
> Wouldn't converting all Markdown files to RST result in still using the _doc submodule, just with all the files converted? Maybe I'm not understanding something here.
redirects. It exists, haven't tried it yet.
Meaning convert qubes-doc, do not create a new doc-rst repo, move
Markdown files like hcl, download etc to the pages submodule due to the
constraints I mentioned below,
as a temp stage or not.
>
>>> The following files will be retained at this stage, due to liquid templating
>>> (needs some tweeking if
>>> one wishes to work with sphinx templating) and also these are perhaps also
>>> just part of the official website not the documenation as I see it at least
>>> atm
>>> - https://www.qubes-os.org/hcl/
>>> - https://www.qubes-os.org/downloads/
>>> - https://www.qubes-os.org/security/xsa/
>>> - https://www.qubes-os.org/security/qsb/
>>> - https://www.qubes-os.org/security/canary/
>>> - https://www.qubes-os.org/security/pgp-keys/
>> Sounds fine. IMO those should be moved to the main repo. In fact, actual
>> data for those pages is already in the main repo (_data/), just not the
>> .md file...
>>
> I think this goes back to the previous discussions we've had over the years about how to organize website vs. doc content, and we landed on organizing it this way for various reasons. I think one of those reasons was to have everything that should be included in offline documentation in qubes-doc. (So, even if the data is in `_data/`, the offline doc generation tool might decide whether to include it based on whether the page referencing that data is in qubes-doc or not.)
>
> Also, it might be a bit weird to have a page (/security/) in qubes-doc, while its "subpages" are in the more general parent repo "above" it. So, organizing based on semantic content vs. source file/data attributes will get you different results.
>
> I'm sure there are many other considerations buried in our archives that I'm forgetting now.
years enough :) (Of course that is also on me)
As soon as we get the localization workflow going and RTD is the way to
go as discussed previously and is stable,
one can still play with sphinx templating and incorporate the above
files in the RST documentation. I already tried it as a simple example
it functions, but is frustrating, and rn I do not have the nerve to sit
through this.
Perhaps some volunteers will come forward at the summit :)
Why I see them as a part of the website is the following: downloads,
xsa, qsb, canary and pgp-keys are somewhat essential to security of the
project, not truly documentation as I see it atm
HCL is documentation yes, there I just make my life easier
Why I brought them to light in the index for discussion is the weird
feeling getting to qubes os documentation on RTD and not having all the
files at one's click.
As part of the official Qubes OS website it made sense and I did not
really notice.
RTD is different story.
I think if RTD should host them, the reader should be able to find them
from the index somehow, or a subindex whatever that may mean
And it is just a dump, it should be reorganized of course :)
mm
Sep 7, 2022, 9:45:58 PM9/7/22
to Andrew David Wong, Marek Marczykowski-Górecki, qubes-devel
Hello again,
here in a clear manner some suggestions.
What do you think?
>>
>> Qubes builder -- probably a subpage linked from a main page
is actually Qubes builder details, could be hosted in the Building section?
>> Join -- intentionally hidden when active recruiting stopped (to be
>> unhidden if/when it resumes)
Should it than be just part of the Qubes OS Website and not the docs?
>> Admin API Table -- it's an "embed" on the Admin API page (just a hack
>> to allow opening that huge table by itself in the "fullscreen" layout)
Could go below Admin API in the index?
>> Disposable VM Implementation -- not sure
Somewhere in developer's sections?
>> Qrexec2 -- not sure
Somewhere in developer's sections?
>> Qfileexchgd -- not sure
Somewhere in developer's sections?
>> Canary Checklist -- same
Go to the official Qubes OS Website?
here in a clear manner some suggestions.
What do you think?
>>
>> Qubes builder -- probably a subpage linked from a main page
>> Join -- intentionally hidden when active recruiting stopped (to be
>> unhidden if/when it resumes)
>> Admin API Table -- it's an "embed" on the Admin API page (just a hack
>> to allow opening that huge table by itself in the "fullscreen" layout)
>> Disposable VM Implementation -- not sure
>> Qrexec2 -- not sure
Somewhere in developer's sections?
>> Qfileexchgd -- not sure
Somewhere in developer's sections?
>> Code of Conduct -- linked from footer, as Marek said; would be okay
>> to link from index
Perhaps in Introduction section ?
>> to link from index
>> Intro -- linked from "nav bar" at the very top of the website
Perhaps in Introduction section ?
>> Privacy -- intentional footer link, like every other website
Perhaps in Introduction section ? for now after that somehow in the footer
>> Screenshots -- linked from Intro; would be okay to link from index
Perhaps in Introduction section ?
>> Statistics -- linked from News page; would be okay to link from index
Perhaps in Introduction section ?
>> Video Tours -- linked from Intro; would be okay to link from index
Perhaps in Introduction section ?
>> QSB Checklist -- checklist for internal use; probably not of interest
>> to users; would just be clutter in the index
Go to the official Qubes OS Website?
>> to users; would just be clutter in the index
>> Canary Checklist -- same
Go to the official Qubes OS Website?
>> Download mirrors -- table embedded on Downloads page
hm ok obsolete
>> Custom install -- linked from Installation Guide; would be okay to
>> link from index
in Downloading, Installing, and Upgrading Qubes section?
>> link from index
>> Install security -- linked from Installation Guide; would be okay to
>> link from index
in Downloading, Installing, and Upgrading Qubes section?
>> link from index
>> How to use the hcl -- linked from HCL page; would be okay to link
>> from index
Choosing your hardware section?
>> from index
>> How to reinstall a template -- subpage linked from Templates page
Templates section?
>> Debian template -- this page is already linked from the index; not
>> sure why it was miscategorized as a hidden gem
ja, idk, mistake
>> sure why it was miscategorized as a hidden gem
>> Debian upgrade -- in-plage upgrade guide subpage linked from Debian page
Templates section?
>> Fedora upgrade -- in-plage upgrade guide subpage linked from Fedora page
Templates section?
>> Fedora -- this page is already linked from the index; not sure why it
>> was miscategorized as a hidden gem
ja, idk, mistake
>> was miscategorized as a hidden gem
>> Update Debian Whonix -- looks like this got missed in the contributor
>> overhaul of troubleshooting documentation that occurred a while back;
>> it might just be outdated now
Templates section?
>> overhaul of troubleshooting documentation that occurred a while back;
>> it might just be outdated now
Andrew David Wong
Sep 8, 2022, 9:00:22 PM9/8/22
to mm, Marek Marczykowski-Górecki, qubes-devel
On 9/7/22 6:30 PM, mm wrote:
> [...]
>>>> https://github.com/maiska/qubes-doc.rtd/commit/0b8529a026da0523355eec55c57c1da2aa110d07
By the way, with this one, the link is not actually broken. It still works, because your browser adds the missing forward slash automatically. Nonetheless, I've committed a fix to make it technically syntactically correct.
> [...]
https://www.qubes-os.org/doc/how-to-update/
After the migration to RtD, wouldn't this still be the same URL for that page? Why would it be any different?
If you're saying it has to be a different URL for some reason, what would the new URL look like?
>> Wouldn't converting all Markdown files to RST result in still using the _doc submodule, just with all the files converted? Maybe I'm not understanding something here.
>
> This could be a possibility yes, perhaps with a sphinx plugin for redirects. It exists, haven't tried it yet.
>
Why only "a possibility"? I figured that would be the obvious thing to do, but clearly I am missing something. If there is a good reason to do otherwise, then that's totally fine! It's just not clear to me what that reason might be.
> [...]
> Why I see them as a part of the website is the following: downloads, xsa, qsb, canary and pgp-keys are somewhat essential to security of the project, not truly documentation as I see it atm
>
Well, I don't know if that holds for all of these, because at least some parts of those pages are clearly documentation explaining what XSAs are, what QSBs are, what canaries are, how we handle them, and so on.
But the other technical reason you gave enough is a separate and probably good enough on its own.
Ok, I've just moved these pages to the main repo and deleted the ones we don't need.
> HCL is documentation yes, there I just make my life easier
>
Yes, making life easier is probably more important at this stage. :)
(Btw, I'm assuming you meant *not* documentation, since above you listed the HCL page as one that you said *should* be moved to the main repo, which makes sense because it is indeed "empty" with all the data stored elsewhere. So, I've just moved it along with the others.)
Well, I think I explained above why that should not be a weird feeling. That's a bit like saying, "I bought a house, but it's a bit weird that I can't easily see and touch the insides of my own walls." o_O
> As part of the official Qubes OS website it made sense and I did not really notice.
>
> RTD is different story.
>
> I think if RTD should host them, the reader should be able to find them from the index somehow, or a subindex whatever that may mean
>
Well, ok, if that makes things easier. I've now made sure all these pages are either incorporated into their parent pages (i.e., eliminated) or listed from the index.
> And it is just a dump, it should be reorganized of course :)
>
Indeed. :)
For reference, here are the two PRs where I do everything I said I did in this email:
https://github.com/QubesOS/qubes-doc/pull/1267
https://github.com/QubesOS/qubesos.github.io/pull/204
> [...]
>>>> https://github.com/maiska/qubes-doc.rtd/commit/0b8529a026da0523355eec55c57c1da2aa110d07
By the way, with this one, the link is not actually broken. It still works, because your browser adds the missing forward slash automatically. Nonetheless, I've committed a fix to make it technically syntactically correct.
> [...]
> other people's articles linking to the official Qubes OS documentation would be broken, since I do not think that there is a proxy
>
> to handle such requests and redirects? Or there is a simpler way?
>
I don't understand why links would get broken. Let's use a concrete example to help me understand. Take this page, for example:
>
> to handle such requests and redirects? Or there is a simpler way?
>
https://www.qubes-os.org/doc/how-to-update/
After the migration to RtD, wouldn't this still be the same URL for that page? Why would it be any different?
If you're saying it has to be a different URL for some reason, what would the new URL look like?
>> Wouldn't converting all Markdown files to RST result in still using the _doc submodule, just with all the files converted? Maybe I'm not understanding something here.
>
> This could be a possibility yes, perhaps with a sphinx plugin for redirects. It exists, haven't tried it yet.
>
> [...]
>>>> The following files will be retained at this stage, due to liquid templating
>>>> (needs some tweeking if
>>>> one wishes to work with sphinx templating) and also these are perhaps also
>>>> just part of the official website not the documenation as I see it at least
>>>> atm
>>>> - https://www.qubes-os.org/hcl/
>>>> - https://www.qubes-os.org/downloads/
>>>> - https://www.qubes-os.org/security/xsa/
>>>> - https://www.qubes-os.org/security/qsb/
>>>> - https://www.qubes-os.org/security/canary/
>>>> - https://www.qubes-os.org/security/pgp-keys/
>>> Sounds fine. IMO those should be moved to the main repo. In fact, actual
>>> data for those pages is already in the main repo (_data/), just not the
>>> .md file...
>>>
>> I think this goes back to the previous discussions we've had over the years about how to organize website vs. doc content, and we landed on organizing it this way for various reasons. I think one of those reasons was to have everything that should be included in offline documentation in qubes-doc. (So, even if the data is in `_data/`, the offline doc generation tool might decide whether to include it based on whether the page referencing that data is in qubes-doc or not.)
>>
>> Also, it might be a bit weird to have a page (/security/) in qubes-doc, while its "subpages" are in the more general parent repo "above" it. So, organizing based on semantic content vs. source file/data attributes will get you different results.
>>
>> I'm sure there are many other considerations buried in our archives that I'm forgetting now.
>
> From my part what I see is more work. It dragged itself through the years enough :) (Of course that is also on me)
>
> As soon as we get the localization workflow going and RTD is the way to go as discussed previously and is stable,
>
> one can still play with sphinx templating and incorporate the above files in the RST documentation. I already tried it as a simple example it functions, but is frustrating, and rn I do not have the nerve to sit through this.
>
> Perhaps some volunteers will come forward at the summit :)
>
Ok, if it's too much trouble to get the tech working correctly, then that's probably a good enough reason to just move these files. I don't mind.
>>>> (needs some tweeking if
>>>> one wishes to work with sphinx templating) and also these are perhaps also
>>>> just part of the official website not the documenation as I see it at least
>>>> atm
>>>> - https://www.qubes-os.org/hcl/
>>>> - https://www.qubes-os.org/downloads/
>>>> - https://www.qubes-os.org/security/xsa/
>>>> - https://www.qubes-os.org/security/qsb/
>>>> - https://www.qubes-os.org/security/canary/
>>>> - https://www.qubes-os.org/security/pgp-keys/
>>> Sounds fine. IMO those should be moved to the main repo. In fact, actual
>>> data for those pages is already in the main repo (_data/), just not the
>>> .md file...
>>>
>> I think this goes back to the previous discussions we've had over the years about how to organize website vs. doc content, and we landed on organizing it this way for various reasons. I think one of those reasons was to have everything that should be included in offline documentation in qubes-doc. (So, even if the data is in `_data/`, the offline doc generation tool might decide whether to include it based on whether the page referencing that data is in qubes-doc or not.)
>>
>> Also, it might be a bit weird to have a page (/security/) in qubes-doc, while its "subpages" are in the more general parent repo "above" it. So, organizing based on semantic content vs. source file/data attributes will get you different results.
>>
>> I'm sure there are many other considerations buried in our archives that I'm forgetting now.
>
> From my part what I see is more work. It dragged itself through the years enough :) (Of course that is also on me)
>
> As soon as we get the localization workflow going and RTD is the way to go as discussed previously and is stable,
>
> one can still play with sphinx templating and incorporate the above files in the RST documentation. I already tried it as a simple example it functions, but is frustrating, and rn I do not have the nerve to sit through this.
>
> Perhaps some volunteers will come forward at the summit :)
>
> Why I see them as a part of the website is the following: downloads, xsa, qsb, canary and pgp-keys are somewhat essential to security of the project, not truly documentation as I see it atm
>
But the other technical reason you gave enough is a separate and probably good enough on its own.
Ok, I've just moved these pages to the main repo and deleted the ones we don't need.
> HCL is documentation yes, there I just make my life easier
>
(Btw, I'm assuming you meant *not* documentation, since above you listed the HCL page as one that you said *should* be moved to the main repo, which makes sense because it is indeed "empty" with all the data stored elsewhere. So, I've just moved it along with the others.)
> As part of the official Qubes OS website it made sense and I did not really notice.
>
> RTD is different story.
>
> I think if RTD should host them, the reader should be able to find them from the index somehow, or a subindex whatever that may mean
>
> And it is just a dump, it should be reorganized of course :)
>
For reference, here are the two PRs where I do everything I said I did in this email:
https://github.com/QubesOS/qubes-doc/pull/1267
https://github.com/QubesOS/qubesos.github.io/pull/204
Andrew David Wong
Sep 8, 2022, 9:00:39 PM9/8/22
to mm, Marek Marczykowski-Górecki, qubes-devel
https://github.com/QubesOS/qubes-doc/pull/1267
https://github.com/QubesOS/qubesos.github.io/pull/204
Please let me know if I missed any.
Andrew David Wong
Sep 8, 2022, 9:14:29 PM9/8/22
to mm, Marek Marczykowski-Górecki, qubes-devel
On 9/8/22 6:00 PM, Andrew David Wong wrote:
> On 9/7/22 6:30 PM, mm wrote:
>> [...]
> On 9/7/22 6:30 PM, mm wrote:
>> [...]
Andrew David Wong
Sep 8, 2022, 9:16:07 PM9/8/22
to mm, Marek Marczykowski-Górecki, qubes-devel
Andrew David Wong
Sep 8, 2022, 9:43:41 PM9/8/22
to mm, qubes-devel
On 9/7/22 6:30 PM, mm wrote:
> and the clone should go recursive if you want to try it out locally
>
> git clone -b new-master --recursive https://github.com/maiska/qubesos.github.io.rtd.git
>
So, how do I serve the site locally after doing this? Can you provide step-by-step instructions?
>
> git clone -b new-master --recursive https://github.com/maiska/qubesos.github.io.rtd.git
>
Andrew David Wong
Sep 8, 2022, 9:49:06 PM9/8/22
to mm, Marek Marczykowski-Górecki, qubes-devel
mm
Sep 9, 2022, 3:52:16 PM9/9/22
to Andrew David Wong, Marek Marczykowski-Górecki, qubes-devel
Hey Andrew,
thank you for your work!!
So one by one:
https://github.com/maiska/qubes-doc.rtd/commit/0b8529a026da0523355eec55c57c1da2aa110d07
By the way, with this one, the link is not actually broken. It still works, because your browser adds the missing forward slash automatically. Nonetheless, I've committed a fix to make it technically syntactically correct.
With the above commit there several links. I suppose you mean the internal one: it is so, yes, but for the convenience of my stupid script..., but let me explain:
Some people, when confronted with a problem, think
“I know, I'll use regular expressions.”
Now they have two problems.
Well.... :)
[...] other people's articles linking to the official Qubes OS documentation would be broken, since I do not think that there is a proxy to handle such requests and redirects? Or there is a simpler way?I don't understand why links would get broken. Let's use a concrete example to help me understand. Take this page, for example: https://www.qubes-os.org/doc/how-to-update/ After the migration to RtD, wouldn't this still be the same URL for that page? Why would it be any different? If you're saying it has to be a different URL for some reason, what would the new URL look like?
The new url when generated will look like:
foo.bar/user/how-to-guides/how-to-update.html from rst will look
like :doc:` foo </user/how-to-guides/how-to-update>`
Domain dependent or whatever...
Wouldn't converting all Markdown files to RST result in still using the _doc submodule, just with all the files converted? Maybe I'm not understanding something here.This could be a possibility yes, perhaps with a sphinx plugin for redirects. It exists, haven't tried it yet.Why only "a possibility"? I figured that would be the obvious thing to do, but clearly I am missing something. If there is a good reason to do otherwise, then that's totally fine! It's just not clear to me what that reason might be.
Nope, you are abs right!
My motivation behind this solution (meaning leave the docs as they are but empty etc.) was take it one step at a time w/o necessarily drain time in fixing possible broken links. Till the documentation is on RTD
and afterwards the website can be dealt with and properly cleared
of any redundant information (such as in _includes research.html,
cause i blank out translated it
into flat markdown or rst),
also than use the sphinx redirect extension, these all are nice to have and can be done afterwards.
The possibility was not the right word in this context...
The HCL and Downloads thus can be addressed later :)
About the pull requests I will look into it latest tomorrow, render and say if smth is 'missing'
One thing though which I did not think about:
The privacy policy should be 2x - one for the Qubes OS website, and one for RTD. The one for RTD should be adjusted and perhaps refer to also their privacy policy:
But maybe that's okay, since once we migrate to RtD, we'll no longer be relying on custom scripts like this to populate the table of contents.
Yes!! Hopefully there will be a lot less clutter :))
git clone -b new-master --recursive https://github.com/maiska/qubesos.github.io.rtd.git
So, how do I serve the site locally after doing this? Can you provide step-by-step instructions?
On a fedora VM:
sudo dnf install ruby ruby-devel openssl-devel redhat-rpm-config @development-tools;
gem install jekyll bundler;
find . -name gem; #(in my case it was /home/user/.local/share/gem/)
git clone -b new-master --recursive https://github.com/maiska/qubesos.github.io.rtd.git; cd qubesos.github.io.rtd/
bundle config set --local path '/home/user/.local/share/gem/'
bundle exec jekyll serve
sudo dnf install ruby ruby-devel openssl-devel redhat-rpm-config @development-tools;
gem install jekyll bundler;
find . -name gem; #(in my case it was /home/user/.local/share/gem/)
git clone -b new-master --recursive https://github.com/maiska/qubesos.github.io.rtd.git; cd qubesos.github.io.rtd/
bundle config set --local path '/home/user/.local/share/gem/'
bundle exec jekyll serve
All the best,
m
mm
Sep 9, 2022, 3:58:23 PM9/9/22
to qubes...@googlegroups.com
I forgot somehting
On a fedora VM:
sudo dnf install ruby ruby-devel openssl-devel redhat-rpm-config @development-tools;
gem install jekyll bundler;
find . -name gem; #(in my case it was /home/user/.local/share/gem/)
git clone -b new-master --recursive https://github.com/maiska/qubesos.github.io.rtd.git; cd qubesos.github.io.rtd/
bundle config set --local path '/home/user/.local/share/gem/'
bundle install;
(if there is some sort of error with webrick, just try: bundle add webrick; bundle exec jekyll serve)
bundle exec jekyll serve
All the best,
m
--
You received this message because you are subscribed to the Google Groups "qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/d1d7a758-b879-4ee2-6ab7-e7ac92deaf04%40maiski.net.
Tobias Killer
Sep 17, 2022, 2:43:27 PM9/17/22
to mm, qubes...@googlegroups.com
Hi, m,
thank you for all the work!
As far as I can remember, you talked with Marek about i18n and l10n on
the Qubes OS summit 2022. Could you (or Marek) please give us a summary
of the results of those talks?
Thank you!
Tobias
thank you for all the work!
As far as I can remember, you talked with Marek about i18n and l10n on
the Qubes OS summit 2022. Could you (or Marek) please give us a summary
of the results of those talks?
Thank you!
Tobias
mm
Sep 25, 2022, 5:31:28 PM9/25/22
to Tobias Killer, qubes...@googlegroups.com
Hey Tobias,
I do not know unfortunately what you are referring to?
I mean the objective is to convert the markdown documentation to rst asap
Unfortunately it did not work out out of the box, due to docutils niceties.
background: I refuse to publish my regular-expressions-monster-convert
script.
I had to write a new one using docutils, as it should be.
Status as now is:
there are some minor fixes to be dealt with, such as container for
video-tours directive, registering of doc and ref roles
and a directive for strike through handling, so that the conversion can
be done with minor headaches by the qubes os team
(hint: I just pushed master branch on qubes-translation-utilz where the
current status can be feared :) )
Regarding translation as a whole: Michael suggested weblate, as tails
have experience with it and it seems very nice as OS alternative to
transifex.
Once there are further infos on that will summarize and write.
Do you mean that or smth else I forget to mention here?
All the best,
m
I do not know unfortunately what you are referring to?
I mean the objective is to convert the markdown documentation to rst asap
Unfortunately it did not work out out of the box, due to docutils niceties.
background: I refuse to publish my regular-expressions-monster-convert
script.
I had to write a new one using docutils, as it should be.
Status as now is:
there are some minor fixes to be dealt with, such as container for
video-tours directive, registering of doc and ref roles
and a directive for strike through handling, so that the conversion can
be done with minor headaches by the qubes os team
(hint: I just pushed master branch on qubes-translation-utilz where the
current status can be feared :) )
Regarding translation as a whole: Michael suggested weblate, as tails
have experience with it and it seems very nice as OS alternative to
transifex.
Once there are further infos on that will summarize and write.
Do you mean that or smth else I forget to mention here?
All the best,
m
Marek Marczykowski-Górecki
Sep 25, 2022, 6:01:53 PM9/25/22
to mm, Tobias Killer, qubes...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi M,
Hash: SHA256
In fact, I'm working on translation-utilz right now too. Marta used her
google-foo and found this gem:
https://github.com/SamWilsn/docutils-rst-writer
So, I replaced most of the qubesrstwriter2.py with just this thing and
it's almost flawless (especially tables, images etc work out of the
box).
I massaged the links fixup a bit, and now the only thing it complains
about is arch-spec-0.3.pdf, and a single "c" lexer issue. That's all.
Lists starting with non-1 number works too.
I'll push my state in few minutes into separate branch.
BR,
Marek
> --
> You received this message because you are subscribed to the Google Groups "qubes-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/75e76fab-da0d-058f-c8e9-4db4006c15ed%40maiski.net.
> You received this message because you are subscribed to the Google Groups "qubes-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel...@googlegroups.com.
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----
1yx55gf+L10gTE4rcdKYNE6QVAZQNfpZvuMiCooS19Ik2hy3cAqEnVtDoxSRnuOr
jr2NY8vroAEFEoD4U1QPRhOe1HeRL81+9zfTdIVZfKXBCxXSiZMlZuh0LHJYvh5X
wE5Skry+ObVjiqA8amb/AeI2Y2LXFAWlV6k0YuwOm5J8/TFu6pe5jKQdaapg5+qa
LLtC5tO+g2BcXEhkhFhhvBS/ABJD6vUjWjvdWdQtdnfhPnOgYTKEYXFiNFMPgs8Q
6ClhzILwVghws7frJJ0eOQ8JvbC+OTHAd3KJJZe5M1kAbu6rQIS7XCSOJ222WSSQ
NhggXUNW8D1sWkLOWeCgA2QaVReNpQ==
=npHQ
-----END PGP SIGNATURE-----
Andrew David Wong
Sep 26, 2022, 4:47:52 AM9/26/22
to mm, qubes-devel
On 9/9/22 12:52 PM, mm wrote:
> Hey Andrew,
>
>
> thank you for your work!!
>
Thank *you*! :)
> Hey Andrew,
>
>
> thank you for your work!!
>
> So one by one:
>
> https://github.com/maiska/qubes-doc.rtd/commit/0b8529a026da0523355eec55c57c1da2aa110d07
>
>> By the way, with this one, the link is not actually broken. It still works, because your browser adds the missing forward slash automatically. Nonetheless, I've committed a fix to make it technically syntactically correct.
>
> With the above commit there several links. I suppose you mean the internal one:
> it is so, yes, but for the convenience of my stupid script..., but let me explain:
>
> Some people, when confronted with a problem, think
> “I know, I'll use regular expressions.”
> Now they have two problems.
>
> Well.... :)
>
>>> [...]
>>> other people's articles linking to the official Qubes OS documentation would be broken, since I do not think that there is a proxy
>>>
>>> to handle such requests and redirects? Or there is a simpler way?
>>>
>> I don't understand why links would get broken. Let's use a concrete example to help me understand. Take this page, for example:
>>
>> https://www.qubes-os.org/doc/how-to-update/
>>
>> After the migration to RtD, wouldn't this still be the same URL for that page? Why would it be any different?
>>
>> If you're saying it has to be a different URL for some reason, what would the new URL look like?
>
> The new url when generated will look like: foo.bar/user/how-to-guides/how-to-update.html from rst will look like :doc:` foo </user/how-to-guides/how-to-update>`
>
> Domain dependent or whatever...
>
>>
>>>> Wouldn't converting all Markdown files to RST result in still using the _doc submodule, just with all the files converted? Maybe I'm not understanding something here.
>>> This could be a possibility yes, perhaps with a sphinx plugin for redirects. It exists, haven't tried it yet.
>>>
>> Why only "a possibility"? I figured that would be the obvious thing to do, but clearly I am missing something. If there is a good reason to do otherwise, then that's totally fine! It's just not clear to me what that reason might be.
>
> Nope, you are abs right!
>
> My motivation behind this solution (meaning leave the docs as they are but empty etc.) was take it one step at a time w/o necessarily drain time in fixing possible broken links. Till the documentation is on RTD
>
> and afterwards the website can be dealt with and properly cleared of any redundant information (such as in _includes research.html, cause i blank out translated it
>
> into flat markdown or rst),
>
> also than use the sphinx redirect extension, these all are nice to have and can be done afterwards.
>
> The possibility was not the right word in this context...
>
> The HCL and Downloads thus can be addressed later :)
>
> About the pull requests I will look into it latest tomorrow, render and say if smth is 'missing'
>
>
> One thing though which I did not think about:
>
> The privacy policy should be 2x - one for the Qubes OS website, and one for RTD. The one for RTD should be adjusted and perhaps refer to also their privacy policy:
>
> https://docs.readthedocs.io/en/stable/privacy-policy.html
>
>
> But maybe that's okay, since once we migrate to RtD, we'll no longer be relying on custom scripts like this to populate the table of contents.
>
> Yes!! Hopefully there will be a lot less clutter :))
>
>
>
> So, how do I serve the site locally after doing this? Can you provide step-by-step instructions?
>
>
> On a fedora VM:
>
> sudo dnf install ruby ruby-devel openssl-devel redhat-rpm-config @development-tools;
> gem install jekyll bundler;
> find . -name gem; #(in my case it was /home/user/.local/share/gem/)
> git clone -b new-master --recursive https://github.com/maiska/qubesos.github.io.rtd.git; cd qubesos.github.io.rtd/
> bundle config set --local path '/home/user/.local/share/gem/'
> bundle exec jekyll serve
>
Ah, ok, it looks similar to the standard old method of serving a Jekyll site. Thanks, I'll give it a try when I find some time.
> So, how do I serve the site locally after doing this? Can you provide step-by-step instructions?
>
>
> On a fedora VM:
>
> sudo dnf install ruby ruby-devel openssl-devel redhat-rpm-config @development-tools;
> gem install jekyll bundler;
> find . -name gem; #(in my case it was /home/user/.local/share/gem/)
> git clone -b new-master --recursive https://github.com/maiska/qubesos.github.io.rtd.git; cd qubesos.github.io.rtd/
> bundle config set --local path '/home/user/.local/share/gem/'
> bundle exec jekyll serve
>
Any chance you've gotten this to work in a Debian minimal VM (plus whatever packages are required, of course)?
mm
Sep 26, 2022, 5:33:30 PM9/26/22
to Marek Marczykowski-Górecki, Tobias Killer, qubes...@googlegroups.com
Hi Marek,
Hi M,
In fact, I'm working on translation-utilz right now too. Marta used her
google-foo and found this gem:
https://github.com/SamWilsn/docutils-rst-writer
this looks very nice!! :) Thanks Marta! :)
So, I replaced most of the qubesrstwriter2.py with just this thing and
it's almost flawless (especially tables, images etc work out of the
box).
OK, thanks for the PR, I'll merge into and get rid of the
obsolete stuff from my branch this days.
I have some weird customized fixes all over, but this should work.
One question though:
have you had success with parsing admin-api.rst (as converted by pandoc from md?)
I get the error when parsing with docutils in order to post-process it for qubes:
docutils.utils.SystemMessage: /home/user/testingfilez/admin-api.rst:74: (SEVERE/4) Unexpected section title
and it cannot get over the table as it is...:/
all the best,
m