Re: Towards solving Qubes OS issue #2930 on GitHub
26 views
Skip to first unread message
Tobias Killer
Aug 27, 2017, 5:56:11 PM8/27/17
to qubes-tr...@googlegroups.com, Marek Marczykowski-Górecki, Andrew David Wong
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Am 27.08.2017 um 22:50 schrieb Marek Marczykowski-Górecki:
> Hi Tobias and Andrew,
>
> On Fri, Aug 18, 2017 at 11:10:15AM +0200, Tobias Killer wrote:
>> Am 13.08.2017 um 20:37 schrieb Andrew David Wong:
>>> On 2017-08-10 12:54, Tobias Killer wrote:
>>>> Hello, Andrew,
>>>>
>>>> How are you? I've mentioned you here [1] two week ago and
>>>> expected an answer, but I didn't get an answer yet. I don't
>>>> want to push by telling it. I just wonder if there are any
>>>> troubles (hopefully not).
>>>>
>>>> (Since people are on vacation or are going to launch a major
>>>> Qubes version release these days, I guess that you are one of
>>>> them. So, it's totally okay.)
>>>>
>>>> Greetings, Tobias
>>>>
>>>> [1]
>>>>
>> https://github.com/QubesOS/qubes-issues/issues/2930#issuecomment-317375022
>>>>
>>>
>>
Hi Tobias,
>>>
>>> Sorry for the delayed response. I am extremely busy and have
>>> not had a chance to look at your prototype until now.
>>
>> It's okay.
>>
>>> I just cloned your repo and tested it. This looks very cool!
>>> This approach seems very promising to me,
>>
>> Nice to hear!
>>
>>> but I must admit that I'm not the most qualified person to say
>>> whether it will break anything (at least not without testing it
>>> on the actual Qubes website).
>>
>> I plan to implement it, but not visible on the actual Qubes
>> website as long as the translation workflow doesn't go like
>> clockwork. The reason is that people could get frustrated,
>> annoyed or demotivated when they visit a perhaps "broken" (not
>> sufficiently tested) or barely translated website (hard to read)
>> or when they (as translators) experience that translated phrases
>> aren't put online soon.
>>
>> The first impression counts. So, currently, there shouldn't be
>> links from the official website to any translated website.
>> However, cross-linking pages within the translated websites and
>> visiting them by entering an appropriate URL by hand should be
>> okay for now.
>>
>> @Marek: What is your assessment? If you like you may assess even
>> now if my prototype is a solid base for further work and wouldn't
>> break anything.
>
> Finally I found some time to look at this subject. Your language
> switch prototype indeed looks cool!
>
> I'm slightly worried about URL splitting code - it split the whole
> URL at slashes, then join it back. While I don't see any error
> there right now, there may be some corner cases. Better would be to
> look at just URL beginning - if does not contain language code -
> add it, otherwise - replace. I'm mostly worried about urlremainder
> variable, which ideally would be just appropriate part of page.url,
> without splitting & merging it at every slash. But this is not
> critical, we can also take care of it when we hit such corner case
> in testing. But at least add a comment about it there.
>
>>> What do you think the next step should be? Would you like to
>>> submit a PR against the official Qubes website repo that
>>> implements this?
>>
>>
>> The next steps I plan to do are:
>>
>> 1. *Extract English phrases from layout and other "meta" files*
>> and put them into (multilingual) YAML files. This is to reduce
>> the need for separate HTML layout files for each language, since,
>> in my view, HTML files (both with and without Liquid
>> instructions) are harder for a translator to read and translate,
>> resulting in more error-prone work. For example, the layout file
>> "/_layouts/doc-full.html", used by "/_doc/services/admin-api.md",
>> contains the readable word "Documentation", which should be moved
>> to a YAML file where it can be translated more easily, at least
>> via Transifex [2], hopefully.
>
> +1
>
>> 2. *Make* the value for the attribute *href* in the concerned
>> HTML elements *multilingual* by adding a prefix like "{{
>> current_lang_url_prefix }}" with a proper definition of
>> "current_lang_url_prefix" (see my prototype).
>
> You mean for example _includes/footer.html? See that most of links
> in footer are sourced from _data/architecture.yml Not all of them
> may have translated version (/news/ ? /team/ ? /partners/ ? ). But
> IMO it is solvable problem, without changing the main idea here.
>
>> 3. *Create a directory structure* that reflects all the
>> available destination languages, including the canonical
>> version.
>
> As for repository layout, we should keep separate: - original
> english files, in qubes-doc repo (basically the current layout) -
> translated files, in some new repo
>
> The second one probably should be just one repo for all languages,
> with language code being top level directory. I don't think it
> affect URLs, because each page have them defined in YAML header.
> Then, it should be connected as a submodule for qubesos.github.io.
> The script that handle submodule updates should be extended with
> special cases for this repo - see point 8.
>
>> 4. Put all the language-dependent MD, YAML and HTML source files
>> to the new directory of the canonical version. Test if the
>> website will still work flawlessly.
>
> While for MD files this is easy, I'm not sure how it will work for
> YAML and HTML - especially those in _includes, _layout, and _data.
> I think some of them have to be in root directory of the website
> repository. IMO the way to approach them is point 1 in your list.
>
>> 5. *Add a language switch* to the concerned layout files etc..
>> The switch shall *not appear in the canonical version*. Thus, (1)
>> a user cannot get to a translated page by accident, but by
>> setting the browser URL manually, and (2), the language switch
>> will be available within the translated pages. Thus, we can work
>> on it and test new things without getting users annoyed by
>> randomly "crashing" or weird web pages.
>
> +1
>
>> 6. Make special modifications like removing the "Edit This Page"
>> button for translated doc versions.
>
> This is mostly about doc-translated layout.
>
>> 7. *Make Transifex uploading translated files* to their matching
>> language directories, with an appropriate file ending.
>
> Yes, Transifex pushing to this repository is one option. The other
> is appropriately configured transifex client and/or a script,
> running periodically or in reaction to some notification from
> Transifex. In any case, files downloaded from Transifex should be
> _only_ placed in separate repository for translated content.
>
>> 8. *Write a bot that corrects internal links and the YAML header*
>> within MD files after being translated. Thus, even if the files
>> are not translated fully then at least the internal links and the
>> YAML header will work correctly. Otherwise, people could feel
>> annoyed by being directed to the canonical version over and over
>> again. Plus, if the translated YAML header is not correct,
>> strange errors could happen. As a bonus, internal links and the
>> YAML header don't need to be translated by human translators,
>> resulting in less error-prone work.
>
> And most importantly: this script should also make sure that
> translated page do not override any other (from other language, or
> a canonical one). This applies to permalink and redirect_from
> fields in YAML header. Generally I think the whole YAML header will
> need to be at least carefully validated, if not re-constructed from
> scratch when downloading files from Transifex.
>
>> 9. Write a program (or at least a manual) that can *set up
>> everything necessary for a new language*.
>
> +1
>
>> After some further steps: The "last" step: *Turn on the language
>> switch for the canonical version*. Thus, a user may visit the
>> website in other languages in a common way (and not by
>> manipulating the web browser URL).
>
> +1
>
> Generally the whole plan looks good. Let me know when you'll need
> some help. I guess point 8 may be quite complex.
>
Have a lot of thanks for your comments, Marek! If I need help, I'll
remember your offer.
I added qubes-translation as a recipient.
I plan to open one issue on QubesOS/qubes-issues for each step of an
appropriate size and to create one or more PR(s). I never did
something like that before and hopefully, everything will work fine.
(Of course, I'll exercise it on my own test repo here in advance.)
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEE9VOPt2yY9xS+XKzULaXvb25AsygFAlmjP80ACgkQLaXvb25A
syg4CRAAo3ohMg717tXYTldaddJqWbhaaLPOlyl5E723NDWITB27xOlM84HW6Q/1
xwzaVSEBkqC5dCo8O0DFvjREjjBpzuqFlqQ3dS/4rgKxvXh+SLxqLAV2JYhM8cD0
U7QigAn2fkFzINRNgwMEpz4ySb0lFGOlXwHmW+7Vd0wYfs9MrhCoug2lp7ATWvB8
ZLykClDmbNIxIpldx0QsepkrBJ8lPRuuW6OSiNzcC8n1HOZQEKuoadbmdO6YKl3o
tXXna/yslYAmMIKTDfY9SoitI+aj1tKlchXdhLQD1Uz6sG6QcVLFhu/ZIeiiP/gt
BugBIEaQ//68VQyeMaKLuRlNNj5EEis1Uz3DwYXoPDN2U/+Pg3YDpt8UKJac4M3X
PW/46ynE1idF3BOCqUO2iTDxZiiIGmn69ahzhau8iivN0e2EvqUdQ0o83g0e0hfc
EtLFJAslm/rVcA1avL+xncKGFnqIKqcbQ9E+S0Dpvsp8BHnZQ2fxOFC9C22Sm7+4
6uFg5qjoPf2KwM1ozJHL3YJgtNIttquf5+3l6wvp0+9F8ncGOu1tnKJhNYHuK6jX
Zj1aCvsR0BGCWzg0H10l/Q/bozf7xclXVn0XDKPHwre7A8aQWkfV6r1mPOKJhNz2
UXv7s9OlW3aL6/tRgD/plIijQBcyfsscTvtFPngiBQvn3qgYTvE=
=uQXT
-----END PGP SIGNATURE-----
Hash: SHA256
Am 27.08.2017 um 22:50 schrieb Marek Marczykowski-Górecki:
> Hi Tobias and Andrew,
>
> On Fri, Aug 18, 2017 at 11:10:15AM +0200, Tobias Killer wrote:
>> Am 13.08.2017 um 20:37 schrieb Andrew David Wong:
>>> On 2017-08-10 12:54, Tobias Killer wrote:
>>>> Hello, Andrew,
>>>>
>>>> How are you? I've mentioned you here [1] two week ago and
>>>> expected an answer, but I didn't get an answer yet. I don't
>>>> want to push by telling it. I just wonder if there are any
>>>> troubles (hopefully not).
>>>>
>>>> (Since people are on vacation or are going to launch a major
>>>> Qubes version release these days, I guess that you are one of
>>>> them. So, it's totally okay.)
>>>>
>>>> Greetings, Tobias
>>>>
>>>> [1]
>>>>
>> https://github.com/QubesOS/qubes-issues/issues/2930#issuecomment-317375022
>>>>
>>>
>>
Hi Tobias,
>>>
>>> Sorry for the delayed response. I am extremely busy and have
>>> not had a chance to look at your prototype until now.
>>
>> It's okay.
>>
>>> I just cloned your repo and tested it. This looks very cool!
>>> This approach seems very promising to me,
>>
>> Nice to hear!
>>
>>> but I must admit that I'm not the most qualified person to say
>>> whether it will break anything (at least not without testing it
>>> on the actual Qubes website).
>>
>> I plan to implement it, but not visible on the actual Qubes
>> website as long as the translation workflow doesn't go like
>> clockwork. The reason is that people could get frustrated,
>> annoyed or demotivated when they visit a perhaps "broken" (not
>> sufficiently tested) or barely translated website (hard to read)
>> or when they (as translators) experience that translated phrases
>> aren't put online soon.
>>
>> The first impression counts. So, currently, there shouldn't be
>> links from the official website to any translated website.
>> However, cross-linking pages within the translated websites and
>> visiting them by entering an appropriate URL by hand should be
>> okay for now.
>>
>> @Marek: What is your assessment? If you like you may assess even
>> now if my prototype is a solid base for further work and wouldn't
>> break anything.
>
> Finally I found some time to look at this subject. Your language
> switch prototype indeed looks cool!
>
> I'm slightly worried about URL splitting code - it split the whole
> URL at slashes, then join it back. While I don't see any error
> there right now, there may be some corner cases. Better would be to
> look at just URL beginning - if does not contain language code -
> add it, otherwise - replace. I'm mostly worried about urlremainder
> variable, which ideally would be just appropriate part of page.url,
> without splitting & merging it at every slash. But this is not
> critical, we can also take care of it when we hit such corner case
> in testing. But at least add a comment about it there.
>
>>> What do you think the next step should be? Would you like to
>>> submit a PR against the official Qubes website repo that
>>> implements this?
>>
>>
>> The next steps I plan to do are:
>>
>> 1. *Extract English phrases from layout and other "meta" files*
>> and put them into (multilingual) YAML files. This is to reduce
>> the need for separate HTML layout files for each language, since,
>> in my view, HTML files (both with and without Liquid
>> instructions) are harder for a translator to read and translate,
>> resulting in more error-prone work. For example, the layout file
>> "/_layouts/doc-full.html", used by "/_doc/services/admin-api.md",
>> contains the readable word "Documentation", which should be moved
>> to a YAML file where it can be translated more easily, at least
>> via Transifex [2], hopefully.
>
> +1
>
>> 2. *Make* the value for the attribute *href* in the concerned
>> HTML elements *multilingual* by adding a prefix like "{{
>> current_lang_url_prefix }}" with a proper definition of
>> "current_lang_url_prefix" (see my prototype).
>
> You mean for example _includes/footer.html? See that most of links
> in footer are sourced from _data/architecture.yml Not all of them
> may have translated version (/news/ ? /team/ ? /partners/ ? ). But
> IMO it is solvable problem, without changing the main idea here.
>
>> 3. *Create a directory structure* that reflects all the
>> available destination languages, including the canonical
>> version.
>
> As for repository layout, we should keep separate: - original
> english files, in qubes-doc repo (basically the current layout) -
> translated files, in some new repo
>
> The second one probably should be just one repo for all languages,
> with language code being top level directory. I don't think it
> affect URLs, because each page have them defined in YAML header.
> Then, it should be connected as a submodule for qubesos.github.io.
> The script that handle submodule updates should be extended with
> special cases for this repo - see point 8.
>
>> 4. Put all the language-dependent MD, YAML and HTML source files
>> to the new directory of the canonical version. Test if the
>> website will still work flawlessly.
>
> While for MD files this is easy, I'm not sure how it will work for
> YAML and HTML - especially those in _includes, _layout, and _data.
> I think some of them have to be in root directory of the website
> repository. IMO the way to approach them is point 1 in your list.
>
>> 5. *Add a language switch* to the concerned layout files etc..
>> The switch shall *not appear in the canonical version*. Thus, (1)
>> a user cannot get to a translated page by accident, but by
>> setting the browser URL manually, and (2), the language switch
>> will be available within the translated pages. Thus, we can work
>> on it and test new things without getting users annoyed by
>> randomly "crashing" or weird web pages.
>
> +1
>
>> 6. Make special modifications like removing the "Edit This Page"
>> button for translated doc versions.
>
> This is mostly about doc-translated layout.
>
>> 7. *Make Transifex uploading translated files* to their matching
>> language directories, with an appropriate file ending.
>
> Yes, Transifex pushing to this repository is one option. The other
> is appropriately configured transifex client and/or a script,
> running periodically or in reaction to some notification from
> Transifex. In any case, files downloaded from Transifex should be
> _only_ placed in separate repository for translated content.
>
>> 8. *Write a bot that corrects internal links and the YAML header*
>> within MD files after being translated. Thus, even if the files
>> are not translated fully then at least the internal links and the
>> YAML header will work correctly. Otherwise, people could feel
>> annoyed by being directed to the canonical version over and over
>> again. Plus, if the translated YAML header is not correct,
>> strange errors could happen. As a bonus, internal links and the
>> YAML header don't need to be translated by human translators,
>> resulting in less error-prone work.
>
> And most importantly: this script should also make sure that
> translated page do not override any other (from other language, or
> a canonical one). This applies to permalink and redirect_from
> fields in YAML header. Generally I think the whole YAML header will
> need to be at least carefully validated, if not re-constructed from
> scratch when downloading files from Transifex.
>
>> 9. Write a program (or at least a manual) that can *set up
>> everything necessary for a new language*.
>
> +1
>
>> After some further steps: The "last" step: *Turn on the language
>> switch for the canonical version*. Thus, a user may visit the
>> website in other languages in a common way (and not by
>> manipulating the web browser URL).
>
> +1
>
> Generally the whole plan looks good. Let me know when you'll need
> some help. I guess point 8 may be quite complex.
>
Have a lot of thanks for your comments, Marek! If I need help, I'll
remember your offer.
I added qubes-translation as a recipient.
I plan to open one issue on QubesOS/qubes-issues for each step of an
appropriate size and to create one or more PR(s). I never did
something like that before and hopefully, everything will work fine.
(Of course, I'll exercise it on my own test repo here in advance.)
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEE9VOPt2yY9xS+XKzULaXvb25AsygFAlmjP80ACgkQLaXvb25A
syg4CRAAo3ohMg717tXYTldaddJqWbhaaLPOlyl5E723NDWITB27xOlM84HW6Q/1
xwzaVSEBkqC5dCo8O0DFvjREjjBpzuqFlqQ3dS/4rgKxvXh+SLxqLAV2JYhM8cD0
U7QigAn2fkFzINRNgwMEpz4ySb0lFGOlXwHmW+7Vd0wYfs9MrhCoug2lp7ATWvB8
ZLykClDmbNIxIpldx0QsepkrBJ8lPRuuW6OSiNzcC8n1HOZQEKuoadbmdO6YKl3o
tXXna/yslYAmMIKTDfY9SoitI+aj1tKlchXdhLQD1Uz6sG6QcVLFhu/ZIeiiP/gt
BugBIEaQ//68VQyeMaKLuRlNNj5EEis1Uz3DwYXoPDN2U/+Pg3YDpt8UKJac4M3X
PW/46ynE1idF3BOCqUO2iTDxZiiIGmn69ahzhau8iivN0e2EvqUdQ0o83g0e0hfc
EtLFJAslm/rVcA1avL+xncKGFnqIKqcbQ9E+S0Dpvsp8BHnZQ2fxOFC9C22Sm7+4
6uFg5qjoPf2KwM1ozJHL3YJgtNIttquf5+3l6wvp0+9F8ncGOu1tnKJhNYHuK6jX
Zj1aCvsR0BGCWzg0H10l/Q/bozf7xclXVn0XDKPHwre7A8aQWkfV6r1mPOKJhNz2
UXv7s9OlW3aL6/tRgD/plIijQBcyfsscTvtFPngiBQvn3qgYTvE=
=uQXT
-----END PGP SIGNATURE-----
Reply all
Reply to author
Forward
0 new messages