Builds docs on MDN

[Gregory Szorc]

MDN is pivoting hard to focus on web docs: https://blog.mozilla.org/opendesign/future-mdn-focus-web-docs/

MDN will actively start de-emphasizing docs that aren't related to the web. You can already see this on things like search results: https://developer.mozilla.org/en-US/search?q=firefox%20build. Note how the "Firefox" topic isn't searched by default. I also heard MDN may start excluding non-web docs from search engine indexing. If they do this, search results for e.g. "contribute to Firefox" will likely go nowhere useful.

There are a ton of build system (and general Firefox development docs) on MDN. e.g. https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions.

We already had fragmented Firefox build docs. We have things scattered between MDN, wiki.mozilla.org, in-tree Sphinx docs, and years of mailing list and blog posts (which sadly are the sole source of some useful info). With MDN pivoting towards the web and being hostile to non-web docs, I think the writing is clear that we should be moving Firefox build/development docs off MDN. Or at the very least we shouldn't continue to invest much effort in the MDN docs.

Personally, I'd like to see us move towards the in-tree docs. Those are currently hosted at https://gecko.readthedocs.org/ (although that's been broken for a few weeks and before that it was only reliable ~50% of the time because our scale breaks RTD). We can certainly improve the robustness. Possibly by hosting ourselves if we need to. However, the in-tree docs aren't a wiki, so the barrier to change is higher - both in terms of process to edit and the knowledge required to use ReST + Sphinx. I've explored some of this in more detail at https://gregoryszorc.com/blog/2015/01/09/firefox-contribution-process-debt/. I wholeheartedly agree that wikis are more user friendly. However, I also feel like the in-tree docs get us nice things like versioning (someone wanting to build Firefox 55 2 years from now will have access to the docs for version 55 via source control), link verification, and a code review process so build peers can prevent bad docs before they are seen by others.

I wanted to start a thread to see what people think we should do. And to be clear, I'm not yet proposing that we incur a bunch of work to move things. But we probably should figure out where are docs efforts should be invested moving forward.

[Andrew Halberstadt]

+1 to preferring in-tree docs.

If we're going to be adding a ton of new docs though, I think it would be worth exploring a different sphinx theme. I find the toctree in the side bar of the default rtd theme is already looking cluttered and getting difficult to navigate. I imagine with an order of magnitude more content, it will be more confusing than helpful.

_______________________________________________
dev-builds mailing list
dev-b...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds

[Nicholas Alexander]

On Thu, Jun 15, 2017 at 1:04 PM, Andrew Halberstadt <ahalbe...@mozilla.com> wrote:

+1 to preferring in-tree docs.

+1 to this from me as well.  Wish it was easier to contribute, but the in-tree docs are the best trade-off we have right now.

Nick

[Gregory Szorc]

On Thu, Jun 15, 2017 at 1:04 PM, Andrew Halberstadt <ahalbe...@mozilla.com> wrote:

+1 to preferring in-tree docs.

If we're going to be adding a ton of new docs though, I think it would be worth exploring a different sphinx theme. I find the toctree in the side bar of the default rtd theme is already looking cluttered and getting difficult to navigate. I imagine with an order of magnitude more content, it will be more confusing than helpful.

Yeah, there is definitely room to improve things.

We have full control over what Sphinx plugins, themes, etc we use. You may find tools/docs/conf.py and `./mach doc` useful for tinkering. The Sphinx bits are an underloved feature. So any contributions would be greatly appreciated. Feel free to flag me for review.

[Sylvestre Ledru]

Le 15/06/2017 à 22:22, Gregory Szorc a écrit :

On Thu, Jun 15, 2017 at 1:04 PM, Andrew Halberstadt <ahalbe...@mozilla.com> wrote:

+1 to preferring in-tree docs.

If we're going to be adding a ton of new docs though, I think it would be worth exploring a different sphinx theme. I find the toctree in the side bar of the default rtd theme is already looking cluttered and getting difficult to navigate. I imagine with an order of magnitude more content, it will be more confusing than helpful.

Yeah, there is definitely room to improve things.

We have full control over what Sphinx plugins, themes, etc we use. You may find tools/docs/conf.py and `./mach doc` useful for tinkering. The Sphinx bits are an underloved feature. So any contributions would be greatly appreciated. Feel free to flag me for review.

 

As most of the people don't like to spend time writing docs, I am a bit concerned that using in-tree docs with our "heavy" review process (bug, reviewers, etc) will even decrease our contributions to the doc (instead of the MDN wiki
where it is quick to contribute).

Sylvestre

[Andrew Halberstadt]

On Fri, Jun 16, 2017 at 4:33 AM, Sylvestre Ledru <sylv...@mozilla.com> wrote:

As most of the people don't like to spend time writing docs, I am a bit concerned that using in-tree docs with our "heavy" review process (bug, reviewers, etc) will even decrease our contributions to the doc (instead of the MDN wiki
where it is quick to contribute).

I suspect overall you're right, but for me personally the opposite is true. I'm much more likely to remember to update the docs for a breaking change I make if they live somewhere near the source, than I am to remember to hunt down all the appropriate wiki/mdn pages.

Awhile back I briefly looked into whether DXR or SF could support some sort of "Edit" functionality a la Github, for quick things like typo fixes and doc edits. It could potentially create a mozreview request (or conduit request) on behalf of the user. I'd love if we could push on that, maybe this would be a good excuse.

-Andrew

[Andreas Tolfsen]

Also sprach Andrew Halberstadt:

> On Fri, Jun 16, 2017 at 4:33 AM, Sylvestre Ledru
> <sylv...@mozilla.com> wrote:
>
>> As most of the people don't like to spend time writing docs, I am a
>> bit concerned that using in-tree docs with our "heavy" review process
>> (bug, reviewers, etc) will even decrease our contributions to the doc
>> (instead of the MDN wiki where it is quick to contribute).
>
> I suspect overall you're right, but for me personally the opposite
> is true. I'm much more likely to remember to update the docs for a
> breaking change I make if they live somewhere near the source, than I
> am to remember to hunt down all the appropriate wiki/mdn pages.

I tend to agree with both points of view: I’m more likely to remember
updating documentation when I grep through the source to find references
to what I’m patching, but perhaps less inclined to file separate bugs
to fix up existing documentation.

However, what concerns me much more than ease of editing is
discoverability. MDN is well-indexed and highly ranked by search
engines and has hyperlinks, making it (tolerably) easy to find what
information it holds on any given subject.

How do we expose publicly-intended non-web API documentation from the
tree and make it discoverable by search engines?

> Awhile back I briefly looked into whether DXR or SF could support some
> sort of "Edit" functionality a la Github, for quick things like typo
> fixes and doc edits. It could potentially create a mozreview request
> (or conduit request) on behalf of the user. I'd love if we could push
> on that, maybe this would be a good excuse.

Our current workflow uses Bugzilla bugs as the central focal point.
I think there would either have to be a change to this policy, or an
exception to this rule for documentation for that to work.

I am otherwise in full agreement that it would be desireable to have a
less “heavy”, as Sylvestre describes it, process.

[Ehsan Akhgari]

FWIW most of my contributions to the documentation hosted on MDN were in
the form of casual edits here and there to pages, mostly in the form of
me noticing something wrong in some documentation and quickly editing
the wiki to fix it. The editing process of the MDN wiki was (well,
still is!) exceptionally good at this workflow, so it is extremely sad
to not be able to continue to use it going forward.

For those types of casual contributions, I certainly feel like the
barrier for filing bugs, creating a patch, figuring out how to use
readthedocs infrastructure, getting reviews, etc. isn't really worth it,
since I'm comparing it to a workflow that used to take a few seconds, so
a switch to a workflow based on in-tree docs would drive someone like me
away. But maybe those types of casual contributions aren't worth
optimizing worth here.

(Also, maybe it's just me, but even though I have done tons of code
archeology, in practice I don't recall having much if any at all of
documentation archeology personally so part of the reason why I'm
reluctant in spending the effort to switch to a workflow based on
in-tree documentation is because I feel like I haven't yet seen how it
benefits me!)

My CDN$0.02,

Ehsan

On 06/15/2017 03:41 PM, Gregory Szorc wrote:
> MDN is pivoting hard to focus on web docs:
> https://blog.mozilla.org/opendesign/future-mdn-focus-web-docs/
>
> MDN will actively start de-emphasizing docs that aren't related to the
> web. You can already see this on things like search results:
> https://developer.mozilla.org/en-US/search?q=firefox%20build. Note how
> the "Firefox" topic isn't searched by default. I also heard MDN may
> start excluding non-web docs from search engine indexing. If they do
> this, search results for e.g. "contribute to Firefox" will likely go
> nowhere useful.
>
> There are a ton of build system (and general Firefox development docs)
> on MDN. e.g.
> https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions.
>
> We already had fragmented Firefox build docs. We have things scattered

> between MDN, wiki.mozilla.org <http://wiki.mozilla.org>, in-tree

[Mike Hoye]

On 6/15/17 3:41 PM, Gregory Szorc wrote:

We already had fragmented Firefox build docs. We have things scattered between MDN, wiki.mozilla.org, in-tree Sphinx docs, and years of mailing list and blog posts (which sadly are the sole source of some useful info). With MDN pivoting towards the web and being hostile to non-web docs, I think the writing is clear that we should be moving Firefox build/development docs off MDN. Or at the very least we shouldn't continue to invest much effort in the MDN docs.

For whatever it's worth, I'm investigating how to move the MediaWiki-based Wikimo over to a non-MDN Kuma instance. I don't know whether or not this is feasible yet - "losing no history" is The Thing - but MDN is _so_ much easier to edit, and in principle maybe we don't need to actively support... four? five? however many different wikis we support.

"Correct the wiki" is a really low-friction way to contribute, and from a community member's first-contact perspective: learning how to submit a patch to fix an error in our patch submission documentation seems like a lot. I also don't know how well an in-codebase solution will play with localization or accessibility tools in comparison.

I agree that we should anoint a single point of authoritative truth for this, but I think that we want the prerequisites for accessing and changing our docs to be as low as possible, preferably as little as "human brain and keyboard".

- mhoye

[Boris Zbarsky]

On 6/16/17 9:33 AM, Ehsan Akhgari wrote:
> I certainly feel like the
> barrier for filing bugs, creating a patch, figuring out how to use
> readthedocs infrastructure, getting reviews, etc. isn't really worth it

I believe we should not require filing bugs, reviews, or any of that for
in-tree docs. Just edit the doc, commit, push. Add "r=documentation"
if needed to placate hooks. Just because it's in-tree doesn't mean it
needs to use the whole heavyweight process. And if we can make these
things auto-DONTBUILD, that's even better, of course.

I agree it's still slower than a wiki. :(

-Boris

[Sylvestre Ledru]

And far from trivial for volunteers who just want to fix one or two
things in the doc :/

S

[Andreas Tolfsen]

Also sprach Sylvestre Ledru:

> Le 16/06/2017 à 16:40, Boris Zbarsky a écrit :
>

>> I believe we should not require filing bugs, reviews, or any of
>> that for in-tree docs. Just edit the doc, commit, push. Add
>> "r=documentation" if needed to placate hooks. Just because it's
>> in-tree doesn't mean it needs to use the whole heavyweight process.
>> And if we can make these things auto-DONTBUILD, that's even better,
>> of course.
>

> And far from trivial for volunteers who just want to fix one or two
> things in the doc :/

Notwithstanding security considerations of commit level access, if
we could agree having a special review flag for doc changes like bz
suggests, then the remainder of the problem is UX.

ahal’s suggestion to have a click-to-edit file editor in the source
browser (maybe in searchfox?) would fix that. The actual case of
landing the change is automatable with Autoland/Conduit.

[J. Ryan Stinnett]

I am not sure if the reasoning applies to a community as large as Mozilla, but lately I've heard that new contributors can view a project wiki very differently from those heavily involved in a project.

Copying directly from a blog post[1] on this topic:

---

• Maintainers believe they're saying "feel free to make any changes you want, the Wiki is owned by the community." By doing that, they are generally hoping to greatly increase collaboration.

• Contributors, however, seem to be intimidated by a Wiki. Most contributors do not feel completely confident in their ability to add correct content, adhere to standards, fit into the right outline, etc. So paradoxically, by making the medium as open as possible, the Wiki discourages contribution.

• Readers of documentation greatly appreciate well structured content, and want to be able to trust the accuracy of the content they're reading. Wikis do not inspire this confidence. Despite my previous comments about contributors, readers are (logically) concerned that Wiki content may have been written by someone uninformed, or may have fallen out of date.

---

The post goes on to suggest in-tree documentation kept up to date via pull requests as the better approach. I guess the trouble is that at least for mozilla-central, there isn't really a low barrier approach to submitting an in-tree change like a pull request currently.

[1]: http://www.yesodweb.com/blog/2015/08/thoughts-on-documentation

- Ryan

On Fri, Jun 16, 2017 at 10:05 AM, Sylvestre Ledru <sylv...@mozilla.com> wrote:

Le 16/06/2017 à 16:40, Boris Zbarsky a écrit :

And far from trivial for volunteers who just want to fix one or two
things in the doc :/

S

[Ehsan Akhgari]

On 06/16/2017 11:22 AM, J. Ryan Stinnett wrote:
> The post goes on to suggest in-tree documentation kept up to date via
> pull requests as the better approach. I guess the trouble is that at
> least for mozilla-central, there isn't really a low barrier approach
> to submitting an in-tree change like a pull request currently.

As someone with commit access, even if we lifted all requirements to
file bugs, get reviews etc from the process, I'd still find the overhead
of contributing to in-tree documentation too high personally unless if
my contribution was something I would really consider "worth it". I
think moving docs into the tree will exclude the kind of casual
contributor who doesn't even know how to bypass all of our process to
get their contribution "accepted". Compare this to the current world
where they don't need to ask for anyone's blessing beforehand in the
first place. :-/

> [1]: http://www.yesodweb.com/blog/2015/08/thoughts-on-documentation
>
> - Ryan
>
> On Fri, Jun 16, 2017 at 10:05 AM, Sylvestre Ledru
> <sylv...@mozilla.com <mailto:sylv...@mozilla.com>> wrote:
>
>
>
> Le 16/06/2017 à 16:40, Boris Zbarsky a écrit :

> And far from trivial for volunteers who just want to fix one or two
> things in the doc :/
>
> S
>

> _______________________________________________
> dev-builds mailing list
> dev-b...@lists.mozilla.org <mailto:dev-b...@lists.mozilla.org>
> https://lists.mozilla.org/listinfo/dev-builds
> <https://lists.mozilla.org/listinfo/dev-builds>

[Benjamin Smedberg]

Coming in late to this thread, but I'd like to strongly support moving our Mozilla internals documentation in the tree.

The fundamental problem with our current doc situation is not that contributing is too hard: the fundamental problem is that our docs are wrong more often than they are right, and there is very poor ability to get rid of old docs or even see all the docs that currently exist. We cannot afford to have documentation be somebody else's problem, where "volunteers will get to it".

I don't think we can have a sustainable codebase and community (paid and volunteer) without some fundamental improvements in the way we handle code documentation. Documentation needs to be part of everyday engineering and needs to happen in step with changes in our internal APIs and code structure, rather than as something that happens after-the-fact. Technical writing is a skill that we should be actively teaching senior engineers in the project.

I'd even go farther and say that we should keep some kind of review requirements on documentation, although with perhaps different rules: e.g. module owners and peers can change docs without external review.

The existing in-tree docs are already pretty well discoverable on gecko.readthedocs.io: as an example, searching for "mozilla main ping" has the readthedocs documentation of the main ping at the top spot.

I agree that as we grow scope and breadth of in-tree docs we're going to need better organization and probably different templates. I don't know if that's something mhoye would be willing to own as community manager? Or if not let's raise that as a need to engineering directors. There are a bunch of options, including things like integrating more directly with DXR, the way markdown checked into github repositories renders automatically in the github repo browser.

I've found it almost automatic to modify existing docs. Adding *new* docs is a bit of a pain, because the build machinery isn't obvious. Perhaps we could write down exactly where the overhead lies and actually make this better?

--BDS

On Fri, Jun 16, 2017 at 10:37 PM, Ehsan Akhgari <ehsan....@gmail.com> wrote:

On 06/16/2017 11:22 AM, J. Ryan Stinnett wrote:

The post goes on to suggest in-tree documentation kept up to date via pull requests as the better approach. I guess the trouble is that at least for mozilla-central, there isn't really a low barrier approach to submitting an in-tree change like a pull request currently.

As someone with commit access, even if we lifted all requirements to file bugs, get reviews etc from the process, I'd still find the overhead of contributing to in-tree documentation too high personally unless if my contribution was something I would really consider "worth it".  I think moving docs into the tree will exclude the kind of casual contributor who doesn't even know how to bypass all of our process to get their contribution "accepted".  Compare this to the current world where they don't need to ask for anyone's blessing beforehand in the first place.  :-/

[1]: http://www.yesodweb.com/blog/2015/08/thoughts-on-documentation

- Ryan

On Fri, Jun 16, 2017 at 10:05 AM, Sylvestre Ledru <sylv...@mozilla.com <mailto:sylv...@mozilla.com>> wrote:



    Le 16/06/2017 à 16:40, Boris Zbarsky a écrit :

    And far from trivial for volunteers who just want to fix one or two
    things in the doc :/

    S

    _______________________________________________
    dev-builds mailing list

    dev-b...@lists.mozilla.org <mailto:dev-b...@lists.mozilla.org>
    https://lists.mozilla.org/listinfo/dev-builds
    <https://lists.mozilla.org/listinfo/dev-builds>

_______________________________________________
dev-builds mailing list
dev-b...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds

[Ehsan Akhgari]

On 06/20/2017 11:21 AM, Benjamin Smedberg wrote:
> Coming in late to this thread, but I'd like to strongly support moving
> our Mozilla internals documentation in the tree.

It seems that the rest of your post is mostly advocating that we should
write documentation, which is hard to argue with. For the record it's
not clear to me how you connect that to it being a better choice for the
said documentation to live in the tree. (Since you mentioned the
anecdotal successful example of telemetry ping documentation, let me
counter it with the anecdotal successful example of WebIDL bindings
documentation which lives out of the tree:
https://developer.mozilla.org/en-US/docs/Mozilla/WebIDL_bindings.)

> The fundamental problem with our current doc situation is not that
> contributing is too hard: the fundamental problem is that our docs are
> wrong more often than they are right, and there is very poor ability
> to get rid of old docs or even see all the docs that currently exist.
> We cannot afford to have documentation be somebody else's problem,
> where "volunteers will get to it".
>
> I don't think we can have a sustainable codebase and community (paid
> and volunteer) without some fundamental improvements in the way we
> handle code documentation. Documentation needs to be part of everyday
> engineering and needs to happen in step with changes in our internal
> APIs and code structure, rather than as something that happens
> after-the-fact. Technical writing is a skill that we should be
> actively teaching senior engineers in the project.

This is a matter that is totally irrelevant to where the documentation
lives, FWIW. :-)

> I'd even go farther and say that we should keep some kind of review
> requirements on documentation, although with perhaps different rules:
> e.g. module owners and peers can change docs without external review.
>
> The existing in-tree docs are already pretty well discoverable on

> gecko.readthedocs.io <http://gecko.readthedocs.io>: as an example,

> searching for "mozilla main ping" has the readthedocs documentation of
> the main ping at the top spot.
>
> I agree that as we grow scope and breadth of in-tree docs we're going
> to need better organization and probably different templates. I don't
> know if that's something mhoye would be willing to own as community
> manager? Or if not let's raise that as a need to engineering
> directors. There are a bunch of options, including things like
> integrating more directly with DXR, the way markdown checked into
> github repositories renders automatically in the github repo browser.

So this thread is about *build system* docs. Now it seems we are
talking about all docs?

> I've found it almost automatic to modify existing docs. Adding *new*
> docs is a bit of a pain, because the build machinery isn't obvious.
> Perhaps we could write down exactly where the overhead lies and
> actually make this better?

For the record, I think the in-tree documentation system could use some
documentation on how it should be used (in case there isn't some already
available for it.) For example I am not sure what's the process for an
update to something in the tree to go to the live site, how would the
different versions of the docs living on the different branches
(central/beta etc) get treated differently (for example how should one
document something that changed on central before the change hits the
rest of the channels?), how to preview the edits I'm making, how to
subscribe to notifications (similar to watching a wiki page), etc.

Cheers,
Ehsan

>
>
> On Fri, Jun 16, 2017 at 10:37 PM, Ehsan Akhgari
> <ehsan....@gmail.com <mailto:ehsan....@gmail.com>> wrote:
>
> On 06/16/2017 11:22 AM, J. Ryan Stinnett wrote:
>
> The post goes on to suggest in-tree documentation kept up to
> date via pull requests as the better approach. I guess the
> trouble is that at least for mozilla-central, there isn't
> really a low barrier approach to submitting an in-tree change
> like a pull request currently.
>
> As someone with commit access, even if we lifted all requirements
> to file bugs, get reviews etc from the process, I'd still find the
> overhead of contributing to in-tree documentation too high
> personally unless if my contribution was something I would really
> consider "worth it". I think moving docs into the tree will
> exclude the kind of casual contributor who doesn't even know how
> to bypass all of our process to get their contribution
> "accepted". Compare this to the current world where they don't
> need to ask for anyone's blessing beforehand in the first place. :-/
>
> [1]:
> http://www.yesodweb.com/blog/2015/08/thoughts-on-documentation
> <http://www.yesodweb.com/blog/2015/08/thoughts-on-documentation>
>
> - Ryan
>
> On Fri, Jun 16, 2017 at 10:05 AM, Sylvestre Ledru
> <sylv...@mozilla.com <mailto:sylv...@mozilla.com>

> <mailto:sylv...@mozilla.com <mailto:sylv...@mozilla.com>>>

> wrote:
>
>
>
> Le 16/06/2017 à 16:40, Boris Zbarsky a écrit :

> And far from trivial for volunteers who just want to fix
> one or two
> things in the doc :/
>
> S
>

> _______________________________________________
> dev-builds mailing list
> dev-b...@lists.mozilla.org

> <mailto:dev-b...@lists.mozilla.org>
> <mailto:dev-b...@lists.mozilla.org

> <mailto:dev-b...@lists.mozilla.org>>
> https://lists.mozilla.org/listinfo/dev-builds
> <https://lists.mozilla.org/listinfo/dev-builds>

> <https://lists.mozilla.org/listinfo/dev-builds
> <https://lists.mozilla.org/listinfo/dev-builds>>
>
>
>
>
> _______________________________________________
> dev-builds mailing list

> dev-b...@lists.mozilla.org <mailto:dev-b...@lists.mozilla.org>
> https://lists.mozilla.org/listinfo/dev-builds
> <https://lists.mozilla.org/listinfo/dev-builds>
>
>

> _______________________________________________
> dev-builds mailing list

[Andrew Halberstadt]

I think it's also worth mentioning that if we get github pull request support (which afaik is still being planned?), then we could set up our doc generation such that each page links back directly to github's edit mode (at the bottom in small font), e.g:

https://github.com/mozilla/gecko-dev/edit/master/README.txt

Maybe DXR could simply provide a link there as well.

On Tue, Jun 20, 2017 at 11:21 AM, Benjamin Smedberg <benj...@smedbergs.us> wrote:

Coming in late to this thread, but I'd like to strongly support moving our Mozilla internals documentation in the tree.

The fundamental problem with our current doc situation is not that contributing is too hard: the fundamental problem is that our docs are wrong more often than they are right, and there is very poor ability to get rid of old docs or even see all the docs that currently exist. We cannot afford to have documentation be somebody else's problem, where "volunteers will get to it".

I don't think we can have a sustainable codebase and community (paid and volunteer) without some fundamental improvements in the way we handle code documentation. Documentation needs to be part of everyday engineering and needs to happen in step with changes in our internal APIs and code structure, rather than as something that happens after-the-fact. Technical writing is a skill that we should be actively teaching senior engineers in the project.

I'd even go farther and say that we should keep some kind of review requirements on documentation, although with perhaps different rules: e.g. module owners and peers can change docs without external review.

The existing in-tree docs are already pretty well discoverable on gecko.readthedocs.io: as an example, searching for "mozilla main ping" has the readthedocs documentation of the main ping at the top spot.

I agree that as we grow scope and breadth of in-tree docs we're going to need better organization and probably different templates. I don't know if that's something mhoye would be willing to own as community manager? Or if not let's raise that as a need to engineering directors. There are a bunch of options, including things like integrating more directly with DXR, the way markdown checked into github repositories renders automatically in the github repo browser.

I've found it almost automatic to modify existing docs. Adding *new* docs is a bit of a pain, because the build machinery isn't obvious. Perhaps we could write down exactly where the overhead lies and actually make this better?

--BDS

On Fri, Jun 16, 2017 at 10:37 PM, Ehsan Akhgari <ehsan....@gmail.com> wrote:

On 06/16/2017 11:22 AM, J. Ryan Stinnett wrote:

The post goes on to suggest in-tree documentation kept up to date via pull requests as the better approach. I guess the trouble is that at least for mozilla-central, there isn't really a low barrier approach to submitting an in-tree change like a pull request currently.

As someone with commit access, even if we lifted all requirements to file bugs, get reviews etc from the process, I'd still find the overhead of contributing to in-tree documentation too high personally unless if my contribution was something I would really consider "worth it".  I think moving docs into the tree will exclude the kind of casual contributor who doesn't even know how to bypass all of our process to get their contribution "accepted".  Compare this to the current world where they don't need to ask for anyone's blessing beforehand in the first place.  :-/

[1]: http://www.yesodweb.com/blog/2015/08/thoughts-on-documentation

- Ryan

On Fri, Jun 16, 2017 at 10:05 AM, Sylvestre Ledru <sylv...@mozilla.com <mailto:sylv...@mozilla.com>> wrote:



    Le 16/06/2017 à 16:40, Boris Zbarsky a écrit :

    And far from trivial for volunteers who just want to fix one or two
    things in the doc :/

    S

    _______________________________________________
    dev-builds mailing list

    dev-b...@lists.mozilla.org <mailto:dev-b...@lists.mozilla.org>
    https://lists.mozilla.org/listinfo/dev-builds
    <https://lists.mozilla.org/listinfo/dev-builds>

_______________________________________________
dev-builds mailing list
dev-b...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds

_______________________________________________
dev-builds mailing list
dev-b...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds

[Benjamin Smedberg]

On Tue, Jun 20, 2017 at 12:40 PM, Ehsan Akhgari <ehsan....@gmail.com> wrote:

On 06/20/2017 11:21 AM, Benjamin Smedberg wrote:

Coming in late to this thread, but I'd like to strongly support moving our Mozilla internals documentation in the tree.

It seems that the rest of your post is mostly advocating that we should write documentation, which is hard to argue with.  For the record it's not clear to me how you connect that to it being a better choice for the said documentation to live in the tree. (Since you mentioned the anecdotal successful example of telemetry ping documentation, let me counter it with the anecdotal successful example of WebIDL bindings documentation which lives out of the tree: https://developer.mozilla.org/en-US/docs/Mozilla/WebIDL_bindings.)

I'll put nuance on this. I do think we should be writing docs, but I think it's much more important that we're maintaining and deleting existing docs as the code changes or becomes irrelevant. That's why I think the choice of in-tree versus external tool is important and not merely a tooling choice. Documentation that is in the tree can/should  be reviewed as the code changes. Partly because it is in the tree, it maps to Mozilla module ownership and module owners can exercise ownership over their slice of documentation explicitly.

 

I

The existing in-tree docs are already pretty well discoverable on gecko.readthedocs.io <http://gecko.readthedocs.io>: as an example, searching for "mozilla main ping" has the readthedocs documentation of the main ping at the top spot.

I agree that as we grow scope and breadth of in-tree docs we're going to need better organization and probably different templates. I don't know if that's something mhoye would be willing to own as community manager? Or if not let's raise that as a need to engineering directors. There are a bunch of options, including things like integrating more directly with DXR, the way markdown checked into github repositories renders automatically in the github repo browser.

So this thread is about *build system* docs.  Now it seems we are talking about all docs?

gps proposed a specific change for build system docs, but the pushback he's getting isn't about build docs in particular but about docs in general. Therefore I think it's important to address this generally.

 

I've found it almost automatic to modify existing docs. Adding *new* docs is a bit of a pain, because the build machinery isn't obvious. Perhaps we could write down exactly where the overhead lies and actually make this better?

For the record, I think the in-tree documentation system could use some documentation on how it should be used (in case there isn't some already available for it.)  For example I am not sure what's the process for an update to something in the tree to go to the live site, how would the different versions of the docs living on the different branches (central/beta etc) get treated differently (for example how should one document something that changed on central before the change hits the rest of the channels?), how to preview the edits I'm making, how to subscribe to notifications (similar to watching a wiki page), etc.

I agree this is good, and I'll file a bug about this specifically, but I'll answer the questions here:

As docs land to mozilla-central they will automatically update gecko.readthedocs.io. There is a taskcluster job which reports to the mozilla-central tree which does this automatically, and it turns red on failure for monitoring.

As far as I know, docs from the other trees do not get automatically published anywhere, and we don't currently host archived release versions of the docs.

To preview local edits, run `./mach doc` which will do local generation

Currently we don't have good notifications for changes to any file in-tree (at least that I know of). This is one of the phabricator features that I am most looking forward to!

--BDS

 

[Ehsan Akhgari]

On 06/20/2017 12:55 PM, Benjamin Smedberg wrote:
>
>
> On Tue, Jun 20, 2017 at 12:40 PM, Ehsan Akhgari
> <ehsan....@gmail.com <mailto:ehsan....@gmail.com>> wrote:
>
> On 06/20/2017 11:21 AM, Benjamin Smedberg wrote:
>
> Coming in late to this thread, but I'd like to strongly
> support moving our Mozilla internals documentation in the tree.
>
> It seems that the rest of your post is mostly advocating that we
> should write documentation, which is hard to argue with. For the
> record it's not clear to me how you connect that to it being a
> better choice for the said documentation to live in the tree.
> (Since you mentioned the anecdotal successful example of telemetry
> ping documentation, let me counter it with the anecdotal
> successful example of WebIDL bindings documentation which lives
> out of the tree:
> https://developer.mozilla.org/en-US/docs/Mozilla/WebIDL_bindings

> <https://developer.mozilla.org/en-US/docs/Mozilla/WebIDL_bindings>.)

>
>
> I'll put nuance on this. I do think we should be writing docs, but I
> think it's much more important that we're maintaining and deleting
> existing docs as the code changes or becomes irrelevant.

Something that a wiki is much more suitable for. ;-)

> That's why I think the choice of in-tree versus external tool is
> important and not merely a tooling choice. Documentation that is in
> the tree can/should be reviewed as the code changes. Partly because
> it is in the tree, it maps to Mozilla module ownership and module
> owners can exercise ownership over their slice of documentation
> explicitly.

This is the part that I don't understand. You are implying that by not
having some text file containing the documentation inside the tree we
can't enforce review over documentation, and that they can't be updated
alongside code changes. I chose the example of WebIDL bindings
documentation very consciously to demonstrate that these are false
assertions. That example clearly demonstrates that reviewers can (and
certainly do) ask patch authors to fix the documentation when they make
code changes, such documentation fixes are reviewed by the right people
and the resulting docs stay in good shape.

The flip side of the coin is that collaborating on writing articles
(which documentation writing is ultimately is all about) is just not
that great of an experience when you approach it as rigidly as
collaborating on writing computer code. There is a reason why people
have invented wikis, etherpads, and software like gdocs! They provide a
better model for collaborating on writing articles. It is true that when
our only tool is a DVCS package, all of our problems start to look like
nails. But they don't have to. :-) (And FWIW I'm gonna give arguing
over this a rest if you won't agree, since this is purely subjective and
I suspect at the end of the day we may not be able to convince each
other here!)

>
> I
>
> The existing in-tree docs are already pretty well discoverable
> on gecko.readthedocs.io <http://gecko.readthedocs.io>
> <http://gecko.readthedocs.io>: as an example, searching for
> "mozilla main ping" has the readthedocs documentation of the
> main ping at the top spot.
>
> I agree that as we grow scope and breadth of in-tree docs
> we're going to need better organization and probably different
> templates. I don't know if that's something mhoye would be
> willing to own as community manager? Or if not let's raise
> that as a need to engineering directors. There are a bunch of
> options, including things like integrating more directly with
> DXR, the way markdown checked into github repositories renders
> automatically in the github repo browser.
>
> So this thread is about *build system* docs. Now it seems we are
> talking about all docs?
>
>
> gps proposed a specific change for build system docs, but the pushback
> he's getting isn't about build docs in particular but about docs in
> general. Therefore I think it's important to address this generally.

FWIW previously I was just expressing sadness of the bar to contributing
to build system docs being raised, but I'd be happy to live with it
since I won't be doing most of the work there. But for more general
discussions, this isn't the right venue.

>
> I've found it almost automatic to modify existing docs. Adding
> *new* docs is a bit of a pain, because the build machinery
> isn't obvious. Perhaps we could write down exactly where the
> overhead lies and actually make this better?
>
> For the record, I think the in-tree documentation system could use
> some documentation on how it should be used (in case there isn't
> some already available for it.) For example I am not sure what's
> the process for an update to something in the tree to go to the
> live site, how would the different versions of the docs living on
> the different branches (central/beta etc) get treated differently
> (for example how should one document something that changed on
> central before the change hits the rest of the channels?), how to
> preview the edits I'm making, how to subscribe to notifications
> (similar to watching a wiki page), etc.
>
>
> I agree this is good, and I'll file a bug about this specifically, but
> I'll answer the questions here:
>
> As docs land to mozilla-central they will automatically update

> gecko.readthedocs.io <http://gecko.readthedocs.io>. There is a

> taskcluster job which reports to the mozilla-central tree which does
> this automatically, and it turns red on failure for monitoring.
>
> As far as I know, docs from the other trees do not get automatically
> published anywhere, and we don't currently host archived release
> versions of the docs.
>
> To preview local edits, run `./mach doc` which will do local generation

Great tips, thank you!

> Currently we don't have good notifications for changes to any file
> in-tree (at least that I know of). This is one of the phabricator
> features that I am most looking forward to!

Yeah I am also looking for ward to this feature, more for regular code
review though. :-)

[ISHIKAWA,chiaki]

I am a volunteer who occasionally submit patches to C-C TB.

In-tree documentation is much easier for volunteers to work with.
The documentation is much closer to the code we are modifying.

Out of tree docs are much harder to deal with.
Just adding a few (actually more) keystrokes to access them make it
rather difficult, or more to the point TIME-CONSUMING to work with.

For unpaid volunteers (and I assume even for paid developers), in-tree
documentation is much easier to work with in terms of overhead of JUST
checking WHERE on earth the relevant documentation exists (IF IT EXISTS
AT ALL) and then out-of-tree documentation in an other system means we
still need work with DIFFERENT set of rules that exist for
accountability for modification.

Given these overhead, out-of-tree documentations are basically
something volunteer patch submitters won't be able to work with: I
suspect "Just forget it" would be what many volunteer submitters would
feel after a while.

Initially, when people start to submit patches, they may think what they
might need to do with the documentation aside from DOCS style
annotations inside the code itself.
But then the current anarchy of documents scattered around makes you
realize finding relevant documentation is hard. You never know if you
have found all relevant documentation there is to the particular module,
etc. after google search, say.

That's when "Forget it" makes sense :-(

Of course, if someone points out relevant docs in an exact manner
showing URLs and such, that is another story.
But I wonder, though, whether OTHER contributors could find THOSE
DOCUMENTS on their own in the future. I bet they don't most of the times.

The result is the almost document-less (in practical terms)
hard-to-modify C-C source tree IMHO.

I suspect M-C is not that different in terms of document-friendliness
given some arcane bugs that were not fixed for many years.

The reliance of web tools for documentation is a little overblown.
Relevant documents should live near the source code.
Just creating a DOC directory for each module will go a long way
- to consolidate the docs in a single place (for a module. We can
obviously have many DOC directory for each module.)
- to make the access to the docs easy, etc.


Success of linux is partly attributable to the in-tree documentatio.
Every single important API and design documents for modules (if any for
the latter case) lives in Documentations directory and that was great
for contributors, especially driver developers.
If major changes take place, it is obvious where to put a new document
or whether an existing document in THAT PLACE needs to be changed.
To be frank, rather than relying on a google search with many noisy
hits, I would prefer to run "grep keyword" under a single Documentations
tree. Of course, in this case, no hits means there is NO DOCUMENTATION
at all. That is life.
With mozilla's source code tree, one never knows if one's google search
failed to produce a link to an unknown important document.

Just a perspective from an occasional patch submitter.

TIA

[Nicholas Nethercote]

There are multiple kinds of docs. One group that I've done a lot of work on is the collection of pages under https://developer.mozilla.org/en-US/docs/Mozilla/Performance. I'm not sure if in-tree would be better for those or not, but they're certainly in a different category to API docs, for example.

Nick

_______________________________________________
dev-builds mailing list
dev-b...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds

[Ehsan Akhgari]

On 06/21/2017 12:04 AM, Nicholas Nethercote wrote:
> There are multiple kinds of docs. One group that I've done a lot of > work on is the collection of pages under >
https://developer.mozilla.org/en-US/docs/Mozilla/Performance. I'm not >
sure if in-tree would be better for those or not, but they're >
certainly in a different category to API docs, for example.

Right, and even depending on the kind of API docs, if you want to exceed
beyond a paragraph or two, keeping the API docs next to the code quickly
loses all of its theoretical benefits. For example, think about how
we'd fit something like
<https://developer.mozilla.org/en-US/docs/Web/API/Background_Tasks_API>
next to
<https://searchfox.org/mozilla-central/rev/714606a8145636d93b116943d3a65a6a49d2acf8/dom/webidl/Window.webidl#495>.

But the discoverability issue you brought up is a real issue, and is a
very easy one to solve for out of tree docs incidentally with a simple
moz.build annotation similar to BUG_COMPONENT, I think.

> On Wed, Jun 21, 2017 at 12:50 PM, ISHIKAWA,chiaki > <ishi...@yk.rim.or.jp <mailto:ishi...@yk.rim.or.jp>> wrote: > > On

2017/06/21 2:19, Ehsan Akhgari wrote: > > On 06/20/2017 12:55 PM,
Benjamin Smedberg wrote: > > > > On Tue, Jun 20, 2017 at 12:40 PM, Ehsan
Akhgari > <ehsan....@gmail.com <mailto:ehsan....@gmail.com> >

<mailto:ehsan....@gmail.com <mailto:ehsan....@gmail.com>>> >

<http://gecko.readthedocs.io> > <http://gecko.readthedocs.io>. There is

a taskcluster job which > reports to the mozilla-central tree which does
this automatically, > and it turns red on failure for monitoring. > > As
far as I know, docs from the other trees do not get automatically >
published anywhere, and we don't currently host archived release >
versions of the docs. > > To preview local edits, run `./mach doc` which
will do local > generation > > Great tips, thank you! > > Currently we
don't have good notifications for changes to any file > in-tree (at
least that I know of). This is one of the phabricator > features that I
am most looking forward to! > > Yeah I am also looking for ward to this
feature, more for regular > code review though. :-) >

_______________________________________________ dev-builds mailing >
list dev-b...@lists.mozilla.org >

<mailto:dev-b...@lists.mozilla.org> >
https://lists.mozilla.org/listinfo/dev-builds >
<https://lists.mozilla.org/listinfo/dev-builds> > > > >

_______________________________________________ dev-builds mailing >
list dev-b...@lists.mozilla.org >

[Gregory Szorc]

I completely agree with you here. If this were in place and we could accept contributions through GitHub Pull Requests, then someone could fork via the GitHub Web UI, edit and commit directly in their browser, and we're *almost* on the same level as a wiki in terms of workflow difficulty. You could practically bridge the entire difference by hosting a wiki-like interface that makes calls to the appropriate GitHub APIs with properly formed commit messages, etc behind the scenes. But that's just icing on the cake.

[ISHIKAWA,chiaki]

(I am posting a response to Ehsan's message which somehow was garbled
and he kindly corrected in a personal exchange.)


On 2017/06/22 6:10, Ehsan Akhgari wrote:
> Oops, not sure how that happened, sorry about that! But that formatting
> corruption seemed to have happened in the text I quoted. Here is the newly
> added text in my email:

>
> Right, and even depending on the kind of API docs, if you want to exceed
> beyond a paragraph or two, keeping the API docs next to the code quickly
> loses all of its theoretical benefits. For example, think about how we'd
> fit something like <https://developer.mozilla.org
> /en-US/docs/Web/API/Background_Tasks_API> next to <
> https://searchfox.org/mozilla-central/rev/714606a8145636d93
> b116943d3a65a6a49d2acf8/dom/webidl/Window.webidl#495>.

Well, I am not advocating the direct inclusion of complex documents in
.idl file or C++/Java/etc. source files.
(Is this implied in the above paragraph. Sorry my native language is NOT
English and so sometimes, I lose the nuances.)

On the other hand, if we can keep the original document files in a well
known fixed directory, say, "Documentation" directory, e.g, in this
case, m-c/dom/webidl/Documentation or
m-c/dom/Documentation, it will be much easier for occasional
contributors to figure out where the related documentation to the source
code they are modifying exists IF ANY.
BTW I wonder if each file pointed by an entry in
https://developer.mozilla.org/en-US/docs/Web/API/Background_Tasks_API
has been generated by javadoc or other documentation tools form the
annotation in source files themselves(?) They look so, but I am not sure.

In the case of Linux, there is a Documentation directory near the top of
the source tree and the directory hierarchy under this "Documenation"
reflects the module hierarchy of the real source tree.
Thus you can find SCSI subsystem documents (especially related to driver
files) below
Documentation/scsi

Documentations applicable the kernel as a whole often resides directly
below Documentation directory.

My point is that a simple command below will find ALL the "official"
linux documentation about a keyword. If there is no hit, then there is NONE.

find /usr/src/linux-source-4.9/Documentations -type f -print | xargs
egrep -i searched_keyword

The documentation files are plain text.
(There ARE a few exceptions. I found Documentation/sound/alsa/seq_oss.html
with the following command.
find Documentation/ -name "*.html" -print

I also found a DocBook stylesheet under this directory with
find Documentation/ -name "*.x*" -print
Documentation/DocBook/stylesheet.xsl

>
> But the discoverability issue you brought up is a real issue, and is a very
> easy one to solve for out of tree docs incidentally with a simple moz.build
> annotation similar to BUG_COMPONENT, I think.
>

I am not sure here.
Are you advocating a keyword value pair in the document to make it easy
to search by Google?

I want to make it easy to access the documentation even without such
trick by creating a single place where such documents reside.
(Yes, this directory could be large. But so is the rest of the source
tree. In this sense, the documentation directory does not have to be
within M-C/C-C, etc. It could be D-C directory with its own repository.
Making it easy to find by having a SINGLE repository is an idea.

In linux's case, that is "Documentation" directory.
For official documents, that is it. It happens to be inside the source
tree as well under same git management (I think.)

In the case of linux, there can be other tutorials written for mortals,
but we can always check the Documentation directory for any final word
from module maintainers, driver writers and such.
I think this has helped the contribution from volunteers.

I wonder if we can consolidate all the documentation pieces in one
directory/repository (or be part of the source tree if it is tenable).

Again this is a perspective from an occasional patch contributor who
always cringe at the lack of overall design documents of C-C TB internal
(this can live in for example C-C/mailnews/Documentation/ top-level IMHO).

My position is that all the bugzilla entries will pale with such
documents MOST the times in terms of accessibility. I have no idea how
many times I search bugzilla for certain information. It would have been
vastly superior that such knowledge is condensed over the years by
someone who need it into documentation files under Documentation
directory and kept up-to-date.
[And it has been done in the case of Linux.]

Plain text works, BTW.

If we need contributors, we should make it easy for contributors to
learn the code. I suspect I should propose this documentation approach
to C-C TB maintainers who need every help they can find to court volunteers.
(Adding cc: dev-apps-t...@lists.mizilla.org )


TIA

> On Wed, Jun 21, 2017 at 3:42 PM, ISHIKAWA,chiaki <ishi...@yk.rim.or.jp>
> wrote:
>
>> Dear Ehsan,
>>
>> Something went wrong with the formating of your e-mail post.
>>
>> Would you like to re-post this if your local copy retains the sane
>> formating somehow?
>>
>> Chiaki Ishikawa

>>
>>
>> On 2017/06/21 23:49, Ehsan Akhgari wrote:
>>
>>> On 06/21/2017 12:04 AM, Nicholas Nethercote wrote:
>>>
>>>> There are multiple kinds of docs. One group that I've done a lot of >
>>>> work on is the collection of pages under >
>>>>
>>> https://developer.mozilla.org/en-US/docs/Mozilla/Performance. I'm not >
>>> sure if in-tree would be better for those or not, but they're > certainly
>>> in a different category to API docs, for example.
>>> Right, and even depending on the kind of API docs, if you want to exceed
>>> beyond a paragraph or two, keeping the API docs next to the code quickly
>>> loses all of its theoretical benefits. For example, think about how we'd
>>> fit something like <https://developer.mozilla.org
>>> /en-US/docs/Web/API/Background_Tasks_API> next to <
>>> https://searchfox.org/mozilla-central/rev/714606a8145636d93
>>> b116943d3a65a6a49d2acf8/dom/webidl/Window.webidl#495>.
>>>
>>> But the discoverability issue you brought up is a real issue, and is a
>>> very easy one to solve for out of tree docs incidentally with a simple
>>> moz.build annotation similar to BUG_COMPONENT, I think.
>>>

[I removed the incorrectly formatted my earlier comment and replaced it
with a correctly formatted one. Thunderbird bug? Hmm...
Hope this shows up correctly. I am using TB myself.]

> On 2017/06/21 2:19, Ehsan Akhgari wrote:
>> On 06/20/2017 12:55 PM, Benjamin Smedberg wrote:
>>>
>>>
>>> On Tue, Jun 20, 2017 at 12:40 PM, Ehsan Akhgari <ehsan....@gmail.com <mailto:ehsan....@gmail.com>> wrote:
>>>
>>> On 06/20/2017 11:21 AM, Benjamin Smedberg wrote:
>>>
>>> Coming in late to this thread, but I'd like to strongly
>>> support moving our Mozilla internals documentation in the tree.
>>>
>>> It seems that the rest of your post is mostly advocating that we
>>> should write documentation, which is hard to argue with. For the
>>> record it's not clear to me how you connect that to it being a
>>> better choice for the said documentation to live in the tree.
>>> (Since you mentioned the anecdotal successful example of telemetry
>>> ping documentation, let me counter it with the anecdotal
>>> successful example of WebIDL bindings documentation which lives
>>> out of the tree:
>>> https://developer.mozilla.org/en-US/docs/Mozilla/WebIDL_bindings

>>> <https://developer.mozilla.org/en-US/docs/Mozilla/WebIDL_bindings>.)
>>>
>>>
>>> I'll put nuance on this. I do think we should be writing docs, but I think it's much more important that we're maintaining and deleting existing docs as the code changes or becomes irrelevant.
>> Something that a wiki is much more suitable for.

>>> That's why I think the choice of in-tree versus external tool is important and not merely a tooling choice. Documentation that is in the tree can/should be reviewed as the code changes. Partly because it is in the tree, it maps to Mozilla module ownership and module owners can exercise ownership over their slice of documentation explicitly.
>> This is the part that I don't understand. You are implying that by not having some text file containing the documentation inside the tree we can't enforce review over documentation, and that they can't be updated alongside code changes. I chose the example of WebIDL bindings documentation very consciously to demonstrate that these are false assertions. That example clearly demonstrates that reviewers can (and certainly do) ask patch authors to fix the documentation when they make code changes, such documentation fixes are reviewed by the right people and the resulting docs stay in good shape.
>>
>
> I am a volunteer who occasionally submit patches to C-C TB.
>
> In-tree documentation is much easier for volunteers to work with.
> The documentation is much closer to the code we are modifying.
>
> Out of tree docs are much harder to deal with.
> Just adding a few (actually more) keystrokes to access them make it rather difficult, or more to the point TIME-CONSUMING to work with.
>
> For unpaid volunteers (and I assume even for paid developers), in-tree documentation is much easier to work with in terms of overhead of JUST checking WHERE on earth the relevant documentation exists (IF IT EXISTS AT ALL) and then out-of-tree documentation in an other system means we still need work with DIFFERENT set of rules that exist for accountability for modification.
>
> Given these overhead, out-of-tree documentations are basically something volunteer patch submitters won't be able to work with: I suspect "Just forget it" would be what many volunteer submitters would feel after a while.
>
> Initially, when people start to submit patches, they may think what they might need to do with the documentation aside from DOCS style annotations inside the code itself.
> But then the current anarchy of documents scattered around makes you realize finding relevant documentation is hard. You never know if you have found all relevant documentation there is to the particular module, etc. after google search, say.
>
> That's when "Forget it" makes sense
>

> Of course, if someone points out relevant docs in an exact manner showing URLs and such, that is another story.
> But I wonder, though, whether OTHER contributors could find THOSE DOCUMENTS on their own in the future. I bet they don't most of the times.
>
> The result is the almost document-less (in practical terms)
> hard-to-modify C-C source tree IMHO.
>
> I suspect M-C is not that different in terms of document-friendliness given some arcane bugs that were not fixed for many years.
>
> The reliance of web tools for documentation is a little overblown.
> Relevant documents should live near the source code.
> Just creating a DOC directory for each module will go a long way
> - to consolidate the docs in a single place (for a module. We can obviously have many DOC directory for each module.)
> - to make the access to the docs easy, etc.
>
>
> Success of linux is partly attributable to the in-tree documentatio. Every single important API and design documents for modules (if any for the latter case) lives in Documentations directory and that was great for contributors, especially driver developers.
> If major changes take place, it is obvious where to put a new document or whether an existing document in THAT PLACE needs to be changed.
> To be frank, rather than relying on a google search with many noisy hits, I would prefer to run "grep keyword" under a single Documentations tree. Of course, in this case, no hits means there is NO DOCUMENTATION at all. That is life.
> With mozilla's source code tree, one never knows if one's google search failed to produce a link to an unknown important document.
>
> Just a perspective from an occasional patch submitter.
>
> TIA
>

>> The flip side of the coin is that collaborating on writing articles (which documentation writing is ultimately is all about) is just not that great of an experience when you approach it as rigidly as collaborating on writing computer code. There is a reason why people have invented wikis, etherpads, and software like gdocs! They provide a better model for collaborating on writing articles. It is true that when our only tool is a DVCS package, all of our problems start to look like nails. But they don't have to. (And FWIW I'm gonna give arguing over this a rest if you won't agree, since this is purely subjective and I suspect at the end of the day we may not be able to convince each other here!)

>>> As docs land to mozilla-central they will automatically update gecko.readthedocs.io <http://gecko.readthedocs.io>. There is a taskcluster job which reports to the mozilla-central tree which does this automatically, and it turns red on failure for monitoring.

>>>
>>> As far as I know, docs from the other trees do not get automatically published anywhere, and we don't currently host archived release versions of the docs.
>>>
>>> To preview local edits, run `./mach doc` which will do local generation
>> Great tips, thank you!
>>> Currently we don't have good notifications for changes to any file in-tree (at least that I know of). This is one of the phabricator features that I am most looking forward to!
>> Yeah I am also looking for ward to this feature, more for regular code review though.

[Ehsan Akhgari]

On 06/22/2017 12:11 AM, ISHIKAWA,chiaki wrote:
>
> (I am posting a response to Ehsan's message which somehow was garbled
> and he kindly corrected in a personal exchange.)
>
>
> On 2017/06/22 6:10, Ehsan Akhgari wrote:
>> Oops, not sure how that happened, sorry about that! But that formatting
>> corruption seemed to have happened in the text I quoted. Here is the
>> newly
>> added text in my email:
>>
>> Right, and even depending on the kind of API docs, if you want to exceed
>> beyond a paragraph or two, keeping the API docs next to the code quickly
>> loses all of its theoretical benefits. For example, think about how
>> we'd
>> fit something like <https://developer.mozilla.org
>> /en-US/docs/Web/API/Background_Tasks_API> next to <
>> https://searchfox.org/mozilla-central/rev/714606a8145636d93
>> b116943d3a65a6a49d2acf8/dom/webidl/Window.webidl#495>.
>
> Well, I am not advocating the direct inclusion of complex documents in
> .idl file or C++/Java/etc. source files.
> (Is this implied in the above paragraph. Sorry my native language is
> NOT English and so sometimes, I lose the nuances.)

Oh, I see. I guess I read too much into "close to code" in your email!
Sorry about that. :-)

Now that I read it again, it doesn't really advocate that at all.

> On the other hand, if we can keep the original document files in a
> well known fixed directory, say, "Documentation" directory, e.g, in
> this case, m-c/dom/webidl/Documentation or
> m-c/dom/Documentation, it will be much easier for occasional
> contributors to figure out where the related documentation to the
> source code they are modifying exists IF ANY.
> BTW I wonder if each file pointed by an entry in
> https://developer.mozilla.org/en-US/docs/Web/API/Background_Tasks_API
> has been generated by javadoc or other documentation tools form the
> annotation in source files themselves(?) They look so, but I am not sure.

(No pretty sure they aren't.)

> In the case of Linux, there is a Documentation directory near the top
> of the source tree and the directory hierarchy under this
> "Documenation" reflects the module hierarchy of the real source tree.
> Thus you can find SCSI subsystem documents (especially related to
> driver files) below
> Documentation/scsi
>
> Documentations applicable the kernel as a whole often resides directly
> below Documentation directory.
>
> My point is that a simple command below will find ALL the "official"
> linux documentation about a keyword. If there is no hit, then there is
> NONE.
>
> find /usr/src/linux-source-4.9/Documentations -type f -print | xargs
> egrep -i searched_keyword
>
> The documentation files are plain text.
> (There ARE a few exceptions. I found
> Documentation/sound/alsa/seq_oss.html
> with the following command.
> find Documentation/ -name "*.html" -print
>
> I also found a DocBook stylesheet under this directory with
> find Documentation/ -name "*.x*" -print
> Documentation/DocBook/stylesheet.xsl

I think the Linux model of in-tree documentation still has the usability
issue of not being very user-friendly for writing documentation (I
personally find the current MDN wiki a very nice documentation editing
UI, but this is subjective, and I understand if some people prefer plain
text files here), but I agree that it does provide some general benefits
around making the documentation more easily searchable and navigatable
locally.

>>
>> But the discoverability issue you brought up is a real issue, and is
>> a very
>> easy one to solve for out of tree docs incidentally with a simple
>> moz.build
>> annotation similar to BUG_COMPONENT, I think.
>>
>
> I am not sure here.
> Are you advocating a keyword value pair in the document to make it
> easy to search by Google?

No, something to make it easier for people to find the relevant
documentation on the web by documenting the URL to the documentation in
a consistent way.

> I want to make it easy to access the documentation even without such
> trick by creating a single place where such documents reside.
> (Yes, this directory could be large. But so is the rest of the source
> tree. In this sense, the documentation directory does not have to be
> within M-C/C-C, etc. It could be D-C directory with its own repository.
> Making it easy to find by having a SINGLE repository is an idea.
>
> In linux's case, that is "Documentation" directory.
> For official documents, that is it. It happens to be inside the source
> tree as well under same git management (I think.)
>
> In the case of linux, there can be other tutorials written for
> mortals, but we can always check the Documentation directory for any
> final word from module maintainers, driver writers and such.
> I think this has helped the contribution from volunteers.
>
> I wonder if we can consolidate all the documentation pieces in one
> directory/repository (or be part of the source tree if it is tenable).

FWIW the Linux model has some nice aspects to it, but our in-tree
documentation model is more geared towards Sphinx documentation
generation, so it's fairly different. I don't really know how to
balance out my desire for a wiki like editing interface with yours of
having all of the documentation in one place, but at any rate we
currently are discussing three alternatives it seems. I think
unsurprisingly different people prefer different solutions here...

Cheers,
Ehsan

>>> code review though. _______________________________________________

[Gregory Szorc]

On Wed, Jun 21, 2017 at 9:11 PM, ISHIKAWA,chiaki <ishi...@yk.rim.or.jp> wrote:

(I am posting a response to Ehsan's message which somehow was garbled and he kindly corrected in a personal exchange.)


On 2017/06/22 6:10, Ehsan Akhgari wrote:

Oops, not sure how that happened, sorry about that!  But that formatting
corruption seemed to have happened in the text I quoted.  Here is the newly
added text in my email:

Right, and even depending on the kind of API docs, if you want to exceed
beyond a paragraph or two, keeping the API docs next to the code quickly
loses all of its theoretical benefits.  For example, think about how we'd
fit something like <https://developer.mozilla.org
/en-US/docs/Web/API/Background_Tasks_API> next to <
https://searchfox.org/mozilla-central/rev/714606a8145636d93
b116943d3a65a6a49d2acf8/dom/webidl/Window.webidl#495>.

Well, I am not advocating the direct inclusion of complex documents in .idl file or C++/Java/etc. source files.
(Is this implied in the above paragraph. Sorry my native language is NOT English and so sometimes, I lose the nuances.)

On the other hand, if we can keep the original document files in a well known fixed directory, say, "Documentation" directory, e.g, in this case, m-c/dom/webidl/Documentation or
m-c/dom/Documentation, it will be much easier for occasional contributors to figure out where the related documentation to the source code they are modifying exists IF ANY.
BTW I wonder if each file pointed by an entry in
https://developer.mozilla.org/en-US/docs/Web/API/Background_Tasks_API
has been generated by javadoc or other documentation tools form the annotation in source files themselves(?) They look so, but I am not sure.

We already kinda have this convention. The SPHINX_TREES moz.build variable allows you to define directories containing docs (https://gecko.readthedocs.io/en/latest/#adding-documentation) and the convention seems to be a "docs" directory (https://dxr.mozilla.org/mozilla-central/search?q=SPHINX_TREES&case=true). (This ability to aggregate files from multiple directories was something I added to the in-tree Sphinx docs because I anticipated people wanting the docs to live closer to the components they are documenting.)

Unfortunately, I don't think a unified docs directory for all of mozilla-central is tenable. We've historically thought of mozilla-central as one large project. In reality, there are multiple projects inside. I think we need to get better about formalizing boundaries between components (including their documentation) because that will make it easier to leverage sparse checkouts, builds/scheduling of specific components, etc. What we can do is improve the tooling for findings docs. I think something like `mach doc --list` or even showing the sources of the docs in the generated docs would go a long way.

 

In the case of Linux, there is a Documentation directory near the top of the source tree and the directory hierarchy under this "Documenation" reflects the module hierarchy of the real source tree.
Thus you can find SCSI subsystem documents (especially related to driver files) below
Documentation/scsi

Documentations applicable the kernel as a whole often resides directly below Documentation directory.

My point is that a simple command below will find ALL the "official" linux documentation about a keyword. If there is no hit, then there is NONE.

find /usr/src/linux-source-4.9/Documentations -type f -print | xargs egrep -i searched_keyword

The documentation files are plain text.
(There ARE a few exceptions. I found Documentation/sound/alsa/seq_oss.html
with the following command.
 find Documentation/ -name "*.html" -print

I also found a DocBook stylesheet under this directory with
find Documentation/ -name "*.x*" -print
Documentation/DocBook/stylesheet.xsl

But the discoverability issue you brought up is a real issue, and is a very
easy one to solve for out of tree docs incidentally with a simple moz.build
annotation similar to BUG_COMPONENT, I think.

I am not sure here.
Are you advocating a keyword value pair in the document to make it easy to search by Google?

I want to make it easy to access the documentation even without such trick by creating a single place where such documents reside.
(Yes, this directory could be large. But so is the rest of the source tree. In this sense, the documentation directory does not have to be within M-C/C-C, etc. It could be D-C directory with its own repository.
Making it easy to find by having a SINGLE repository is an idea.

In linux's case, that is "Documentation" directory.
For official documents, that is it. It happens to be inside the source tree as well under same git management (I think.)

In the case of linux, there can be other tutorials written for mortals, but we can always check the Documentation directory for any final word from module maintainers, driver writers and such.
I think this has helped the contribution from volunteers.

I wonder if we can consolidate all the documentation pieces in one directory/repository (or be part of the source tree if it is tenable).

Yeah I am also looking for ward to this feature, more for regular code review though. _______________________________________________

dev-builds mailing list
dev-b...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds