overview questions about new doc changes (coming with v 0.4)

[jgabri...@gmail.com]

Hi,

Looks like exciting doc changes are afoot with Julia! I'd like to get some more understanding of what's coming. Had a look at some of the github issues tagged "doc", but I'm still missing some basics (note, I'm still quite new to Julia). Questions:

  * Is code from Docile.jl, Lexicon.jl, and Markdown.jl being used / incorporated into Julia proper?

  * Will the new syntax be `doc "..."`, `@doc "..." ->`, or something else?

  * What is `md"Some *text* here`? Will Julia support and/or require that for the new docstrings? If so, what is the benefit of `md"this"` over `"this"`?

  * Regarding the docs currently at <http://docs.julialang.org/en/release-0.3/>, does all of that content currently come only from the contents of julia/doc and below?

  * Will the docstrings in 0.4 be online at, say, http://docs.julialang.org/en/release-0.4/ , integrated with the rendered .rst docs? Or are they intended to be strictly available via the repl? Hm... to avoid duplication, are any files in julia/doc slated to be diced up, reformatted into markdown, and inserted into source as docstrings?

Thanks,
-- John

[jgabri...@gmail.com]

  * Regarding the docs currently at <http://docs.julialang.org/en/release-0.3/>, does all of that content currently come only from the contents of julia/doc and below?

  * Will the docstrings in 0.4 be online at, say, http://docs.julialang.org/en/release-0.4/ , integrated with the rendered .rst docs? Or are they intended to be strictly available via the repl? Hm... to avoid duplication, are any files in julia/doc slated to be diced up, reformatted into markdown, and inserted into source as docstrings?

Is julia/doc/stdlib/base.rst currently a manually-curated catch-all for everything in julia/base which doesn't have it's own dedicated doc?

[ele...@gmail.com]

When will the documentation system be documented?

[Ivar Nesje]

> Hi,

Hello.

> Looks like exciting doc changes are afoot with Julia! I'd like to get some more understanding of what's coming. Had a look at some of the github issues tagged "doc", but I'm still missing some basics (note, I'm still quite new to Julia). Questions:

* Is code from Docile.jl, Lexicon.jl, and Markdown.jl being used / incorporated into Julia proper?

Yes.

* Will the new syntax be `doc "..."`, `@doc "..." ->`, or something else?

The -> is probably going away, but final syntax is not yet set in stone (nor in code).

* What is `md"Some *text* here`? Will Julia support and/or require that for the new docstrings? If so, what is the benefit of `md"this"` over `"this"`?

The benefit is that `md"this"` has an explicit format, so that we can have more formats in the future. The value has been discussed and you can have different formats by other means. I like the way it makes markdown optional, but others want to save two characters to type.

* Regarding the docs currently at <http://docs.julialang.org/en/release-0.3/>, does all of that content currently come only from the contents of julia/doc and below?

Yes

* Will the docstrings in 0.4 be online at, say, http://docs.julialang.org/en/release-0.4/ , integrated with the rendered .rst docs? Or are they intended to be strictly available via the repl? Hm... to avoid duplication, are any files in julia/doc slated to be diced up, reformatted into markdown, and inserted into source as docstrings?

Maybe, but it's hard to predict the future. Many files in Base are too long already, and detailed docs will not make them shorter. For huge codebases, I think it makes sense to fit as much code as possible on a screen, and search in separate docs if I need to know more about a function.

Thanks,
-- John

[Mike Innes]

Actually the @doc macro will still interpret plain strings as markdown by default. There are some caveats with escaping that make it good practice to write doc"" anyway, but those will go away once the parser changes are implemented.

I'm in the process of writing documentation documentation, so the manual should be up to date reasonably soon.

[ele...@gmail.com]

Since the @doc is 0.4, is it possible to backport a "do nothing" version that will allow documented code to still compile in 0.3?

Cheers

Lex

[Mike Innes]

It's not really that worthwhile since (a) you can use Docile and (b) the future syntax

"""

foo

"""

foo() ...

is backwards-compatible already. I just use that.

[ele...@gmail.com]

On Wednesday, December 17, 2014 8:41:00 AM UTC+10, Mike Innes wrote:

It's not really that worthwhile since (a) you can use Docile and (b) the future syntax

"""

foo

"""

foo() ...

is backwards-compatible already. I just use that.

Oh, ok, I thought an @doc macro was needed in 0.4 https://github.com/JuliaLang/julia/blob/d0a951ccb3a7ebae7909665f4445a019f2ee54a1/base/basedocs.jl.

Cheers

Lex

[Mike Innes]

It is needed if you want the docs to show up in the repl etc. It's just that the plain string won't break anything (it won't do anything, either, for now).

[ele...@gmail.com]

So if otherwise unchanged code is documented with @doc (which it will be, who doesn't want it to show in the repl :) then it won't compile on 0.3?

If it won't compile it makes maintaining backward compatibility harder, and its hard enough between 0.4 and 0.3 already.

[jgabri...@gmail.com]

On Tuesday, December 16, 2014 4:55:39 PM UTC-5, Ivar Nesje wrote:

  * Is code from Docile.jl, Lexicon.jl, and Markdown.jl being used / incorporated into Julia proper?

Yes.

  * Will the new syntax be `doc "..."`, `@doc "..." ->`, or something else?

The -> is probably going away, but final syntax is not yet set in stone (nor in code).

  * What is `md"Some *text* here`? Will Julia support and/or require that for the new docstrings? If so, what is the benefit of `md"this"` over `"this"`?

The benefit is that `md"this"` has an explicit format, so that we can have more formats in the future. The value has been discussed and you can have different formats by other means. I like the way it makes markdown optional, but others want to save two characters to type.

  * Regarding the docs currently at <http://docs.julialang.org/en/release-0.3/>, does all of that content currently come only from the contents of julia/doc and below?

Yes

  * Will the docstrings in 0.4 be online at, say, http://docs.julialang.org/en/release-0.4/ , integrated with the rendered .rst docs? Or are they intended to be strictly available via the repl? Hm... to avoid duplication, are any files in julia/doc slated to be diced up, reformatted into markdown, and inserted into source as docstrings?

Maybe, but it's hard to predict the future. Many files in Base are too long already, and detailed docs will not make them shorter. For huge codebases, I think it makes sense to fit as much code as possible on a screen, and search in separate docs if I need to know more about a function.

Thanks, Ivar.

Regarding concerns of longish docstrings being cumbersome to edit around, one solution might be to use your editor's code-folding to by-default hide docstrings. Another might be to support having docstrings in separate files (e.g., foo.jl and an optional corresponding foo.jldoc for detailed docstrings).

-- John

[Michael Hatherly]

Hi John,

Another might be to support having docstrings in separate files (e.g., foo.jl and an optional corresponding foo.jldoc for detailed docstrings).

Docile.jl does support this feature already with:

@doc meta(file = "foobar-docs.md") ->
foobar(x) = x

Granted the syntax is slightly bulky, but it does allow arbitrary metadata to be associated with any Julia objects via keyword arguments. This isn’t available in the doc system in Base since it was best to start off with a smaller feature set and extend it later.

Anecdotally, I’ve found that storing long docs in external files to be much more pleasant than wading through them in source files.

— Mike

​

[Tim Holy]

Do you need one file per docstring, or can you pack the docs for many functions
in one file? I think you'd need to be able to do the latter if we want there to
be some kind of automatic correspondence between foo.jl and foo.jldoc.

--Tim

On Wednesday, December 17, 2014 01:23:45 AM Michael Hatherly wrote:
> Hi John,
>
> Another might be to support having docstrings in separate files (e.g.,
> foo.jl and an optional corresponding foo.jldoc for detailed docstrings).
>
> Docile.jl does support this feature already with:
>
> @doc meta(file = "foobar-docs.md") ->
> foobar(x) = x
>
> Granted the syntax is slightly bulky, but it does allow arbitrary metadata

> to be associated with any Julia objects via keyword arguments. This *isn’t*

[Michael Hatherly]

Do you need one file per docstring, or can you pack the docs for many functions in one file?

One file per docstring currently. My thinking was that docstrings that are short enough to warrant packing
several into a single file are probably short enough that they could go directly into the source code. Also,
how would different “subdocstrings” be associated with different methods? Syntax such as:

@doc meta(file = "docs.md#foobar") ->
foobar(x) = x

with a corresponding # foobar in docs.md to link the two together might work.

I’ll probably add this at some point soon.

Some relevant links:

• https://github.com/MichaelHatherly/Docile.jl/issues/48
• https://github.com/MichaelHatherly/Lexicon.jl/issues/18

— Mike

​

[jgabri...@gmail.com]

On Wednesday, December 17, 2014 6:47:03 AM UTC-5, Michael Hatherly wrote:

Do you need one file per docstring, or can you pack the docs for many functions in one file?

One file per docstring currently. My thinking was that docstrings that are short enough to warrant packing

several into a single file are probably short enough that they could go directly into the source code. Also,
how would different “subdocstrings” be associated with different methods? Syntax such as:

@doc meta(file = "docs.md#foobar") ->
foobar(x) = x

with a corresponding # foobar in docs.md to link the two together might work.

Perhaps the convention could be: each foo.jl source file may have a corresponding foo.md file (alongside it, in the source directory), and each thing documented with @doc may have its own H1 in the .md file.

This way, rather than having to explicitly/verbosely say `@doc meta(file = "foo.md#foobar" -> foobar(x) = x` (that is, come up with id's and add them explicitly to the .md file), the tools would know right where to look for external docstrings.

-- John

[Seth]

+1. Please reconsider making a @doc (at least a NOP) for 0.3.x - this way we can start writing repl-printable docstrings that will be useful in 0.4 but not have our code break in earlier versions.

[John Myles White]

This debate seems a little premature to me since the definition of @doc is not totally finished yet and we need to finalize that before anyone should be adding documentation to 0.3 packages.

 -- John

[Tim Holy]

Just add this to your modules:

if VERSION < v"0.4.0-dev"
using Docile
end

and use the subset of @doc common to 0.3 and 0.4.

--Tim

[Seth]

I respectfully disagree. I would agree with you if your statement concluded "before anyone should be adding documentation to ANY packages", but the thing is, @doc is available for use in 0.4 and, while the definition may change, it's merely a matter of break-fix if the format changes so much between now and when it's "ready" that @docstrings fail to compile properly. However, RIGHT NOW, any code written with @doc in a 0.4 environment will simply fail in 0.3, Compat or no Compat.

My suggestion would be to create, if possible, a @doc noop in Compat.jl so that seasoned developers who are using the latest and greatest 0.4 code can start creating documentation now* while not breaking their packages for those users who are on the release version, and folks who are developing on 0.3 can include basic @docstrings that will work as intended on 0.4.

*the reality is that documentation is frequently seen as an irritation for developers, especially once a package is released. Getting them to add @doc after they've moved on to the next thing requiring their attention, and getting end users to update their packages just to get new docstrings, is going to be a bigger challenge than providing the capability at the front end and encouraging its use.

[Jay Weisskopf]

Necrothread after six months. Has the @doc format been finalized and what is the best way to start using it without breaking 0.3?