proposal: freeze 0.4 soon; short, intense 0.5 "Arraymageddon" release

[Stefan Karpinski]

There are several important breaking changes planned for arrays – #3701, #4774, #7568, among others. Several of these have been slated to go into the 0.4 release, which is getting a bit long in the tooth and already has plenty of material. So I'd like to propose that we do the following:

• jettison the planned array changes from 0.4
• feature freeze 0.4 soon – like in a week or so, release around JuliaCon
  
• focus on array changes for a very short 0.5 release – around october
• postpone the remaining major non-array changes to a 0.6 release

This plan allows us to wrap up 0.4 sooner rather than later. It also allows us to do do all of the things that are likely to break array-heavy code bases all at once in a fairly short period of time, which seems like it will minimize the pain caused by those changes. I don't think that 0.5 needs to be just array changes (if anything else gets changed in that timeframe, that's fine), but non-array changes shouldn't hold up the release. The 0.5 release will be known as "Arraymageddon", which looks something like this:

Thoughts?

[Elliot Saba]

Do we have a list of any non-array related changes that absolutely, positively must make it into 0.4?  I would also point out that random segfaults and stack errors have been happening at least once a day on the buildbots, (Example 1, Example 2) so once we call a feature freeze and tag an RC, I'll do something like run the test suites 10 times on all the buildbots to make sure that things are actually well and truly "good".

-E

[Scott Jones]

What about getting all of the Unicode related bug fixes in #11004 in?

[Tim Holy]

On Tuesday, May 26, 2015 03:15:13 PM Elliot Saba wrote:
> Do we have a list of any non-array related changes that absolutely,
> positively must make it into 0.4?

Yes, and it has exactly one item on it:
- Package precompilation (https://github.com/JuliaLang/julia/pull/8745)

If we want to prevent small furry rodents the world over from leaping off
cliffs, this absolutely, positively, must make it into 0.4. Absolutely.
Positively. Really. If you want Julia to grow its package ecosystem, and
especially to finally make graphics a routine part of the julia experience,
there's nothing else anywhere close.

Regarding the array changes, I heartily agree with Stefan's plan. There's a
curious point: most folks seem view the important transition as occurring when
A[:,3] returns a SubArray rather than an Array (#3701). In most respects, this
will indeed be the disruptive transition. However, almost all the underlying
components are already in place, and the A[:,3] transition is "just" syntax.
Syntax is important, don't get me wrong; but releasing things as they are now
has the advantage that anyone who wants the benefits of SubArrays can get them
now, but it's "opt-in."

There is one exception: #10525 is seriously amazing and, to a package
developer, game-changing in a way that A[:, 3]/#3701 frankly isn't (even if
the latter will be more disruptive). It will be a real shame if the codegen
bugs can't be worked out in time for 0.4.

Best,
--Tim

> I would also point out that random
> segfaults and stack errors have been happening at least once a day on the
> buildbots, (Example 1

> <http://buildbot.e.ip.saba.us:8010/builders/build_ubuntu12.04-x86/builds/152
> 0/steps/shell_2/logs/stdio>, Example 2
> <http://buildbot.e.ip.saba.us:8010/builders/build_centos7.1-x64/builds/66/st
> eps/shell_2/logs/stdio>) so once we call a feature freeze and tag an RC,

> I'll do something like run the test suites 10 times on all the buildbots to
> make sure that things are actually well and truly "good".
> -E
>
> On Tue, May 26, 2015 at 2:59 PM, Stefan Karpinski <ste...@karpinski.org>
>
> wrote:
> > There are several important breaking changes planned for arrays – #3701

> > <https://github.com/JuliaLang/julia/issues/3701>, #4774
> > <https://github.com/JuliaLang/julia/issues/4774>, #7568
> > <https://github.com/JuliaLang/julia/pull/7568>, among others. Several of

> > these have been slated to go into the 0.4 release, which is getting a bit
> > long in the tooth and already has plenty of material. So I'd like to
> >
> > propose that we do the following:

> > - jettison the planned array changes from 0.4
> > - feature freeze 0.4 soon – like in a week or so, release around
> > JuliaCon
> > - focus on array changes for a very short 0.5 release – around october
> > - postpone the remaining major non-array changes to a 0.6 release

> >
> > This plan allows us to wrap up 0.4 sooner rather than later. It also
> > allows us to do do all of the things that are likely to break array-heavy
> > code bases all at once in a fairly short period of time, which seems like
> > it will minimize the pain caused by those changes. I don't think that 0.5
> > needs to be just array changes (if anything else gets changed in that
> > timeframe, that's fine), but non-array changes shouldn't hold up the
> > release. The 0.5 release will be known as "Arraymageddon", which looks
> > something like this:
> >
> > [image: Inline image 1]

> > <http://artofjosevega.deviantart.com/art/Armageddon-376300293>
> >
> > Thoughts?

[Stefan Karpinski]

On Tue, May 26, 2015 at 6:29 PM, Scott Jones <scott.pa...@gmail.com> wrote:

What about getting all of the Unicode related bug fixes in #11004 in?

Yes, that should be doable.

[Stefan Karpinski]

If Jameson and Jeff think that static precompilation can get done soon, then yes, that would be a great thing to have for 0.4. Non-breaking features that aren't too dangerous could be added in a 0.4.1 release too. I agree that it would be really nice to get #10525 merged, especially since it paves the way for Arraymageddon while not itself being a breaking change. It might even make backporting changes to array behavior via Compat feasible.

[catc...@bromberger.com]

Could we get #9147 implemented in 0.4?

[Stefan Karpinski]

That seems like a good candidate as well.

[Tobi]

Stefan, I agree with your plan but also lets take package precompilation into account if there is any way to hammer the remaining issues out. And while I also look forward to the array changes precompilation will be the absolute game changer. It should get an even more dramatic release name ;-)

Regarding last minute features other than precompilation: Only if one is really sure to not break things. It is really not a big deal if a features arrive later as long as we would release "early and often".

Tobi

[Tony Kelman]

Backporting features is almost always a bad idea. Packages start depending on them without adequately testing and specifying exact lower bounds on julia versions, in ways that cause things to break for innocent people who just do Pkg.update() while staying on an existing release version of Julia.

Master's still pretty unstable in the random-CI-failures and buildbot segfaults sense, so some major bug hunting (and PR reviewing, for that matter - there's a growing backlog and not as much review as there should be) remains to be done. The amount of time between feature freeze and rc1, between rc1 and final, and before the next release will all take longer than you're estimating.

One thing that will absolutely need to be done before releasing 0.4.0, not on Julia itself but on Travis, is https://github.com/JuliaCI/travis-build/issues/2 - otherwise packages will lose the ability to test on 0.3 as soon as 0.4.0 comes out. All it should take is a bit of Ruby to add in testing against specific version numbers of Julia, both patch releases and 0.4-latest, 0.3-latest, etc.

[Christian Peel]

> > Do we have a list of any non-array related changes that absolutely, positively must make it into 0.4?
>
> Yes, and it has exactly one item on it:
> - Package precompilation (https://github.com/JuliaLang/julia/pull/8745)

+1  

This is important and would lower a big barrier to adoption.  I've been counting on it.

--

chris...@ieee.org

[Scott Jones]

Good, once the fixes for UTF-16 and UTF-32 are in, I can quickly fix the ones in UTF-8 as well (wouldn't introduce almost any new code, since the validation routines are in #11004).  There are also some bugs with handling BOM markers that should be addressed.

[Tim Holy]

On Tuesday, May 26, 2015 09:45:34 PM Stefan Karpinski wrote:
> If Jameson and Jeff think that static precompilation can get done soon,
> then yes, that would be a great thing to have for 0.4.

To be clear on my own (very personal) opinion, I would state it differently:
0.4 can go into feature freeze if and only if static precompilation gets
merged. 0.4 has been a year in the making; a feature that many users will rate
as the single most important development of 0.4 (and which should not take
much more time to finish off) is worth the wait.

Best,
--Tim

[Scott Jones]

Even as a relative newcomer to Julia, I'd have to agree, static precompilation would be critical...

[Iain Dunning]

+1 to the absolute essentialness of a package compilation solution, I can't imagine the current situation going on for another year.

In fact, pretty much everything Tim Holy said is exactly what I believe.

[Jiahao Chen]

+1 to package compilation being the highest priority 0.4 issue.

[Tobi]

So there is only two people that can realistically say when 0.4 can happen ...

[Tim Holy]

Well, I've also been trying to cross the remaining items off the list, see
https://github.com/JuliaLang/julia/pull/11426
and I fixed
https://github.com/JuliaLang/julia/issues/8652
explicitly because it was blocking for precompilation. I think it's only fair
to put my money where my mouth is. I was working on that even before this
thread got started, because I was guessing that folks would be interested in
advancing 0.4 to the next phase.

Overall, I'd discourage taking a fatalistic assessment of the schedule.
Presumably with a few days of investment (which admittedly is quite a lot of
time), there are many people, myself included, who could learn enough to fix
the remaining bugs. This is open-source, after all. But if Jameson really can
get back to this next week, it will be most efficient to wait.

--Tim

[Stefan Karpinski]

On Thu, May 28, 2015 at 4:08 AM, Tobi <tobias...@googlemail.com> wrote:

So there is only two people that can realistically say when 0.4 can happen ...

Jeff is on board with the Arraymageddon plan – we discussed it before I started this thread. Jameson is on vacation at the moment, but seems to be checking email now and then, and hasn't objected. He is spearheading the static compilation work, but there are lots of bits that the rest of us can help with (like the issues that Tim linked to). If we make a concerted effort in the next couple of weeks, I suspect that we get this done.

[Stefan Karpinski]

To put it another way, the real point of this idea is to stop thinking of the big array changes as going into 0.4 and to change our collective perspective to wrapping up the 0.4 changes as soon as possible. It seems that this has already been accomplished by this discussion.

[Jutho]

Is #7128 still on the table for v0.4? It fits better with the concatenation / array construction changes that are already in v0.4 than with the big array changes planned for v0.5. I am happy to find some time to take a look at the concatenation functions, but am not capable of making the required changes in the interpreter. Also, it won't be until after next week.

Op donderdag 28 mei 2015 14:40:13 UTC+2 schreef Stefan Karpinski:

[Viral Shah]

+1 to 0.5 as an arraymageddon release.

I would love to see migration to the new documentation system for 0.4, so that packages uniformly use that, and so does Base. I am looking forward to doing help(Package.foo) uniformly across all packages.

-viral

[Stefan Karpinski]

On Thu, May 28, 2015 at 10:48 AM, Jutho <juthoh...@gmail.com> wrote:

Is #7128 still on the table for v0.4? It fits better with the concatenation / array construction changes that are already in v0.4 than with the big array changes planned for v0.5. I am happy to find some time to take a look at the concatenation functions, but am not capable of making the required changes in the interpreter. Also, it won't be until after next week.

Yes, I think we should probably flip the switch on this. The current state of affairs where you can't use [x] to collect a range but you also can't construct a range array is kind of lousy – worst of all worlds. My suspicion is that changing this behavior will cause fairly obvious failures. 

[Scott Jones]

Someone would have to do the work to split out doc's dependence on Markdown in order to make Base be able to use the doc system... I'd tried to do this earlier, but didn't know enough about how to change doc to just store the information for later processing by Markdown...  I wish I knew how... For now, I'm documenting all my changes to Base something like this:

#=

@doc """!

@brief Doxygen with markdown style comments

@param  sillyvariable::MyType

@returns Int64

@throws ArgumentError

""" ->

=#

[Stefan Karpinski]

Yes, to make the docstring thing happen requires these changes:

1. Make markdown parsing lazy. Only save the text at parse time; do markdown parsing on-demand and cache it if necessary.
2. Make top-level string literals in void context the syntax for docstrings in some default configuration.
3. Move documentation for Base into docstrings.

[Tony Kelman]

Has anyone tried building on a new Broadwell CPU yet? Judging by https://github.com/xianyi/OpenBLAS/issues/583 it looks like we'd need the next version of openblas for that to build correctly. We could backport an openblas version bump if necessary, but it's a pretty big dependency to be doing that on.

[Mike Innes]

Getting OT I guess but I'm not sure how much lazy parsing would actually buy us. We could move the doc system to slightly earlier in the bootstrap process, I guess, but you still have the problem of documenting things that come before it anyway.

I figure that for things that can't be documented inline we can just load a separate file (e.g. basedocs.jl) later on. It's no worse than what we have now and still lets you do `?foo` in the repl which is the most important thing IMO.

Anyway – we can probably save the discussion for 0.5. I have an implementation in mind which I can play around with then and we'll see what people think of the tradeoffs.

[Scott Jones]

On Friday, May 29, 2015 at 11:31:43 AM UTC+2, Mike Innes wrote:

Getting OT I guess but I'm not sure how much lazy parsing would actually buy us. We could move the doc system to slightly earlier in the bootstrap process, I guess, but you still have the problem of documenting things that come before it anyway.

I would think that if @doc did nothing more than save the string, no Markdown parsing done, and associated it with the method, type, or whatever, it could be done very very early on... and maybe fancy Markdown features could even be moved out of Base...  I attempted to move doc earlier, but it's dependence on Markdown made that impossible...

and IMO, Markdown brings in a lot of stuff that really shouldn't be in Base... (I think there should only be enough in Base to support ?foo at the REPL on a terminal, everything else should be extended in a package)

I figure that for things that can't be documented inline we can just load a separate file (e.g. basedocs.jl) later on. It's no worse than what we have now and still lets you do `?foo` in the repl which is the most important thing IMO.

It's very hard to keep documentation and code in sync in that fashion, IMO, and having the documentation also right next to the function helps people actually trying to work on Base...

 

Anyway – we can probably save the discussion for 0.5. I have an implementation in mind which I can play around with then and we'll see what people think of the tradeoffs.

I'd love to hear your thoughts on an implementation! 

 

[Mike Innes]

I'm not saying there are no advantages to keeping Markdown.jl outside of Base. But they have to outweigh reading and writing Base's documentation in plain text. We can't even handle code samples properly that way, let alone more advanced things like equations, references and structured docs.

Try `?fft` vs. `@doc(fft) |> Markdown.plain |> println` in the repl and see if you'd honestly prefer the latter.

We should move any further discussion of this to an issue, though (there's probably one around already).

[catc...@bromberger.com]

On Thursday, May 28, 2015 at 8:02:19 PM UTC-7, Tony Kelman wrote:

Has anyone tried building on a new Broadwell CPU yet? Judging by https://github.com/xianyi/OpenBLAS/issues/583 it looks like we'd need the next version of openblas for that to build correctly. We could backport an openblas version bump if necessary, but it's a pretty big dependency to be doing that on.

I've built on Broadwell - specifically, a Macbook Pro 12,1 - and had to specify "override OPENBLAS_TARGET_ARCH=HASWELL" in my Make.user for it to work. It does, indeed, work:

julia> versioninfo()

Julia Version 0.4.0-dev+5008

Commit 0855ec9 (2015-05-26 16:08 UTC)

Platform Info:

  System: Darwin (x86_64-apple-darwin14.3.0)

  CPU: Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz

  WORD_SIZE: 64

  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Prescott)

  LAPACK: libopenblas

  LIBM: libopenlibm

  LLVM: libLLVM-3.3

 

[Stefan Karpinski]

On Fri, May 29, 2015 at 5:30 AM, Mike Innes <mike.j...@gmail.com> wrote:

Getting OT I guess but I'm not sure how much lazy parsing would actually buy us. We could move the doc system to slightly earlier in the bootstrap process, I guess, but you still have the problem of documenting things that come before it anyway.

On Fri, May 29, 2015 at 6:42 AM, Scott Jones <scott.pa...@gmail.com> wrote:

and maybe fancy Markdown features could even be moved out of Base.

Yes, this is the point. Markdown parsing should not be in the Base language. It's fine for it to be something that we ship with for the standard interactive Julia environment, but it should not be part of the language. See https://github.com/JuliaLang/julia/issues/5155.

[Stefan Karpinski]

On Fri, May 29, 2015 at 7:00 AM, Mike Innes <mike.j...@gmail.com> wrote:

I'm not saying there are no advantages to keeping Markdown.jl outside of Base. But they have to outweigh reading and writing Base's documentation in plain text. We can't even handle code samples properly that way, let alone more advanced things like equations, references and structured docs.

I don't understand why Markdown parsing needs to be in Base for Base's documentation to be in Markdown format. All the docstring syntax needs to do is associate chunks of data with objects. Markdown parsing is *only* necessary for presenting that documentation, e.g. at the REPL or rendering it as HTML.

[Tony Kelman]

> Try `?fft` vs. `@doc(fft) |> Markdown.plain |> println` in the repl and see if you'd honestly prefer the latter.

The only difference to me is the former shows a few things in color. I'm unconvinced, especially considering the number of pull requests that have been exclusively fixing bugs in the markdown parser since it was brought into base, relative to its usage so far. Not worth making a markdown parser a dependency of all code written in the language.

[Mike Innes]

Markdown parsing should not be in the Base language. It's fine for it to be something that we ship with for the standard interactive Julia environment ... Markdown parsing is *only* necessary for presenting that documentation, e.g. at the REPL or rendering it as HTML.

Don't get me wrong, I'm all for this. But right now we can't actually ship packages with Base Julia, right? Which means we either keep Markdown in Base or forgo nice presentation by default. Not saying that won't change in future, but it's the situation we have now.

It's worth pointing out that Markdown.jl doesn't strictly only do presentation. If we want to do things like references, handle relative paths to images, embed plots and other graphics etc. that gets a lot harder if you add laziness as a requirement, although it's not strictly impossible. It's nice to be able to handle those things without creating Julia-flavoured markdown.

[Tony Kelman]

We *don't* currently ship packages with Base Julia, there's no reason we *can't* except that no one has figured out the right distribution method to do that. You've been making Juno bundles, haven't you? Don't those include several packages?

[Mike Innes]

I'm unconvinced, especially considering the number of pull requests that have been exclusively fixing bugs in the markdown parser since it was brought into base, relative to its usage so far.

Yeah, the fact that I have no time to improve/maintain it is one of the major downsides, I guess. That will change this summer, but not in time for 0.4, unfortunately.

My thinking is that even though we don't support every markdown feature, it doesn't really matter because whatever you do support is a bonus, and everything else will just show up as readable plain text (by the nature of markdown). If people in general don't agree with that reasoning we should definitely discuss it, because the situation certainly won't be different before 0.4 is released.

That said – have there been that many PRs fixing bugs? The PRs I've been dealing with recently are mostly adding features like tables. If there are a issues going unattended please ping/assign me so I can take a look.

We *don't* currently ship packages with Base Julia, there's no reason we *can't* except that no one has figured out the right distribution method to do that.

"Not saying that won't change in future, but it's the situation we have now."

You've been making Juno bundles, haven't you? Don't those include several packages?

No.

[Tony Kelman]

Apologies, it's been hard to tell bug fixes from feature additions on the Markdown stuff, all I know is Andy Hayden's been contributing up a storm (which is awesome, but probably could have been done at the package stage just as well as in base).

No you haven't been making Juno bundles, or no those don't include any packages?

[Mike Innes]

Absolutely, Andy's contributions have been fantastic. We have a bunch of useful extras like tables and rulers thanks to him, as well as being a lot closer to CommonMark compatibility, which will be nice to have.

Juno comes in a bundle with Julia, but when it first boots it just invokes the package manager (through the usual Pkg.add("Jewel")) to get the necessary packages. I have some ideas for how to bundle packages, but haven't gotten round to playing around with them yet – obviously you want something like that to be really robust, and I'm not as confident with the package manager's internals or the details of handling build steps and binary dependencies etc. It'll be great to have this functionality working, though, not least so we can split up Base a bit.

[Jim Garrison]

On Friday, May 29, 2015 at 8:23:11 AM UTC-7, Mike Innes wrote:

I have some ideas for how to bundle packages, but haven't gotten round to playing around with them yet – obviously you want something like that to be really robust, and I'm not as confident with the package manager's internals or the details of handling build steps and binary dependencies etc. It'll be great to have this functionality working, though, not least so we can split up Base a bit.

I have thoughts package distribution too, but don't want to take this thread completely off topic.  Is there an existing issue for this, or should we create one?  I'd be interested to know more about what you have planned.

[Viral Shah]

What is the recommended course of action for 0.4? Leave things as they are? I would be fine with that since it helps achieve the bigger goal of package documentation. One possibility is to have the Markdown module live by itself, instead of in Base. 

-viral

[Mike Innes]

I would recommend leaving things as they are for now – it seems like the main concern over having Markdown in Base is aesthetic, and stripping it out, implementing the lazy loading of doc strings, making sure it's all tested and robust etc, can't all happen in the next month.

Once I get started in July I'll be able to get that stuff out of the way fairly quickly, though. Call it a short, intense 0.5 "Markdownmageddon" release.

Is there an existing issue for this, or should we create one?

I don't know of one that focusses on the technical details. Feel free to start one and ping me if there isn't.

[Scott Jones]

On Saturday, May 30, 2015 at 7:42:33 PM UTC+2, Mike Innes wrote:

I would recommend leaving things as they are for now – it seems like the main concern over having Markdown in Base is aesthetic, and stripping it out, implementing the lazy loading of doc strings, making sure it's all tested and robust etc, can't all happen in the next month.

Once I get started in July I'll be able to get that stuff out of the way fairly quickly, though. Call it a short, intense 0.5 "Markdownmageddon" release.

Sounds like a very wonderful plan!  +1

[Scott Jones]

I just had an idea, wondered what people might think of it...

Why can't we add, very early on in building the system image, a macro called `@basedoc`, that simply eats the two arguments that @doc would, and just throws them away?

That could be done right away, and would allow code (like mine) that wants to start adding documentation to methods/types in Base, to do so, now, in 0.4, and later, during 0.5 "Markdownmageddon", @basedoc

could be made to do the lazy loading of doc strings, etc., and then all that new Base documentation would magically be available...

On Saturday, May 30, 2015 at 7:42:33 PM UTC+2, Mike Innes wrote:

[Mike Innes]

There's no real need for @basedoc since you can just add a void string and eventually it will work. That said, during 0.4 let's treat the doc system as being for packages and use the manual as the primary source for Base. Otherwise we're going to end up having docs scattered around both systems.

In order to transfer Base to the new system, we need to (1) figure out how to generate basedocs.jl from the manual pages, (2) figure out how to generate the manual pages from the doc system, and (3) move the basedocs.jl docstrings into Base. Technically speaking it's not rocket surgery but it's definitely not going to happen by magic, and I'd like to avoid making it an harder organisational problem than it already is.

[Michael Hatherly]

For the last month or so Docile has had support for the features being
discussed here about docstrings. Namely, strings in void context being
taken as docstrings and lazy markdown parsing.

This isn't limited to packages that are currently importing Docile.
Any package that's imported can have it's docstrings collected by Docile now --
Lazy.jl is a good example of that, hope you don't mind Mike ;).

It also works with Base, though it's currently off by default pending
additional testing. (``Docile.Cache.togglebase()`` turns it on).

There's still some work to be done on the presentation side in Lexicon, but
I figured I'd better mention it here so we don't duplicate effort unnecessarily.

-- Mike

[Michael Hatherly]

For the last month or so Docile has had support for the features being
discussed here about docstrings. Namely, strings in void context being
taken as docstrings and lazy markdown parsing.

This isn't limited to packages that are currently importing Docile.
Any package that's imported can have it's docstrings collected by Docile now --
Lazy.jl is a good example of that, hope you don't mind Mike ;).

It also works with Base, though it's currently off by default pending
additional testing. (``Docile.Cache.togglebase()`` turns it on).

There's still some work to be done on the presentation side in Lexicon, but
I figured I'd better mention it here so we don't duplicate effort unnecessarily.

-- Mike

On Sunday, 31 May 2015 09:05:54 UTC+2, Mike Innes wrote:

[Scott Jones]

On Sunday, May 31, 2015 at 9:05:54 AM UTC+2, Mike Innes wrote:

There's no real need for @basedoc since you can just add a void string and eventually it will work. That said, during 0.4 let's treat the doc system as being for packages and use the manual as the primary source for Base. Otherwise we're going to end up having docs scattered around both systems.

Well, I've added documentation to all the new functions that I wrote for PR #11004, but I had to hide them in #= =#, because @doc and doc""" didn't work until near the very end of building Julia.

When you say, just add a void string, do you mean as the first expression in the function definition, a la Python?  Otherwise, don't you need the @doc string -> or @doc string name macros?

The problem I see with what you say, is that the manual does not document a large part of Base... (and it shouldn't).   This is why I want to have doxygen support added (both to comments, like C/C++/Java and like Python, which also uses the # for comment syntax).

If the new docs can just pick up arbitrary strings in the source, it would be good if ! and < would indicate a doxygen formatted doc string.

(it could produce the information needed for doxygen on one hand, and strip or modify the doxygen commands, so they would produce nicely formatted documentation also.

 

In order to transfer Base to the new system, we need to (1) figure out how to generate basedocs.jl from the manual pages, (2) figure out how to generate the manual pages from the doc system, and (3) move the basedocs.jl docstrings into Base. Technically speaking it's not rocket surgery but it's definitely not going to happen by magic, and I'd like to avoid making it an harder organisational problem than it already is.

Well, I've already got it mostly working, I was going to put in into a WIP or RFC PR, to get comments and advice, when I saw your message...

I don't think it makes things harder, organizationally, I think that the splitting of of parts of strings.jl, and splitting up the setting of a doc string from the one arg form to get the documentation actually makes things clearer.

;

[Tomas Lycken]

Just to be clear here - would https://github.com/JuliaLang/julia/pull/10525 be part of 0.4 or 0.5? If the latter, pretty-pleeease can we make sure that the wait is not so long?

That is about arrays, sure, but really not so much about changing the underlying behavior of arrays as about making it easier for others to gain something from inheriting AbstractArray...

// T

[Stefan Karpinski]

I would like to see if we can get #10525 into 0.4.

[David Anthoff]

I am very worried that having two breaking releases (0.4 and 0.5) in a relatively short time frame (say July/August and then October) will create a real mess in the package landscape. I think realistically it will take weeks, if not months, to move the long tail of packages over to a new version like 0.4, and having something like that happen in short succession twice seems unwise to me. I think it would be a good rule of thumb to have at least 6 months, but ideally more, time between breaking releases, just to not give package developers the impression that they have to constantly keep up with breaking changes in the language.

 

I also think that at this point it is actually really key that we manage to have smooth major version transitions in the package landscape, if we want to convince people that they can use julia for real work.

 

From: juli...@googlegroups.com [mailto:juli...@googlegroups.com] On Behalf Of Stefan Karpinski
Sent: Tuesday, May 26, 2015 3:00 PM
To: juli...@googlegroups.com
Subject: [julia-dev] proposal: freeze 0.4 soon; short, intense 0.5 "Arraymageddon" release

 

There are several important breaking changes planned for arrays – #3701, #4774, #7568, among others. Several of these have been slated to go into the 0.4 release, which is getting a bit long in the tooth and already has plenty of material. So I'd like to propose that we do the following:

• jettison the planned array changes from 0.4
• feature freeze 0.4 soon – like in a week or so, release around JuliaCon
• focus on array changes for a very short 0.5 release – around october
• postpone the remaining major non-array changes to a 0.6 release

This plan allows us to wrap up 0.4 sooner rather than later. It also allows us to do do all of the things that are likely to break array-heavy code bases all at once in a fairly short period of time, which seems like it will minimize the pain caused by those changes. I don't think that 0.5 needs to be just array changes (if anything else gets changed in that timeframe, that's fine), but non-array changes shouldn't hold up the release. The 0.5 release will be known as "Arraymageddon", which looks something like this:

 

 

Thoughts?

[John Myles White]

FWIW, I'd say the opposite: it's better that Julia reach a mature state faster than that people believe that pre-1.0 is stable enough to commit to.

 -- John

<image001.jpg>

 

Thoughts?

[Tobi]

It is a little hypothetical to now plan when 0.5 will happen if its not even clear when 0.4 can be released. 

But I am on the "release early and often" front although I maintain a larger Julia code base. What is IMHO  to be avoided is that release cycles are so long that master gets to a "rolling release" (as we had for 0.3 due to the improved startup time)

Tobi

[Randy Zwitch]

Agreed. After trying to work through using Compat.jl, I'd rather get to where we're going as fast as possible rather than linger.