People (and not just book authors :) often ask - whither 1.0? [Ok,
maybe they don't use 'whither']. The fact remains, some people want a
1.0 designation, and I'm not unwilling, providing we as a community
can come to an understanding as to what that means, the process it
implies, and the work it will require (and who will do it). Here are
some of the relevant issues, IMO:
- Stability/completeness/robustness
This is mostly about - does it work? Is it relatively free of bugs? Is
it free of gaping holes in core functionality? I think Clojure is in a
pretty good place right now, but am obviously biased. This in no way
implies there isn't room for improvement.
- API stability
With the semantic changes of fully lazy sequences behind us, I think
the syntax and semantics of existing functions is largely stable.
- Development process stability
Currently all new work (fixes and enhancements) occurs in trunk.
There's no way to get fixes without also getting enhancements. I think
this is the major missing piece in offering stable numbered releases.
While I've cut a branch for each of the prior two releases, no one has
ever submitted a bugfix patch for either. If people are going to want
to work with a particular release version for an extended period of
time, someone (other than me) will have to produce patches of (only!)
fixes from the trunk for the release branch, and occasionally produce
point releases (1.0.x) from that branch. I'd like to continue to do
the bulk of my work in trunk, without messing anyone up or forcing
everyone to follow along.
- Freedom from change
Numbered releases are most definitely not about absence of change in
general. There are more things I want to add and change, and there
will be for some time. That will keep Clojure a living language. 1.0
or any numbered release can't and won't constitute a promise of no
further change. But there are different kinds of change, changes that
fix bugs and changes that add new capabilities or break existing code.
People need to be able to choose the type of change they can tolerate,
and when to incur it.
- Perception
Obviously, a 1.0 designation impacts perception. I am not interested
in pursuing it just to influence perception, but rather to
(collectively) acknowledge a milestone in usability and stability.
However there may be other perceptions, good/bad or simply wrong (e.g.
that Clojure is "finished"). Will the general perception engendered
by 1.0 be met in the large, or a mismatch?
What does 1.0 mean to you? Are we there yet? Any recommendations for
the organization of the release branches, patch policy etc?
"There's no way to get fixes without also getting enhancements" ----
unless ---- you use a non-linear source control like Git. (please
switch?) :)
Ok no flames please - but since we have switched to Git nearly 2 years
ago we have been blessed with it's abilities to keep a stable branch
"master" and a RC branch "next" going with the freedom to try anything
in the background (topic branches). Branching is easy, cheap and fun
and I create a branch for every new bugfix or feature.
It's a thing of beauty and I could never go back to SVN or CVS. I
would switch to Hg first (badomp bomp *ching*) :) Just my $0.02
On Apr 16, 6:53 am, Rich Hickey <richhic...@gmail.com> wrote:
> Currently all new work (fixes and enhancements) occurs in trunk.
> There's no way to get fixes without also getting enhancements. I think
> this is the major missing piece in offering stable numbered releases.
> While I've cut a branch for each of the prior two releases, no one has
> ever submitted a bugfix patch for either. If people are going to want
> to work with a particular release version for an extended period of
> time, someone (other than me) will have to produce patches of (only!)
> fixes from the trunk for the release branch, and occasionally produce
> point releases (1.0.x) from that branch. I'd like to continue to do
> the bulk of my work in trunk, without messing anyone up or forcing
> everyone to follow along.
Also a benefit of being on Git for contrib would mean I don't have to
pull ClojureCLR and other stuff I don't want into my clone. It would
make it less "kitchen junk drawer".
Another benefit of being on Git is people can fork, fix and send you
pull requests (which you can accept or not at your discretion). It
just encourages easy fork/branch/collaborate.
On Apr 16, 7:33 am, dysinger <dysin...@gmail.com> wrote:
> "There's no way to get fixes without also getting enhancements" ----
> unless ---- you use a non-linear source control like Git. (please
> switch?) :)
> Ok no flames please - but since we have switched to Git nearly 2 years
> ago we have been blessed with it's abilities to keep a stable branch
> "master" and a RC branch "next" going with the freedom to try anything
> in the background (topic branches). Branching is easy, cheap and fun
> and I create a branch for every new bugfix or feature.
> It's a thing of beauty and I could never go back to SVN or CVS. I
> would switch to Hg first (badomp bomp *ching*) :) Just my $0.02
> On Apr 16, 6:53 am, Rich Hickey <richhic...@gmail.com> wrote:
> > - Development process stability
> > Currently all new work (fixes and enhancements) occurs in trunk.
> > There's no way to get fixes without also getting enhancements. I think
> > this is the major missing piece in offering stable numbered releases.
> > While I've cut a branch for each of the prior two releases, no one has
> > ever submitted a bugfix patch for either. If people are going to want
> > to work with a particular release version for an extended period of
> > time, someone (other than me) will have to produce patches of (only!)
> > fixes from the trunk for the release branch, and occasionally produce
> > point releases (1.0.x) from that branch. I'd like to continue to do
> > the bulk of my work in trunk, without messing anyone up or forcing
> > everyone to follow along.
On Apr 16, 12:53 pm, Rich Hickey <richhic...@gmail.com> wrote:
> What does 1.0 mean to you? Are we there yet? Any recommendations for
> the organization of the release branches, patch policy etc?
I would like to see, concurrent with 1.0, some kind of library
management system. As noted before, contrib is already getting
hairy. We have dozens of little libraries floating around on GitHub.
Having you, Rich, maintain the list of libraries on clojure.org is not
sustainable. We don't need a fully-realized CPAN/RubyGem but we need
something that can scale to a few hundred authors. Dependency
management and documentation markup are components of this.
It sounds nice, but I experienced massive amounts of pain trying to get the
eclipse git plugin to work on mac ... eventually punted back to SVN. To me
version control should be well integrated with an editor ... bottom line ...
much more important than the given features of the version control system.
On Thu, Apr 16, 2009 at 10:33 AM, dysinger <dysin...@gmail.com> wrote:
> "There's no way to get fixes without also getting enhancements" ----
> unless ---- you use a non-linear source control like Git. (please
> switch?) :)
> Ok no flames please - but since we have switched to Git nearly 2 years
> ago we have been blessed with it's abilities to keep a stable branch
> "master" and a RC branch "next" going with the freedom to try anything
> in the background (topic branches). Branching is easy, cheap and fun
> and I create a branch for every new bugfix or feature.
> It's a thing of beauty and I could never go back to SVN or CVS. I
> would switch to Hg first (badomp bomp *ching*) :) Just my $0.02
> On Apr 16, 6:53 am, Rich Hickey <richhic...@gmail.com> wrote:
> > - Development process stability
> > Currently all new work (fixes and enhancements) occurs in trunk.
> > There's no way to get fixes without also getting enhancements. I think
> > this is the major missing piece in offering stable numbered releases.
> > While I've cut a branch for each of the prior two releases, no one has
> > ever submitted a bugfix patch for either. If people are going to want
> > to work with a particular release version for an extended period of
> > time, someone (other than me) will have to produce patches of (only!)
> > fixes from the trunk for the release branch, and occasionally produce
> > point releases (1.0.x) from that branch. I'd like to continue to do
> > the bulk of my work in trunk, without messing anyone up or forcing
> > everyone to follow along.
On Thu, Apr 16, 2009 at 10:39 AM, dysinger <dysin...@gmail.com> wrote:
> Also a benefit of being on Git for contrib would mean I don't have to
> pull ClojureCLR and other stuff I don't want into my clone. It would
> make it less "kitchen junk drawer".
> Another benefit of being on Git is people can fork, fix and send you
> pull requests (which you can accept or not at your discretion). It
> just encourages easy fork/branch/collaborate.
making the common case easy is important. I like good branching, too, for
when you need it, but is it the common case, or does it fly in the face of
what we are talking about here?
> On Apr 16, 7:33 am, dysinger <dysin...@gmail.com> wrote:
> > "There's no way to get fixes without also getting enhancements" ----
> > unless ---- you use a non-linear source control like Git. (please
> > switch?) :)
> > Ok no flames please - but since we have switched to Git nearly 2 years
> > ago we have been blessed with it's abilities to keep a stable branch
> > "master" and a RC branch "next" going with the freedom to try anything
> > in the background (topic branches). Branching is easy, cheap and fun
> > and I create a branch for every new bugfix or feature.
> > It's a thing of beauty and I could never go back to SVN or CVS. I
> > would switch to Hg first (badomp bomp *ching*) :) Just my $0.02
> > > Currently all new work (fixes and enhancements) occurs in trunk.
> > > There's no way to get fixes without also getting enhancements. I think
> > > this is the major missing piece in offering stable numbered releases.
> > > While I've cut a branch for each of the prior two releases, no one has
> > > ever submitted a bugfix patch for either. If people are going to want
> > > to work with a particular release version for an extended period of
> > > time, someone (other than me) will have to produce patches of (only!)
> > > fixes from the trunk for the release branch, and occasionally produce
> > > point releases (1.0.x) from that branch. I'd like to continue to do
> > > the bulk of my work in trunk, without messing anyone up or forcing
> > > everyone to follow along.
> What does 1.0 mean to you? Are we there yet? Any recommendations for > the organization of the release branches, patch policy etc?
To me, beside what was already said, it means a deprecation policy. I like Python's. First release after deprecated changes are decided, code works as is but produces warning if you run it under warning mode. Second release, warning will come warning mode or not and third release, the feature is cut.
Ideally, since backward compatibility is a big selling point of Java, I'd like to be able to tell clojure I require say version 1.0 and get the 1.0 behaviour even if I'm running on a newer version. It would enable the language to grow while providing a nice compatibility layer. Unless that's too hard or there is ramifications I'm not seeing.
> Ideally, since backward compatibility is a big selling point of Java.
my view of Java's backward compatibility is that it is kind of a bunch of hot air that restricts the ecosystem from being better. i vastly prefer the fact that .net is willing to make real changes to get real benefits.
> On Apr 16, 12:53 pm, Rich Hickey <richhic...@gmail.com> wrote: >> What does 1.0 mean to you? Are we there yet? Any recommendations for >> the organization of the release branches, patch policy etc?
> I would like to see, concurrent with 1.0, some kind of library > management system. As noted before, contrib is already getting > hairy. We have dozens of little libraries floating around on GitHub. > Having you, Rich, maintain the list of libraries on clojure.org is not > sustainable. We don't need a fully-realized CPAN/RubyGem but we need > something that can scale to a few hundred authors. Dependency > management and documentation markup are components of this.
+1. I worry about the cpan/gem mention, though. We're still in the java world w.r.t. build and packaging, which generally means using something like ivy/maven/svn:externals/git-submodule/etc ahead of one's build process, rather than using cpan/gem/easy_install/etc at deployment time.
That said, I have no concrete suggestion, as we'll always separately pull our projects' dependencies into whatever we happen to be using as a dependency management repo (it's a bummer to not be able to run a build if some third-party repo is down, etc).
Regarding documentation, I remember throwing around ideas in irc some months ago about how to fold documentation from gen-class'ed libs into a library's broader javadoc. That would be a huge boon to those using clojure to build libraries that would be transparently usable by Java developers.
I am all for a standard packaging/build system but what ever it is it
needs to not ignore the 10s of thousands of libraries tucked away in
maven2 repos. Something like Ties w/ compile support would be cool.
Git submodules, SVN externals & Hg forrest won't work either because
everyone uses different source control (and sorry I am not switching
off git :P ).
On Apr 16, 9:10 am, Chas Emerick <cemer...@snowtide.com> wrote:
> On Apr 16, 2009, at 1:56 PM, Stuart Sierra wrote:
> > On Apr 16, 12:53 pm, Rich Hickey <richhic...@gmail.com> wrote:
> >> What does 1.0 mean to you? Are we there yet? Any recommendations for
> >> the organization of the release branches, patch policy etc?
> > I would like to see, concurrent with 1.0, some kind of library
> > management system. As noted before, contrib is already getting
> > hairy. We have dozens of little libraries floating around on GitHub.
> > Having you, Rich, maintain the list of libraries on clojure.org is not
> > sustainable. We don't need a fully-realized CPAN/RubyGem but we need
> > something that can scale to a few hundred authors. Dependency
> > management and documentation markup are components of this.
> +1. I worry about the cpan/gem mention, though. We're still in the
> java world w.r.t. build and packaging, which generally means using
> something like ivy/maven/svn:externals/git-submodule/etc ahead of
> one's build process, rather than using cpan/gem/easy_install/etc at
> deployment time.
> That said, I have no concrete suggestion, as we'll always separately
> pull our projects' dependencies into whatever we happen to be using as
> a dependency management repo (it's a bummer to not be able to run a
> build if some third-party repo is down, etc).
> Regarding documentation, I remember throwing around ideas in irc some
> months ago about how to fold documentation from gen-class'ed libs into
> a library's broader javadoc. That would be a huge boon to those using
> clojure to build libraries that would be transparently usable by Java
> developers.
> my view of Java's backward compatibility is that it is kind of a bunch > of hot air that restricts the ecosystem from being better. i vastly > prefer the fact that .net is willing to make real changes to get real > benefits.
> sincerely. > $0.02
And that requires shoe-horning new stuff in the old framework like using that $ convention for inner classes for instance, I agree.
I did not suggest the language and API should not change and remain forever compatible, only that if possible, instead of doing something like this:
OSGi is becoming the de facto standard for solving the runtime issues
around versioning and classpath management in the standard Java
world. As for development versioning issues, Maven is the de facto
standard.
While I certainly don't think that Clojure 1.0 should have any
dependency on OSGi, I'd personally like to see the libraries evolve
that direction with some library API support. The API support would
likely parallel the OSGi jar manifest declarations. That API could
serve both as build time code to generate the static manifest files
necessary to play in an OSGi container as well as runtime declarations
of services and dynamic dependencies.
On Apr 16, 12:28 pm, Dan <redalas...@gmail.com> wrote:
> > my view of Java's backward compatibility is that it is kind of a bunch
> > of hot air that restricts the ecosystem from being better. i vastly
> > prefer the fact that .net is willing to make real changes to get real
> > benefits.
> > sincerely.
> > $0.02
> And that requires shoe-horning new stuff in the old framework like using
> that $ convention for inner classes for instance, I agree.
> I did not suggest the language and API should not change and remain forever
> compatible, only that if possible, instead of doing something like this:
On Thu, Apr 16, 2009 at 9:53 AM, Rich Hickey <richhic...@gmail.com> wrote:
> People (and not just book authors :) often ask - whither 1.0? [Ok,
> maybe they don't use 'whither']. The fact remains, some people want a
> 1.0 designation, and I'm not unwilling, providing we as a community
> can come to an understanding as to what that means, the process it
> implies, and the work it will require (and who will do it). Here are
> some of the relevant issues, IMO:
> - Stability/completeness/robustness
> This is mostly about - does it work? Is it relatively free of bugs? Is
> it free of gaping holes in core functionality? I think Clojure is in a
> pretty good place right now, but am obviously biased. This in no way
> implies there isn't room for improvement.
> - API stability
> With the semantic changes of fully lazy sequences behind us, I think
> the syntax and semantics of existing functions is largely stable.
Version numbering should reflect stability and compatibility.
Clojure x.y.z
z increments when a release changes functionality but not (public) APIs.
y increments when a release adds new public APIs.
x increments when public APIs change in a non-compatible way.
People should have the expectation that an upgrade from 1.0.2 to 1.0.3
should be painless (and you should be able
to back down from 1.0.3 to 1.0.2 without any compilation errors). An
upgrade from 1.0.3 to 1.1.0 may not be reversable
(if you start using new APIs in 1.1.0, your code won't compile is you
revert to 1.0.3).
However, this is very hard to achieve in practice (so far we haven't
pulled this off for Tapestry);
just knowing how a particular change affects the y or z digit takes
experience. In addition,
there's a drive from end users who want pre-compiled snapshots of
versions short of a fully endorsed release. That's one of the reasons
I've put some effort into the Clojure nightly builds and Maven
repository: to allow people to track the latest without building it
themselves,
or asking Rich to make more frequent releases.
Clojure has an advantage here that functions, rather than objects, are
extremely fine grained. In addition, macros and multimethods allow
API compatibility to be maintained even as new features are added.
Finally, my experience with final releases is that they rarely are.
Drawing a line in the sand and saying "this is the 1.0 release"
usually results
in a frantic batch of patch releases. Instead, release a candidate,
say "1.0.1". If you find bugs, release a new "1.0.2". When bugs stop
being deal-busters,
announce that "1.0.2" is the GA release. In other words, let a release
prove itself before being anointed the final release. Having a fixed
release version
number is no different than having a fixed release date: those are
impositions by marketing, not an engineering decision.
I think there is a definite need, however, to ** get tests into
clojure-lang **. The tests will be the best way to determine how a
change affects
compatibility. Regressions are very hard to predict, and I don't trust
myself to identify which changes will break client code, and which
will not ... short of having a test
to represent client code. The lack of tests and the sorry state of
Java code documentation are daunting to many, including myself. Rich
is obviously brilliant, but any successful
project has to scale beyond its creator. The lack of tests and
documentation borders on arrogance.
> Currently all new work (fixes and enhancements) occurs in trunk.
> There's no way to get fixes without also getting enhancements. I think
> this is the major missing piece in offering stable numbered releases.
> While I've cut a branch for each of the prior two releases, no one has
> ever submitted a bugfix patch for either. If people are going to want
> to work with a particular release version for an extended period of
> time, someone (other than me) will have to produce patches of (only!)
> fixes from the trunk for the release branch, and occasionally produce
> point releases (1.0.x) from that branch. I'd like to continue to do
> the bulk of my work in trunk, without messing anyone up or forcing
> everyone to follow along.
> - Freedom from change
> Numbered releases are most definitely not about absence of change in
> general. There are more things I want to add and change, and there
> will be for some time. That will keep Clojure a living language. 1.0
> or any numbered release can't and won't constitute a promise of no
> further change. But there are different kinds of change, changes that
> fix bugs and changes that add new capabilities or break existing code.
> People need to be able to choose the type of change they can tolerate,
> and when to incur it.
> - Perception
> Obviously, a 1.0 designation impacts perception. I am not interested
> in pursuing it just to influence perception, but rather to
> (collectively) acknowledge a milestone in usability and stability.
> However there may be other perceptions, good/bad or simply wrong (e.g.
> that Clojure is "finished"). Will the general perception engendered
> by 1.0 be met in the large, or a mismatch?
This is an important perception!
> What does 1.0 mean to you? Are we there yet? Any recommendations for
> the organization of the release branches, patch policy etc?
> Feedback welcome,
> Rich
-- Howard M. Lewis Ship
Creator of Apache Tapestry
Director of Open Source Technology at Formos
On Thu, Apr 16, 2009 at 9:10 PM, Chas Emerick <cemer...@snowtide.com> [...]
> That said, I have no concrete suggestion, as we'll always separately > pull our projects' dependencies into whatever we happen to be using as > a dependency management repo (it's a bummer to not be able to run a > build if some third-party repo is down, etc).
Yes, please, I don't want to be forced to work around automated downloaders. e.g. Like OpenWrt's build system that wants to download huge amounts of code if you don't watch it instead of just failing so you can tell it to look *over there* where I've already downloaded 90% of it! Of course, after it happens the first time you learn to be more careful, but I would prefer to be told what the dependencies are so that I can get them myself, or to be asked whether to download them than just automatically downloading them.
> Regarding documentation, I remember throwing around ideas in irc some > months ago about how to fold documentation from gen-class'ed libs into > a library's broader javadoc. That would be a huge boon to those using > clojure to build libraries that would be transparently usable by Java > developers.
About documentation: I believe this has been mentioned before, but I haven't been able to find the thread. Some functions are documented like this:
"assoc[iate]. When applied to a map[...]"
This makes it difficult if you're looking for some function and don't know the exact name.
e.g.: user=> (find-doc "quotient") nil user=> (find-doc "quot\\[ient") ------------------------- clojure.core/quot ([num div]) quot[ient] of dividing numerator by denominator. nil
Also, while looking for the thread mentioning the above, I found another unresolved documentation-related thread:
> On Apr 16, 12:53 pm, Rich Hickey <richhic...@gmail.com> wrote: >> What does 1.0 mean to you? Are we there yet? Any recommendations for >> the organization of the release branches, patch policy etc?
> I would like to see, concurrent with 1.0, some kind of library > management system. As noted before, contrib is already getting > hairy. We have dozens of little libraries floating around on GitHub. > Having you, Rich, maintain the list of libraries on clojure.org is not > sustainable. We don't need a fully-realized CPAN/RubyGem but we need > something that can scale to a few hundred authors. Dependency > management and documentation markup are components of this.
Instead of stating the problem as a problem of arrogance, it would be
better to state it as without X, you can't get Y.
Specifically, without better documentation there exists a class of
users that will not use clojure and there exists a class of problems
that will take a lot longer to solve than they would otherwise take
with better documentation.
Without more testing it will be impossible to say that any given
change in subversion won't break a program. Really, this is an
impossible statement anyway although I realize that tests mitigate the
problem somewhat. Thus we can't use x.y.z. Every change would be
just x (somewhat absurd but at least correct).
Your point release theory sounds good if the use cases of your library
are very will defined and tested. A programming language doesn't fit
the testability coverage scenario very well as it is, well, a turing
complete language and thus it would be difficult at best to prove it
was working as it was before after *any* non-trivial change (difficult
meaning it is unlikely anyone here is a good enough mathematician to
do it in the average case).
I agree that more tests would be a good thing; I personally am not
going to touch clojure source without more tests that show a little
more of the intent of what the code is doing. But clients of clojure
*should* have sufficient tests to ensure that upgrading clojure will
not present a massive problem; it is impossible for Rich or anyone
else to provide this guarantee.
So the question is, what level of documentation and what level of test
coverage is important for a 1.0 release? What would you like to see
documented and tested?
I would like to see the datastructures' memory and performance bounds
tested, for instance.
Chris
On Apr 16, 2:58 pm, Howard Lewis Ship <hls...@gmail.com> wrote:
> On Thu, Apr 16, 2009 at 9:53 AM, Rich Hickey <richhic...@gmail.com> wrote:
> > People (and not just book authors :) often ask - whither 1.0? [Ok,
> > maybe they don't use 'whither']. The fact remains, some people want a
> > 1.0 designation, and I'm not unwilling, providing we as a community
> > can come to an understanding as to what that means, the process it
> > implies, and the work it will require (and who will do it). Here are
> > some of the relevant issues, IMO:
> > - Stability/completeness/robustness
> > This is mostly about - does it work? Is it relatively free of bugs? Is
> > it free of gaping holes in core functionality? I think Clojure is in a
> > pretty good place right now, but am obviously biased. This in no way
> > implies there isn't room for improvement.
> > - API stability
> > With the semantic changes of fully lazy sequences behind us, I think
> > the syntax and semantics of existing functions is largely stable.
> Version numbering should reflect stability and compatibility.
> Clojure x.y.z
> z increments when a release changes functionality but not (public) APIs.
> y increments when a release adds new public APIs.
> x increments when public APIs change in a non-compatible way.
> People should have the expectation that an upgrade from 1.0.2 to 1.0.3
> should be painless (and you should be able
> to back down from 1.0.3 to 1.0.2 without any compilation errors). An
> upgrade from 1.0.3 to 1.1.0 may not be reversable
> (if you start using new APIs in 1.1.0, your code won't compile is you
> revert to 1.0.3).
> However, this is very hard to achieve in practice (so far we haven't
> pulled this off for Tapestry);
> just knowing how a particular change affects the y or z digit takes
> experience. In addition,
> there's a drive from end users who want pre-compiled snapshots of
> versions short of a fully endorsed release. That's one of the reasons
> I've put some effort into the Clojure nightly builds and Maven
> repository: to allow people to track the latest without building it
> themselves,
> or asking Rich to make more frequent releases.
> Clojure has an advantage here that functions, rather than objects, are
> extremely fine grained. In addition, macros and multimethods allow
> API compatibility to be maintained even as new features are added.
> Finally, my experience with final releases is that they rarely are.
> Drawing a line in the sand and saying "this is the 1.0 release"
> usually results
> in a frantic batch of patch releases. Instead, release a candidate,
> say "1.0.1". If you find bugs, release a new "1.0.2". When bugs stop
> being deal-busters,
> announce that "1.0.2" is the GA release. In other words, let a release
> prove itself before being anointed the final release. Having a fixed
> release version
> number is no different than having a fixed release date: those are
> impositions by marketing, not an engineering decision.
> I think there is a definite need, however, to ** get tests into
> clojure-lang **. The tests will be the best way to determine how a
> change affects
> compatibility. Regressions are very hard to predict, and I don't trust
> myself to identify which changes will break client code, and which
> will not ... short of having a test
> to represent client code. The lack of tests and the sorry state of
> Java code documentation are daunting to many, including myself. Rich
> is obviously brilliant, but any successful
> project has to scale beyond its creator. The lack of tests and
> documentation borders on arrogance.
> > - Development process stability
> > Currently all new work (fixes and enhancements) occurs in trunk.
> > There's no way to get fixes without also getting enhancements. I think
> > this is the major missing piece in offering stable numbered releases.
> > While I've cut a branch for each of the prior two releases, no one has
> > ever submitted a bugfix patch for either. If people are going to want
> > to work with a particular release version for an extended period of
> > time, someone (other than me) will have to produce patches of (only!)
> > fixes from the trunk for the release branch, and occasionally produce
> > point releases (1.0.x) from that branch. I'd like to continue to do
> > the bulk of my work in trunk, without messing anyone up or forcing
> > everyone to follow along.
> > - Freedom from change
> > Numbered releases are most definitely not about absence of change in
> > general. There are more things I want to add and change, and there
> > will be for some time. That will keep Clojure a living language. 1.0
> > or any numbered release can't and won't constitute a promise of no
> > further change. But there are different kinds of change, changes that
> > fix bugs and changes that add new capabilities or break existing code.
> > People need to be able to choose the type of change they can tolerate,
> > and when to incur it.
> > - Perception
> > Obviously, a 1.0 designation impacts perception. I am not interested
> > in pursuing it just to influence perception, but rather to
> > (collectively) acknowledge a milestone in usability and stability.
> > However there may be other perceptions, good/bad or simply wrong (e.g.
> > that Clojure is "finished"). Will the general perception engendered
> > by 1.0 be met in the large, or a mismatch?
> This is an important perception!
> > What does 1.0 mean to you? Are we there yet? Any recommendations for
> > the organization of the release branches, patch policy etc?
> > Feedback welcome,
> > Rich
> --
> Howard M. Lewis Ship
> Creator of Apache Tapestry
> Director of Open Source Technology at Formos
> Yes, please, I don't want to be forced to work around automated > downloaders. e.g. Like OpenWrt's build system that wants to download > huge amounts of code if you don't watch it instead of just failing so > you can tell it to look *over there* where I've already downloaded 90% > of it! Of course, after it happens the first time you learn to be > more careful, but I would prefer to be told what the dependencies are > so that I can get them myself, or to be asked whether to download them > than just automatically downloading them.
To start collecting some Pros/Cons for the different possibilities at there:
From my experiments with Ivy: You can just remove any public repository from the search paths. So any download will fail, telling you which dependencies could not be met.
In fact I run such a setup at work, where all external dependencies are mirrored locally. So there is no network access required.
Finally, Ivy makes it easy to setup such a mirror by virtue of the ivy:install task.
On Thu, Apr 16, 2009 at 2:40 PM, chris <cnuern...@gmail.com> wrote:
> That is putting it quite strongly, Howard.
> Instead of stating the problem as a problem of arrogance, it would be
> better to state it as without X, you can't get Y.
> Specifically, without better documentation there exists a class of
> users that will not use clojure and there exists a class of problems
> that will take a lot longer to solve than they would otherwise take
> with better documentation.
> Without more testing it will be impossible to say that any given
> change in subversion won't break a program. Really, this is an
> impossible statement anyway although I realize that tests mitigate the
> problem somewhat. Thus we can't use x.y.z. Every change would be
> just x (somewhat absurd but at least correct).
> Your point release theory sounds good if the use cases of your library
> are very will defined and tested. A programming language doesn't fit
> the testability coverage scenario very well as it is, well, a turing
> complete language and thus it would be difficult at best to prove it
> was working as it was before after *any* non-trivial change (difficult
> meaning it is unlikely anyone here is a good enough mathematician to
> do it in the average case).
Only a small portion of the code is the language; most of Clojure is
the standard library, which can
be unit tested. Testability is one of the foundations of functional
programming, so its
painful to see no actual tests.
> I agree that more tests would be a good thing; I personally am not
> going to touch clojure source without more tests that show a little
> more of the intent of what the code is doing. But clients of clojure
> *should* have sufficient tests to ensure that upgrading clojure will
> not present a massive problem; it is impossible for Rich or anyone
> else to provide this guarantee.
There are no guarantees, but there are measures you can take to
inspire confidence.
> So the question is, what level of documentation and what level of test
> coverage is important for a 1.0 release? What would you like to see
> documented and tested?
> I would like to see the datastructures' memory and performance bounds
> tested, for instance.
> Chris
> On Apr 16, 2:58 pm, Howard Lewis Ship <hls...@gmail.com> wrote:
>> On Thu, Apr 16, 2009 at 9:53 AM, Rich Hickey <richhic...@gmail.com> wrote:
>> > People (and not just book authors :) often ask - whither 1.0? [Ok,
>> > maybe they don't use 'whither']. The fact remains, some people want a
>> > 1.0 designation, and I'm not unwilling, providing we as a community
>> > can come to an understanding as to what that means, the process it
>> > implies, and the work it will require (and who will do it). Here are
>> > some of the relevant issues, IMO:
>> > - Stability/completeness/robustness
>> > This is mostly about - does it work? Is it relatively free of bugs? Is
>> > it free of gaping holes in core functionality? I think Clojure is in a
>> > pretty good place right now, but am obviously biased. This in no way
>> > implies there isn't room for improvement.
>> > - API stability
>> > With the semantic changes of fully lazy sequences behind us, I think
>> > the syntax and semantics of existing functions is largely stable.
>> Version numbering should reflect stability and compatibility.
>> Clojure x.y.z
>> z increments when a release changes functionality but not (public) APIs.
>> y increments when a release adds new public APIs.
>> x increments when public APIs change in a non-compatible way.
>> People should have the expectation that an upgrade from 1.0.2 to 1.0.3
>> should be painless (and you should be able
>> to back down from 1.0.3 to 1.0.2 without any compilation errors). An
>> upgrade from 1.0.3 to 1.1.0 may not be reversable
>> (if you start using new APIs in 1.1.0, your code won't compile is you
>> revert to 1.0.3).
>> However, this is very hard to achieve in practice (so far we haven't
>> pulled this off for Tapestry);
>> just knowing how a particular change affects the y or z digit takes
>> experience. In addition,
>> there's a drive from end users who want pre-compiled snapshots of
>> versions short of a fully endorsed release. That's one of the reasons
>> I've put some effort into the Clojure nightly builds and Maven
>> repository: to allow people to track the latest without building it
>> themselves,
>> or asking Rich to make more frequent releases.
>> Clojure has an advantage here that functions, rather than objects, are
>> extremely fine grained. In addition, macros and multimethods allow
>> API compatibility to be maintained even as new features are added.
>> Finally, my experience with final releases is that they rarely are.
>> Drawing a line in the sand and saying "this is the 1.0 release"
>> usually results
>> in a frantic batch of patch releases. Instead, release a candidate,
>> say "1.0.1". If you find bugs, release a new "1.0.2". When bugs stop
>> being deal-busters,
>> announce that "1.0.2" is the GA release. In other words, let a release
>> prove itself before being anointed the final release. Having a fixed
>> release version
>> number is no different than having a fixed release date: those are
>> impositions by marketing, not an engineering decision.
>> I think there is a definite need, however, to ** get tests into
>> clojure-lang **. The tests will be the best way to determine how a
>> change affects
>> compatibility. Regressions are very hard to predict, and I don't trust
>> myself to identify which changes will break client code, and which
>> will not ... short of having a test
>> to represent client code. The lack of tests and the sorry state of
>> Java code documentation are daunting to many, including myself. Rich
>> is obviously brilliant, but any successful
>> project has to scale beyond its creator. The lack of tests and
>> documentation borders on arrogance.
>> > - Development process stability
>> > Currently all new work (fixes and enhancements) occurs in trunk.
>> > There's no way to get fixes without also getting enhancements. I think
>> > this is the major missing piece in offering stable numbered releases.
>> > While I've cut a branch for each of the prior two releases, no one has
>> > ever submitted a bugfix patch for either. If people are going to want
>> > to work with a particular release version for an extended period of
>> > time, someone (other than me) will have to produce patches of (only!)
>> > fixes from the trunk for the release branch, and occasionally produce
>> > point releases (1.0.x) from that branch. I'd like to continue to do
>> > the bulk of my work in trunk, without messing anyone up or forcing
>> > everyone to follow along.
>> > - Freedom from change
>> > Numbered releases are most definitely not about absence of change in
>> > general. There are more things I want to add and change, and there
>> > will be for some time. That will keep Clojure a living language. 1.0
>> > or any numbered release can't and won't constitute a promise of no
>> > further change. But there are different kinds of change, changes that
>> > fix bugs and changes that add new capabilities or break existing code.
>> > People need to be able to choose the type of change they can tolerate,
>> > and when to incur it.
>> > - Perception
>> > Obviously, a 1.0 designation impacts perception. I am not interested
>> > in pursuing it just to influence perception, but rather to
>> > (collectively) acknowledge a milestone in usability and stability.
>> > However there may be other perceptions, good/bad or simply wrong (e.g.
>> > that Clojure is "finished"). Will the general perception engendered
>> > by 1.0 be met in the large, or a mismatch?
>> This is an important perception!
>> > What does 1.0 mean to you? Are we there yet? Any recommendations for
>> > the organization of the release branches, patch policy etc?
>> > Feedback welcome,
>> > Rich
>> --
>> Howard M. Lewis Ship
>> Creator of Apache Tapestry
>> Director of Open Source Technology at Formos
-- Howard M. Lewis Ship
Creator of Apache Tapestry
Director of Open Source Technology at Formos
> What does 1.0 mean to you? Are we there yet? Any recommendations for > the organization of the release branches, patch policy etc?
What I tacitly expect from a 1.0 release (or any official, numbered release) is - bug fixes without imposed changes in functionality for at least a year - the promise not to introduce breaking changes in future releases unless there is a very good reason to do so - a clear indication of potentially breaking changes by a change in version number - an outline of how future changes will be handled so that I can prepare for them in my own code management
For a 1.0 release in particular, I expect sufficient stability and completeness for use in real-life projects.
From my own experience with today's Clojure, I'd say the stability is there. I am less convinced about completeness, but that's mainly because there are many things I haven't yet tried to do. Overall, the discussions on the group don't leave the impression that there are glaring holes anywhere. However, it is possible that many users are relying on clojure-contrib for completeness, and if that is the case, the status of clojure-contrib needs to be considered as well before a 1.0 release.
What I miss most for a 1.0 release is some idea of how future changes will be handled, and what Clojure users can safely count on. For example, every new function added to clojure.core will break code that has chosen to use the same name for something else. While such incompatibilities are easy to fix (with an exclusion in refer- clojure), they are still a pain if they happen frequently. Another point is multimethod dispatch. Various ideas have been discussed here, and most of them, if adopted, are likely to break a lot of code relying on the current approach.
As for bug fixes for a stable release, you pointed out that someone would have to take care of them (mostly by backporting fixes in trunk I guess). I don't think it will be hard to find a team of volunteers, but it would be nice to arrange this before an official release. The same team could also maintain a stable subset of clojure-contrib as a kind of standard library.
What I'd really like to see are better command line tools. Make it
easy to compile, merge with java code, (or jython, or jruby, etc), and
a repl that came closer to something like IPython. A prototype lint
would be nice too, assuming it's possible for a lisp. And of course,
easier install.
> What does 1.0 mean to you? Are we there yet? Any recommendations for
> the organization of the release branches, patch policy etc?
I see 1.0 as indicating there will be a significant period (months)
between breaking changes. It seems we are there, based on your
comments on API stability. HLShip's recommended x.y.z release
numbering policy would be fine, but from my perspective any release/
patch policy you adopt will be fine as long as it's clearly documented
what changes will be breaking vs new api's and bug fixes. At this time
I don't think there is much value in a system to manage patches to old
releases. Bugs could be reported and the latest version patched, and
then users could make the decision of whether to backport the patch
themselves or upgrade to the latest rev. If a significant user base of
an older release develops, this could be revisited (ie cross that
bridge when you come to it).
I just wanted to give some thoughts on what I think are the main points coming from this discussion. It seems like most agree that "Clojure the language" is ready for a 1.0 release (and all that comes with it). The main issues are A) choice of VCS and B) the state of clojure-contrib. Here's my thoughts on both:
A) VCS
Python has just changed from Subversion to Hg (there is a great PEP article that details reasons for doing so here: http://www.python.org/dev/peps/pep-0374/). They did this because they had a need to and at the moment Clojure doesn't have this need (because there is only one person doing commits!). While it seems pointless changing technology for the sake of it, I also think - if it's an inevitable change in the future - why not just do it now? Rich - is changing VCS something you would consider at this stage?
B) clojure-contrib
For me, the purpose of clojure-contrib is a little confusing. Is it the Clojure standard library? well a lot of "standard" libraries are not in clojure-contrib so I think the answer is no. Is it a special holder repos for code that might someday be moved up into the core language? Well, that's what I thought it was for but there is currently stuff in there that will never be in the core language IMO. I think we need to decide what clojure-contrib is before a 1.0 release (or maybe it's already very clear and I have just missed something!).
In general, I think there should be only two "things" - the core language and a set of libraries. Whether the "libraries" are managed CPAN-style or whether they are delivered as part of a standard library (like with Python) I don't know at this point - but having a "special third place" for libraries that may or may not go into core seems unnecessary to me.
Paul. -- Iode Software Ltd, registered in England No. 6299803.
This message is intended only for the use of the person(s) ("the intended recipient(s)") to whom it is addressed. It may contain information which is privileged and confidential within the meaning of applicable law. If you are not the intended recipient, please contact the sender as soon as possible. The views expressed in this communication may not necessarily be the views held by The Company.
Thanks all for the feedback. One impression I get is that it seems the
existing community is getting along ok on trunk, so perhaps we also
need to consider those not yet using Clojure, possibly even because of
a lack of 1.0.
I joked about book authors, but I'd like to make it clear that I think
it is very important that Stuart's book correspond to a release of
Clojure. It would be huge for newcomers coming to Clojure find a book
that says "Covers Clojure 1.0", and a compatible download for 1.0.x
with the latest fixes.
Stuart has worked hard on tracking the latest changes, as have I in
trying to get in those changes that would be breaking as soon as I
could before 1.0. I'm presuming it's not too late to get "Covers
Clojure 1.0" in there (Stuart?), and, if so, it is a factor in this
decision.
I'll also add that there is plenty more I'd like to do, and as soon as
I get into that trunk will again be changing rapidly. There simply has
to be a branch for fixes only.
As to the feedback:
A library management system seems like a tall order for a language
1.0. It is certainly an interesting and useful pursuit, but given the
variety of approaches and opinions therein, it seems a bit out in
front of us. Advantages of Maven vs Ivy vs whatever should be
separate. Git is not going to happen any time soon, great as it may
be, given the current lack of infrastructure (google code) and tools
support. Is there some respect in which this impacts the core? It
would seem dangerous to marry any single approach in the language
itself.
A deprecation policy is a good idea. Backward compatibility mode is
unlikely.
Overall, I'm getting feature requests (more change!) and not a strong
drive for 1.0 stability. If you feel otherwise, please speak up.
Otherwise, my conclusion is that 1.0 may be more important for not-yet-
users wary of working from source.
One possible approach that just occurred to me waking up this morning is to
just do it. The very idea that now is a good time to ask the question is a
milestone. 1.0 marks the time that the question was asked as to what it
would take for there to be a 1l0! That was a typo, I meant 1.0, but why use
a decimal point? call it 1|0, and then you can REALLY have your own
definition as to what the version meant.
Seriously, I agree that there would be power in having a book that
corresponds with 1.0 even to the extent that it were part of a title, like
"Clojure 1.0".
*
Ok, here's another perspective now:*
People talk about Erlang as being fairly unique in that distributed
computing was taken into consideration as part of the core language design
rather than being bootstrapped on top as an after thought. Clojure has a
lot of nice design considerations along these lines, too. If the 1.0 line
is drawn in the sand, and there are a lot of nice design ideas coming around
the corner, while baselining now (after moving a few things from the core
like file io) could make things feel more sane, if any of these changes
really are revolutionary, they might not have the same pizaz in a post 1.0
release. They could with the right marketting, like "redesigned from the
ground up", or something, but it could be somewhat like Perl 6.0. I hear
it's nice and maybe taking a look at, but when it first, finally came out,
the message I got (mostly from Perl people) was "don't bother". That could
happen with a 2.0 or post 1.0, too.
On Fri, Apr 17, 2009 at 6:21 AM, Rich Hickey <richhic...@gmail.com> wrote:
> Thanks all for the feedback. One impression I get is that it seems the
> existing community is getting along ok on trunk, so perhaps we also
> need to consider those not yet using Clojure, possibly even because of
> a lack of 1.0.
> I joked about book authors, but I'd like to make it clear that I think
> it is very important that Stuart's book correspond to a release of
> Clojure. It would be huge for newcomers coming to Clojure find a book
> that says "Covers Clojure 1.0", and a compatible download for 1.0.x
> with the latest fixes.
> Stuart has worked hard on tracking the latest changes, as have I in
> trying to get in those changes that would be breaking as soon as I
> could before 1.0. I'm presuming it's not too late to get "Covers
> Clojure 1.0" in there (Stuart?), and, if so, it is a factor in this
> decision.
> I'll also add that there is plenty more I'd like to do, and as soon as
> I get into that trunk will again be changing rapidly. There simply has
> to be a branch for fixes only.
> As to the feedback:
> A library management system seems like a tall order for a language
> 1.0. It is certainly an interesting and useful pursuit, but given the
> variety of approaches and opinions therein, it seems a bit out in
> front of us. Advantages of Maven vs Ivy vs whatever should be
> separate. Git is not going to happen any time soon, great as it may
> be, given the current lack of infrastructure (google code) and tools
> support. Is there some respect in which this impacts the core? It
> would seem dangerous to marry any single approach in the language
> itself.
> A deprecation policy is a good idea. Backward compatibility mode is
> unlikely.
> Anyone who wants more tests can contribute them.
> Overall, I'm getting feature requests (more change!) and not a strong
> drive for 1.0 stability. If you feel otherwise, please speak up.
> Otherwise, my conclusion is that 1.0 may be more important for not-yet-
> users wary of working from source.
I feel the urge to drop a couple more pennies into this thread.
Trunk should NOT be used for day-to-day development and
experimentation. There should be a branch for that.
Trunk should NEVER be broken. Comprehensive tests need to run and pass
on the development branch before those changes are merged into trunk.
Subversion should NOT be used. Branching and merging are too
difficult. Distributed version contol makes this much easier.
Now, the issue of Rich's preferences comes in. Obviously, the current
development practices have worked pretty damn well so far. Perhaps
there is a way to allow the current SVN trunk to stand, and have
changes from there merged into another system for testing and
distribution. That would make SVN a private repository, for the most
part. Details would need to be worked out, but I think it's an avenue
worth exploring.
