The Guava conundrum.

[Kevin Bourrillion]

Hello esteemed would-be users of Guava,

We're facing a conundrum or two in our effort to produce an initial binary release of the Guava libraries.  I'd like to describe my thinking on the matter so you can tell me if you think I've made any errors in my logic, assumptions or judgment.  To resolve this in the best way for our users we may need some outside-the-box thinking that I haven't come up with yet.

Background: As you know, Google has open-sourced several core Java libraries, such as com.google.common.collect.  First these were released as the "Google Collections Library 1.0" (http://google-collections.googlecode.com), but then imported into a broader project called Guava (http://guava-libraries.googlecode.com).

Guava is currently open-sourced in source form only; we do not yet provide a binary.  This confuses and frustrates our would-be users -- isn't it trivially easy for us to build a jar?  Sure it is.  The problem is what happens next.

Consider what happens if a released library needs to change one of its APIs in some source- or binary-incompatible way.  If all the users of that library are end user applications, life is a breeze!  When they go to upgrade -- if they go to upgrade -- they'll just see some errors, and fix them, and then all is peachy again.

But when the user of our library is itself a library, which is itself used by applications and other libraries -- this can cause serious trouble!  A library that used the old version cannot coexist in the same classpath with a library that uses the new one.  This is going to be the case with Guava in the extreme, because of  the kind of building-block library that we are.

A common solution to this is for each library to encapsulate its dependencies, for example re-packaging them using a tool like JarJar.  In this scenario, the foo library will actually load classes called things like com.acme.foo.internal.repackaged.com.google.common.base.Objects, while the bar library will use some similar garbage under com.spiffyco.bar.  

This works for some cases, but our Guava project contains plenty of the kinds of datatypes that libraries would really like to use in their public APIs. This clearly couldn't work. Also, the runtime cost adds up of loading and initializing many separate copies of the same library.

Here's how we solved this problem of incompatible changes for Google Collections: we promised to never ever make them (well, modulo a lengthy deprecation cycle).  As a result, the library was a gigantic effort to release, spanning multiple years (!) and forcing us to skew our priorities far from where they'd otherwise have been.  It became unreasonably important to get everything "perfect" while we still could.  We also released much less than we could have otherwise.  I'm resolved that we are never going through that again.

So, Guava needs to reserve the right to make incompatible changes; changes which can be catastrophic to its users.  That's the heart of the conundrum.

What we have done so far is to mark each Guava class, and some methods, as being in one of two states: API-frozen, and not API-frozen.  The words we used as a first pass are "tentative" and "stable" (disrespectively).  As a result, the extremely conscientious developer has the ability to avoid the unfrozen APIs if he or she chooses.  However, I think it may be asking too much of a developer to pay such fine-grained attention.  It would be greatly preferable if they could set their build to fail (or at least warn) if an unfrozen API were used.

Perhaps the binary should contain only the stable APIs, while the tentative ones are still available in source form?  Sounds decent, but then we'd need the means to create such a binary from current sources. Complicating this fact is that the APIs we don't want to be accessible in that binary will often still need to be there, because we use them ourselves in our implementations.  I also fear that this would mean that no one would use the unstable APIs, even application developers who should have little to lose by doing so.  If they don't use them, we can't get the feedback on them that would help us get to their stable release.

Perhaps the jar should be complete, but an annotation processor can warn or fail when unstable APIs are used?  If this is workable, I'd happily switch from javadoc tagging to annotations in our source files.

There are a few other ideas, but I'd like to leave this message of here, for now.  Do you agree with my reasoning thus far?  Have I missed something?  Am I (characteristically) worrying too much, and I should just get over it?  Your feedback would be much appreciated.  Thanks!

-- 
Kevin Bourrillion @ Google
internal:  http://goto/javalibraries
external: http://guava-libraries.googlecode.com

[Paul Lindner]

I wonder if it's possible to use jarjar style tricks during the build-process?  I assume that the stable classes do not reference any of the unstable classes (otherwise they're transitively unstable!)  In that case package up all the stable classes into a slowly moving stable release.

For the unstable stuff repackage them in a version namespaced set of classes.  This would allow two separate versions of the guava classes to live in the same classloader.  You could then make the unstable jar depend on the more slowly moving core jar.

  com.google.gauva.MyFoo  --> com.google.guava.unstable_r1014.MyFoo

This puts more burden on the dev when the upgrades the library, small price to pay..

Right now I'm trying to work out how to upgrade a complex system based on google-collections 0.9 to 1.0 -- it's been a royal pain.  Contrast this with moving from httpclient 3.1 to httpclient 4.0 -- it's been a breeze, both packages can live together and I can upgrade much more easily.

--
guava-...@googlegroups.com.
http://groups.google.com/group/guava-discuss?hl=en
unsubscribe: guava-discus...@googlegroups.com
 
This list is for discussion; for help, post to Stack Overflow instead:
http://stackoverflow.com/questions/ask
Use the tag "guava".
 
To unsubscribe from this group, send email to guava-discuss+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.

[Nikolas Everett]

On Wed, Mar 24, 2010 at 2:09 PM, Kevin Bourrillion <kev...@google.com> wrote:

Perhaps the jar should be complete, but an annotation processor can warn or fail when unstable APIs are used?  If this is workable, I'd happily switch from javadoc tagging to annotations in our source files.

Your reasoning seems sound to me.  The release process for Google Collections was arduous from my perspective so it must have been tons worse for the people working on it.

I mostly build applications that don't mind the unstable nature of the API but for if I were making a library I'd be happy to have a FindBugs or PMD rule to yell at me for using the unstable APIs.

--Nik Everett

[Bob Lee]

I really like the idea of fine-grained freezing.

Like you said, the primary audience of this is other libraries.

Writing an annotation processor should be pretty simple. Just traverse the AST looking for any com.google.xxx elements that aren't annotated as frozen. You can easily generate a warning or error (you could make it configurable).

You'll have to use Sun's internal tree API because you need full AST access. You get to this from a JSR-269 annotation processor via the Trees class.

You might create a separate lib for the "frozen" annotations so IDEs and compatibility checking tools can use it and other libraries can adopt it. You could annotate the package with @Freezable so tools know that a subset of the API is frozen and the rest should be considered experimental.

Android does something similar. They hide experimental APIs. Then, they generate non-functional stub classes that have the hidden parts removed (all methods throw UnsupportedOperationException, so they compile w/ no problem). You compile against the stubs and run against the real thing. If you want to use an experimental API, compile against the real thing. It would be nice to see better tool support for this sort of setup.

Bob

[Kamlesh Sangani]

I don't understand why frozen and unfrozen APIs are in the same jar. API under development is unfrozen by definition and can be released under different version and library developers using that are aware that they are using unfrozen API. Other application developers can use the released STABLE version until the new API becomes stable again.

--

[Bob Lee]

On Wed, Mar 24, 2010 at 11:51 AM, Kamlesh Sangani <ksan...@gmail.com> wrote:

I don't understand why frozen and unfrozen APIs are in the same jar. API under development is unfrozen by definition and can be released under different version and library developers using that are aware that they are using unfrozen API. Other application developers can use the released STABLE version until the new API becomes stable again.

This isn't so easy when frozen and unfrozen parts are mixed in the same type (which is something we want to do).

Bob 

[Kevin Bourrillion]

On Wed, Mar 24, 2010 at 11:24 AM, Paul Lindner <lin...@inuus.com> wrote:

I wonder if it's possible to use jarjar style tricks during the build-process?  I assume that the stable classes do not reference any of the unstable classes (otherwise they're transitively unstable!)

Misunderstanding:  it is only the API that is unstable.  The better description is "not API-frozen".  Meaning, it may be renamed, its parameter order could be switched -- heck, just refer to the gigantic pile of refuse that is http://code.google.com/p/google-collections/wiki/Releases!  However, these methods being used by the implementations of methods that are API-frozen ("stable") presents no problem at all.

I'm not sure to what extent this misunderstanding colored the rest of your message, so please update me on what you think on these issues now that it's been cleared up.

[Kevin Bourrillion]

Some process by which a stable-subset JAR could be produced is one of the avenues we're considering.  How to do it is a big question, though.  I touched on this in the initial message.

On Wed, Mar 24, 2010 at 11:51 AM, Kamlesh Sangani <ksan...@gmail.com> wrote:

[Paul Lindner]

Consider the following thought experiment:

We have the following jars

  guava-core-1.0.jar

  guava-unstable_r100-1.0.jar

  guava-unstable_r200-1.0.jar

Note that the release is part of the jar name, it is not part of the version numbering.  This means that ivy/maven can include both in a project.  In both cases guava-ustable_r100/r200 depend on the stable library.

Now assume there is a FancyMultimap class in the unstable jars.  Each class is namespaced by version:

In guava-unstable_r100 it will be found at:

   com.google.r100.collect.FancyMultimap

In guava-unstable_r200 it will be found at:

   com.google.r200.collect.FancyMultimap

Now assume there are two libraries using guava named blend.jar and pour.jar.  Each of them pins the unstable library at a different versions and uses FancyMultimap

  blend.jar -> guava-unstable_r100

     Blend.blend(FancyMultimap drinks)

  pour.jar -> guava-unstable_r200

     Pour.pour(FancyMultimap drinks)

Now let's say I want to create a derivative work called barkeeper.war that needs to 'blend' and 'pour'

In this case my code would still work and I could link together the r100, r200 and stable jars without linking errors.  If I had code that needed to 'blend' and 'pour' in the same class it might need lots of explicit class naming, but things would still work.

class Barkeeper {

   ...

     com.google.r100.FancyMultimap blendDrinks = ....;

     com.google.r200.FancyMultimap pourDrinks = .....;

    Blend.blend(blendDrinks);

    Pour.pour(pourDrinks);

[Kevin Bourrillion]

On Wed, Mar 24, 2010 at 12:47 PM, Paul Lindner <lin...@inuus.com> wrote:

Consider the following thought experiment:

We have the following jars

  guava-core-1.0.jar

  guava-unstable_r100-1.0.jar

  guava-unstable_r200-1.0.jar

Note that the release is part of the jar name, it is not part of the version numbering.  This means that ivy/maven can include both in a project.  In both cases guava-ustable_r100/r200 depend on the stable library.

Remember that the stable APIs need to depend on the unstable, and that single classes (like Lists, Sets, etc.) will contain a mix of both kinds.  That seems to break your starting assumptions here.

[Kevin Bourrillion]

On Wed, Mar 24, 2010 at 11:33 AM, Bob Lee <crazy...@gmail.com> wrote:

Android does something similar. They hide experimental APIs. Then, they generate non-functional stub classes that have the hidden parts removed (all methods throw UnsupportedOperationException, so they compile w/ no problem). You compile against the stubs and run against the real thing. If you want to use an experimental API, compile against the real thing. It would be nice to see better tool support for this sort of setup.

In fact, to the extent that I understand how Android's droiddoc tool works, it strikes me as a very promising possibility for us.  Jesse actually spent some time trying to make it work for us, but it turned out to be nontrivial and he didn't have enough time.  I liked that it addresses not only the problems I've mentioned in this thread, but also provides the ability to switch your javadoc view between stable and full, and between versions of the API as well.  One possible outcome of this thread is that we decide to resume work on adopting droiddoc; however, I want to see how well my assumptions and reasoning bear out.  The internal discussion we're having on this is going along different lines, and at some point I need to try to bring the two threads together somehow.

[Tom Gibara]

There are many dimensions to this, here are a few things based on my experience of large java projects. I'm not suggesting that you haven't considered them, but they may be worth listing.

The first practical problem for a developer is often identifying that you have a dependency conflict, and which versions of what are conflicting. A number of simple practical measures help: published jars that clearly indicate the release in their name eg. guava-stable-1.2.3.jar, proper jar meta information in the form of a manifest or a maven POM. It can be valuable to have access to static methods that can provide key version information at runtime too.

This issue can often be addressed without resorting to bytecode manipulation, by appropriately segmenting the classloading; ensuring that libraries with conflicting dependencies are loaded from separate classloaders. This won't always work in cases where the libraries expose mutual types, however, though I've been involved with several projects that have suffered dependency conflicts, I've never seen one 'sunk' by them.

There are a number of more crafty ways that library compatibility can be problematic, serialization is one. I personally tend to become very conservative when evolving APIs that contain serializable classes; maintaining compatibility is difficult and hard to test (the guava code actually appears to implement serialization very well). My point in relation to the current topic is that 'experimentally' exposing state on an object that one then wants to recind may no longer be just an obvious compilation error if the state gets buried in a serialized object and has nowhere to go when its finally deserialized. A policy on the degree of serialization compatibility for unstable binary releases would be necessary (my suggestion: none).

On a different note, another issue that frequently causes compatibility issues is the introduction of new methods to interfaces, something that is necessary in some libraries (DOM implementations are a prominent example). Implementations compiled against the newer library interfaces will break (for want of the newer methods) and can usually be straightforwardly patched. More problematically, libraries that fully honour backward compatibility often need to be interoperable with binary classes compiled against older versions of the library interfaces.

I guess my general conclusion is that there are lots of issues that influence 'overall' compatibility of a library, conflicting binary compatibility (in the Java sense) is actually just one of many, and something that a developer can often do something about - even when the problems arise transitively. I'm not convinced that any of the systems outlined will provide benefits that outweigh the drag they will generate on the development of Guava itself. Of course there's a balance to be struck - stability is a very desirable attribute in any such library - but it's in the interests of every developer that the guava library evolves to meet new demands with the least impediment possible; that may not be served by trying to mitigate some of the problems of the wider Java environment.

Tom.

-- 
Tom Gibara
email: m...@tomgibara.com
web: http://www.tomgibara.com
blog: http://blog.tomgibara.com
twitter: tomgibara

--

[Paul Lindner]

Regarding evolving APIs -- the eclipse folks have a good writeup of the tradeoffs and some techniques for maintaining backwards compatibility:

http://wiki.eclipse.org/index.php/Evolving_Java-based_APIs

http://wiki.eclipse.org/Evolving_Java-based_APIs_2
http://wiki.eclipse.org/Evolving_Java-based_APIs_3

[ray.j.greenwell]

A truly ugly possibility:

When you first add a tentative method, declare it as:

public boolean _methodName_v1(String param);

Later you change your mind and simply @Deprecate that first one and
add:

public boolean _methodName_v2(String param, Predicate<Foo>
predicate);

If possible, adapt v1 to call v2. Perhaps in this case the
implementation moves to v2, modified of course to accept the
Predicate, and v1 becomes:

@Deprecated
public boolean _methodName_v1(String param) {
return _methodName_v2(param, Predicates.<Foo>alwaysTrue());
}

When one day it all settles down you move the implementation to
methodName(String, Predicate), and @Deprecate _v2 and have it call the
final implementation.

You could include an annotation processor for application developers
that removes the @Deprecated methods, or based on another annotation
that could also be on the versioned methods.

Classes are more difficult. SomeClass extends _SomeClass_v2 extends
_SomeClass_v1. There's not really a good way to strip the cruft for an
application developer, but this could be crafted to keep binary
compatibility for library users.

This is a total mess, but I think it could work.

[Johan Van den Neste]

Perhaps not terribly helpful, but this is why osgi should have been
built into the jdk from the start.

Using osgi pretty much solves those dependency problems. Which is, not
surprisingly, why most projects use osgi as soon as they hit a certain
size. (and that is sooner than you might think)

Again, I'm really sorry if this is not helpful, but all of the
proposed solutions will have a painful impact on your productivity
(and perhaps worse, possibly also on the productivity of your library
users).

Johan Van den Neste

[Tom Gibara]

I've spent a lot of time with the Android javadocs, and clarity of their API level is definitely very useful. Something similar for Guava would be beneficial, but I think this is a separate concern from how the process by which the library is built or used.  

Developing against the redacted jar is very effective for android development only because your application is automatically deployed against the full implementation, I can't see how this could be so in the vast majority the environments where Guava is used.

It's also worth pointing out that at least one source-incompatible change has been made to the Android APIs, and the sky didn't fall in:

http://groups.google.com/group/android-developers/browse_thread/thread/f5603049a7f85d01

Tom.

-- 
Tom Gibara
email: m...@tomgibara.com
web: http://www.tomgibara.com
blog: http://blog.tomgibara.com
twitter: tomgibara

--

[Nikolas Everett]

On Thu, Mar 25, 2010 at 4:10 AM, Johan Van den Neste <jvdn...@gmail.com> wrote:

Perhaps not terribly helpful, but this is why osgi should have been
built into the jdk from the start.

Using osgi pretty much solves those dependency problems. Which is, not
surprisingly, why most projects use osgi as soon as they hit a certain
size. (and that is sooner than you might think)

Again, I'm really sorry if this is not helpful, but all of the
proposed solutions will have a painful impact on your productivity
(and perhaps worse, possibly also on the productivity of your library
users).

I can't speak to OSGI but I really think that the annotation based solution solves this problem with a minimum of fuss.  Adding the annotations to the project isn't a huge burden.

Using them should be easy:  add some canned Findbugs or PMD check.  It'll warn me if I'm doing anything incompatible.  I can even make it fail my build.  If we built the whole thing properly then you could set it up by adding one dependency and pasting a small chunk into your Findbugs or PMD config.  That just doesn't sound like a big impact on my productivity.

As far as I'm concerned the hits in productivity is adding the annotations, writing the checks, and documenting how to use them.  That really doesn't sound that bad.  Deciding that this is the right solution is much more work.

--Nik

[Joshua O'Madadhain]

Here's my two cents' worth, wearing my JUNG hat.

JUNG development has proceeded in fits and starts, especially since I left graduate school.  During the 6+ years since its first release, there have been quite a lot of minor revisions to the API, and one big huge major one (the 2.0 release).  Ironically, I'm now contemplating another major revision, this time to use Guava instead of the Larvalabs generics collections API.

JUNG is certainly more specialized than Guava is, and it's probably fair to guess that many of the people using it are of a more experimental bent (and thus willing to tolerate changes), and/or were using it for a one-off project (and thus not needing to maintain their code).  A higher proportion, in both cases, than users of Guava may be, perhaps.

But my experience has been that the vast majority of JUNG users that have expressed an opinion found even the 1.x to 2.0 transition to be relatively painless...and we actually got a lot of positive feedback on that change because 2.0 was perceived as being such an improvement over 1.x.  More concretely, I can think of _one_ person who complained about the porting burden, and that was because there was a specific class that I hadn't yet ported over to 2.0.  (The porting itself would have been fairly straightforward, but I ended up rewriting that class from scratch because the original design--not mine!--was unnecessarily complicated.)  I've written a "this is where that class/capability went" guide, but considering how often people ignore the documentation, I'd bet that it doesn't get used much.  *wry smile*

Looking at it from the other side, the Apache commons-collections API changed a few times.  It was never a big deal to me when it happened.

So, some things to consider:

Many of your users will not be affected by some changes that you will make in future.

Of those that would be affected, many will no longer be maintaining the code that used the old APIs and can continue to use their old jars indefinitely.

Of those that are still maintaining their old code, many will probably find that the changes themselves are perhaps tedious, but not overly burdensome.

There will be some remaining.  They will complain, especially if they (unreasonably, of course :) ) liked the old version better.  But I'm guessing that this will be a pretty small subset of the Guava user base.

My advice: put some jars out there, do a good job of communicating changes to your API, make the changes good ones, and you will be forgiven the disruptions.  If putting annotations in there makes you happy, go for it, but I'd recommend that you _definitely_ keep the Javadoc tags in there; anyone can look through those even if they don't speak annotations.

Joshua

--

guava-...@googlegroups.com.
http://groups.google.com/group/guava-discuss?hl=en
unsubscribe: guava-discus...@googlegroups.com
 
This list is for discussion; for help, post to Stack Overflow instead:
http://stackoverflow.com/questions/ask
Use the tag "guava".
 
To unsubscribe from this group, send email to guava-discuss+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.

--
   Joshua O'Madadhain: Information Scientist, Musician, Philosopher-At-Tall
  "It's that moment of dawning comprehension that I live for" -- Bill Watterson

[Colin Decker]

Joshua,

It seems like you're looking at it completely from the perspective of applications using the API, where the user can just change their code to accommodate API changes in new versions. Kevin already said that isn't really an issue.

The issue is when other libraries or frameworks depend on Guava and applications depend on those. Let's say there's a Guava 1.0 with unfrozen parts that could be changed in the future. Let's say we have an application that uses Guava and two other libraries that themselves also use Guava. Now let's say a Guava 2.0 is released, with breaking changes in the unfrozen parts of the API. If either of the other libraries using Guava use anything from 1.0 that was incompatibly changed in 2.0, we can neither upgrade to Guava 2.0 in our application nor upgrade either library to a new version that uses 2.0 while the other does not have a version out that supports it.

Colin

[Kevin Bourrillion]

Hello again all,

This thread (and its internal counterpart) have now surpassed my ability to comment on each and every point raised, but I'm gradually putting together the following conclusion:

Library developers, in general, should stay far away from our unstable APIs.  We'll help them by marking them all clearly and stating as unequivocally as we can that we do reserve the right to change them at any time.  If they do want to use one, it would be best for them to contact us and beg us to hurry up and make it stable already.  As a second resort, they can weigh the trade-offs of a jarjar-like solution.  I would not be happy if they decided "whatever, my users can just use OSGi to keep my library separate from the others"; I think this is an extremely suboptimal approach, but I can't stop them from taking that attitude if that's the attitude they have.

I would love tools to be created that such developers could use to alert them (warning, or build-failing) when they stray outside the well-lit path.  We may not have time to create these tools ourselves -- it would be ideal if an enterprising reader of this thread would decide this sounds like a fun project.  I would personally start with Bob's advice, as his method will make it so that javac itself takes care of it all!

What to call the annotations?  I do believe that both cases must be marked explicitly. No annotation on an element means it has the same state as the containing element (class, outer class, package).  I'd like to stay away from "unstable"... it sounds wrong, like something that will have flaky behavior at runtime.

@Frozen vs. @Unfrozen?

@ApiFrozen vs. @ApiUnfrozen?

@ApiFrozen(true) vs. @ApiFrozen(false)?

@ApiState(ApiStates.FROZEN) vs. @ApiState(ApiStates.UNFROZEN) (ok, yuck)

Note that this is all orthogonal to the @since tag, which will always tell the version since which the API has been present in its exact (binary-compatible) present form. If a method doesn't change for a year before it's marked frozen, then for all practical purposes it's sort of retroactively frozen as of a year prior.

Tiny side point:

On Thu, Mar 25, 2010 at 10:05 AM, Joshua O'Madadhain <jr...@google.com> wrote:

If putting annotations in there makes you happy, go for it, but I'd recommend that you _definitely_ keep the Javadoc tags in there; anyone can look through those even if they don't speak annotations.

Meh, annotations show up in the javadoc too; no special know-how required.

[Bob Lee]

On Thu, Mar 25, 2010 at 10:38 AM, Kevin Bourrillion <kev...@google.com> wrote:

@Frozen vs. @Unfrozen?

@ApiFrozen vs. @ApiUnfrozen?

@ApiFrozen(true) vs. @ApiFrozen(false)?

@ApiState(ApiStates.FROZEN) vs. @ApiState(ApiStates.UNFROZEN) (ok, yuck)

@Published(since="1.0") vs. @Experimental

"experimental" would be the default, so we could do without @Experimental entirely. Thoughts?

Bob

[Kevin Bourrillion]

On Thu, Mar 25, 2010 at 10:49 AM, Bob Lee <crazy...@gmail.com> wrote:

@Frozen vs. @Unfrozen?

@ApiFrozen vs. @ApiUnfrozen?

@ApiFrozen(true) vs. @ApiFrozen(false)?

@ApiState(ApiStates.FROZEN) vs. @ApiState(ApiStates.UNFROZEN) (ok, yuck)

@Published(since="1.0") vs. @Experimental

I'm curious why "since <version>" is useful information.  Let's say that we created foo(Bar) in version 2010.05.01, then changed it to foo(Baz) in version 2010.06.01, then marked it as frozen/published/whatever in version 2010.07.01.  From that point on, the only version number that is of any interest is the middle one -- 2010.06.01.  That is the minimum version of the library that one needs to have in order to use the method.  The fact that 2010.07.01 is when the stable badge was applied is kind of meaningless information, is it not?

On the terms published vs. experimental -- meh.  I feel like people would have to dig in further to what exactly that means, and we'd finally end up speaking in terms of whether the API is frozen or not.  I feel like API "frozenness" is the most precise description of what we're talking about here that I've heard.

 

"experimental" would be the default, so we could do without @Experimental entirely. Thoughts?

Again, we need both to be explicit.  Picture a class like Maps.java having 50 frozen methods and one unfrozen/experimental one.  If we only had @Published, we'd have to put it on all 49 of the others.  With both, either can override the other.

[Kevin Bourrillion]

On Thu, Mar 25, 2010 at 10:58 AM, Kevin Bourrillion <kev...@google.com> wrote:

I'm curious why "since <version>" is useful information.  Let's say that we created foo(Bar) in version 2010.05.01, then changed it to foo(Baz) in version 2010.06.01, then marked it as frozen/published/whatever in version 2010.07.01.  From that point on, the only version number that is of any interest is the middle one -- 2010.06.01.  That is the minimum version of the library that one needs to have in order to use the method.  The fact that 2010.07.01 is when the stable badge was applied is kind of meaningless information, is it not?

Whoops, to finish this thought: What I mean is, if only one version number is meaningful (as I believe), and there is already a well-known place for that number to go (in the javadoc @since clause), then there's no reason for it to also go in the annotation, is there?

--

Kevin Bourrillion @ Google

[Kevin Bourrillion]

One very tiny disadvantage of these:

@Frozen vs. @Unfrozen?

@ApiFrozen vs. @ApiUnfrozen?

compared to these:

   

@ApiFrozen(true) vs. @ApiFrozen(false)?

@ApiState(ApiStates.FROZEN) vs. @ApiState(ApiStates.UNFROZEN) (ok, yuck)

is that someone could theoretically accidentally put both @Frozen and @Unfrozen on the same method/class, which would be nonsense.  Not a big deal, but it's sort of intellectually pleasing to know that wouldn't be possible for the latter, single-annotation-with-parameter approach.

Of that latter category, the one I labelled "yuck" has the slight advantage that we cooould introduce a third state one day if needed.  But I really don't foresee us wanting to go to that degree of craziness here.

Finally, about Bob's suggested term "Experimental", I think it's a much stronger statement than we are really trying to make.  The method and its behavior might be quite tried and true but just pending a final decision on its name -- that doesn't sound like an experimental feature to me, just a not-API-frozen one.

-- 

[Tim Peierls]

On Thu, Mar 25, 2010 at 1:58 PM, Kevin Bourrillion <kev...@google.com> wrote:

"experimental" would be the default, so we could do without @Experimental entirely. Thoughts?

Again, we need both to be explicit.  Picture a class like Maps.java having 50 frozen methods and one unfrozen/experimental one.  If we only had @Published, we'd have to put it on all 49 of the others.  With both, either can override the other.

I still don't understand why @Published/@Frozen is necessary. How useful/likely is the opposite case: a class with 50 experimental methods and 1 published method? Not very, right?

So the real default is frozen -- that's what we assume about an interface that makes no claims currently, right? 

In which case, how about just @Experimental (on class or method) and not @Published/@Frozen/etc.? I like @Experimental -- doesn't seem too strong at all. 

--tim

[Joshua O'Madadhain]

I prefer @Experimental.  Or some other name, if you like, but it seems to me that what we're primarily concerned with is people using APIs that we _haven't_ set in stone.

That is, if that annotation is missing, use it, you're golden; if it's present, you're accepting a risk.  Using both annotations, or even annotations with both senses, seems redundant.  There isn't a third category, is there?

I don't like "Unfrozen" because that implies that it was once frozen.  How about "Liquid"?  "Fluid"?  :)

Joshua

--

guava-...@googlegroups.com.
http://groups.google.com/group/guava-discuss?hl=en
unsubscribe: guava-discus...@googlegroups.com
 
This list is for discussion; for help, post to Stack Overflow instead:
http://stackoverflow.com/questions/ask
Use the tag "guava".
 
To unsubscribe from this group, send email to guava-discuss+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.

[Joshua O'Madadhain]

On Thu, Mar 25, 2010 at 10:24, Colin Decker <cgde...@gmail.com> wrote:

Joshua,

It seems like you're looking at it completely from the perspective of applications using the API, where the user can just change their code to accommodate API changes in new versions.

I didn't say this explicitly, but there are a few different libraries that use JUNG, as well as lots of applications.   Never had any complaints from any of the library maintainers in response to JUNG version changes.

But like I said, I recognize that JUNG != Guava and its user base (and uses) may be different, as well.

 

Kevin already said that isn't really an issue.

The issue is when other libraries or frameworks depend on Guava and applications depend on those. Let's say there's a Guava 1.0 with unfrozen parts that could be changed in the future. Let's say we have an application that uses Guava and two other libraries that themselves also use Guava. Now let's say a Guava 2.0 is released, with breaking changes in the unfrozen parts of the API. If either of the other libraries using Guava use anything from 1.0 that was incompatibly changed in 2.0, we can neither upgrade to Guava 2.0 in our application nor upgrade either library to a new version that uses 2.0 while the other does not have a version out that supports it.

I agree that adding an intermediate library makes things more complicated.  However, I'm not sure that it really fundamentally changes the issues involved.  The library maintainer has a choice to continue using the existing Guava 1.0 jar and API, or to provide a new version using 2.0.  If the library maintainer provides a new 2.0-using jar, then applications using that library will have the option of using the old or new libraries.

Obviously, the more Guava-using libraries that an application depends on, the more annoying the situation gets from the perspective of the application developer.  But as a designer of a library, if I see a feature that I really like in Guava, I'm going to use it.  If it works well for me, I'm going to try hard to get it to stay part of Guava.  The worst case is that Guava has a feature that a lot of people really like but that gets (re)moved in a future version.  Kevin (and others) have put in a lot of time, and will continue to put in a lot of time, to try to anticipate and prevent this.

As I said, changes happened in the commons-collections APIs, too.  As a writer of a library that used those APIs, I coped.  To me it wasn't a big deal, as the changes were clearly documented--and I always had the option of continuing to use the old version.

Joshua

[Bob Lee]

On Thu, Mar 25, 2010 at 10:58 AM, Kevin Bourrillion <kev...@google.com> wrote:

I'm curious why "since <version>" is useful information.  Let's say that we created foo(Bar) in version 2010.05.01, then changed it to foo(Baz) in version 2010.06.01, then marked it as frozen/published/whatever in version 2010.07.01.  From that point on, the only version number that is of any interest is the middle one -- 2010.06.01.  That is the minimum version of the library that one needs to have in order to use the method.  The fact that 2010.07.01 is when the stable badge was applied is kind of meaningless information, is it not?

Agreed. Are you really considering using dates? I'd prefer incrementing numbers, in which case, we'd go back and fill in missing "since" attributes when we increment the version #.

On the terms published vs. experimental -- meh.  I feel like people would have to dig in further to what exactly that means, and we'd finally end up speaking in terms of whether the API is frozen or not.  I feel like API "frozenness" is the most precise description of what we're talking about here that I've heard.

Yeah, I guess it's all "published", and "frozen" is more precise. How about @FullyBaked vs. @HalfBaked? Or @SetInStone. ;-) 

Again, we need both to be explicit.  Picture a class like Maps.java having 50 frozen methods and one unfrozen/experimental one.  If we only had @Published, we'd have to put it on all 49 of the others.  With both, either can override the other.

Good point. I was thinking that everything should be experimental by default so we don't accidentally freeze things, but an API compatibility checker could catch these mistakes. All we need to do is create a doclet wrapper for JDiff that filters out non-published elements (I personally did just this for Android).

Bob 

[Bob Lee]

On Thu, Mar 25, 2010 at 11:13 AM, Tim Peierls <t...@peierls.net> wrote:

I still don't understand why @Published/@Frozen is necessary. How useful/likely is the opposite case: a class with 50 experimental methods and 1 published method? Not very, right?

So the real default is frozen -- that's what we assume about an interface that makes no claims currently, right? 

In which case, how about just @Experimental (on class or method) and not @Published/@Frozen/etc.? I like @Experimental -- doesn't seem too strong at all. 

+1

This is nicely analogous to Android's @hidden.

Bob 

[Kevin Bourrillion]

These are valid points.  We would need to be awfully careful not to forget to add the annotation when creating new experimental methods, though.  I can see us forgetting this more than once.  However, this is already an issue anyway if the class we're adding to is marked @Frozen.

I was predisposed against a "assume everything's fine unless we specifically say otherwise" model, thinking it would be safer to say "assume nothing unless we explicitly say so".  But in reality, developers are by and large going to do the former anyway; it's the way we are.

On Thu, Mar 25, 2010 at 11:14 AM, Joshua O'Madadhain <jr...@google.com> wrote:

I don't like "Unfrozen" because that implies that it was once frozen.  How about "Liquid"?  "Fluid"?  :)

@NotFrozen, then.  But it doesn't work so well when not paired with @Frozen.  I still feel like @Experimental overstates the case.

[Nikolas Everett]

Have both annotations lets you run a test to make sure that every method/class/whatever has one of the two annotations.  I wouldn't trust myself not to forget to add frozen.

[Sam Berlin]

If @Experimental is the only annotation (going with a 'assume it's OK unless stated otherwise' model), I could imagine @Experimental(stage=COULD_CHANGE_AT_ANY_MOMENT), or @Experimental(stage=REALLY_CLOSE_TO_BEING_FROZEN) kind of thing.  (With better names, of course.  The idea being that an enum within the annotation could describe just how experimental it really is.)

Sam

On Thu, Mar 25, 2010 at 2:38 PM, Kevin Bourrillion <kev...@google.com> wrote:

[Kevin Bourrillion]

On Thu, Mar 25, 2010 at 11:41 AM, Nikolas Everett <nik...@gmail.com> wrote:

Have both annotations lets you run a test to make sure that every method/class/whatever has one of the two annotations.  I wouldn't trust myself not to forget to add frozen.

I almost made that point myself, except that we don't actually want to annotate every single method in a class like Sets or Maps -- under the two-annotation model, it would be essential that an unannotated method would inherit the annotation of its containing element.  So no checker is possible in that model either.

[Nikolas Everett]

Wow I just sent an email that was barely English.  What I meant was that I wouldn't trust myself to mark the parts of the API appropriately so I'd want something coming in after me.  Having both annotations makes that possible.

[Kevin Bourrillion]

I think tweaking those would be too high-maintenance.  I'd rather this were an on/off switch.  Also, if we have any tangible goal of getting the API frozen in the foreseeable future, then the best place to capture information about our progress on that front would be in our issues db on code.google.com.

At some point I'll probably go through and file one issue for each "experimental" API we have.

[Nikolas Everett]

On Thu, Mar 25, 2010 at 2:44 PM, Kevin Bourrillion <kev...@google.com> wrote:

On Thu, Mar 25, 2010 at 11:41 AM, Nikolas Everett <nik...@gmail.com> wrote:

Have both annotations lets you run a test to make sure that every method/class/whatever has one of the two annotations.  I wouldn't trust myself not to forget to add frozen.

I almost made that point myself, except that we don't actually want to annotate every single method in a class like Sets or Maps -- under the two-annotation model, it would be essential that an unannotated method would inherit the annotation of its containing element.  So no checker is possible in that model either.

That checker is really the only reason I can think of to have the two annotations.  Without it just save yourself the trouble and go with one of @Frozen or @Experimental.  I don't really have an opinion on whether its more natural to assume everything is fine unless otherwise marked or the other way around.

[Joshua O'Madadhain]

On Thu, Mar 25, 2010 at 11:38, Kevin Bourrillion <kev...@google.com> wrote:

These are valid points.  We would need to be awfully careful not to forget to add the annotation when creating new experimental methods, though.  I can see us forgetting this more than once.  However, this is already an issue anyway if the class we're adding to is marked @Frozen.

The more annotations (and types of annotations) that you have to add, then the more mistakes that will be made, both on the annotating side and on the reading side.

 

To those that are saying that both are necessary: remember that the _worst_ case involves someone failing to realize that something _isn't_ frozen.  If we had (only) a 'frozen' annotation and we forgot to use it, then someone might grumble that they would use that method if only they were sure it would still be there later, oh well, wait for it.  That's not a big problem in my book.  

You do not want to overload people's brains.  Keep it simple.

I was predisposed against a "assume everything's fine unless we specifically say otherwise" model, thinking it would be safer to say "assume nothing unless we explicitly say so".  But in reality, developers are by and large going to do the former anyway; it's the way we are.

Yup.  :)

 

On Thu, Mar 25, 2010 at 11:14 AM, Joshua O'Madadhain <jr...@google.com> wrote:

I don't like "Unfrozen" because that implies that it was once frozen.  How about "Liquid"?  "Fluid"?  :)

@NotFrozen, then.  But it doesn't work so well when not paired with @Frozen.  I still feel like @Experimental overstates the case.

I'm now going to back slowly away from the linguistic madlibs before I get sucked into it (more).  Good luck.

 

Joshua

[Bob Lee]

On Thu, Mar 25, 2010 at 11:38 AM, Kevin Bourrillion <kev...@google.com> wrote:

These are valid points.  We would need to be awfully careful not to forget to add the annotation when creating new experimental methods, though.  I can see us forgetting this more than once.  However, this is already an issue anyway if the class we're adding to is marked @Frozen.

How about this?

  1. You make an API change.

  2. The build generates current-api.xml using JDiff.

  3. If current-api.xml is different from expected-api.xml, return an error.

  4. If the change was intentional, copy current-api.xml to expected-api.xml.

  5. Re-run the build.

  6. Check in.

The API diff tool generates an error for new APIs, compatibile or not. The tool allows:

  1) New @Experimental elements.

  2) Changes to @Experimental elements so long as @since changed, too. (This could be tricky.)

The API diff tool could also detect missing @since tags.

Bob

[Kevin Bourrillion]

On Thu, Mar 25, 2010 at 11:52 AM, Joshua O'Madadhain <jr...@google.com> wrote:

To those that are saying that both are necessary: remember that the _worst_ case involves someone failing to realize that something _isn't_ frozen.  If we had (only) a 'frozen' annotation and we forgot to use it, then someone might grumble that they would use that method if only they were sure it would still be there later, oh well, wait for it.  That's not a big problem in my book.  

This was my first instinct at the suggestion of having only one.  However, it does mean that to add one new experimental method to a formerly-frozen class, we've got to add @Frozen to aallllll the existing methods of the class. Ugh.

But here's another problem with the opposite approach -- if we have only @RiskyStayAway, it's pretty unreasonable to ask the user of Foo.bar() to check for this both on the method and on the class.  So, putting the annotation on classes would be useless.  We'd always have to add it to eeeach and eeeevery single method.  Ugh again!

One good thing about the two-annotation model is that we're never stuck adding dozens of annotations to a file.  There is no default.  If the method doesn't tell you, the class will.  There will always be something in the file you can ctrl-click or whatever to see a fuller explanation of what this freeze status business is all about.

[Kevin Bourrillion]

On Thu, Mar 25, 2010 at 12:02 PM, Kevin Bourrillion <kev...@google.com> wrote:

But here's another problem with the opposite approach -- if we have only @RiskyStayAway, it's pretty unreasonable to ask the user of Foo.bar() to check for this both on the method and on the class.

That is unless we want to put all our faith in the tools we come up with, in which case it's fine for a tool to check both.

[Steve Reed]

On Thu, Mar 25, 2010 at 12:02 PM, Kevin Bourrillion <kev...@google.com> wrote:

But here's another problem with the opposite approach -- if we have only @RiskyStayAway, it's pretty unreasonable to ask the user of Foo.bar() to check for this both on the method and on the class.  So, putting the annotation on classes would be useless.  We'd always have to add it to eeeach and eeeevery single method.  Ugh again!

I'm not sure it's useless. This is what developers need to do today with @Deprecated. If you're not relying on some sort of tool to scan for these annotations, then it's still up in the air whether the developer is going to bother checking the annotations or apidoc any way.

[Joshua O'Madadhain]

On Thu, Mar 25, 2010 at 12:02, Kevin Bourrillion <kev...@google.com> wrote:

On Thu, Mar 25, 2010 at 11:52 AM, Joshua O'Madadhain <jr...@google.com> wrote:

To those that are saying that both are necessary: remember that the _worst_ case involves someone failing to realize that something _isn't_ frozen.  If we had (only) a 'frozen' annotation and we forgot to use it, then someone might grumble that they would use that method if only they were sure it would still be there later, oh well, wait for it.  That's not a big problem in my book.  

This was my first instinct at the suggestion of having only one.  However, it does mean that to add one new experimental method to a formerly-frozen class, we've got to add @Frozen to aallllll the existing methods of the class. Ugh.

Possibly I was unclear in my motivating example, but I am proposing that we have only @RiskyStayAway, not only @Frozen.

 

But here's another problem with the opposite approach -- if we have only @RiskyStayAway, it's pretty unreasonable to ask the user of Foo.bar() to check for this both on the method and on the class.  So, putting the annotation on classes would be useless.  We'd always have to add it to eeeach and eeeevery single method.  Ugh again!

I guess it depends on whether you're thinking tool-checked or visual checking.  If most or all of the class is @Malleable/@PossiblyEphemeral, this should be reflected in the Javadoc.  If you're writing a library without reading the Javadoc, especially the stuff in LARGE BOLD CAPITALIZED SCARY LETTERS, then I can't help you.

How often do you think it's going to be the case that an entire class is going to be marked as @Limbo?

Food for thought: 

(1) To my mind, a class with lots of methods that are all @Malleable is probably that way because we haven't yet decided _where_ to put it more than because we've got serious doubts about all those methods being useful.  This may suggest that a separate annotation to indicate that it might be _moved_ might be useful.  Maybe.  Really not sure about that.

(2) We should keep separate track of each individual item that may be moved/removed, so each should be tagged.  Even if--perhaps because!--it's painful to do that.  We should be a little reluctant to add a bunch of @Hypothetical stuff.

(Hmm.  Useful to create groups of items that will stand/fall together, e.g. @Malleable(CapabilityName)?)

 

One good thing about the two-annotation model is that we're never stuck adding dozens of annotations to a file.  There is no default.  

That, in a nutshell, is the problem: "there is no default".  Which means that if the notation is missing then people don't know what to assume (or, more precisely, they know that they don't know whether it's frozen or not).  Embrace the power of defaults!  :)

Joshua

 

If the method doesn't tell you, the class will.  There will always be something in the file you can ctrl-click or whatever to see a fuller explanation of what this freeze status business is all about.

--
Kevin Bourrillion @ Google
internal:  http://goto/javalibraries
external: http://guava-libraries.googlecode.com

--
guava-...@googlegroups.com.
http://groups.google.com/group/guava-discuss?hl=en
unsubscribe: guava-discus...@googlegroups.com
 
This list is for discussion; for help, post to Stack Overflow instead:
http://stackoverflow.com/questions/ask
Use the tag "guava".
 
To unsubscribe from this group, send email to guava-discuss+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.

[Tom Gibara]

I'm curious why "since <version>" is useful information.  Let's say that we created foo(Bar) in version 2010.05.01, then changed it to foo(Baz) in version 2010.06.01, then marked it as frozen/published/whatever in version 2010.07.01.  From that point on, the only version number that is of any interest is the middle one -- 2010.06.01.  That is the minimum version of the library that one needs to have in order to use the method.  The fact that 2010.07.01 is when the stable badge was applied is kind of meaningless information, is it not?

 

A method can change its contract without changing its binary compatibility, choosing the version at which it was first considered stable is, I think, more meaningful. For example, suppose the method was initially added as a stub that throws an UnsupportedOperationException, is later implemented and then frozen, is it really meaningful to say that it was available from the first? And if not, how would one determine that the method implementation was good enough to qualify?

Also, I think it would be much better to have ordinals for versions instead of dates. Given a version like 2010.06.01, how would a checker (or developer for that matter) easily determine the next version, or the previous version, or identify that an invalid version number had been applied? Ordinals are more useful in this regard, if you want to know the date for a version you can look it up, it's much harder to go the other way.

Tom.

-- 

 

Tom Gibara
email: m...@tomgibara.com
web: http://www.tomgibara.com
blog: http://blog.tomgibara.com
twitter: tomgibara

[Kevin Bourrillion]

On Thu, Mar 25, 2010 at 12:22 PM, Tom Gibara <m...@tomgibara.com> wrote:

Also, I think it would be much better to have ordinals for versions instead of dates.

We could still decide to do that.  We could rename "2009.09.15" to "1" and "2010.01.04" to "2", and the next cut will be "3".  If all goes well and we release, say, monthly, then in two years we'll be at version "27".  Do people like this?  All that's important to me is that the numbers be meaningless; dates were one way to get that.

 

Given a version like 2010.06.01, how would a checker (or developer for that matter) easily determine the next version, or the previous version, or identify that an invalid version number had been applied? Ordinals are more useful in this regard,

Yet you see ordinal version numbers used so very rarely.  I can't think of many examples.  Are these benefits really important, I wonder?

[Tom Gibara]

Yet you see ordinal version numbers used so very rarely.  I can't think of many examples.  Are these benefits really important, I wonder?

That's true. I don't think I had encountered it until I started developing for Android, and I'm basing my preference for it on my experience there. I happen to know that the versions of Android are 1.0, 1.1, 1.5, 1.6, 2.0, 2.0.1, 2.2. And I think in terms of these, but mentally I distinguish between these and the API levels 1-7 they correspond to. And perhaps that's a distinction that's needed here: an API definition (the API level in Android parlance) distinguished by a number and the version (a particular implementation of that API level) distinguished by a date.

Then the distinction between frozen/unfrozen may perhaps be reducible to an @APILevel annotation (or something better named) that declares which level the class or method was frozen in. Anything without a declared level is, as a consequence, not stabilized.

Tom

[Torbjorn Gannholm]

On Thu, Mar 25, 2010 at 7:38 PM, Kevin Bourrillion <kev...@google.com> wrote:

These are valid points.  We would need to be awfully careful not to forget to add the annotation when creating new experimental methods, though.  I can see us forgetting this more than once.  However, this is already an issue anyway if the class we're adding to is marked @Frozen.

I was predisposed against a "assume everything's fine unless we specifically say otherwise" model, thinking it would be safer to say "assume nothing unless we explicitly say so".  But in reality, developers are by and large going to do the former anyway; it's the way we are.

On Thu, Mar 25, 2010 at 11:14 AM, Joshua O'Madadhain <jr...@google.com> wrote:

I don't like "Unfrozen" because that implies that it was once frozen.  How about "Liquid"?  "Fluid"?  :)

@NotFrozen, then.  But it doesn't work so well when not paired with @Frozen.  I still feel like @Experimental overstates the case.

How about @Tentative
 

[Graham Allan]

>Finally, about Bob's suggested term "Experimental", I think it's a much
> stronger statement than we are really trying to make. The method and its
> behavior might be quite tried and true but just pending a final decision on
> its name -- that doesn't sound like an experimental feature to me, just a
> not-API-frozen one.

If that's a good way to describe your intent you might not find a better
adjective than the one you used...

@Pending
@ApiState(ApiStates.PENDING) ?

~ Graham

[Kevin Bourrillion]

And on further reflection, I think we'll simply need to be able to count on those tools.  (Assuming a volunteer appears who wants to create them!)  So I take back this objection to the mark-only-unfrozen proposal.  As well, there's another reason to favor it: years from now, we all hope that the frozen portion of the library will be much larger than the unfrozen portion.  So it makes sense to consider the unfrozen parts as the ones worth flagging as being the exceptions to the norm.

Currently I am liking @Beta (hey, it's Google tradition!) or @Draft.  Or, because what we're talking about is just like what you see at http://code.google.com/labs/, only at a smaller granularity, there's @Labs. But that seems a little more awkward to me.

[Torbjorn Gannholm]

@Experimental - seems a lot less stable than we intended
@Beta - makes me wonder if the code might not work, rather than that the API might change.
@Draft - OK, but still seems a little less thought-through than it really is.
@Labs - the problem of both @Experimental and @Beta

@Tentative - this is what is planned, but we reserve the right to change it if something comes up.

[Etienne Neveu]

*** On having one / multiple annotations ***
I like the idea of having an unique annotation to describe "unfrozen"
API.
Unfrozen API is where we want people to really pay attention.
I think @Tentative or @TentativeAPI would be good, but I'm not opposed
to @Beta or @Labs.

I don't like the idea of having two annotations (@Frozen and
@Unfrozen, for example) and having "method-level annotation overriding
the class-level annotation". While this sounds very flexible on paper,
the developer would be forced to always remember the "context" he is
in when stumbling on an unannotated method ("context" meaning whether
the class holding the method he wants to use is frozen or not):

** Developer thinking: "Maps.newGodlyMap(), this method sounds really
cool! But wait, is it tentative? Hmmm it's not annotated with either
@Frozen or @Unfrozen. Was the Maps class frozen or not? Mmh let's
check. *scrolls up* Oh yeah, the map class is frozen, so this means
this method is frozen too. Cool, let's use it then!" **

When coding, we already have to juggle with many abstractions and
contexts, let's not add another one.

Some question to ask ourselves before using @Frozen/@Unfrozen and such
an overriding mechanism:
- would it make sense "semantically" to have an @Unfrozen class with
@Frozen methods inside? If the class itself is unfrozen, how could a
method inside it be frozen?
- let's say you have a com.google.common.collect.Maps class with both
frozen and unfrozen static utility methods. Should you annotate the
class with @Frozen or @Unfrozen?

I think we should have frozen classes / methods by default, and we
should draw attention to unfrozen methods with an annotation.

You may forget to add the @Tentative annotation on new methods, but
the build/tools should tell you so in this case.

On Mar 25, 9:30 pm, Kevin Bourrillion <kev...@google.com> wrote:
> As well,
> there's another reason to favor it: years from now, we all hope that the
> frozen portion of the library will be much larger than the unfrozen portion.
> So it makes sense to consider the unfrozen parts as the ones worth flagging
> as being the exceptions to the norm.

Very good point. In the end, when most of the API is frozen, having
@Frozen annotations all over the code would be just clutter.
Readability is important, and limiting annotations would help
readability.

--------------------------------------------------------

*** On the @Frozen/@Unfrozen versus @Frozen(true)/@Frozen(false)
debate ***

If we still choose to go with multiple annotations, I don't like the
idea of having something like @Frozen(true) / @Frozen(false) or
@ApiState(States.Frozen) / ApiState(States.UnFrozen).

Yes, it would be more flexible, and it would stop us from accidentally
annotating the same method with both @Frozen and @Unfrozen.
But:
- it's less readable: @Unfrozen is shorter/cooler than @Frozen(false)
- using such language tricks is very useful and elegant, especially
when it stops *your API users* from making mistakes. But in this case,
your users will only "read" these annotations (1). You are the ones
who control the Guava code and annotate methods, and you will not make
these mistakes because...
- your build may be configured to alert you when you annotate a member
is annotated with both annotations

(1) - Unless the plan is to create @Frozen/@Unfrozen annotation which
will then be used by other library developers in their own APIs? But I
think the goal is mainly to document the guava code, not to create a
"versioning framework".

--------------------------------------------------------

*** On versioning ***
Instead of using dates/numbers, why not add an enum somewhere, such
as:

public enum Version {
/**
* Version 1.0
* Added 2009-03-31.
*/
1_0,
/**
* Version 1.1
* Added 2009-04-22.
*/
1_1,
/**
* Version 1.5.
* Added 2009-05-13.
*/
1_5;
}

Then in your code:
@Since(Version.1_1)

Advantages:
- user will see useful informations in the javadoc (by hovering over
the version enum). Of course, you shouldn't add too much information
there if you don't want the enum to become really big, but some basic
info about the version, as well as a link to the release notes, would
be great. Modern IDEs (at least Eclipse) show the javadoc on hovering.
- you can very simply find all methods added in a specific version
(ctrl + alt + H in Eclipse - find usages)
- type safe: no typo allowed

Disadvantages:
- you may not use dots in a java variable, so you have to write 1_0 /
1_2_31 instead of 1.0 / 1.2.31

[fishtoprecords]

Can I suggest we back up a bit. Think about the claim during the
recent banking crisis that some banks were "too big to fail" or at
least some power brokers claimed it was so. To me, the answer is, OK,
bail them out, and then break them up so they are no longer "too big"
by whatever metric you like.

From first principals, what is the real goal of Guava? I've been a
user of Google-Collections for a long time, and love them, and if
there were more depth and breath to this new Guava thing, that sounds
cool to me.

But the complexity that Kevin mentions from the start of this thread
is real. Its real in any software project. And there are billions of
lines of code and billions of dollars spent on projects that flat out
fail because of their complexity.

Perhaps we need, instead of some trick with annotations, to use a more
fundamental part of "agile development" and not try to make the
greatest most google-teriffic library in the universe. Partition the
problem, with clearly defined modules and clearly defined internal
walls. If the Google Matrix Math library has to rely upon a specific
version of the Google-Collections library, is it really all that bad
to bind them closely and internally?

[Vladimír Oraný]

@Experimental is fair enough. We don't have any @NonDeprecated
annotation too, so why would we have @Publish/@Frozen ones?

On 25 bře, 19:13, Tim Peierls <t...@peierls.net> wrote:

> On Thu, Mar 25, 2010 at 1:58 PM, Kevin Bourrillion <kev...@google.com>wrote:
>
> > "experimental" would be the default, so we could do without @Experimental
> >> entirely. Thoughts?
>
> > Again, we need both to be explicit.  Picture a class like Maps.java having
> > 50 frozen methods and one unfrozen/experimental one.  If we only had
> > @Published, we'd have to put it on all 49 of the others.  With both, either
> > can override the other.
>

> I still don't understand why @Published/@Frozen is necessary. How
> useful/likely is the opposite case: a class with 50 experimental methods and
> 1 published method? Not very, right?
>
> So the real default is frozen -- that's what we assume about an interface
> that makes no claims currently, right?
>
> In which case, how about just @Experimental (on class or method) and not
> @Published/@Frozen/etc.? I like @Experimental -- doesn't seem too strong at
> all.
>

> --tim

[Ray Conner]

Having read 50+ responses so far...

I agree with the annotation approach, but only one for @Beta (or
whatever you name it). As the library becomes more stable, I would
rather see fewer annotations than @Frozen on every class.

As an alternative/additional approach, you should seriously consider
putting experimental code in a separate package. I'm in the process of
rewriting one of my projects, and using different packages is
definitely the cleanest (simple) way to keep users of the old and new
libraries from being incompatible. Especially since both need to live
in the same class loader. But that doesn't help too much with the "one
experimental method on Maps" problem.

Most of you are thinking about the problems you'll encounter using
Guava directly (since that's who's on this forum), but you control the
Guava-calling code. So it's not really a huge problem. A much tougher
problem would be if your project depended on both
1) Widget, which depends (possibly transitively) on Guava 1.0
2) Gizmo, which depends (possibly transitively) on Guava 2.0
and both Widget and Gizmo use parts of Guava that are API-incompatible
with each other. The user has very few options here, other than
abandoning or forking one of their dependencies.

Now, multiply that by a few more dependencies, since Guava is a very
general library and could be used by many other projects.

Using the same package bit me recently. We were on Java 1.5 and
(transitively) using a 3rd party implementation of javax.xml.bind.
Then we moved to Java 1.6 and everything broke, because the 3rd party
implementation was API-incompatible with the classes/interfaces in the
new JDK. IMHO, Sun made a huge mistake allowing 3rd parties use of the
javax package namespace. We've had the same problem previously with
org.w3c.dom implementations.

- Ray A. Conner

[Kevin Bourrillion]

On Fri, Mar 26, 2010 at 10:39 AM, Ray Conner <ray.a....@gmail.com> wrote:

I agree with the annotation approach, but only one for @Beta (or
whatever you name it).

I'm going down this road so far.  Calling it @Beta.

As an alternative/additional approach, you should seriously consider
putting experimental code in a separate package.

We do this internally, with packages called com.google.common.labs.*.  It's hell.  All the users have to change when things graduate, and even worse, we can't access other parts of the library that are package-private, so things have to be made public that shouldn't be.  I'm not eager to replicate this practice further!

 

I'm in the process of
rewriting one of my projects, and using different packages is
definitely the cleanest (simple) way to keep users of the old and new
libraries from being incompatible.

So this assumes that each time an incompatible change is made, you're always changing the package.  I don't see this working out.

Most of you are thinking about the problems you'll encounter using
Guava directly (since that's who's on this forum), but you control the
Guava-calling code. So it's not really a huge problem. A much tougher
problem would be if your project depended on both

Yes, I hoped to draw exactly this distinction in my original message, though it may have been lost among all the other verbosity.

[Bob Lee]

Java Modules (coming in Java 7) will really help here. We'll be able to have two modules: guava and guava-experimental

We can use the same package names in both modules (no painful graduation). "guava" can depend on "guava-experimental" without exposing the experimental APIs to modules that depend just on "guava".

It should be possible for "guava" and "guava-experimental" to share private APIs between each other so you don't have to make stuff public unnecessarily.

I'm not sure how adding experimental methods to stable classes would work. Maybe we could have the same type in both guava and guava-experimental. The version in guava-experimental would override the version in guava.

We could generate both modules from one source tree. We'd use "#ifdefs" to delineate the experimental parts. We'd strip these parts out to generate the stable module. 

Bob

[Bob Lee]

Hey, we could benefit from the "ifdef" approach even without Java modules. We don't need an annotation. We can use vanilla JDIff (no need for a wrapper).

If you have an experimental method that you want to use from the stable code, you can do this:

public class StableClass {

  ...

#ifdef BETA

  public

#endif

  static void betaMethod() {

    ...

  }

}

Bob

[Raymond Conner]

On Fri, Mar 26, 2010 at 1:46 PM, Kevin Bourrillion <kev...@google.com> wrote:

On Fri, Mar 26, 2010 at 10:39 AM, Ray Conner <ray.a....@gmail.com> wrote:

As an alternative/additional approach, you should seriously consider
putting experimental code in a separate package.

We do this internally, with packages called com.google.common.labs.*.  It's hell.  All the users have to change when things graduate, and even worse, we can't access other parts of the library that are package-private, so things have to be made public that shouldn't be.  I'm not eager to replicate this practice further!

Yeah, there may be no good solution to this, just a least bad solution.
 

 

I'm in the process of
rewriting one of my projects, and using different packages is
definitely the cleanest (simple) way to keep users of the old and new
libraries from being incompatible.

So this assumes that each time an incompatible change is made, you're always changing the package.  I don't see this working out.

I'm not thinking so much of the case where you have a few methods that are unstable, but when you have an entire sub-framework or usage pattern that is. As a thought experiment, what if apache commons-collections were actually GCL 0.9? (they really serve different use cases, but it's the best I could think of) The two approaches are so different that a package change is not really a stretch to justify. Or maybe migrating from a pojo pattern to fluent builders.

I have changed my project in ways that are API-incompatible in the past, with no package change. But this next version is going to be so different that keeping the same classes and interfaces, let alone packages, makes no sense. That and I don't control all the code that uses my library; I can't force others to upgrade, and I have to use their code. A package change is really my only option without some kind of class loader separation.

 

--

Kevin Bourrillion @ Google
internal:  http://goto/javalibraries
external: http://guava-libraries.googlecode.com

--