Closure Prep for java7?

[Reinier Zwitserloot]

java7 isn't going to have closures, but I don't think the topic has
been dropped entirely; just delayed, right?

Via Neil Gafter's twitter stream, the simple idea that you need to
entirely retrofit the Collections API (with methods like 'filter',
'map', 'map.foreach' and other functionally inspired methods) and also
parts of the file API (with methods like 'readLines', 'doWith',
etcetera).

Unless of course, you add extension methods. The ability to declare in
an interface a method along with a default implementation. There's no
issues with multiple inheritance - if that ever happens (one class
gets 2 different default implementations by 2 different interface
chains), then the rule is simple: Don't allow compilation, or even
loading of, the class. Instead, the java file needs to be explicit (by
implementing the method itself. They can refer to any default
implementation by full name in the method body). Then it'll compile
and run fine.


Given the sheer amount of work you'd have to do re-engineering the
java API, breaking backwards compatibility, or adding extension
methods, is really the only way.

So, given that closures are likely coming in java8, wouldn't it be
nice to add the relatively low-impact extension method system right
now?

[Casper Bang]

+1. What you're really after though are mixins, not extension methods
no?

/Casper

[Edward]

Java closures discussion is dead end, quite that simple. There are two
opposite forces and no consensus on that issue in a foresseable
future.:-( I was eagerly hoping for new features like closures in Java
language, but any real evolution I've seen is being directed towards
the vm, tools, specs and alike. The big mammoth (aka JEE) is an
outstanding example of this.

Maybe we can see closures in Java *someday*, but don't count on that.
It will be long after part of the community has adopted other
languages (Scala, Ruby, Clojure, etc) in addition to Java, and as a
desperate attempt to regain market position.

/Edward

[Viktor Klang]

But Reinier had a "supersolution"?

--
Viktor Klang
Senior Systems Analyst

[Alex Buckley]

On Feb 21, 8:25 am, Reinier Zwitserloot <reini...@gmail.com> wrote:
> So, given that closures are likely coming in java8, wouldn't it be
> nice to add the relatively low-impact extension method system right
> now?

What evidence do you have for your statement about closures? Moreover,
where can I read about your proposal for extension methods?

Alex

[Alex Buckley]

On Feb 21, 8:25 am, Reinier Zwitserloot <reini...@gmail.com> wrote:

> Given the sheer amount of work you'd have to do re-engineering the
> java API, breaking backwards compatibility, or adding extension
> methods, is really the only way.

Happily, the existence of a versioned module system in JDK7 allows
incompatible API evolution where it is considered necessary.

Alex

[Reinier Zwitserloot]

Still working out the details on the supersolution. And figuring out
if I have the time. At this point, without a prototype, I'm not sure
if you can (or, in fact, should) be taken seriously.

I don't think you can just evolve incompatible APIs because you have a
module system, Alex.

Let's say we add a 'filter' method to list. How? We can only change
List if we add List to the versioned set. But if we do that, then any
two modules that require different versions can no longer talk to each
other if the talk protocol involves Lists, which are almost as 'near-
primitive' as Strings. That's not going to fly. You still need a way
for an old 'view' on Lists, or for two separate List libraries to be
able to hand each other wrappers or some other solution to cross this
bridge.

Or you use extension methods and you don't have this problem.


NB: If you want a full proposal, it's very simple:

A) allow static methods in interfaces. There's no sane reason I can
think of for why this is disallowed in the first place.

B) Use an annotation or keyword (@Extension), legal only on static
methods inside interfaces where the first parameter of the method has
a type that is equal to the interface itself. So, this would be legal,
in java.util.List:

@Extension public static <T> void sort(List<T> list, Comparator<T>
comparator) { /* code */ }

C) Whenever the compiler encounters a call which cannot be resolved
(no such method), then the compiler will back up and look through the
chain of interfaces implemented by the type of the object for which
the call cannot be resolved. If it can match the call to an extension
by moving the object into the first parameter position, then it'll
compile that instead. The fact that this will not behave in a virtual
manner (will not runtime-find implementations on subclasses) is
intentional.


Thus, something like this:

list.sort(someComparator); //not legal now

will then be compiled as: List.sort(list, someComparator);

If there are multiple choices to make, then: If there's one interface
that is hierarchically closer to the target type than any of the
others, that one wins. If you can't make this determination (diamond
pattern; type A implements both B and C, where B and C both have a sort
() extension), then do not choose at all and instead generate a
compile-time error. This can be solved by either calling one explicit
version [use B.sort(me) instead of me.sort()] or fix it in A by adding
a wrapper method, like so:

@Extension public static <T> void sort(A<T> list, Comparator<T>
comparator) {
C.sort(list, comparator); //We'll explicitly choose C
implementation.

[Reinier Zwitserloot]

Well, I don't have proof that closures are 'definitely' coming in
java8, but I have heard no statement from anyone in any sort of
official capacity that closures are over. Just that java 7 already has
more than its fair share of language changes as is, and closures are
too big (and there's too much arguing between the various proposals).
It also seems, from admittedly not statistically reliable information
(asking people at java conferences about java doesn't really give you
a view of the average joe's opinions), that there are significantly
more people in favour of some sort of closure proposal vs. not, so if
the stalemate can be resolved, which is perhaps somewhat more likely
with the 'shock' that closures still won't be in java until at least
2011 or so as a best case scenario (guesstimating java 8's release)
that the stand-offish nature of the current closure situation will
diminish somewhat.

In other words, I have hope, that's all.

[Mark Derricutt]

How about stealing/reusing the C# idiom of a "this" parameter:

public class MyList {
  public static void sort(this List<T>, Comparator<T> comparator) {
   ..
  }
}

Which compiles to the same as you suggest, being

new List().sort(comparator) being

MyList.sort(new List(), comparator);

This could also play well with introducing "this" as a return type for chaining.

...and then Buffy staked Edward.  The End.

[Alex Buckley]

On Feb 23, 7:33 pm, Reinier Zwitserloot <reini...@gmail.com> wrote:
> I don't think you can just evolve incompatible APIs because you have a
> module system, Alex.

What is the point of versioning in a module system if not to allow
multiple INcompatible versions to co-exist? (That doesn't require
every upgrade to introduce an incompatibility, only that a versioning
scheme describes when and how compatibility can be affected.)

> Let's say we add a 'filter' method to list. How? We can only change
> List if we add List to the versioned set. But if we do that, then any
> two modules that require different versions can no longer talk to each
> other if the talk protocol involves Lists, which are almost as 'near-
> primitive' as Strings. That's not going to fly. You still need a way
> for an old 'view' on Lists, or for two separate List libraries to be
> able to hand each other wrappers or some other solution to cross this
> bridge.

Of course java.util.List is in a versioned module. Every piece of code
you ever write is versioned, whether you write down the version or
not. Two modules that require disjoint versions of List cannot pass
Lists between themselves, but that is likely to be rare. What is far
more likely for core JDK classes is that we will promise to evolve
them in a way that is a) binary-compatible, b) behavior-compatible,
and c) not source-compatible. In other words, we will add methods to
List but (probably) not redefine existing methods. Again, this is what
a versioning scheme is for. Putting it all together, a module which
requires version >=7 of List and a module which requires version >=8
of List (since the module uses List.filter) can both get version 9 of
List and cooperate at runtime.

> Or you use extension methods and you don't have this problem.
>
> NB: If you want a full proposal, it's very simple:

Few things in the Java language are as simple as they seem.

Alex

[Alex Buckley]

Given that Mark has targeted JDK7 for early 2010, your estimate for
JDK8 is almost certainly a year too early at least.

I am sad that "closures" have come to be seen as some kind of panacea,
where people think having them gives Java a bright and secure future
while not having them gives Java a quick and certain death. If we did
the full BGGA proposal tomorrow, people would quickly determine that
properties, or reified higher-kinded generics, or traits, or
multimethods, or pattern matching, or keyword methods, or operator
overloading, or data binding, or any number of features drawn from
other languages must be shoe-horned into Java too, or else it will
die. But one language cannot be an amalgam of every other language,
especially when Java's vast base of existing libraries and fondness
for migration compatibility are taken into account.

Alex

[Viktor Klang]

Jorge Ortiz, Christos Loverdos and myself have a slumbering Scala Improvement Proposal to include what you call "extension methods" to Scala code.

We've experimented with different syntax and it depends on how you want it to work.

I was pulling toward a more "type selector" based mechanism like the following:

def [T <: String].isDigitsOnly() = this.matches("^\\d$")

where the prefixing type declaration selects which types are allowed to be attached to

basically it's syntactic sugar over:

def isDigitsOnly[T <: String]( t : T) = t.matches("^\\d$")


so it's basically possible to call it using:

isDigitsOnly("blah")

and as

"blah".isDigitsOnly()


Well well, I'm crazy-talking again.

Cheers,
Viktor

[Reinier Zwitserloot]

Nope, if you want version 7 of List, you can't even use version 9 of
List, because version 9 of List declares 'filter', which you can't
work with. The module system does not,as far as I can tell, allow you
to make the distinction: "I'm just using these" and "I'm extending
list, and therefore you can't give me version 9, or there's a runtime
error when I don't have my filter method".


Your rant on closures being seen as the key to the life of java: Who
here in this thread has claimed that java is going to die without
closures, and is going to be the best of the best with them? You're
arguing against a straw man.

Still, you've given me an idea. If java never gets closures, then java
really is dead*. Not because closures are a panacea, but, for an
entirely different reason: The majority of the community wants them.
If java never delivers something that everyone wants, it's dead.
Simple. Doesn't matter what the feature being asked for is. As long as
the feature is reasonably well defined and there's a way to get there
without too much collateral damage, java should get there if the
community has made up its mind. It's okay if this takes careful
deliberation and years of design considerations, but if it does happen
within a reasonable time-span after community consensus, java is
dead**.

*) Assumptions:

1. The community is convinced closures are a good idea (which at java
events seems to be sort of true, but that's not the ENTIRE community,
just the most active members, though their say certainly should count
more. The internet is a meritocracy, not a democracy, because the
meritocracy model is more efficient, the internet is ruthless
competition, and going against this grain will eventually get you
killed.

2. At some point the community starts focusing on a single closure
proposal, which might simply mean that the focus shifts a little from
'What's the best closure proposal' to 'which one(s) are good enough
for you?'

If those assumptions don't pan out, then java not getting closures is
no cause to say java is dead.

**) 'dead' in the sense that cobol is dead. Due to java's legacy,
we'll have java code running critical systems and a few old farts
earning a pretty penny keeping them running for a very long time
indeed, but we all have our gut instincts regarding languages like
cobol, ada, pl/1, fortan, etcetera. When I say those languages are
'dead', I mean: They don't get taught at universities and you'd be
laughed out of the room for suggesting it there, no one is offering
courses, and nobody is lining up to learn them autodidactically
either. No new projects are being written in them, and deployments are
ever going down, never going up.

NB: Viktor - a key difference between many of the extension point
proposals is where you are allowed to declare the extension. In many
proposals, you use a variant of static import (something like: import
static Collections.sort into List), but now we either need to
configure our IDEs to add a busload of static import statements on
every source file, which can't be the right answer, or we need a 'read
this file, and import everything IT imports' kind of system, or we
need a module-level import system. Other proposals allow the target
type to declare its own extensions, which doesn't allow you to 'hack'
existing code, but it does allow existing code, particularly
interfaces, to add new stuff without breaking backwards compatibility,
and to add functionality on interfaces which doesn't usually change
between implementations thereof, such as most of the stuff in
java.util.Collections and java.util.Arrays. There are advantages and
disadvantages to each, and you could even combine the proposals quite
easily.

[Alex Buckley]

On Feb 24, 6:15 am, Reinier Zwitserloot <reini...@gmail.com> wrote:
> Nope, if you want version 7 of List, you can't even use version 9 of
> List, because version 9 of List declares 'filter', which you can't
> work with. The module system does not,as far as I can tell, allow you
> to make the distinction: "I'm just using these" and "I'm extending
> list, and therefore you can't give me version 9, or there's a runtime
> error when I don't have my filter method".

Again, where is your evidence for these statements? Let me inject some
reality:

- A module which asks for version >=7 of List because it uses (but
does not extend) List can work with version 9, no problem. The
presence of additional List methods makes no difference.

- A module which asks for version >=7 of List and extends List cannot
work with version 9 because adding methods to an interface is source-
incompatible. (The usual javac error will result when compiling the
extension of List.) A module containing a type that extends List
should ask for version 7.* of List, since source-incompatible changes
should not be made under the same major version (7).

Therefore, the module system can discriminate between "safe" use
(references to List) and slightly more "dangerous" reuse (subclassing/
subtyping).

> Your rant on closures being seen as the key to the life of java: Who
> here in this thread has claimed that java is going to die without
> closures, and is going to be the best of the best with them? You're
> arguing against a straw man.

One paragraph does not a rant make. The gist of many discussions on
"closures" is that "everyone" wants them and that Java is dead without
them because people will flock to Fan/Groovy/Scala/Ruby. I am an
enormous fan of Scala, but I recognize that Java and Scala have
complementary, not competing, roles. I look forward post-JDK7 to
better interoperation of the statically typed JVM languages; perhaps
that will come for free if the dynamic typing guys agree on a MOP.

Alex

[Reinier Zwitserloot]

Right, so you agree that there are situations where a module must ask
specifically for List[1..7] and cannot just go for [1..*]. I contend
that these situations are sufficiently common that there's going to be
a LOT of headache with incompatible modules, if this happens.

You also won't get any errors when you start implementing list when
you compile this with java7, which is presumably the state of the art
when the library is written. In other words, we have the following fun
situation:

A tonne of libraries out there will all of a sudden break when the new
java8 (or whatever) is released with new methods on java.util.List,
because they mistakenly specified compatibility with 7+. There's no
way to test for this, certainly not at first (at some point I expect
findbugs and other tools can find likely problems in this area). Also
known as a clustersomething.

Can't be done.

> Alex Buckley said: "One paragraph does not a rant make"

When you're attacking a strawman with hyperbole terms, then, yes, it
was a rant.

[Alex Buckley]

On Feb 24, 12:26 pm, Reinier Zwitserloot <reini...@gmail.com> wrote:
> Right, so you agree that there are situations where a module must ask
> specifically for List[1..7] and cannot just go for [1..*]. I contend
> that these situations are sufficiently common that there's going to be
> a LOT of headache with incompatible modules, if this happens.

I agree versioning is a big deal and all module systems - Jigsaw,
OSGi, Maven, NetBeans - have to educate their users about it
carefully. Almost no module should depend on a version with an
infinite upper bound. The common practice in OSGi today is to use
bounded ranges like [1.0,2.0). A module using or reusing List should
ask for >=7 or 7.* respectively, which is not possible today and
prevents Sun and others from sanely evolving APIs.

> You also won't get any errors when you start implementing list when
> you compile this with java7, which is presumably the state of the art
> when the library is written. In other words, we have the following fun
> situation:
>
> A tonne of libraries out there will all of a sudden break when the new
> java8 (or whatever) is released with new methods on java.util.List,
> because they mistakenly specified compatibility with 7+. There's no
> way to test for this, certainly not at first (at some point I expect
> findbugs and other tools can find likely problems in this area). Also
> known as a clustersomething.
>
> Can't be done.

Legacy libraries implicitly have dependencies on the JDK up to and
including 7.0. Such libraries are easily identified at compile-time
and runtime, and bound forever to JDK 7.*. (As I said earlier, the 7.*
family is very very unlikely to make source-incompatible changes to
official APIs.) I am pretty sure the Jigsaw module system will default
to x.* version ranges rather than >=x (i.e. no upper bound) version
ranges.

In years to come, a library (legacy or otherwise) which likes new JDK8
List methods can implement List and depend on JDK >=8 specifically.

Alex

[robilad]

On Feb 24, 9:26 pm, Reinier Zwitserloot <reini...@gmail.com> wrote:
> A tonne of libraries out there will all of a sudden break when the new
> java8 (or whatever) is released with new methods on java.util.List,
> because they mistakenly specified compatibility with 7+. There's no
> way to test for this, certainly not at first (at some point I expect
> findbugs and other tools can find likely problems in this area). Also
> known as a clustersomething.

I'd assume that it's rather easy to test for such situations
automatically.

I mean, tools like https://sigtest.dev.java.net/faq.html and
http://clirr.sourceforge.net/ have existed for a while out there, and
any sufficiently large project has their own built from scratch.

The missing component to make it work smoothly is being able to
given a version constraint figure out where to find the source code
for
the dependency or the corresponding binary object or a representation
of
its exported API. Given a module system, it shouldn't take much effort
to mine the metadata and put it to such use.

You can do all kinds of fun things automatically once you have a
metadata graph, where nodes carry actually useful annotations,
like tests, sources, etc. rather then just a bunch of binary blobs.
See http://www.cpantesters.org/show/JSON.html#JSON-2.14 for a
tiny example.

A fun blog to read on the tangetial subject of modernizing a platform
is chromatic's at http://www.modernperlbooks.com/mt/index.html

[Reinier Zwitserloot]

Okay, so, now, you're saying that the default for just about every
library out there is going to be java[7,8), whereas before you said
that updating list is no problem, because most modules will say they
want the list from java[7,*].

You're not making any sense.

[Reinier Zwitserloot]

robilad: Absolutely, you can do wonderful things. If java wants to
take the official position that serious developers will be using
serious tools, and non-serious developers don't really need the
serious backwards compatibility, I'd be very happy. Couldn't happen
soon enough. But its one of those commitments that I doubt are going
to be made, so from a realistic standpoint, I'm just saying that this
feeling of 'We'll get modules and then we can evolve whatever APIs we
want! Freedom!!!!' is just not going to happen, or if it is, we'll
need to start advocating the mandatory use of such tools, work on
getting findbugs integrated into javac, start including things like
how certain code features will be rendered in IDEs (and come up with
an IDE standard of sorts), and more.

On Feb 25, 2:30 am, robilad <dalibor.to...@googlemail.com> wrote:
> On Feb 24, 9:26 pm, Reinier Zwitserloot <reini...@gmail.com> wrote:
>
> > A tonne of libraries out there will all of a sudden break when the new
> > java8 (or whatever) is released with new methods on java.util.List,
> > because they mistakenly specified compatibility with 7+. There's no
> > way to test for this, certainly not at first (at some point I expect
> > findbugs and other tools can find likely problems in this area). Also
> > known as a clustersomething.
>
> I'd assume that it's rather easy to test for such situations
> automatically.
>

> I mean, tools likehttps://sigtest.dev.java.net/faq.htmlandhttp://clirr.sourceforge.net/have existed for a while out there, and

> any sufficiently large project has their own built from scratch.
>
> The missing component to make it work smoothly is being able to
> given a version constraint figure out where to find the source code
> for
> the dependency or the corresponding binary object or a representation
> of
> its exported API. Given a module system, it shouldn't take much effort
> to mine the metadata and put it to such use.
>
> You can do all kinds of fun things automatically once you have a
> metadata graph, where nodes carry actually useful annotations,
> like tests, sources, etc. rather then just a bunch of binary blobs.

> Seehttp://www.cpantesters.org/show/JSON.html#JSON-2.14for a

[Alex Buckley]

On Feb 25, 8:54 am, Reinier Zwitserloot <reini...@gmail.com> wrote:
> Okay, so, now, you're saying that the default for just about every
> library out there is going to be java[7,8), whereas before you said
> that updating list is no problem, because most modules will say they
> want the list from java[7,*].

Unmodularized code - which claims no module membership and gives no
explicit dependencies - implicitly depends on JDK 6. Assuming no
breaking changes in JDK 7, such code can be bound when running on a 7
VM to JDK 7.*. 7.* is also known as known as [7,8).

Modularized code - which gives explicit dependencies - can be split
into code which a) uses or b) reuses "core" APIs like List. I said "A
module using or reusing List should ask for >=7 or 7.* respectively".
In fact, it's still pretty naughty to rely on >=7 even if you simply
refer to List, since methods might be removed or made less accessible
(binary incompatible change), but I can imagine ways the risk might be
mitigated by the provider, e.g. JDK 8.

> You're not making any sense.

What we're mainly discussing is migration compatibility - how an
unversioned library uses or reuses versioned libraries. Back in JDK
1.5, the most subtle part of generics was migration compatibility -
how a non-generic client uses or reuses generic types. You are keeping
me honest by posing your questions, keep them coming :-)

Alex

[robilad]

On Feb 25, 5:58 pm, Reinier Zwitserloot <reini...@gmail.com> wrote:
> need to start advocating the mandatory use of such tools, work on
> getting findbugs integrated into javac,

A. Why would you want to integrate findbugs into javac?

B. Mandatory use of tools wouldn't work too well. Yes, you can make
tools that tell developers that they aren't perfect, euphemistically
speaking, but if they are sufficiently capable with any programming
language, they'll find a way to work around the litany of the
mandatory
tools, for example by logging it to /dev/null .. ;)

In a galaxy far away, a long time ago, a compiler called jikes tried
to
do that - their last or so release basically made a -Wall mode the
default one,
and showered the users with pedantic warnings about the quality of
their
code. Unsurprisingly, faced with hundreds or thousands of warnings on
code bases that compiled cleanly with other compilers, developers
blamed
the compiler, and turned the pedantic mode off, rather then spending
their
time fixing the issues the compiler complained about.

Static analysis tools can tell you which of the code out there is
below average
quality - Freud can tell you that its developers likely don't want to
know that,
anyway.

[Mark Derricutt]

Maybe integrate findbugs into the javac process chain, much like APT and annotation processing is now part of the chain, but not PART of.

...and then Buffy staked Edward.  The End.

[Reinier Zwitserloot]

Alex, you haven't explained how we can migrate list safely, then.

If lots and lots of legacy code cannot accept new-style lists, then
trying to integrate with the older code is going to be a very
frustrating exercise.

Which brings us back to my original point:

Modularization is very cool, and we really need it, but without some
serious wrapping/view/translation tricks, modularization is not going
to allow us to fix bad core API. Some work on the fringe might be
possible if sun is okay with losing backwards compatibility in very
rare circumstances.

[Reinier Zwitserloot]

Integrating findbugs into javac is good because javac currently does a
half-arsed job. It warns on some strange situations, produces stupid
errors on others ('unreachable code', which has 'this should have
been a warning' written all over it!), catches certain code mistakes
but doesn't catch similar code mistakes elsewhere. It's a mess. The
notion that java is statically typed is often claimed to be good for
finding bugs at compile time.

Live the story. Make that happen. What are the downsides of
integrating findbugs into javac, especially if it is done in a way
that you can easily replace findbugs or upgrade that part of it
separately?

I don't think we're on the same line on tool usage. I'm talking about
using an IDE that helps you with auto-complete and is capable of
rendering texts a certain way (which opens the door to eliminating
arguments of 'but that'll be very confusing' with: Okay, so we mandate
that certain usages get rendered differently from other usages which
look the same in plain ascii, but do something entirely different,
hence lowering the confusion. That kind of argument doesn't really fly
today. Or, findbugs. Or, taking for granted that any java user can
call up a full object graph. Or, taking for granted that a new module
system will force people to properly sort out dependencies, but not
caring too much about that because the notion that any java programmer
uses tools is accepted. (in this case, a tool that analyses a set of
modules and makes suggestions about making them work together)

It has absolutely nothing to do with logging?

[robilad]

On Feb 26, 5:04 am, Reinier Zwitserloot <reini...@gmail.com> wrote:
> What are the downsides of
> integrating findbugs into javac, especially if it is done in a way
> that you can easily replace findbugs or upgrade that part of it
> separately?

Well, the point of javac is that it operates on Java source code. The
idea
behind findbugs is that it operates on Java bytecode - because for
most
binaries out there, there is no way to reliably rebuild them even if
you
want to, so bytecode is what a typical static analysis tool works
with, if it
wants to reach an audience outside the top development shops that
have
their full dependency chain's source code under their control and can
rebuild it on a spot. :)

The real value of tools like findbugs is not in checking your own,
hopefully
spotless code that you have the source code for .- it's in checking
your
potential dependencies before you commit to using them, so that you
can
weed out the bad choices in advance. 'L'enfer, c'est les autres.'

So yeah, one could rewrite findbugs to do its checking on source code
instead, and thereby limit its usefulness to people who always have
all
the sources they need, or one could leave it as it is, and run it on
bytecode produced by javac - but that's something everyone can do
(and hopefully does) already anyway.

Should my IDE run findbugs for me and tell me that my dependencies
are quite likely junk software? Yes, please. Should my IDE be able
to
tell me that the tomcat JAR I'm using in my project through Maven
hasn't been updated in the Maven repo to fix a dozen serious CVEs?
Hell, yeah. Should an IDE force someone to do something about it?
Well,
probably not - a lot of people are stuck with their or their
predecessors
legacy of mvn-snapshot jars, code without adequate test coverage, and
what not - they can't change their environments overnight, sometimes
not
at all (never touch a running system).

[Reinier Zwitserloot]

Yes, findbugs works on class files right now, but there's no reason
why it couldn't work on source files. Some projects have tried
(arguscodewatch for example) and while they're mostly inactive at this
point, source-level scanning is better in many ways; you can have
quick fix suggestions and certain findbugs-esque things can only be
detected on the source level and no longer in the class level.

So, perhaps not findbugs per se, but some sort of serious static
analysis tool, at least.

findbugs as a 'check somebody else's already compiled stuff' can stay
separate.

Also, the programming world as a whole is steadily moving towards more
open source. You can see this in the tooling as well: obfuscators have
virtually disappeared from the map and nobody cares enough about
'decompiling' java 1.5+ class files to write a good decompiler. So, if
you're going to add new tooling, it should definitely focus on source
files and not on class files.

Nobody is talking about forcing anything. @SuppressWarnings exists for
a reason. The compiler is there to tell you there's a problem you
should look at, and when you've looked at it, you can easily tell the
compiler to shut up henceforth. At least, that's the idea.

[Tom Copeland]

On Feb 26, 2009, at 12:18 PM, Reinier Zwitserloot wrote:

>
> Yes, findbugs works on class files right now, but there's no reason
> why it couldn't work on source files. Some projects have tried
> (arguscodewatch for example) and while they're mostly inactive at this
> point, source-level scanning is better in many ways; you can have
> quick fix suggestions and certain findbugs-esque things can only be
> detected on the source level and no longer in the class level.
>
> So, perhaps not findbugs per se, but some sort of serious static
> analysis tool, at least.

PMD does a good bit of source code analysis, including some limited
data flow analysis. As you say, compilation removes some things from
the source code, so PMD can find a few things that a bytecode checker
can't. What PMD doesn't do yet (if I recall correctly) is a complete
type resolution, e.g., this reference to "List" is a java.util.List,
not a com.mycompany.List. But the tools are in place for it to do
that....

Yours,

Tom

[Alex Buckley]

On Feb 25, 7:59 pm, Reinier Zwitserloot <reini...@gmail.com> wrote:
> Alex, you haven't explained how we can migrate list safely, then.
>
> If lots and lots of legacy code cannot accept new-style lists, then
> trying to integrate with the older code is going to be a very
> frustrating exercise.

In general, legacy code CANNOT use new-style lists. If JDK8 makes
binary-incompatible changes to List, e.g. drops size(), then only 8-
aware clients must get it. If JDK8 makes source-incompatible changes
to List, e.g. adds filter(), then only 8-aware extenders must get it.

There is no middle ground here; if incompatible changes are made in
version n (an enormous decision!), then some legacy code is stuck
forever in the pre-n world. Legacy code is implicitly bound to 7.*
classes on a 7 VM, an 8 VM, a 9 VM, etc.

For generics, there was a middle ground: legacy code was permitted to
use generic types with warnings where type correctness was
compromised. Those warnings were necessary even if the program is
ultimately type-safe. At least for modules, there is no such ambiguity
because it's clear which features are used by legacy code that are not
provided by a new List.

> Which brings us back to my original point:
>
> Modularization is very cool, and we really need it, but without some
> serious wrapping/view/translation tricks, modularization is not going
> to allow us to fix bad core API. Some work on the fringe might be
> possible if sun is okay with losing backwards compatibility in very
> rare circumstances.

Let's think about wrappers. Suppose JDK8 wants to drop a class. To
offer migration to a legacy client, the only possible option is to
leave the class in place but to annotate all its methods with
@CarefulThisMethodIsREALLYDeprecatedAsOf("8") and make the bodies
throw RuntimeException. We could imagine strategies like "javac: give
warnings in version n and errors in n+1" but ultimately the client is
going to have change their code. The provider wants that change to
happen ASAP. The easiest change for the client is to modularize itself
- this doesn't have to mean much more than declaring a dependency on
JDK 7.*. (And javac + a module system can assume this dependency
automatically, as I said.)

But look: if the client can be recoded to not use the dropped class,
then the "skeleton" class was unnecessary in the first place, since
it'll be obvious that a formerly usable class has become unusable. If
the client cannot be recoded to avoided the dropped class, then 7.* is
its dependency forever and the skeleton class adds nothing.

Other changes are harder. Suppose JDK8 changes a class from public to
module-private. Classes in the same module can still use it so the
method bodies need to stay intact. The trick of having a method check
who calls it at runtime (as proposed in http://wiki.apidesign.org/wiki/AlternativeBehaviour)
won't work since access control applies at compile-time.

Once the provider has made the choice to break binary compatibility,
the only option seems to be to let multiple versions (old "unbroken"
and new "broken") coexist.

Alex

[Reinier Zwitserloot]

I was more talking about e.g. a classloader that will on-the-fly
rewrite 'old style' code to 'new style' code. For example, it would on-
the-fly rewrite something like:

Point p = foo.getSize();
p.x = 10;
bar.setSize(p);

to:

MutablePoint p = MutablePoint.of(foo.getSize());
p.x = 10;
bar.setSize(p.toPoint());


Where the code to do that rewrite ships along with the library itself
(e.g. the above 'template solution' would be a part of Point v8.

Crazy stuff, perhaps. I don't think any programming language has gone
here before. But that's what I'm talking about: If something like that
was added, we really could unshackle almost all the old API and fix
it, thinking -only- of making sure the legions of java coders can keep
up with the changes mentally.

> who calls it at runtime (as proposed inhttp://wiki.apidesign.org/wiki/AlternativeBehaviour)

[Alex Buckley]

OK, I see now. I don't think it's crazy stuff. It's the kind of
creative thinking not done by existing module systems because they
don't have to handle unversioned and versioned code simultaneously.

As we start to split up the JRE, we'll see what kind of binary
incompatibilities look desirable either for 7 or post-7. I'll get back
to you :-)

Alex

[Reinier Zwitserloot]

True, such a crazy rewrite scheme would be most useful when handling
unversioned code,

but even 5 years down the line when most java code out there is
versioned and modularized properly, such a scheme - especially for
code which lies 'on the boundary' between modules would be very useful
still. A way to take code which is written for version X of a certain
library and make it semi-magically work with version Y (which is
newer, better, and not entirely compatible with X) is very useful.

[mbien]

this reminds me to this project: https://lang.dev.java.net/
it uses annotations to specify necessary refactorings to update a user
lib to a incompatible api version. It supports also annotations in a
class "overlay" (looks like a interface) in case you don't want to
have them in your api code.

would fitt pretty well to the 'compatibility classloader' proposal ;)

regards,
michael

[mbien]

wait, please ignore my last post - this won't work anyway without
additional efforts since classloaders don't deal with source code
bytecode transformation would be required to glue to a incompatible
api... not sourcecode refactorings - my fault

[robilad]

On Feb 26, 6:18 pm, Reinier Zwitserloot <reini...@gmail.com> wrote:
> Nobody is talking about forcing anything. @SuppressWarnings exists for
> a reason. The compiler is there to tell you there's a problem you
> should look at, and when you've looked at it, you can easily tell the
> compiler to shut up henceforth. At least, that's the idea.

Yeah, I understand - but that's the point I made about showering
developers
writing questionable code with warnings by default using Jikes as an
example
- they'll find it easier to turn them off, then to go and fix them. ;)

[Reinier Zwitserloot]

I dunno - what with the vocal 'use findbugs!' exclamations all around,
I'd imagine that 'not just blanket-ignoring all those warnings'
quickly becomes one of those things managers demand.

[robilad]

On Feb 28, 7:15 pm, Reinier Zwitserloot <reini...@gmail.com> wrote:
> I dunno - what with the vocal 'use findbugs!' exclamations all around,
> I'd imagine that 'not just blanket-ignoring all those warnings'
> quickly becomes one of those things managers demand.

But then people would hardly be using the JDK, Eclipse, NetBeans,
JBoss,
etc. as much they do - all those have hundreds of potential issues as
reported by findbugs - so clearly

[robilad]

On Mar 2, 11:30 am, robilad <dalibor.to...@googlemail.com> wrote:
> On Feb 28, 7:15 pm, Reinier Zwitserloot <reini...@gmail.com> wrote:
>
> > I dunno - what with the vocal 'use findbugs!' exclamations all around,
> > I'd imagine that 'not just blanket-ignoring all those warnings'
> > quickly becomes one of those things managers demand.
>
> But then people would hardly be using the JDK, Eclipse, NetBeans,
> JBoss,
> etc. as much they do - all those have hundreds of potential issues as

> reported by findbugs - so clearly it's not as much of a cut-off criterion
as one would suspect it to be. ;)