Option

[Cédric Beust ♔]

I just listened to the latest podcast and I'm a bit surprised by the explanation that Dick gave about Scala's Option type, so I thought I'd make a few comments (and congratulations to Tor and Carl for pressing Dick to clarify some of the things he said).

First of all, Option's effect is 100% runtime, not static. It's nothing like @Nonnull or @Nullable. Dick, can you explain why you said that Option added static checking?

If you want to get an intuition for Option, it's described in details in the GOF book (1994!) under the name "Null Object Design Pattern" (here is the wikipedia entry, although it doesn't mention the GOF book so maybe I'm misremembering).

The idea behind Option is for the program to continue working even if a method is invoked on a "null" (called "None" in Scala and "Nothing" in Haskell) object. Obviously, null checks are still present but since they are hidden inside the Option object, code that operates on these objects can be oblivious to them. In other words, if f is an Option of any type, f.foo() will never crash. Either it will actually call foo if there is an object inside the Option or it will do nothing if there is no object.

Here are some of the problems with this approach.

Let's say that you have an HTTP request object and you want to see if a GET parameter is present. Since this parameter could not be there, it's reasonable to use an Option[String]:

class HttpRequest {

  def getParameter(val name: String) : Option[String] { ...

Now that you have this parameter wrapped in an option, you want to pass it to another method. Either that method takes a String or an Option[String]:

1) def process(val param: String) { ... }

2) def process(val param: Option[String]) { ... }

In the first case, you have two choices: 1a) either you extract that string from the option or, if that process() method belongs to you and you can change it, 1b) you can modify its signature to accept an Option[String] instead of a String (and obviously, you need to change its implementation as well).

The 1a) case is basically checking against null, like Java. You will do something like:

Option[String] p = req.getParameter("foo")

p match {

  case Some(s) : process(s)

  case None: _

}

You have gained nothing (I touched on this particular scenario in this blog post).

If 1b) is an option (no pun intended) to you, you go ahead and modify your process() method to accept an Option, and your modified code will probably have some matching code as shown above. And if you don't do it at this level, you will, eventually, have to extract that value from the Option.

Scenario 2) is the best of both worlds: you have two methods that communicate with each other in terms of Option, and this buys you some nice properties (composability).

As you can see from this short description, the benefit of Option is directly proportional to how many of your API's support Options. This is often referred to as the "Midas effect", something that used to plague the C++ world with the `const` keyword. The direct consequence is that you end up reading API docs with method signatures that contain a lot of Options in them.

The problem with this rosy scenario is that you can't ignore the Java world, and as soon as you try to interoperate with it, you will be dealing with raw (non Option) values). So the scenario 1) above is, sadly, common.

Another dark side of Options is that they tend to sweep errors under the rug. You definitely see a lot less NPE's when you use options, because what is happening is that your code is happily working on nonexistent objects (by doing nothing) instead of blowing up. That's fine if your logic is expected to sometimes return nonexistent objects but there are times when you are dealing with Options that should never contain None. The correct thing to do here would be to leave the Option world, extract the underlying object and keep passing this, but then you are back to the interop problems described above between raw objects and boxed ones.

Debugging applications that have misbehaving Option objects is quite challenging because the symptoms are subtle. You put a break point into your code, look into the Option and realize that it's a None while you would expect a Some. And now, you have to figure out what part of your code returned a None instead of a Some, and Option doesn't have enough contextual data to give you this information (some third party libraries attempt to fix that by providing improved Options, e.g. scalaz's Validation).

This is why I think of Options as having a tendency to "sweep errors under the rug".

From a practical standpoint, I think that the "Elvis" approach which I first saw in Groovy and which is also available in Fantom and Kotlin is the best of both worlds. It doesn't offer the nice monadic properties that Option gives you, but it comes at a much cheaper price and less boiler plate.

Finally, Option is still an interesting beast to study since it's a gateway drug to monads. You probably guessed it by now but Option is a monad, and understanding some of the properties that it exhibits (e.g. you can chain several options) is a good first step toward getting a good understanding of the appeal of monads, and from then on, functors and applicatives are just a short block away.

Dick, would love to get more details on what you were explaining during that podcast!

-- 

Cédric

[Josh Berry]

On Sun, Jun 3, 2012 at 10:43 PM, Cédric Beust ♔ <ced...@beust.com> wrote:
> First of all, Option's effect is 100% runtime, not static. It's nothing like
> @Nonnull or @Nullable. Dick, can you explain why you said that Option added
> static checking?

I think it is safe to assume what was referred to was the fact that
you can not use an Option[String] like a String. You have to
statically do something special to get the value out of it. (Now,
there is no static analysis that guarantees you did not put a null in
the option, but that is another beast, isn't it?)

[Casper Bang]

On Monday, June 4, 2012 4:43:48 AM UTC+2, Cédric Beust ♔ wrote:

From a practical standpoint, I think that the "Elvis" approach which I first saw in Groovy and which is also available in Fantom and Kotlin is the best of both worlds. It doesn't offer the nice monadic properties that Option gives you, but it comes at a much cheaper price and less boiler plate.

Also known as the null-coalescing operator in C#.

[Ricky Clarkson]

Option isn't the same as the null object pattern, because null objects get used as if they were real objects whereas code dealing with Options looks different.  It has to contain some form of check or lifting of a function.

Option is the wrong type if you think you might ever ask 'why is it None?'.  Before I realised that I had my own version in Java that stored a stack trace at the point of creating a None.

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.
To post to this group, send email to java...@googlegroups.com.
To unsubscribe from this group, send email to javaposse+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

[Kevin Wright]

On 4 June 2012 03:43, Cédric Beust ♔ <ced...@beust.com> wrote:

I just listened to the latest podcast and I'm a bit surprised by the explanation that Dick gave about Scala's Option type, so I thought I'd make a few comments (and congratulations to Tor and Carl for pressing Dick to clarify some of the things he said).

First of all, Option's effect is 100% runtime, not static. It's nothing like @Nonnull or @Nullable. Dick, can you explain why you said that Option added static checking?

It's *everything* like @Nullable, insofar as it tells you at compile time (i.e. statically) that "Here is a thing which might not have a value".  Nulls, on the other hand, work on the basis of a type system where *any* object might not have a value (note: primitives are not objects), you therefore have to risk null pointer exceptions anywhere.

The important part here is whether the declaration of possible emptiness is static or dynamic, not the whether the actual representation of an empty value is static or dynamic.  Option has the advantage of being universally supported within Scala whereas @Nullable/@NotNull support is far less widespread.

Of course, if you use @NotNull or Option then you must also have the discipline that you won't put a null value into such an entity[1], because there's nothing at the bytecode level to protect you here.

[1] It can sometimes make sense in rare cases to have a Some(null).  For example, if a map contains the key-value pair "a" -> null then a lookup against "a" would correctly return Some(null), None would be the correct return value if the key "a" wasn't present at all.  In reality, you should never design data structures to allow for such a possibility, but it could well happen if dealing with a map produced by some java library.  flatMap makes it possible to still keep such designs very clean, less so with @Nullable and the elvis operator.

 

If you want to get an intuition for Option, it's described in details in the GOF book (1994!) under the name "Null Object Design Pattern" (here is the wikipedia entry, although it doesn't mention the GOF book so maybe I'm misremembering).

Not so.  A "Null Object" is treated like any other instance of the same type.  An Optional value still requires some form of explicit handling in your code path, although this is typically very lightweight and with minimal ceremony thanks to methods such as map and flatMap.  It's a great deal less intrusive than explicit runtime checks for null values.

The idea behind Option is for the program to continue working even if a method is invoked on a "null" (called "None" in Scala and "Nothing" in Haskell) object. Obviously, null checks are still present but since they are hidden inside the Option object, code that operates on these objects can be oblivious to them. In other words, if f is an Option of any type, f.foo() will never crash. Either it will actually call foo if there is an object inside the Option or it will do nothing if there is no object.

It's not a null check.  At the bytecode level it normally boils down to an isInstanceOf check.  Different mechanism, but with the same ultimate effect.

f.foo() is also invalid syntax, foo() isn't defined on Options.  Instead, you would:

f map { _.foo() }

 

Here are some of the problems with this approach.

Let's say that you have an HTTP request object and you want to see if a GET parameter is present. Since this parameter could not be there, it's reasonable to use an Option[String]:

class HttpRequest {

  def getParameter(val name: String) : Option[String] { ...

Now that you have this parameter wrapped in an option, you want to pass it to another method. Either that method takes a String or an Option[String]:

1) def process(val param: String) { ... }

2) def process(val param: Option[String]) { ... }

In the first case, you have two choices: 1a) either you extract that string from the option or, if that process() method belongs to you and you can change it, 1b) you can modify its signature to accept an Option[String] instead of a String (and obviously, you need to change its implementation as well).

The 1a) case is basically checking against null, like Java. You will do something like:

Option[String] p = req.getParameter("foo")

p match {

  case Some(s) : process(s)

  case None: _

}

So process only gets called if the foo parameter is present?  I'd use:

req.getParameter("foo") foreach { process(_) }

or, in the so-called "point free" style:

req.getParameter("foo") foreach { process }

Only if it makes sense to call process with no value would I define it to accept an optional value.  Or, if defined in java to make use of nulls, I'd call it as:

process(req.getParameter("foo").orNull)

Look ma! No explicit checks!

 

You have gained nothing (I touched on this particular scenario in this blog post).

If 1b) is an option (no pun intended) to you, you go ahead and modify your process() method to accept an Option, and your modified code will probably have some matching code as shown above. And if you don't do it at this level, you will, eventually, have to extract that value from the Option.

Scenario 2) is the best of both worlds: you have two methods that communicate with each other in terms of Option, and this buys you some nice properties (composability). 

Again, why?  If it makes no sense for process to be called with no value, then why define it so it can accept no value?  It's disingenuous to anyone using the method and it goes against the principles of self-documenting code.

 

As you can see from this short description, the benefit of Option is directly proportional to how many of your API's support Options. This is often referred to as the "Midas effect", something that used to plague the C++ world with the `const` keyword. The direct consequence is that you end up reading API docs with method signatures that contain a lot of Options in them.

The problem with this rosy scenario is that you can't ignore the Java world, and as soon as you try to interoperate with it, you will be dealing with raw (non Option) values). So the scenario 1) above is, sadly, common.

So you either use "foreach" or "orNull" as appropriate to handle this interop, which is still typically very lightweight.

 

Another dark side of Options is that they tend to sweep errors under the rug. You definitely see a lot less NPE's when you use options, because what is happening is that your code is happily working on nonexistent objects (by doing nothing) instead of blowing up. That's fine if your logic is expected to sometimes return nonexistent objects but there are times when you are dealing with Options that should never contain None. The correct thing to do here would be to leave the Option world, extract the underlying object and keep passing this, but then you are back to the interop problems described above between raw objects and boxed ones.

Debugging applications that have misbehaving Option objects is quite challenging because the symptoms are subtle. You put a break point into your code, look into the Option and realize that it's a None while you would expect a Some. And now, you have to figure out what part of your code returned a None instead of a Some, and Option doesn't have enough contextual data to give you this information (some third party libraries attempt to fix that by providing improved Options, e.g. scalaz's Validation).

This is why I think of Options as having a tendency to "sweep errors under the rug".

From a practical standpoint, I think that the "Elvis" approach which I first saw in Groovy and which is also available in Fantom and Kotlin is the best of both worlds. It doesn't offer the nice monadic properties that Option gives you, but it comes at a much cheaper price and less boiler plate.

Finally, Option is still an interesting beast to study since it's a gateway drug to monads. You probably guessed it by now but Option is a monad, and understanding some of the properties that it exhibits (e.g. you can chain several options) is a good first step toward getting a good understanding of the appeal of monads, and from then on, functors and applicatives are just a short block away.

And this is the real power of Options!  imagine you have multiple params to be used in the preciously mentioned process method.  I'll also assume that req.getParameters returns a map of params to values.

val params = req.getParameters

for{

  foo <- params get "foo"

  bar <- params get "bar"

  baz <- params get "baz"

} { process(foo,bar,baz) }

Easy peasy, process now only gets called if all three params are present.  Still no explicit checks required.

[Casper Bang]

On Monday, June 4, 2012 9:35:50 AM UTC+2, Ricky Clarkson wrote:

Option isn't the same as the null object pattern, because null objects get used as if they were real objects whereas code dealing with Options looks different.  It has to contain some form of check or lifting of a function.

I didn't say it was. I just added C# to Cédric's list of languages (Groovy, Fantom and Kotlin) which adopted an up-front way of dealing with null, rather than burying the concept as a NOP.

[Kevin Wright]

Just in case anybody here hasn't yet seen the talk, here's what the inventor of nulls currently thinks of his creation:

http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare

[Reinier Zwitserloot]

This. Especially the midas problem tells me that Option<T> is not at all a decent substitute for 'null'.

I personally feel like types need saner defaults, but they still have to be optional. For example, strings should default to "", lists should default to an immutable empty list, and so on and so forth. This should make it sufficiently less painful to work with variables that can never be null, that you can reasonably expect to default to 'never null' behaviour and use some sort of special marker to indicate that a value is allowed to contain null. It would then be necessary to initialize any field that is consider 'never null', assuming its type has no default empty value.

Also, to do the concept underlying option right, AND introduce it to java (or at least introduce it so that you can interoperate with java), you need not 2 nullities (can-be-null and never-null) but 4 nullities:

* RAW - this basically warns on everything and errors on nothing and is analogous to using java.util.List instead of java.util.List<T> in java 1.5+. Needed only for backwards compatibility.

* never-null. For example, if you have a List<[never-null] String>, then when you get strings out, you don't need to null-check them (in fact, if you do, the compiler should warn you that you're doing pointless work), and you cannot put strings into this list unless the compiler can confirm that the reference can't possibly be null at this juncture. That, or a runtime check is added which throws an NPE if you attempt to add a null to this thing.

* can-be-null. For example, if you have a List<[can-be-null] String>, then when you get strings out, you must null-check them if you try to pass them to things that want [never-null] Strings. On the plus side, you can shove whatever string you want in these, no need for null checks.

* dunno-null. This is a special case which is different from the previous case - it means that it is not known if nulls are allowed here or not. If you have a List<[dunno-null] String>, then when you get these out, you have to null-check, and you also can't put nulls in. The great advantage is that if you have a method with a parameter of type List<[dunno-null] String>, you can pass both List<[never-null] String> and List<[can-be-null] String> into it and that will work as expected without heap corruption. This solves Option's major downfall: it is not feasible to write a method, even one that never writes nulls in and always null-checks when reading, which accepts both a List<T> and a List<Option<T>> transparently, if it needs to do things to the inner T (if the inner T is not null, of course).

The above 4 kinds of null are roughly anologous to the way any given type can show up in 4 different styles in generics: List (raw), List<String>, List<? extends String>, List<? super String>.

Such a type system would eliminate all runtime bugs associated with NPEs and turn them into compiler errors. The downside is that this is very complicated: We have 4 separate nullities so how do we specify them for each type? Keywords? That might turn out rather wordy. Symbols, like "String?" for can-be-null, "String!" for never-null, and "String" for dunno-null, with raw types impossible (but files compiled on old versions are all raw?) - that's probably gonna look rather ugly, and if the goal is to turn most refs into never-null, "String!" should probably be the default and we need to come up with another symbol to indicate dunno-null.

I have no solution for this dilemma, other than introducing an IDE which exceeds the ascii character set for symbols, and which introduces certain keyboard shortcuts to change nullity. But that's an entirely different can of worms.

On Monday, June 4, 2012 4:43:48 AM UTC+2, Cédric Beust ♔ wrote:

[Reinier Zwitserloot]

For what it's worth, we're adapting Philipp Eichhorn's work on static import extension methods in java to Project Lombok. The idea is that you can use an annotation to do extension point imports - i.e. if you have a class filled with a bunch of static methods whose first parameter is always string, you can use an annotation to say: For all code in this class, assume that all these static methods exist as instance methods on any string instance, so, if you have public static String toTitleCase(String in) {}, you can call "foobar".toTitleCase().

An interesting trick is that the replacement does not care about null, it is just translated to a static method call. Thus, you can define this method:

public static <T> T or(T object, T ifNull) {

    return object != null ? object : ifNull;

}

and if statically extension-pointed, then all instances of T (and T erases to Object in the above example, so, every expression of type object, i.e. all non-primitive expressions) have the or method, and it is exactly like the elvish operator:

String x = null;

String y = x.or("Hello, World"); //y is now 'Hello, World'.

I'm excited to see if this makes programming java easier. Kind of annoying that you'd have to put @Extension(Objects.class) everywhere. Maybe we'll allow annotations on package-info.java to count for all files in that package at some point to reduce that annoyance, but that's for another day.

On Monday, June 4, 2012 4:43:48 AM UTC+2, Cédric Beust ♔ wrote:

[clay]

The null coalescing operator aka the Elvis operator is a simple shorthand for the traditional approach seen in C-derived languages:

String something = something ? something : "default value"; // Ternary operator in C, Java, JavaScript, C#, an many others

val something = something ?: "default value" // Groovy coalescing operator (Elvis Operator)

String something = something ?? "default value"; // C# coalescing operator 

The advantage of Scala's Option is that it works properly with map and fold type functions without any special null checking/coalescing syntax. 

Fantom and Kotlin (and Haskell) take this much further. By default, references are non-nullable, and the compiler guarantees that they can never contain null values at runtime. When nulls make sense, you can choose to use a nullable reference.

IMO, the best is the Fantom/Kotlin/Haskell route. Scala has the second best option that is based on an intentional trade off with better Java interop. A simple null coalescing operator is the weaker solution and that is only slightly better than Java which requires slightly more syntax.

[Ricky Clarkson]

I disagree on the defaults issue.  It's better to have an error of some kind that points you directly at the source of a problem than simply a missing word or a nonsensical one.

Dear unknown,

Your bank account null has a $NaN balance.

Thank you,
$DEPT

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/2IMakRSvHSoJ.

[Kevin Wright]

On 4 June 2012 15:07, Reinier Zwitserloot <rein...@gmail.com> wrote:

This. Especially the midas problem tells me that Option<T> is not at all a decent substitute for 'null'.

I personally feel like types need saner defaults, but they still have to be optional. For example, strings should default to "", lists should default to an immutable empty list, and so on and so forth. This should make it sufficiently less painful to work with variables that can never be null, that you can reasonably expect to default to 'never null' behaviour and use some sort of special marker to indicate that a value is allowed to contain null. It would then be necessary to initialize any field that is consider 'never null', assuming its type has no default empty value.

Absolutely, yes!  One man's default is another man's Null Object, it's still essentially the same pattern and something that I very much advocate in preference to Option[T] wherever it makes sense to do so.

 

Also, to do the concept underlying option right, AND introduce it to java (or at least introduce it so that you can interoperate with java), you need not 2 nullities (can-be-null and never-null) but 4 nullities:

* RAW - this basically warns on everything and errors on nothing and is analogous to using java.util.List instead of java.util.List<T> in java 1.5+. Needed only for backwards compatibility.

* never-null. For example, if you have a List<[never-null] String>, then when you get strings out, you don't need to null-check them (in fact, if you do, the compiler should warn you that you're doing pointless work), and you cannot put strings into this list unless the compiler can confirm that the reference can't possibly be null at this juncture. That, or a runtime check is added which throws an NPE if you attempt to add a null to this thing.

* can-be-null. For example, if you have a List<[can-be-null] String>, then when you get strings out, you must null-check them if you try to pass them to things that want [never-null] Strings. On the plus side, you can shove whatever string you want in these, no need for null checks.

* dunno-null. This is a special case which is different from the previous case - it means that it is not known if nulls are allowed here or not. If you have a List<[dunno-null] String>, then when you get these out, you have to null-check, and you also can't put nulls in. The great advantage is that if you have a method with a parameter of type List<[dunno-null] String>, you can pass both List<[never-null] String> and List<[can-be-null] String> into it and that will work as expected without heap corruption. This solves Option's major downfall: it is not feasible to write a method, even one that never writes nulls in and always null-checks when reading, which accepts both a List<T> and a List<Option<T>> transparently, if it needs to do things to the inner T (if the inner T is not null, of course).

This now has alarm bells going off in my head.  These (as described) would only make sense if specified at both declaration AND use site, that's an awful lot of boilerplate and ceremony to be added!  We already tried the same thing with checked exceptions, and you know how well that succeeded...

 

The above 4 kinds of null are roughly anologous to the way any given type can show up in 4 different styles in generics: List (raw), List<String>, List<? extends String>, List<? super String>.

Such a type system would eliminate all runtime bugs associated with NPEs and turn them into compiler errors. The downside is that this is very complicated: We have 4 separate nullities so how do we specify them for each type? Keywords? That might turn out rather wordy. Symbols, like "String?" for can-be-null, "String!" for never-null, and "String" for dunno-null, with raw types impossible (but files compiled on old versions are all raw?) - that's probably gonna look rather ugly, and if the goal is to turn most refs into never-null, "String!" should probably be the default and we need to come up with another symbol to indicate dunno-null.

Then there's the issue of subtyping and the LSP to consider.  It's easy enough to say that String is a subtype of both String! and String?, but it massively changes the Java spec (which only allows subtyping through inheritance).  It looks like we'd be in a similar position to having Array[Object] being a supertype of Array[T], and the problems that caused.  Then you still have the midas problem if you need to pass a String where a String! is demanded.  And how does it work inside collections and generics (especially wildcarded ones)? and through reflection?

I have no solution for this dilemma, other than introducing an IDE which exceeds the ascii character set for symbols, and which introduces certain keyboard shortcuts to change nullity. But that's an entirely different can of worms.

Not just the IDE.  You have javadoc, static analysis tools, code coverage, etc. etc.

 

It's a bold solution, to be sure.  But the work and complexity required to retrofit it look more complicated than Option[T] at this stage.  That's before you even consider composability.

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/t_ihunElI3MJ.

To post to this group, send email to java...@googlegroups.com.
To unsubscribe from this group, send email to javaposse+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

--
Kevin Wright
mail: kevin....@scalatechnology.com
gtalk / msn : kev.lee...@gmail.com

quora: http://www.quora.com/Kevin-Wright

google+: http://gplus.to/thecoda

twitter: @thecoda

vibe / skype: kev.lee.wright

steam: kev_lee_wright

"My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger" ~ Dijkstra

[Kevin Wright]

On 4 June 2012 15:25, clay <clayt...@gmail.com> wrote:

The null coalescing operator aka the Elvis operator is a simple shorthand for the traditional approach seen in C-derived languages:

String something = something ? something : "default value"; // Ternary operator in C, Java, JavaScript, C#, an many others

val something = something ?: "default value" // Groovy coalescing operator (Elvis Operator)

String something = something ?? "default value"; // C# coalescing operator 

The advantage of Scala's Option is that it works properly with map and fold type functions without any special null checking/coalescing syntax. 

and, most importantly, flatMap.  This is the key to full composability and the one that simply can't be done with something like the elvis-operator solutions.

 

Fantom and Kotlin (and Haskell) take this much further. By default, references are non-nullable, and the compiler guarantees that they can never contain null values at runtime. When nulls make sense, you can choose to use a nullable reference.

IMO, the best is the Fantom/Kotlin/Haskell route. Scala has the second best option that is based on an intentional trade off with better Java interop. A simple null coalescing operator is the weaker solution and that is only slightly better than Java which requires slightly more syntax.

 

Haskell uses Maybe, which is directly equivalent to Scala's Option.  It just does it a bit more elegantly because the lack of subtyping allows for full H-M type inference.  

[clay]

On Monday, June 4, 2012 10:14:07 AM UTC-5, KWright wrote:

Haskell uses Maybe, which is directly equivalent to Scala's Option.  It just does it a bit more elegantly because the lack of subtyping allows for full H-M type inference.  

The major difference between Scala's Option and Haskell's Mabye is that Scala's Option is optional (I tried to avoid the pun) in that you can simply use null if you choose. Haskell doesn't let you use a null value for any reference type, so the language affords you that non-null guarantee at the compiler level like Kotlin and Fantom.

Java has third party libraries like Functional Java that supply an Option class. The problem is that any other Java library or API that you use won't use that.

[Kevin Wright]

The *main* difference is that Scala is very much designed for maximum compatibility with existing Java bytecode, whereas Haskell has its own ecosystem.  Hence, null has to be supported.

Given an effects system (it'll come one day, honest...), Scala could easily enforce non-nullability without any need for annotations or trailing punctuation on type names.  In the meantime, we're already getting value types in Scala 2.10 which, like Java's primitives*, cannot be subclassed by Null.  I dearly hope to see more work in this direction with subsequent released!

* Scala's type system has Any at the top, then AnyVal (equivalent to primitives) and AnyRef (equivalent to Object) as the only two subtypes of Any.  Then there's a whole bunch of stuff in the middle, then Null which can subtype any AnyRef, then Nothing**, which can subtype anything.  Prior to 2.10, the only possible subclasses of AnyVal were Int, Double, etc. (equivalent to Java's primitives), this is about to change...

** A good example of a statement returning Nothing is "throw new Exception()", it really does return nothing, not even Unit (which is equivalent to void)

[Gabriel Claramunt]

But (unless you're learning) nobody does that. Is not idiomatic and is unnecessary.

As Kevin said, you just do p.map ( process(_) ), there's no need to touch process's code

 

On Sunday, June 3, 2012 11:43:48 PM UTC-3, Cédric Beust ♔ wrote:

...

Option[String] p = req.getParameter("foo")

p match {

  case Some(s) : process(s)

  case None: _

}

You have gained nothing (I touched on this particular scenario in this blog post).

...

-- 

Cédric

[Fabrizio Giudici]

On Mon, 04 Jun 2012 16:43:20 +0200, Ricky Clarkson
<ricky.c...@gmail.com> wrote:

> I disagree on the defaults issue. It's better to have an error of some
> kind that points you directly at the source of a problem than simply a
> missing word or a nonsensical one.
>
> Dear unknown,
>
> Your bank account null has a $NaN balance.
>
> Thank you,
> $DEPT

+1: all boils down to proper testing and get the output you expect.

--
Fabrizio Giudici - Java Architect, Project Manager
Tidalwave s.a.s. - "We make Java work. Everywhere."
fabrizio...@tidalwave.it
http://tidalwave.it - http://fabriziogiudici.it

[Ricky Clarkson]

We all know that testing can't cover everything even under ideal circumstances, and even if it did and you caught this in testing it would be better to have a stack trace than an empty string.  Wouldn't you agree,?

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.

To post to this group, send email to java...@googlegroups.com.

To unsubscribe from this group, send email to javaposse+unsubscribe@googlegroups.com.

[Fabrizio Giudici]

On Mon, 04 Jun 2012 19:53:26 +0200, Ricky Clarkson
<ricky.c...@gmail.com> wrote:

> We all know that testing can't cover everything even under ideal
> circumstances, and even if it did and you caught this in testing it would
> be better to have a stack trace than an empty string. Wouldn't you
> agree,?

In general, +1 again

[Reinier Zwitserloot]

On Monday, June 4, 2012 5:09:43 PM UTC+2, KWright wrote:

This now has alarm bells going off in my head.  These (as described) would only make sense if specified at both declaration AND use site, that's an awful lot of boilerplate and ceremony to be added!  We already tried the same thing with checked exceptions, and you know how well that succeeded...

Yes, you need to specify both at declaration and use. Like _ANY_ type. Why does that have alarm bells going off in your head?

String x = "Hello"; //site 1

System.out.println(x);

public void println(String in) { ... } //site 2

That's perfectly normal, no need for alarm bells.

Then there's the issue of subtyping and the LSP to consider.

No, the 4-nulls thing actually solves this. Technically, you could treat [dunno-null] as "? extends String! super String?" if you really wanted to, but you can simplify matters considerably by taking nullity out of the type inheritance system. String is a type, and where inheritance is concerned, String is just String, and the nullity of it does not enter into the equation whatsoever; String, String!, String?, String[raw], it's all the same to the type system. As in, == equals - the exact same. If there's a mismatch between a type's nullity state and the operation you do on it, then the compiler will emit an error (you're treating a could-be-null as if it's never-null, or you're passing a could-be-null to something that wants a never-null) or a warning (you're doing null-checks on a never-null). This errors-and-warnings system is like a compiler plugin, it has no actual effect on the meaning of any of the code nor of how any of the code is compiled, the ONLY thing that it could possibly cause is errors and warnings. It's analogous to @Override in that sense. Just compiler-checked documentation is all.

If you want to be technical about it, nullity is its own tiny little non-expandable strictly and statically defined hardcoded type system. The upshot is that it's all very easy to reason about and there's no risk of conflicting with existing specs. For example, you cannot write both:

public static void test1(String? foo) {}

public static void test1(String! foo) {}

in the same file, but that would have been possible if these were actually different types.

 

  It's easy enough to say that String is a subtype of both String! and String?, but it massively changes the Java spec (which only allows subtyping through inheritance).  It looks like we'd be in a similar position to having Array[Object] being a supertype of Array[T], and the problems that caused.  Then you still have the midas problem if you need to pass a String where a String! is demanded.  And how does it work inside collections and generics (especially wildcarded ones)? and through reflection?

See above - no changes needed whatsoever. There is also no midas problem here; just because I use this equivalent of Option somewhere does NOT mean my code is going to end up with every type in every parameter replaced with Option<T> everywhere! Generalized APIs such as List will most likely roll with dunno-Ts everywhere but this is completely transparent to ALL nullities: You can pass never-nulls, could-be-nulls, and dunno-nulls to such an API and it would all work. Maybe you don't understand the midas problem? With Option, then I start having List<Option<X>> everywhere, severely complicating my API documentation and requiring me to also start using option. The entire _point_ of the 4-nullity concept is that null safety is transparent yet compile-time checked. Contrast to Option, which is compile-time checked but not transparent, and java's system, which is transparent but not compile-time checked.

 

I have no solution for this dilemma, other than introducing an IDE which exceeds the ascii character set for symbols, and which introduces certain keyboard shortcuts to change nullity. But that's an entirely different can of worms.

Not just the IDE.  You have javadoc, static analysis tools, code coverage, etc. etc.

Nope, those can just use the long-form ASCII symbol that is in the actual source file. It's fine in all such tools, but where it gets real tedious is in day-to-day edit work, but if your IDE can let you enter the appropriate nullity state very easily, and render it in an unobtrusive manner, you've gotten your 90% in and it's fine then.

 

 

It's a bold solution, to be sure.  But the work and complexity required to retrofit it look more complicated than Option[T] at this stage.  That's before you even consider composability.

Hah, just... no. It is not possible to retrofit java to Option<T> because that is not transparent; APIs are set in stone, you can't change them. 4-nullities is transparent which means that's actually an option, al though it is very complex.

[Kevin Wright]

On 5 June 2012 11:05, Reinier Zwitserloot <rein...@gmail.com> wrote:

On Monday, June 4, 2012 5:09:43 PM UTC+2, KWright wrote:

This now has alarm bells going off in my head.  These (as described) would only make sense if specified at both declaration AND use site, that's an awful lot of boilerplate and ceremony to be added!  We already tried the same thing with checked exceptions, and you know how well that succeeded...

Yes, you need to specify both at declaration and use. Like _ANY_ type. Why does that have alarm bells going off in your head?

String x = "Hello"; //site 1

System.out.println(x);

public void println(String in) { ... } //site 2

That's perfectly normal, no need for alarm bells.

Not quite.  In addition to the regular type system, we also have the shadow type systems of annotations and checked exceptions.  This would be a third shadow system and it *would* add complexity.

I suspect that Nullity in this form could be internally implemented through annotations to minimise the effort required, yet it would have to follow different rules in the presence of subtyping (more later).  In practice, it would feel a lot more like generics - optional, erased, yet still part of the type system.

Nullity is close to variance in this regard.  With use-site (Java) or declaration-site(C#, Scala) approaches both being possible.  If I could statically type Map<K!, V!> when *defining* the class, then using a guava-esque factory we might have:

public static final Map<String, Integer> example = Map.of("a", 1, "c", 2)

declaration-site nullity.  No extra ceremony or boilerplate in the client code :)

 

Then there's the issue of subtyping and the LSP to consider.

No, the 4-nulls thing actually solves this. Technically, you could treat [dunno-null] as "? extends String! super String?" if you really wanted to, but you can simplify matters considerably by taking nullity out of the type inheritance system. String is a type, and where inheritance is concerned, String is just String, and the nullity of it does not enter into the equation whatsoever; String, String!, String?, String[raw], it's all the same to the type system. As in, == equals - the exact same. If there's a mismatch between a type's nullity state and the operation you do on it, then the compiler will emit an error (you're treating a could-be-null as if it's never-null, or you're passing a could-be-null to something that wants a never-null) or a warning (you're doing null-checks on a never-null). This errors-and-warnings system is like a compiler plugin, it has no actual effect on the meaning of any of the code nor of how any of the code is compiled, the ONLY thing that it could possibly cause is errors and warnings. It's analogous to @Override in that sense. Just compiler-checked documentation is all.

If you want to be technical about it, nullity is its own tiny little non-expandable strictly and statically defined hardcoded type system. The upshot is that it's all very easy to reason about and there's no risk of conflicting with existing specs. For example, you cannot write both:

public static void test1(String? foo) {}

public static void test1(String! foo) {}

in the same file, but that would have been possible if these were actually different types.

Here's a thought experiment, I have:

public class Foo { ... }

public class Bar extends Foo { ... }

public void doSomething(List<? super Bar!> xs)

What would be a valid argument to doSomething?

List<Bar!>

List<Bar>

List<Foo!>

List<Foo>

This is definitely something outside the current spec, so adding it would be a mammoth task - possibly the same order of magnitude as both generics and annotations.

 

 

  It's easy enough to say that String is a subtype of both String! and String?, but it massively changes the Java spec (which only allows subtyping through inheritance).  It looks like we'd be in a similar position to having Array[Object] being a supertype of Array[T], and the problems that caused.  Then you still have the midas problem if you need to pass a String where a String! is demanded.  And how does it work inside collections and generics (especially wildcarded ones)? and through reflection?

See above - no changes needed whatsoever. There is also no midas problem here; just because I use this equivalent of Option somewhere does NOT mean my code is going to end up with every type in every parameter replaced with Option<T> everywhere! Generalized APIs such as List will most likely roll with dunno-Ts everywhere but this is completely transparent to ALL nullities: You can pass never-nulls, could-be-nulls, and dunno-nulls to such an API and it would all work. Maybe you don't understand the midas problem? With Option, then I start having List<Option<X>> everywhere, severely complicating my API documentation and requiring me to also start using option. The entire _point_ of the 4-nullity concept is that null safety is transparent yet compile-time checked. Contrast to Option, which is compile-time checked but not transparent, and java's system, which is transparent but not compile-time checked.

public String! turtleA() { return turtleB(); }

public String turtleB() { return turtleC(); }

public String turtleC() { return ...; }

... continue until you run out of turtles ...

Every method below the top one now has to be changed to return a String! (unless you provide some form of nullity cast).  Is this not the essence of the midas problem?

 

 

I have no solution for this dilemma, other than introducing an IDE which exceeds the ascii character set for symbols, and which introduces certain keyboard shortcuts to change nullity. But that's an entirely different can of worms.

Not just the IDE.  You have javadoc, static analysis tools, code coverage, etc. etc.

Nope, those can just use the long-form ASCII symbol that is in the actual source file. It's fine in all such tools, but where it gets real tedious is in day-to-day edit work, but if your IDE can let you enter the appropriate nullity state very easily, and render it in an unobtrusive manner, you've gotten your 90% in and it's fine then.

 

 

It's a bold solution, to be sure.  But the work and complexity required to retrofit it look more complicated than Option[T] at this stage.  That's before you even consider composability.

Hah, just... no. It is not possible to retrofit java to Option<T> because that is not transparent; APIs are set in stone, you can't change them. 4-nullities is transparent which means that's actually an option, al though it is very complex.

Sure.  Retrofitting is *always* harder than getting a clean design in the first place.  This is why C# forked the collections when they added generics.  I also consider Scala's collections to be a similar fork - not just adding option awareness, but also immutability at the root of the hierarchy and all sorts of monadic goodness.  Other such examples are Google Guava and the totallylazy library.  This can happen at the library level without requiring sweeping changes to the type system in the language spec.

[Eric Jablow]

What about Objective-C's treatment of nil (or NIL).  In Objective-C, messages cannot be overloaded; the argument types and return type are fixed on first declaration.  The same message can be sent to instances of any class, of course. The system then ignores messages sent to nil: Assuming reasonable declarations:

id foo = nil;

[foo shout:@"I don't exist."]; // Does nothing.

double weight = [foo weight]; // Does nothing but returns 0.0;

id bar = [foo parent]; // Does nothing but returns nil;

None of these would cause a stack trace or raise any sort of error condition.

Respectfully,

Eric Jablow

[Ricky Clarkson]

That sounds terrible.  How do you work out the original source of nil?

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.
To post to this group, send email to java...@googlegroups.com.

To unsubscribe from this group, send email to javaposse+...@googlegroups.com.

[Mark Derricutt]

On 6/06/12 12:46 AM, Ricky Clarkson wrote:

That sounds terrible.  How do you work out the original source of nil?

You don't.  I've heard from several people that this is the worst thing about writing iOS apps, or Objective-C in general.  Silent, delayed, untraceable null errors.

[Cédric Beust ♔]

On Tue, Jun 5, 2012 at 5:51 AM, Mark Derricutt <ma...@talios.com> wrote:

You don't.  I've heard from several people that this is the worst thing about writing iOS apps, or Objective-C in general.  Silent, delayed, untraceable null errors.

JQuery does the same thing: selectors return arrays of matching elements, but if no elements were found, you receive an empty array instead of null. Anyone who thinks this is a better idea hasn't practically worked with the concept.

Failing fast (with an NPE or, in the case above, with a Javascript error regarding 'undefined') saves much more time than silently proceeding with an unexpected result.

-- 

Cédric

[Josh Berry]

On Tue, Jun 5, 2012 at 10:57 AM, Cédric Beust ♔ <ced...@beust.com> wrote:
> JQuery does the same thing: selectors return arrays of matching elements,
> but if no elements were found, you receive an empty array instead of null.
> Anyone who thinks this is a better idea hasn't practically worked with the
> concept.
>
> Failing fast (with an NPE or, in the case above, with a Javascript error
> regarding 'undefined') saves much more time than silently proceeding with an
> unexpected result.

Isn't this really just making nil/null a true bottom type? I see
Groovy explicitly introduced the "safe navigation" operator to allow
this. I can see where it would have its uses.

[Ricky Clarkson]

There is value in being *able* to go from null to null (or nil to nil or None to None).  The problem is when it's unintentional and that's what we're saying is bad in JQuery and Objective-C, that there's no visual cue that you might be operating on a nil.

Instead you see a nil 5 miles away in code that has been working for years, with no real clue as to where it comes from.

[Cédric Beust ♔]

On Tue, Jun 5, 2012 at 8:13 AM, Ricky Clarkson <ricky.c...@gmail.com> wrote:

There is value in being *able* to go from null to null (or nil to nil or None to None).  The problem is when it's unintentional and that's what we're saying is bad in JQuery and Objective-C, that there's no visual cue that you might be operating on a nil.

Absolutely. There are times where I want to be able to write a.foo.bar.baz without inserting null checks at every step along the way (and please spare me the Demeter nonsense :-)) and times when each of these should always return something, so a null return should blow up right away.

Like we discussed, @Nullable and @Nonnull do exactly that. Groovy and Kotlin offer similar flexibility (Kotlin with sure()).

Option is actually the one that gives you the least choices because if you decide to unbox the option to enable the "fail fast" mode, you find yourself in the mixed world of Option/Raw values with all the drawbacks I described.

-- 

Cédric

[ebresie]

Curious if the the originally planned coin null-safe method invocation feature for Java 7 comes in to play on this discussion?

Eric

Reference:
- http://mail.openjdk.java.net/pipermail/coin-dev/2009-March/000047.html
- http://metoojava.wordpress.com/2010/11/15/java-7-awesome-features/

On Sunday, June 3, 2012 9:43:48 PM UTC-5, Cédric Beust ♔ wrote:

I just listened to the latest podcast and I'm a bit surprised by the explanation that Dick gave about Scala's Option type, so I thought I'd make a few comments (and congratulations to Tor and Carl for pressing Dick to clarify some of the things he said).

First of all, Option's effect is 100% runtime, not static. It's nothing like @Nonnull or @Nullable. Dick, can you explain why you said that Option added static checking?

If you want to get an intuition for Option, it's described in details in the GOF book (1994!) under the name "Null Object Design Pattern" (here is the wikipedia entry, although it doesn't mention the GOF book so maybe I'm misremembering).

The idea behind Option is for the program to continue working even if a method is invoked on a "null" (called "None" in Scala and "Nothing" in Haskell) object. Obviously, null checks are still present but since they are hidden inside the Option object, code that operates on these objects can be oblivious to them. In other words, if f is an Option of any type, f.foo() will never crash. Either it will actually call foo if there is an object inside the Option or it will do nothing if there is no object.

Here are some of the problems with this approach.

Let's say that you have an HTTP request object and you want to see if a GET parameter is present. Since this parameter could not be there, it's reasonable to use an Option[String]:

class HttpRequest {

  def getParameter(val name: String) : Option[String] { ...

Now that you have this parameter wrapped in an option, you want to pass it to another method. Either that method takes a String or an Option[String]:

1) def process(val param: String) { ... }

2) def process(val param: Option[String]) { ... }

In the first case, you have two choices: 1a) either you extract that string from the option or, if that process() method belongs to you and you can change it, 1b) you can modify its signature to accept an Option[String] instead of a String (and obviously, you need to change its implementation as well).

The 1a) case is basically checking against null, like Java. You will do something like:

Option[String] p = req.getParameter("foo")

p match {

  case Some(s) : process(s)

  case None: _

}

You have gained nothing (I touched on this particular scenario in this blog post).

If 1b) is an option (no pun intended) to you, you go ahead and modify your process() method to accept an Option, and your modified code will probably have some matching code as shown above. And if you don't do it at this level, you will, eventually, have to extract that value from the Option.

Scenario 2) is the best of both worlds: you have two methods that communicate with each other in terms of Option, and this buys you some nice properties (composability).

As you can see from this short description, the benefit of Option is directly proportional to how many of your API's support Options. This is often referred to as the "Midas effect", something that used to plague the C++ world with the `const` keyword. The direct consequence is that you end up reading API docs with method signatures that contain a lot of Options in them.

The problem with this rosy scenario is that you can't ignore the Java world, and as soon as you try to interoperate with it, you will be dealing with raw (non Option) values). So the scenario 1) above is, sadly, common.

Another dark side of Options is that they tend to sweep errors under the rug. You definitely see a lot less NPE's when you use options, because what is happening is that your code is happily working on nonexistent objects (by doing nothing) instead of blowing up. That's fine if your logic is expected to sometimes return nonexistent objects but there are times when you are dealing with Options that should never contain None. The correct thing to do here would be to leave the Option world, extract the underlying object and keep passing this, but then you are back to the interop problems described above between raw objects and boxed ones.

Debugging applications that have misbehaving Option objects is quite challenging because the symptoms are subtle. You put a break point into your code, look into the Option and realize that it's a None while you would expect a Some. And now, you have to figure out what part of your code returned a None instead of a Some, and Option doesn't have enough contextual data to give you this information (some third party libraries attempt to fix that by providing improved Options, e.g. scalaz's Validation).

This is why I think of Options as having a tendency to "sweep errors under the rug".

From a practical standpoint, I think that the "Elvis" approach which I first saw in Groovy and which is also available in Fantom and Kotlin is the best of both worlds. It doesn't offer the nice monadic properties that Option gives you, but it comes at a much cheaper price and less boiler plate.

Finally, Option is still an interesting beast to study since it's a gateway drug to monads. You probably guessed it by now but Option is a monad, and understanding some of the properties that it exhibits (e.g. you can chain several options) is a good first step toward getting a good understanding of the appeal of monads, and from then on, functors and applicatives are just a short block away.

Dick, would love to get more details on what you were explaining during that podcast!

-- 

Cédric

On Sunday, June 3, 2012 9:43:48 PM UTC-5, Cédric Beust ♔ wrote:

I just listened to the latest podcast and I'm a bit surprised by the explanation that Dick gave about Scala's Option type, so I thought I'd make a few comments (and congratulations to Tor and Carl for pressing Dick to clarify some of the things he said).

First of all, Option's effect is 100% runtime, not static. It's nothing like @Nonnull or @Nullable. Dick, can you explain why you said that Option added static checking?

If you want to get an intuition for Option, it's described in details in the GOF book (1994!) under the name "Null Object Design Pattern" (here is the wikipedia entry, although it doesn't mention the GOF book so maybe I'm misremembering).

The idea behind Option is for the program to continue working even if a method is invoked on a "null" (called "None" in Scala and "Nothing" in Haskell) object. Obviously, null checks are still present but since they are hidden inside the Option object, code that operates on these objects can be oblivious to them. In other words, if f is an Option of any type, f.foo() will never crash. Either it will actually call foo if there is an object inside the Option or it will do nothing if there is no object.

Here are some of the problems with this approach.

Let's say that you have an HTTP request object and you want to see if a GET parameter is present. Since this parameter could not be there, it's reasonable to use an Option[String]:

class HttpRequest {

  def getParameter(val name: String) : Option[String] { ...

Now that you have this parameter wrapped in an option, you want to pass it to another method. Either that method takes a String or an Option[String]:

1) def process(val param: String) { ... }

2) def process(val param: Option[String]) { ... }

In the first case, you have two choices: 1a) either you extract that string from the option or, if that process() method belongs to you and you can change it, 1b) you can modify its signature to accept an Option[String] instead of a String (and obviously, you need to change its implementation as well).

The 1a) case is basically checking against null, like Java. You will do something like:

Option[String] p = req.getParameter("foo")

p match {

  case Some(s) : process(s)

  case None: _

}

You have gained nothing (I touched on this particular scenario in this blog post).

If 1b) is an option (no pun intended) to you, you go ahead and modify your process() method to accept an Option, and your modified code will probably have some matching code as shown above. And if you don't do it at this level, you will, eventually, have to extract that value from the Option.

Scenario 2) is the best of both worlds: you have two methods that communicate with each other in terms of Option, and this buys you some nice properties (composability).

As you can see from this short description, the benefit of Option is directly proportional to how many of your API's support Options. This is often referred to as the "Midas effect", something that used to plague the C++ world with the `const` keyword. The direct consequence is that you end up reading API docs with method signatures that contain a lot of Options in them.

The problem with this rosy scenario is that you can't ignore the Java world, and as soon as you try to interoperate with it, you will be dealing with raw (non Option) values). So the scenario 1) above is, sadly, common.

Another dark side of Options is that they tend to sweep errors under the rug. You definitely see a lot less NPE's when you use options, because what is happening is that your code is happily working on nonexistent objects (by doing nothing) instead of blowing up. That's fine if your logic is expected to sometimes return nonexistent objects but there are times when you are dealing with Options that should never contain None. The correct thing to do here would be to leave the Option world, extract the underlying object and keep passing this, but then you are back to the interop problems described above between raw objects and boxed ones.

Debugging applications that have misbehaving Option objects is quite challenging because the symptoms are subtle. You put a break point into your code, look into the Option and realize that it's a None while you would expect a Some. And now, you have to figure out what part of your code returned a None instead of a Some, and Option doesn't have enough contextual data to give you this information (some third party libraries attempt to fix that by providing improved Options, e.g. scalaz's Validation).

This is why I think of Options as having a tendency to "sweep errors under the rug".

From a practical standpoint, I think that the "Elvis" approach which I first saw in Groovy and which is also available in Fantom and Kotlin is the best of both worlds. It doesn't offer the nice monadic properties that Option gives you, but it comes at a much cheaper price and less boiler plate.

[Ricky Clarkson]

If you unbox the Option then you don't have an Option any longer and can't really blame Option for what happens next.  Option gives you the same choices plus can be used in for-comprehensions like any other monad.

[Cédric Beust ♔]

On Tue, Jun 5, 2012 at 9:18 AM, Ricky Clarkson <ricky.c...@gmail.com> wrote:

Option gives you the same choices

How do I get Option to blow up if I try to map on a None?

-- 

Cédric

[Ricky Clarkson]

Option.map returns another Option, which you clearly don't want if you want something to blow up.  Instead of option.map(_ + 1) you'd write option.get + 1

If you require that the Option contains a value then you probably didn't want an Option in the first place but a direct (non-null) value.

[Josh Berry]

On Tue, Jun 5, 2012 at 11:29 AM, Cédric Beust ♔ <ced...@beust.com> wrote:
> On Tue, Jun 5, 2012 at 8:13 AM, Ricky Clarkson <ricky.c...@gmail.com>
> wrote:
>>
>> There is value in being *able* to go from null to null (or nil to nil or
>> None to None).  The problem is when it's unintentional and that's what we're
>> saying is bad in JQuery and Objective-C, that there's no visual cue that you
>> might be operating on a nil.
>
>
> Absolutely. There are times where I want to be able to write a.foo.bar.baz
> without inserting null checks at every step along the way (and please spare
> me the Demeter nonsense :-)) and times when each of these should always
> return something, so a null return should blow up right away.
>
> Like we discussed, @Nullable and @Nonnull do exactly that. Groovy and Kotlin
> offer similar flexibility (Kotlin with sure()).

I'm not sure how those annotations help here. In fact, they seem
specifically not to help. The cases this covers are where you are
passing a @Nullable and would like the result to remain null through
calls. So, you're back to checking for it. With Option and the like,
so long as you don't "unbox" the value, you get that ability through
the various methods mentioned.

[Kevin Wright]

Just call .get on the thing, it'll go bang fast enough :)

Often though, that simply isn't what you want.  If I'm running some form of 5 hour long big data / batch processing task and it goes bang on item 38,976 / 39,207 then I am *NOT* going to be happy.

"Going bang" really isn't your friend if processing large data sets.  Especially if you're processing them in parallel, even more especially if the processing involves side effects - at which point you've made your behaviour non-deterministic.

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.
To post to this group, send email to java...@googlegroups.com.
To unsubscribe from this group, send email to javaposse+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

[Fabrizio Giudici]

I'm pretty much with Rick and Cédric on this topic...

On Tue, 05 Jun 2012 18:42:57 +0200, Kevin Wright
<kev.lee...@gmail.com> wrote:


> "Going bang" really isn't your friend if processing large data sets.
> Especially if you're processing them in parallel, even more especially
> if
> the processing involves side effects - at which point you've made your
> behaviour non-deterministic.

... so how would it be better to have the whole data set processed without
problems until the end, but with the doubt that it just produced (some)
nonsense?

[Ricky Clarkson]

That's one reason why many batch processing systems have ways of rerunning parts of a job.

If you design the system for it, of course it could continue processing and then raise some alert pointing at the problem data.

If you haven't yet considered this failure mode though it's probably best to just stop.  If you carry on maybe you're rendering garbage to the screen, allocating buffers many times bigger than you should because you read 4 bytes that don't mean what you think they will..

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.
To post to this group, send email to java...@googlegroups.com.

To unsubscribe from this group, send email to javaposse+unsubscribe@googlegroups.com.

[Cédric Beust ♔]

On Tue, Jun 5, 2012 at 9:42 AM, Kevin Wright <kev.lee...@gmail.com> wrote:

"Going bang" really isn't your friend

This is like saying "having bugs is not your friend".

Well, doh. The question is not about having bugs or not, it's about detecting them as soon as possible.

-- 

Cédric

[Josh Berry]

It is easy to contrive the situation where you don't want something to
just go bang on the first error, though. Does anyone like the
scenario where you get an error and fix it, only to be shown a new
error that could have been additionally been reported in the first
pass?

[Fabrizio Giudici]

On Tue, 05 Jun 2012 21:48:09 +0200, Josh Berry <tae...@gmail.com> wrote:

> It is easy to contrive the situation where you don't want something to
> just go bang on the first error, though. Does anyone like the
> scenario where you get an error and fix it, only to be shown a new
> error that could have been additionally been reported in the first
> pass?

It's easy to contrive that situation too, thinking of a consequentiality
so that the second error only happens in consequence of the previous one,
or it will happen only after you fix the previous one :-)

But I'd say that both examples are inappropriate. The point is that I want
perhaps something that processes the batch up to the end, then I go to the
log file and see the NPE where they happened, versus the fact that I have
a result set which is apparently ok, but now I have to dig into it to 1)
search for errors and then 2) rebuild the sequence of things that happened
back to the error.

[Josh Berry]

On Tue, Jun 5, 2012 at 3:55 PM, Fabrizio Giudici
<Fabrizio...@tidalwave.it> wrote:
> It's easy to contrive that situation too, thinking of a consequentiality so
> that the second error only happens in consequence of the previous one, or it
> will happen only after you fix the previous one :-)
>
> But I'd say that both examples are inappropriate. The point is that I want
> perhaps something that processes the batch up to the end, then I go to the
> log file and see the NPE where they happened, versus the fact that I have a
> result set which is apparently ok, but now I have to dig into it to 1)
> search for errors and then 2) rebuild the sequence of things that happened
> back to the error.

I wasn't meaning to argue that the one way is the only way. Rather
trying to give my +1 to the argument of "use the correct tool."
Ideally, a language should make either choice a fairly easy to spot
and conscious choice by the programmer. (Well, in my ideal of the
moment. :) )

[Mark Derricutt]

On 6/06/12 2:57 AM, Cédric Beust ♔ wrote:

JQuery does the same thing: selectors return arrays of matching elements, but if no elements were found, you receive an empty array instead of null. Anyone who thinks this is a better idea hasn't practically worked with the concept.

Failing fast (with an NPE or, in the case above, with a Javascript error regarding 'undefined') saves much more time than silently proceeding with an unexpected result.

Isn't this variation the one we WANT tho - the normal case it returns an array, but it returns EMPTY when nothing is found, rather than null.   Given the contract says "i return 0 or more found things" thats preferred IMHO.  getElementById() silently returning null however....

If we had Option<>'s here I'd say "get" methods throw exceptions when failing, "find" variations return an Option<?> of whatever.

[Graham Allan]

My 0.02c, FWIW. 

I have found using a variant of the Option notion quite satisfying in some use cases. In my case we use Nat Price's Maybe for Java, but it's essentially equivalent[1]. (I'll use the term Option here for consistency, though the API of the two things are not the same) 

What I have found is it tends to be more useful as return values rather than parameters. For example, it is possible to have methods where they can be invoked correctly, in the right state, but still have no 'correct' return value. As a trivial example consider a Person class which contains an optional middleName property. Throwing an exception when you ask for the middle name in this case would be really unfriendly, since nothing exceptional has occurred. You could return null, but it could be easy to forget to check. Having the compiler tell the callers that they may not get something they're expecting, rather than it being a runtime null value has been quite elegant. 

Of course, if the same Person class has a firstName property, and you've decided it's mandatory (possibly at the database level as well) then it is

just a plain bug if it returns null, I wouldn't expect to find a usage of Option here, and I'd be upset if someone used it to mask the bug.

When callers get the Option returned, *they* decide on the correct behaviour when it's absent. That can be choosing a default value instead (e.g. empty string, empty iterable); throwing an exception if it makes sense for them; applying a function to the value if it exists; or propagating it as a return value up to the point where the decision should be made. For me the use of Option essentially boiled down to distributing responsibilities: the thing that produced the Option result shouldn't necessarily know how to handle the cases where a value is present or absent. Using a type to document that is much clearer than Javadoc/annotations etc.

There's nuances and trade-offs, of course, but I've found using an Option-like thing in Java to be pretty useful.

Kind regards,

Graham

[1] https://github.com/npryce/maybe-java 

-- 

Cédric

[Cédric Beust ♔]

On Tue, Jun 5, 2012 at 1:20 PM, Mark Derricutt <ma...@talios.com> wrote:

On 6/06/12 2:57 AM, Cédric Beust ♔ wrote:

JQuery does the same thing: selectors return arrays of matching elements, but if no elements were found, you receive an empty array instead of null. Anyone who thinks this is a better idea hasn't practically worked with the concept.

Failing fast (with an NPE or, in the case above, with a Javascript error regarding 'undefined') saves much more time than silently proceeding with an unexpected result.

Isn't this variation the one we WANT tho - the normal case it returns an array, but it returns EMPTY when nothing is found, rather than null.   Given the contract says "i return 0 or more found things" thats preferred IMHO.  getElementById() silently returning null however....

Absolutely, and sadly, the Java collections make this mistake in various places (returning null instead of empty collections).

In an ideal world, the default case never crashes (empty collections) but the language gives me a way to crash immediately if I receive null/None/Nothing (without me having to explicitly test for it).

> If we had Option<>'s here I'd say "get" methods throw exceptions when failing,

> "find" variations return an Option<?> of whatever. 

Yes, this would make Option much more useful in my opinion.

-- 

Céric

[Kevin Wright]

This is the existing behaviour in scala:

val m = Map(1->"a", 2->"b")

m(1) // "a"

m.get(1) // Some("a")

m(3) // Exception!

m.get(3) // None

For indexed collections (anything subclassing Seq), you can achieve the same by using the lift method.

val xs = Seq("a", "b", "c", "d")

xs(1) // "a"

xs lift 1 // Some("a")

xs(99) // Exception!

xs lift 99 // None

The reason it's named so oddly is because Seq[T] is a subclass of PartialFunction[Int,T], where lift is a method that converts the PartialFunction into a Function1[Int,Option[T]]

Note: PartialFunction is a function that isn't defined for all possible input values, a classic example being square root where the input and output must both be real numbers.  In Scala, Maps and indexed collections can be transparently suppled as arguments anywhere that a PartialFunction of the same signature would otherwise be expected.  It comes in handy for implementing trivial table-based logic.

[Kevin Wright]

You can use *any* collection type if you want, just be careful to guard against mutation and not allow it to grow beyond one element.  That said, Arrays offer the best performance if taking this approach (no boxing of primitives!).

Nat Price's solution does offer some nice extra goodies though, `otherwise` is very handy for providing default values and `to` for working with guava functions is a godsend.

All of this stuff makes a lot more sense and becomes a lot more powerful once you have higher-order functions in your toolbox, so I reckon we can expect to see a lot more of it once lambdas go mainstream.

[Fabrizio Giudici]

On Tue, 05 Jun 2012 22:28:41 +0200, Cédric Beust ♔ <ced...@beust.com>
wrote:

> In an ideal world, the default case never crashes (empty collections) but
> the language gives me a way to crash immediately if I receive
> null/None/Nothing (without me having to explicitly test for it).

In my own stuff, I have finder methods that, when a result is expected,
throw a NotFoundException. It all depends on the case. OTOH there's a
point in returning an empty collection rather than null: that people, "in
doubt", would anyway write

List<> l = ...

if ((l != null) && !l.isEmpty())

which I hate. Being sure that l is not null, you can at least omit the (l
!= null).

[Cédric Beust ♔]

On Tue, Jun 5, 2012 at 2:11 PM, Kevin Wright <kev.lee...@gmail.com> wrote:

Nat Price's solution does offer some nice extra goodies though, `otherwise` is very handy for providing default values

Providing default values seems to be antithetical to Option, though. What's the point of creating an Option if you know it will never be None?

-- 

Cédric

[Kevin Wright]

On 5 June 2012 20:55, Fabrizio Giudici <Fabrizio...@tidalwave.it> wrote:

On Tue, 05 Jun 2012 21:48:09 +0200, Josh Berry <tae...@gmail.com> wrote:

It is easy to contrive the situation where you don't want something to
just go bang on the first error, though.   Does anyone like the
scenario where you get an error and fix it, only to be shown a new
error that could have been additionally been reported in the first
pass?

It's easy to contrive that situation too, thinking of a consequentiality so that the second error only happens in consequence of the previous one, or it will happen only after you fix the previous one :-)

Sure, we see it all the time when compiling code.  Doesn't stop we wanting to see more than one error when it fails though :)

 

But I'd say that both examples are inappropriate. The point is that I want perhaps something that processes the batch up to the end, then I go to the log file and see the NPE where they happened, versus the fact that I have a result set which is apparently ok, but now I have to dig into it to 1) search for errors and then 2) rebuild the sequence of things that happened back to the error.

Which is where you'd use `Either`.  Errors can still be propagated up the call stack through `Left` instances, but in a deterministic way that doesn't bail out in the middle of a long-running process.  

[Kevin Wright]

I have a Seq[Option[String]], let's say the values are surnames to be used in a mail merge.  If we don't know the surname then we just want to "print" an empty string in its place:

val surnames: Seq[Option[String]] = ...

val default = ""

val surnamesForMerge = surnames map {_ getOrElse default}

I don't know that any given entry will never be None, but I *do* know what default I want if it is.

[Kevin Wright]

val surnames: Seq[String] = ...

val page1 = surnames filter {_.head.toLower == 'a'} take 50

val page1uc = page1 map {_.toUpperCase}

It doesn't matter if surnames is empty, or no entry starts with the letter 'a', or if less than 50 entries map.  It all just works and flows effortlessly with no explicit checking for nulls or emptiness.

page1uc will contain a sequence of between 0 and 50 uppercased names starting with 'A', it's fully typesafe and statically guaranteed.

[Mark Derricutt]

As I mentioned on Google+ the other day after listening to this episode, one of the best things that Smalltalk offered with its optional system was/is the ifNil:/ifNotNil: and isNil:/isNotNil messages ( Nil is an Object in its own right, and offers negative variations of the four messages) .

The if variations take blocks meaning you can write code like:

(customerManager findCustomerNamed: 'Tor') ifNotNil: [ | cust |
  cust increaseLevelBy: 5].

The block gets executed ONLY if the result is not nil/null, and the value is passed along as a parameter to the block/closure.

No NullPointerExceptions, no temporary variables.

On 6/06/12 8:28 AM, Graham Allan wrote:

[Graham Allan]

The default value is decided by the caller who is unwrapping the Option, not when the Option is constructed.

I think that's perhaps the point I struggled most with when adopting Option: never use it if you "know" it will never be None, just use the type itself.  When I say "know" in inverted commas, I mean, it *could* be null, due to a bug. But if you define such a case to be a bug, then it's a bug. For me that's not the use case of an Option-like thing*.

Failing fast, or collecting the problems and continuing, is a different, orthogonal discussion.

Regards,

Graham

* But if I'm at the point where I'm incorrectly foisting my own definition/perspective into the discussion, then that's worse than useless :)

[Dick Wall]

Well, It's late, and I'm tired, and I haven't read the rest of this post yet, so I am sure some of this has been answered. In brief though.

Option is absolutely a static type solution to the problem. Option[String] is not a String. You can't do o.charAt(1) on an Option[String], but you can do an o.map(_.charAt(1)) on it, and it will return a None if it was None to begin with, you can also do stuff like this

for (p <- person, a <- p.address, z <- a.zipCode) yield zipCode

Which very neatly will handle an optional person, with an optional address, with an optional zipCode, and either return a Some(zipCode) or a None depending on whether person, address or zipCode are actually there (any of them can be None, the result will be None, if they are all there you get a zipCode). Combine that with a loop and a flatten, and you get a very compact way of pulling out all of the zipCodes that are available without a single null check or NPE. Cedric, I invite you to come to our next training session, we have exercises that really demonstrate this stuff very nicely.

You say they sweep errors under the rug, but that's a pessimist's viewpoint. From experience on the massive calc engines I work on now, the use of Option and Either are a massive improvement on what was available before. Not every calculation has a valid (or possible) answer. The Java way to handle that is to bail when you get one that doesn't (throw an exception) and you end up with nothing. Now in genetics, where you have millions of calculations, and maybe 5% will fail, using either Option or Either is a far better way to go. Instead you get 95% of the answers completed as expected, and 5% of them have an exceptional outcome which can be reported at the end. When I think of the exception throwing way we are used to, I am pleased to have a much better option.

At the end of the day though, the real proof in the pudding (so to speak) as I mentioned on the podcast is that when I started out with Scala I thought the same as you did in your article, that because get could throw an exception, it really didn't give you anything more than null did. It took me a good 6 months or so to realize that I had not had a single NPE in runtime code (or its equivalent get exception I should point out) since I started using option. It makes you think about the fact that the value might not be there when you use the value. In one fell swoop, it has eliminated NPEs from my runtimes - fully a third of my runtime errors at a conservative estimate, not to mention eliminating all of the null checks littered all over my code. I totally stand by what I said on the podcast, and I am happy to discuss it here until the cows come home. The rule of thumb (and it's a simple one) is never use get (unless you know by some means like isDefined, that the value has to be there). Most of the time you leave things in the Option space until the last possible moment, and then provide either a default alternative, or an explanation of why it's missing. The libraries do that too - it's the real value of having something like this as a core feature of the libraries.

For extra credit, I highly recommend everyone goes and looks at Either as well - that's a real gem (it has either a left side or a right side - one with the expected answer, one with the exceptional one - at least that's the conventional pattern). You can then map functions over the "right" answer (get it) and leave the left side alone :-)

Dick

[Dick Wall]

Oops - that should have been

for (p <- person; a <- p.address; z <- a.zipCode) yield z

Said I was tired didn't I? :-)

[Mark Derricutt]

I'd say the only reason its not a static solution to the nullable types
issue is you're example can still easily fail:

a = null;
....

for (p <- person, a <- p.address, z <- a.zipCode) yield zipCode

BLAMO - NPE :-)

[Cédric Beust ♔]

Hi Dick,

Thanks for taking the time to jump in.

A few thoughts:

Option is absolutely a static type solution to the problem. Option[String] is not a String.

This was my point: if it's okay to change types, then it's hard to pretend that Option is a revolutionary improvement over what we have today. "Replace String with SafeString everywhere in your code base and your code will be more robust" is a simplistic way of solving problems (the same kind that led to the fallacy of using auto_ptr everywhere in the C++ world).

First of all, I've been in this industry for long enough to know that there are no silver bullet solution. No technical approach comes with zero costs, and while I'm fully able to understand the benefit of a technical solution, I'm more interested in hearing the trade offs of said solution, which are, oddly enough, hardly ever mentioned by the people trying to sell me on that new hot thing. I understand the benefits of Option, now please tell me what the costs are? This is what I've been trying to address in my previous email and also the older blog post I pointed to.

In particular, the "I haven't seen an NPE in months" justification irks me. I have no doubt it's true, by the very definition of Option, but I ask you (and anyone who's defending Option/Maybe): what do you think you lost? I have asked this question a few times and I have never received anything but blank stares, as if most people who jumped on the Option bandwagon never even considered the trade offs.

Make no mistake: you *have* lost something while using Option, the question should really be "did you lose more than you gained?".

My experience with Option within the JVM has been quite mixed, as I explained. I can totally see how Haskell's Maybe is a clearer win, but within the JVM and necessary interoperability with Java, the track record of Option is much more mixed.

 

Which very neatly will handle an optional person, with an optional address, with an optional zipCode, and either return a Some(zipCode) or a None depending on whether person,

Like I said earlier, I find

person.?address.?zipcode

much more useful and less boiler-platey than the Option approach (then again, you said so yourself on the podcast, so I think we're in agreement there).

address or zipCode are actually there (any of them can be None, the result will be None, if they are all there you get a zipCode). Combine that with a loop and a flatten, and you get a very compact way of pulling out all of the zipCodes that are available without a single null check or NPE. Cedric, I invite you to come to our next training session, we have exercises that really demonstrate this stuff very nicely.

I would love to attend one of your sessions, but my family obligations make this a bit difficult. Hope we can make this happen one day.

 

You say they sweep errors under the rug, but that's a pessimist's viewpoint. From experience on the massive calc engines I work on now, the use of Option and Either are a massive improvement on what was available before. Not every calculation has a valid (or possible) answer. The Java way to handle that is to bail when you get one that doesn't (throw an exception) and you end up with nothing. Now in genetics, where you have millions of calculations, and maybe 5% will fail, using either Option or Either is a far better way to go. Instead you get 95% of the answers completed as expected, and 5% of them have an exceptional outcome which can be reported at the end. When I think of the exception throwing way we are used to, I am pleased to have a much better option.

You probably already know this, but Either's default bias is currently under heavy discussion on the Scala list, and the community seems to be squarely split on the right default.

 In one fell swoop, it has eliminated NPEs from my runtimes

I believe I covered this above. The "pudding proof" is not whether it eliminated NPE's in your code base but whether it made your code base more robust. If you just traded NPE's for obscure errors that escape tests and are hard to track, you haven't gained much.

Myself, I *love* it when I get NPE's because I know that I can fix them in less than a minute. I am much more interested in finding a systematic way to fix ConcurrentModificationException (to name just one) than NullPointerExceptions.

NPE's are really not a huge deal in practice.

 

- fully a third of my runtime errors at a conservative estimate, not to mention eliminating all of the null checks littered all over my code.

No argument there. I totally agree that null (or "no value") checks should be handled by the compiler.

-- 

Cédric

[Fabrizio Giudici]

On Wed, 06 Jun 2012 07:28:47 +0200, Cédric Beust ♔ <ced...@beust.com>
wrote:

> I understand the benefits of Option, now please tell me what
> the costs are? This is what I've been trying to address in my previous
> email and also the older blog post I pointed to.
> In particular, the "I haven't seen an NPE in months" justification irks
> me.

Exactly. Because I can say that "no exceptions for months" is also mine
and my customers' experience when other proper things have been done, in
plain Java, which don't include Option. So, now the point is to understand
what strategy costs more.

[Dick Wall]

I think the costs are probably something along these lines:

When using an Option type, you cannot use methods on that type directly any more (that's the point) so you have to understand enough about the functional approach to use stuff like:

o.map(_.length)

or

for (s <- o) yield s.length

which will still be compile time checkable, and will turn an Option[String] into an Option[Int] with a possible None outcome if it is missing.

It's also extra characters to type in type signatures (fortunately though Scala is pretty good at inferring types so that overhead is reduced) 

The biggest liability, as you point out, is that because Scala runs on the JVM, references can still be null. That's a big problem, but (even though you find the statement irksome - sorry but I can't help what you find irksome) I have eliminated null in my code. I never set variables to null, and anything that comes out of Java can be wrapped in an option immediately:

Option("hello") gives an Some("hello")

Option(null) gives a None

so you can take the result of any java method and immediately make it safe using Option.

In my opinion and experience of working on a rapidly constructed, extensive calculation engine, the result of using Option throughout has been a far more reliable system, easier to make agile changes to quickly, and which produces large numbers of results reliably even when some operations fail (as they will - the one thing that is consistent in genetic science is its inconsistency). Our scientists want all the answers that can be calculated, and to know about the ones that can't, not simply to be told that it can't do it.

If you work under those criteria, you will end up with a SafeString, or an option-like construct yourself - it's the logical way to do it. The part that makes it the most valuable is that it is implemented at the core library level and hence used by all of the libraries. In the same way that lambdas in a language make the library APIs tighter and more integrated, so patterns like this when adopted by the libraries throughout have a similar advantage.

Another way to look at it is that you say yourself, the down sides "...which are, oddly enough, hardly ever mentioned by the people trying to sell me on that...". Two things on this - Cedric - I never tried to sell Scala to you at all, it seems to me that all of our conversations on the subject have been started by you, not by me. Secondly, have you ever stopped to ask yourself why so many people seem to like Option, or indeed Scala. Perhaps, like me, they just find it really, really good and want to let others know that. I don't think of the downsides of Option ever when I use it, I am just grateful that it is there.

Sure there is no silver bullet, of course not (neither are the null safe operators, or the @nullable annotations - Tor talked about the downsides of the annotations already on the podcast) but mostly people see that these improve on what went before. Nullsafe operator actually adds syntax to the language, which I am not against, but also don't think is necessary here - sure it saves characters, but if that is what you want Scala provides on that front all over the place (check out case classes for an enormous win on that front, or simply having lambdas and the loan pattern).

Dick

[Reinier Zwitserloot]

On Tuesday, June 5, 2012 1:19:52 PM UTC+2, KWright wrote:

Here's a thought experiment, I have:

public class Foo { ... }

public class Bar extends Foo { ... }

public void doSomething(List<? super Bar!> xs)

What would be a valid argument to doSomething?

List<Bar!>

List<Bar>

List<Foo!>

List<Foo>

Type-wise it's all legal, as both Bar and Foo are equal to or a supertype of Bar. For the nullity aspect, all types are again fine - the idea is that I can write any expression of type 'Bar!' into this list and be 'safe'. That means ? super T! is as accepting as it gets - anything goes.

This isn't that hard; certainly not more difficult than generics (with super and extends and wildcards and all that).

 

This is definitely something outside the current spec, so adding it would be a mammoth task - possibly the same order of magnitude as both generics and annotations.

Why would it be a mammoth task? This stuff writes itself; the difficult footwork has all been done in 1.5 as part of generics. 

public String! turtleA() { return turtleB(); }

public String turtleB() { return turtleC(); }

public String turtleC() { return ...; }

... continue until you run out of turtles ...

Every method below the top one now has to be changed to return a String! (unless you provide some form of nullity cast).  Is this not the essence of the midas problem?

This is (A) not true (you don't have to change turtleB or turtleC if you don't want to) and (B) even if it was, not the essence of the midas problem. If you do change turtleB and turtleC, you have an effect on subclass implementations of this thing, but NOT on callers - they can go on as if nothing changed. Contrast this to changing it to Option<T> and that would actually be very annoying.

 

[Reinier Zwitserloot]

'Going bang' is an option on any one of the 39,207 items. The appropriate response is for the code that runs the closure that operates on the data to catch this exception and transport it to the calling frame in a way that does not imply killing the result of the other 39,206 operations. While you're catching exceptions anyway, also catch the NPE and everything is fine. Why does null get special 'ignore it' vs. 'throw an exception' behaviour?

If I _WANT_ nulls to be 'ignored', then I should be explicit about it, with i.e. an elvis operator or whatnot.

On Tuesday, June 5, 2012 6:42:57 PM UTC+2, KWright wrote:

Just call .get on the thing, it'll go bang fast enough :)

Often though, that simply isn't what you want.  If I'm running some form of 5 hour long big data / batch processing task and it goes bang on item 38,976 / 39,207 then I am *NOT* going to be happy.

"Going bang" really isn't your friend if processing large data sets.  Especially if you're processing them in parallel, even more especially if the processing involves side effects - at which point you've made your behaviour non-deterministic.

On 5 June 2012 17:25, Cédric Beust ♔ <ced...@beust.com> wrote:

On Tue, Jun 5, 2012 at 9:18 AM, Ricky Clarkson <ricky.clarkson@gmail.com> wrote:

Option gives you the same choices

How do I get Option to blow up if I try to map on a None?

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.
To post to this group, send email to java...@googlegroups.com.

To unsubscribe from this group, send email to javaposse+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

--
Kevin Wright
mail: kevin.wright@scalatechnology.com
gtalk / msn : kev.lee...@gmail.com

quora: http://www.quora.com/Kevin-Wright

google+: http://gplus.to/thecoda

twitter: @thecoda

vibe / skype: kev.lee.wright

steam: kev_lee_wright

"My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger" ~ Dijkstra

On Tuesday, June 5, 2012 6:42:57 PM UTC+2, KWright wrote:

Just call .get on the thing, it'll go bang fast enough :)

Often though, that simply isn't what you want.  If I'm running some form of 5 hour long big data / batch processing task and it goes bang on item 38,976 / 39,207 then I am *NOT* going to be happy.

"Going bang" really isn't your friend if processing large data sets.  Especially if you're processing them in parallel, even more especially if the processing involves side effects - at which point you've made your behaviour non-deterministic.

On 5 June 2012 17:25, Cédric Beust ♔ <ced...@beust.com> wrote:

On Tue, Jun 5, 2012 at 9:18 AM, Ricky Clarkson <ricky.clarkson@gmail.com> wrote:

Option gives you the same choices

How do I get Option to blow up if I try to map on a None?

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.
To post to this group, send email to java...@googlegroups.com.

To unsubscribe from this group, send email to javaposse+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

--
Kevin Wright
mail: kevin.wright@scalatechnology.com
gtalk / msn : kev.lee...@gmail.com

quora: http://www.quora.com/Kevin-Wright

google+: http://gplus.to/thecoda

twitter: @thecoda

vibe / skype: kev.lee.wright

steam: kev_lee_wright

"My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger" ~ Dijkstra

[Ricky Clarkson]

Could some or all of this be inferred so we can statically detect possible NPEs without having to add punctuation or annotations to our method signatures?

IDEs can already do some of that but can't cross method boundaries as far as I know.

--

You received this message because you are subscribed to the Google Groups "Java Posse" group.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/E0lE0_tdsOwJ.

To post to this group, send email to java...@googlegroups.com.

To unsubscribe from this group, send email to javaposse+...@googlegroups.com.

[Fabrizio Giudici]

On Wed, 06 Jun 2012 23:12:46 +0200, Ricky Clarkson
<ricky.c...@gmail.com> wrote:

> Could some or all of this be inferred so we can statically detect
> possible
> NPEs without having to add punctuation or annotations to our method
> signatures?

@Nonnull etc... have runtime retention, that means that they make it
possible to check for NPEs across methods. To me it's part of the value of
those annotations, which is pretty good to avoid problems without
searching for more complex stuff.

[Tobias Neef]

Some of you may be interested that Google Guava also contains a Type to represent optional values http://docs.guava-libraries.googlecode.com/git-history/v12.0/javadoc/index.html. I use this approach a lot in the most recent java projects I have done. In my own code this is a nice thing, but in contrast to the scala word we have a lot of places in java which do not follow this pattern. As a result we still have issues with NPEs from time to time.

In the most recent version it is also possible to apply a function on the Optional object using the transform function which is similar to the scala 'map' behavior which was described by Dick. Because the guava library is already used in a lot of places, some Java projects will maybe start using this way of dealing with optional values. Another advantage of guava is, that it uses the JSR 305 annotations for static type checking which also helps to avoid nulls. This is especially helpful for methods those methods of the Guava Collections or the Optional type which do not allow null parameters.

On Tuesday, June 5, 2012 1:19:52 PM UTC+2, KWright wrote:

On 5 June 2012 11:05, Reinier Zwitserloot <rein...@gmail.com> wrote:

On Monday, June 4, 2012 5:09:43 PM UTC+2, KWright wrote:

This now has alarm bells going off in my head.  These (as described) would only make sense if specified at both declaration AND use site, that's an awful lot of boilerplate and ceremony to be added!  We already tried the same thing with checked exceptions, and you know how well that succeeded...

Yes, you need to specify both at declaration and use. Like _ANY_ type. Why does that have alarm bells going off in your head?

String x = "Hello"; //site 1

System.out.println(x);

public void println(String in) { ... } //site 2

That's perfectly normal, no need for alarm bells.

Not quite.  In addition to the regular type system, we also have the shadow type systems of annotations and checked exceptions.  This would be a third shadow system and it *would* add complexity.

I suspect that Nullity in this form could be internally implemented through annotations to minimise the effort required, yet it would have to follow different rules in the presence of subtyping (more later).  In practice, it would feel a lot more like generics - optional, erased, yet still part of the type system.

Nullity is close to variance in this regard.  With use-site (Java) or declaration-site(C#, Scala) approaches both being possible.  If I could statically type Map<K!, V!> when *defining* the class, then using a guava-esque factory we might have:

public static final Map<String, Integer> example = Map.of("a", 1, "c", 2)

declaration-site nullity.  No extra ceremony or boilerplate in the client code :)

 

Then there's the issue of subtyping and the LSP to consider.

No, the 4-nulls thing actually solves this. Technically, you could treat [dunno-null] as "? extends String! super String?" if you really wanted to, but you can simplify matters considerably by taking nullity out of the type inheritance system. String is a type, and where inheritance is concerned, String is just String, and the nullity of it does not enter into the equation whatsoever; String, String!, String?, String[raw], it's all the same to the type system. As in, == equals - the exact same. If there's a mismatch between a type's nullity state and the operation you do on it, then the compiler will emit an error (you're treating a could-be-null as if it's never-null, or you're passing a could-be-null to something that wants a never-null) or a warning (you're doing null-checks on a never-null). This errors-and-warnings system is like a compiler plugin, it has no actual effect on the meaning of any of the code nor of how any of the code is compiled, the ONLY thing that it could possibly cause is errors and warnings. It's analogous to @Override in that sense. Just compiler-checked documentation is all.

If you want to be technical about it, nullity is its own tiny little non-expandable strictly and statically defined hardcoded type system. The upshot is that it's all very easy to reason about and there's no risk of conflicting with existing specs. For example, you cannot write both:

public static void test1(String? foo) {}

public static void test1(String! foo) {}

in the same file, but that would have been possible if these were actually different types.

Here's a thought experiment, I have:

public class Foo { ... }

public class Bar extends Foo { ... }

public void doSomething(List<? super Bar!> xs)

What would be a valid argument to doSomething?

List<Bar!>

List<Bar>

List<Foo!>

List<Foo>

This is definitely something outside the current spec, so adding it would be a mammoth task - possibly the same order of magnitude as both generics and annotations.

 

 

  It's easy enough to say that String is a subtype of both String! and String?, but it massively changes the Java spec (which only allows subtyping through inheritance).  It looks like we'd be in a similar position to having Array[Object] being a supertype of Array[T], and the problems that caused.  Then you still have the midas problem if you need to pass a String where a String! is demanded.  And how does it work inside collections and generics (especially wildcarded ones)? and through reflection?

See above - no changes needed whatsoever. There is also no midas problem here; just because I use this equivalent of Option somewhere does NOT mean my code is going to end up with every type in every parameter replaced with Option<T> everywhere! Generalized APIs such as List will most likely roll with dunno-Ts everywhere but this is completely transparent to ALL nullities: You can pass never-nulls, could-be-nulls, and dunno-nulls to such an API and it would all work. Maybe you don't understand the midas problem? With Option, then I start having List<Option<X>> everywhere, severely complicating my API documentation and requiring me to also start using option. The entire _point_ of the 4-nullity concept is that null safety is transparent yet compile-time checked. Contrast to Option, which is compile-time checked but not transparent, and java's system, which is transparent but not compile-time checked.

public String! turtleA() { return turtleB(); }

public String turtleB() { return turtleC(); }

public String turtleC() { return ...; }

... continue until you run out of turtles ...

Every method below the top one now has to be changed to return a String! (unless you provide some form of nullity cast).  Is this not the essence of the midas problem?

 

 

I have no solution for this dilemma, other than introducing an IDE which exceeds the ascii character set for symbols, and which introduces certain keyboard shortcuts to change nullity. But that's an entirely different can of worms.

Not just the IDE.  You have javadoc, static analysis tools, code coverage, etc. etc.

Nope, those can just use the long-form ASCII symbol that is in the actual source file. It's fine in all such tools, but where it gets real tedious is in day-to-day edit work, but if your IDE can let you enter the appropriate nullity state very easily, and render it in an unobtrusive manner, you've gotten your 90% in and it's fine then.

 

 

It's a bold solution, to be sure.  But the work and complexity required to retrofit it look more complicated than Option[T] at this stage.  That's before you even consider composability.

Hah, just... no. It is not possible to retrofit java to Option<T> because that is not transparent; APIs are set in stone, you can't change them. 4-nullities is transparent which means that's actually an option, al though it is very complex.

Sure.  Retrofitting is *always* harder than getting a clean design in the first place.  This is why C# forked the collections when they added generics.  I also consider Scala's collections to be a similar fork - not just adding option awareness, but also immutability at the root of the hierarchy and all sorts of monadic goodness.  Other such examples are Google Guava and the totallylazy library.  This can happen at the library level without requiring sweeping changes to the type system in the language spec.

[Dale Wijnand]

The class Tobias is talking about (which I can't believe no one mentioned yet, thanks Tobias) is Optional: http://docs.guava-libraries.googlecode.com/git-history/v12.0/javadoc/com/google/common/base/Optional.html

I also like the use of JSR 305 annotations, which I think work well for parameters, while using Optional for return types when not returning a value is an expected result (as opposed to an exception case, in which case I prefer to design the API to throw an exception).

Dale

[Reinier Zwitserloot]

On Tuesday, June 12, 2012 8:52:30 AM UTC+2, Dale Wijnand wrote:

The class Tobias is talking about (which I can't believe no one mentioned yet, thanks Tobias) is Optional: http://docs.guava-libraries.googlecode.com/git-history/v12.0/javadoc/com/google/common/base/Optional.html

Nobody mentioned it, probably because it's frankly unusable in java. As has been covered before, if you use Option, you _NEED_ closures all over the place; there needs to be a way to map your type for ANY operation that runs on a collection of said type, such that you can easily pass a collection of Option<type> along with a mapping.

[Ricky Clarkson]

That doesn't stop JdbcTemplate from being used, so it shouldn't and doesn't stop Option.

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/NR37xDf42BgJ.

[Dale Wijnand]

I've found it very useful to design my API, making the intent evident

from the type signature ("this method might not have a return value

for you").

Unfortunately it's a bit verbose and of limited use in Java than in Scala

but it still beats ambiguity and having to resort to reading the Javadoc

(or, even worst, making assumptions)..

Dale

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.

[Reinier Zwitserloot]

I'd use annotations for this. If you give me an Option<T>, that's not TOO bad, but if you give me i.e. a List<Option<T>>, you're pretty much forcing me to jump through awkward hoops.

Dale

To unsubscribe from this group, send email to javaposse+unsubscribe@googlegroups.com.

[Kevin Wright]

For example, converting said list to a list of strings:

myList map { _ map {_.toString} getOrElse "(null)" }

compared to a version using nulls

myList map { x => if (null == x) x else x.toString }

or if you just want to drop all the Nones:

myList.flatten

same again, using nulls:

myList filter { _ != null }

There's no hoopiness from using Options here, not so long as you have lambdas.  Now compare a lambda-free version.  I'm using Google guava here to maintain immutability characteristics and so keep it a like-for-like comparison.  Also using both the enhanced-for notation and the tertiary operator to reduce the boilerplate as much as possible:

ImmutableList.Builder<String> out = ImmutableList.builder<String>

for(T x : myList) {

  out.add((null==x) ? "null" : x.toString)

}

out.build()

Conclusion:  If your goal is to avoid jumping through hoops, then picking on Option isn't exactly fertile ground.  There are *much* better candidates for improving the developer experience.

[Dale Wijnand]

On the fly I can't think of a reason to return a List<Option<T>>, that's just ridiculous.

On Friday, July 6, 2012 12:41:26 PM UTC+2, Reinier Zwitserloot wrote:

I'd use annotations for this. If you give me an Option<T>, that's not TOO bad, but if you give me i.e. a List<Option<T>>, you're pretty much forcing me to jump through awkward hoops.

On Tuesday, July 3, 2012 10:27:57 AM UTC+2, Dale Wijnand wrote:

I've found it very useful to design my API, making the intent evident

from the type signature ("this method might not have a return value

for you").

Unfortunately it's a bit verbose and of limited use in Java than in Scala

but it still beats ambiguity and having to resort to reading the Javadoc

(or, even worst, making assumptions)..

Dale

[Dale Wijnand]

Nope, my goal is to make intent clearer in the type signature.

However I'm interested to know what candidates would you suggest to improve developer experience?

On Friday, July 6, 2012 1:08:02 PM UTC+2, KWright wrote:

For example, converting said list to a list of strings:

myList map { _ map {_.toString} getOrElse "(null)" }

compared to a version using nulls

myList map { x => if (null == x) x else x.toString }

or if you just want to drop all the Nones:

myList.flatten

same again, using nulls:

myList filter { _ != null }

There's no hoopiness from using Options here, not so long as you have lambdas.  Now compare a lambda-free version.  I'm using Google guava here to maintain immutability characteristics and so keep it a like-for-like comparison.  Also using both the enhanced-for notation and the tertiary operator to reduce the boilerplate as much as possible:

ImmutableList.Builder<String> out = ImmutableList.builder<String>

for(T x : myList) {

  out.add((null==x) ? "null" : x.toString)

}

out.build()

Conclusion:  If your goal is to avoid jumping through hoops, then picking on Option isn't exactly fertile ground.  There are *much* better candidates for improving the developer experience.

On 6 July 2012 11:41, Reinier Zwitserloot wrote:

I'd use annotations for this. If you give me an Option<T>, that's not TOO bad, but if you give me i.e. a List<Option<T>>, you're pretty much forcing me to jump through awkward hoops.

On Tuesday, July 3, 2012 10:27:57 AM UTC+2, Dale Wijnand wrote:

I've found it very useful to design my API, making the intent evident

from the type signature ("this method might not have a return value

for you").

Unfortunately it's a bit verbose and of limited use in Java than in Scala

but it still beats ambiguity and having to resort to reading the Javadoc

(or, even worst, making assumptions)..

Dale

[Reinier Zwitserloot]

On Friday, July 6, 2012 1:12:18 PM UTC+2, Dale Wijnand wrote:

On the fly I can't think of a reason to return a List<Option<T>>, that's just ridiculous.

Why is that ridiculous? If I have a method which emits an Option<T>, and I have a list of inputs and I run a map operation, I'll get a List<Option<T>> out. Either Option is part of the type system (which also means it can be used in generics too), or it's not. There's no halfwaysies on this, IMO. 

[Dale Wijnand]

Of course you can map a List<A> to a List<Optional<B>> via your mapping method, however I don't see why you would return that List<Optional<B>>, just return List<B> with the non-present optionals removed.

[Ricky Clarkson]

Stranger things have happened at sea.

Let's say you're converting some code that returned a List<T> to use Options because sometimes nulls were getting added and you're fed up of the NPEs.  As a final point a List<Option<T>> might not be what you want but it's probably a pretty handy intermediate step in cleaning that kind of code up.  As an extra, you could make a List implementation that overrides add, rejecting null, to help spot any that you didn't know about.

Can Java.next please not support null?  Thanks.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/alKPZIyxOZUJ.

To post to this group, send email to java...@googlegroups.com.

To unsubscribe from this group, send email to javaposse+...@googlegroups.com.

[Ricky Clarkson]

A map operation that removes empties is generally called a flatMap, because it flattens while it maps, so you get Foo<Bar> instead of Foo<Baz<Bar>>.  It's helpful to call a map a map, and a flatMap a flatMap, just to have some common terminology.

--

You received this message because you are subscribed to the Google Groups "Java Posse" group.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/3NZ0gDscvlQJ.

To post to this group, send email to java...@googlegroups.com.

To unsubscribe from this group, send email to javaposse+...@googlegroups.com.

[Cédric Beust ♔]

Right, it's not ridiculous. The overall idea is that once you start manipulating monadic values and you are using a language whose syntax and libraries supports monads pervasively, you are better off keeping these and working with them than flattening them. Note that I abstracted away from Option: such API's don't accept just an Option<Person> in their API but a Monad<?>, which Option happens to satisfy (and a bunch of others).

In Java, you will most likely need to flatten more often because all the API's accept non-monadic values, e.g. your API accepts a Person, not an Option<Person>, so you have to extract the value before calling that API.

-- 

Cédric

[Kevin Wright]

More normally, you'd just map over your option:

val p = Some(person)

p foreach { myMethod }

This works precisely because you're keeping the optional nature of such constructs away from your functions, allowing them to just get on with their job ab not have to muck about checking for null and suchlike.

You DON'T weave monads through functions, you weave functions through monads.  This is the crucial distinction between Option/Maybe and nullability, but it can also be a tricky concept until your default mental model of functions is that they're first-class entities in their own right, and can be freely composed, manipulated, passed as arguments, etc.

So it's your Option[Person] that accepts your method.  Not the other way around.

[Ricky Clarkson]

I like your ideas and wish to subscribe to your newsletter.

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.

To post to this group, send email to java...@googlegroups.com.

To unsubscribe from this group, send email to javaposse+...@googlegroups.com.

[Kevin Wright]

Lol, thanks for the vote of confidence!

Been thinking about getting back on the blog scene, even been pushing zeebox into doing something formal, watch this space...

[Reinier Zwitserloot]

I completely agree:

p foreach {

    myThing

}

is ever so much cleaner and easier to follow compared to

if (p != null) {

    myThing

}

It's, frankly, a breath of fresh air.

[Kevin Wright]

Even if you're being ironic, this is right, whether or not you meant for it to be :)

In the first case, p could just as well be a collection, or a stream, or a future, or almost anything else that can be classed as a functor.  A functor is most easily described as "anything with a map operation", and `foreach` is nothing more than a map operation where you discard the results.  (You may be more familiar with the term monad, which is most easily described as "an extension of the functor concept that adds the flatMap operation".)

Let's consider that p represents a printer attached to your computer.  Initially, you represent it as an Option[Printer], later it might be several printers and so you use Set[Printer], still later still it could be networked printers that have a high latency to enumerate and so you use SetOfFutures[Printer] to take advantage of asynchronous logic.

In all cases, the central logic remains totally unaltered even as the type of `p` evolves:

p foreach {

    myThing

}

nullable values just fail to deliver at this level of abstraction.

(note: SetOfFutures isn't a built-in type, but there are techniques to flatten nested monads such as Set[Future[T]] and treat them as the hypothetical SetOfFutures[T].  This sort of composability is one of the driving forces towards using monads in the first place!)

[Dale Wijnand]

Just throwing this out there, here's another way to use Guava's Optional:

        for (String format : logFormat.asSet()) {

            formatter.setPattern(format);

        }

as an alternative to

        if (logFormat.isPresent()) {

            formatter.setPattern(logFormat.get());

        }

which is somewhat similar to the "foreach" above.

On Sunday, July 8, 2012 4:42:46 PM UTC+2, KWright wrote:

Even if you're being ironic, this is right, whether or not you meant for it to be :)

In the first case, p could just as well be a collection, or a stream, or a future, or almost anything else that can be classed as a functor.  A functor is most easily described as "anything with a map operation", and `foreach` is nothing more than a map operation where you discard the results.  (You may be more familiar with the term monad, which is most easily described as "an extension of the functor concept that adds the flatMap operation".)

Let's consider that p represents a printer attached to your computer.  Initially, you represent it as an Option[Printer], later it might be several printers and so you use Set[Printer], still later still it could be networked printers that have a high latency to enumerate and so you use SetOfFutures[Printer] to take advantage of asynchronous logic.

In all cases, the central logic remains totally unaltered even as the type of `p` evolves:

p foreach {

    myThing

}

nullable values just fail to deliver at this level of abstraction.

(note: SetOfFutures isn't a built-in type, but there are techniques to flatten nested monads such as Set[Future[T]] and treat them as the hypothetical SetOfFutures[T].  This sort of composability is one of the driving forces towards using monads in the first place!)

On 8 July 2012 15:09, Reinier Zwitserloot wrote:

I completely agree:

p foreach {

    myThing

}

is ever so much cleaner and easier to follow compared to

if (p != null) {

    myThing

}

It's, frankly, a breath of fresh air.

On Friday, July 6, 2012 5:05:51 PM UTC+1, KWright wrote:

More normally, you'd just map over your option:

val p = Some(person)

p foreach { myMethod }

This works precisely because you're keeping the optional nature of such constructs away from your functions, allowing them to just get on with their job ab not have to muck about checking for null and suchlike.

You DON'T weave monads through functions, you weave functions through monads.  This is the crucial distinction between Option/Maybe and nullability, but it can also be a tricky concept until your default mental model of functions is that they're first-class entities in their own right, and can be freely composed, manipulated, passed as arguments, etc.

So it's your Option[Person] that accepts your method.  Not the other way around.

On 6 July 2012 16:47, Cédric Beust ♔ wrote:

[Kevin Wright]

Yes.  I never did understand why Google chose not to have Optional inherit from Iterable

So that it could be used in for loops without the superfluous asSet conversion.

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/1IQ5RGMlnd4J.

To post to this group, send email to java...@googlegroups.com.

To unsubscribe from this group, send email to javaposse+...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

--
Kevin Wright
mail: kevin....@scalatechnology.com

[clay]

Here is a simple example that illustrates the benefit of Option beyond avoiding null exceptions:

Here is a traditional piece of logic using traditional null if-checks. This is using Scala syntax but would be equivalent in Java.

val variableAMayBeNull: A = methodThatMayProduceA();

if (variableAMayBeNull != null) {

val variableBMayBeNull: B = tryToGetBFromA(variableAMayBeNull);

if (variableBMayBeNull != null) {

doSomethingWithNonNullB(variableBMayBeNull);

}

}

Same logic using Option. This is Scala syntax. Java can do this but not as nicely until it gets lambdas.

val variableAMayBeNull: Option[A] = methodThatMayProduceA();

val variableBMayBeNull: Option[B] = variableAMayBeNull.flatMap(tryToGetBFromA);

variableBMayBeNull.foreach(doSomethingWithNonNullB);

Both blocks are logically equivalent, but the Option route is much more concise, elegant, and maintainable. The if-logic is moved from the end application code to inside of the Option class. Since this type of logic is so common in application code, the code simplification benefits are quite large.

[Cédric Beust ♔]

Yes but that's the most trivial case. For example, experiment with your Option code when the original code has else's.

-- 

Cédric

[Kevin Wright]

That's terribly non-idiomatic though, the Scala version would more normally be written as:

for {

  a <- methodThatMayProduceA

  b <- tryToGetBFrom(a)

} {

  doSomethingWith(b)

}

I also stripped out the semicolons and empty parameter list (which is typically only used to denote a method with side effects)

This style won't be possible in Java even after it gets lambdas.  Though I have high hopes that comprehensions will be proposed for the language some point around Java 12 (a may even be added by the time we reach Java 15)

[Kevin Wright]

That's a bit open-ended, are you thinking of a particular construct that you think would be challenging to convert?

As another trivial example, something like this:

final A a = getA();

if (a != null) {

    final B b = getBFrom(a);

    if (b != null) {

  doSomethingWithB(b);

  } else {

        doSomethingIfBWasAbsent();

    }

} else {

    doSomethingIfAWasAbsent();

}

Or with the tertiary operator:

final A a = getA();

(a == null) ? doSomethingIfAWasAbsent() : {

    final B b = getBFrom(a);

    (b == null) ? doSomethingIfBWasAbsent() : {

  doSomethingWithB(b);

}

};

Could be written in Scala as:

val a = getA

val b = a flatMap {getBFrom}

a getOrElse {doSomethingIfAWasAbsent()}

b getOrElse {doSomethingIfBWasAbsent()}

b forEach {doSomethingWithB}

[Ricky Clarkson]

Kevin, I believe you don't need any of the (){} characters there, but maybe I'm missing something.

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.

[Kevin Wright]

I think you do for the (a==null) check, because there are two statements against the failure condition (the assignment to b, and the second check).

if a is absent, then neither doSomethingIfBWasAbsent nor doSomethingWithB should be called!

The second block can go though, and taking advantage of precedence for == gives:

final A a = getA();

a == null ? doSomethingIfAWasAbsent() : {

    final B b = getBFrom(a);

    b == null ? doSomethingIfBWasAbsent() :

doSomethingWithB(b);

};

Though my personal preference would be to retain the bracketing for the equality checks, I find it easier to read that way. Past exposure to LISP may have some influence on this though :)

[Ricky Clarkson]

I meant in the Scala code but forgot to say so.  Also, blocks can't be part of expressions in Java like in Scala so your Java code won't compile.

[Kevin Wright]

doSomethingIfAWasAbsent() and doSomethingIfBWasAbsent() are clearly impure, side-effecting methods.  The fact that we're not using any return values is a dead giveaway!

By convention they *should* have the empty param block to indicate this.

Always using braces around a closure is more of a personal convention.  Just like the parentheses around the conditions in the Java code, I find it helps readability my more clearly delimiting the intent of the code.

[Ricky Clarkson]

Without a => it's not a closure.  I find it confusing sometimes to have extra punctuation; I keep thinking I'm missing something.  return (x + 5); bugs me in Java too :)

[Kevin Wright]

There's definitely closures here.  They're just hiding, if you don't know where to look!

Here's one...

val b = a flatMap {getBFrom}

which is point-free notation for

val b = a flatMap {getBFrom(_)}

Which, in turn, is just syntactic sugar for

val b = a flatMap {x => getBFrom(x)}

So that's your desire for reduced punctuation nicely taken care of :)

[Ricky Clarkson]

1 to 10 map { println("Yo"); println }

Yo gets printed once, println happens 10 times.  Just because you're providing a function doesn't mean you're in a closure.  If it was a closure (and certain other magic happened to make it well-typed) you'd see Yo 10 times with a blank line between each.

[Josh Berry]

On Fri, Jul 27, 2012 at 8:32 AM, Ricky Clarkson
<ricky.c...@gmail.com> wrote:
> 1 to 10 map { println("Yo"); println }
>
> Yo gets printed once, println happens 10 times. Just because you're
> providing a function doesn't mean you're in a closure. If it was a closure
> (and certain other magic happened to make it well-typed) you'd see Yo 10
> times with a blank line between each.

I'm lost. A closure simply means it captures the local environment, right? So:

var y = 0
1 to 10 map {y+= 1; println}
println(y)

Now, I confess I am surprised that it appears this closure is called
once to get a function from Int => Any. I'm assuming it has always
been this way in Scala?

Of course, this does as expected, and looks similar.

for (x <- 1 to 10) {y+=1; println("hello")}

Is this is not a closure, as well?

[Kevin Wright]

You can certainly capture/"close over" the surrounding environment with any Scala anonymous function.  So you can say quite correctly state that any method accepting a function will accept a closure.

As to whether or not you must actually exercise this capability before you're allowed to call it a closure - I guess that's a matter of semantics.

Going on the definition that you must "use it or lose it", then your example is a closure, but probably not in the way you're thinking:

• {y+= 1; println} is evaluated.  It adds one to y and returns a String=>Unit function
• This function is then executed within the for(x <- 1 to 10) comprehension

At first glance, nothing is obviously captured from the surrounding environment.  But... println is about as impure as you can get, working by pure side effect; it has behaviour that depends very much on the surrounding system.

Don't believe me?  Then try this:

def foo(): Unit = {

  var y = 0

  1 to 10 map {y+= 1; println}

  println(y)

}

foo()

System.setOut(someOtherPrintStream)

foo()

[Ricky Clarkson]

Like I said, without a => there's no closure.

The for comprehension works because it gets converted into code containing a =>.

A block in a position where a function is expected is just a block whose value needs to be a function.  Does that make sense now?  I can probably find some specspeak that explains it better if not.

[Ricky Clarkson]

We've already got enough definitions of closures flying around without also calling all functions closures.  Why not be precise and say that you have a method that takes a function as a parameter?

You can provide that function via any expression that yields a function, including but not limited to a lambda expression (one containing =>).  A lambda expression can close over its scope, i.e., forming a closure, meaning that any values, mutated or otherwise, in its scope can be seen from within the lambda expression.

The takeaway is that you don't need to know any of this crap (and a lot of other crap we all already know from Java) if you're not performing side effects.

[Kevin Wright]

The takeaway is that you don't need to know any of this crap (and a lot of other crap we all already know from Java) if you're not performing side effects.

+1 to that!

[Josh Berry]

On Fri, Jul 27, 2012 at 10:16 AM, Ricky Clarkson
<ricky.c...@gmail.com> wrote:
> Like I said, without a => there's no closure.
>
> The for comprehension works because it gets converted into code containing a
> =>.

These two sentences don't make sense together. Either you have to
provide the => or you don't. That one will be implied for you goes
against what you are saying. In fact, I think the confusing thing
here is that the "closure" in the first section is a function from
unit to whatever the closure returns. It seems the compiler works
with you by converting that to what the closure returns through
evaluation. (I am more than willing to admit I'm likely wrong here.)

> A block in a position where a function is expected is just a block whose
> value needs to be a function. Does that make sense now? I can probably
> find some specspeak that explains it better if not.

This doesn't make sense because this only worked since you have it
return a function that can work there. Specifically, the unapplied
println. That is, this didn't work because it was a block where a
function is needed, but because it was a block "returning" a function
that was needed. Something a block doesn't do. (return, that is.)


The simple rule of thumb I have is if a new class file gets created to
support a block, then it was a closure. :) Not always obvious from
just looking at the source. Using this guide, you'll find that
changing from:

object Test {
def main(args:Array[String]) {
println("Hello")
}
}

to


object Test {
def main(args:Array[String]) {
1 to 10 map println
}
}

Results in a new classfile generated. For the anonfun that is the
unapplied println.