2.10 regression Option.flatten no longer works for Option(Iterable)

[Chris Marshall]

It looks like the signature of flatten has changed when used from Option (it's been added to Option, rather than being inferred from the conversion to Iterable). It looks like this was made by Paul a year ago in this commit:  https://github.com/scala/scala/commit/11ebee09918841b5b4175b60399af4e04fade833. It breaks existing code (and commonly-used code at that) - was it meant to?

C:\Scala\sdk\scala-2.10.0-M6\bin>scala

Welcome to Scala version 2.10.0-M6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_04).

Type in expressions to have them evaluated.

Type :help for more information.

scala> Option(Stream continually 1).flatten

<console>:8: error: Cannot prove that scala.collection.immutable.Stream[Int] <:< Option[B].

              Option(Stream continually 1).flatten

                                           ^

This works in 2.9

C:\Scala\sdk\scala-2.9.2\bin>scala

Welcome to Scala version 2.9.2 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_04).

Type in expressions to have them evaluated.

Type :help for more information.

scala> Option(Stream continually 1).flatten

java.lang.OutOfMemoryError: Java heap space

Which brings me to what I was originally going to complain about: "flatten" (from Option) should work where the inner sequence has no definite size (Option(Stream continually 1).flatten works just fine)

Chris

[Chris Marshall]

OK, I found this:  https://issues.scala-lang.org/browse/SI-4474 

Case closed

---

From: oxbow...@hotmail.com
To: scala...@googlegroups.com
Subject: [scala-user] 2.10 regression Option.flatten no longer works for Option(Iterable)
Date: Wed, 22 Aug 2012 07:12:41 +0000

[Antoine Gourlay]

Actually it's worse than that, it has nothing to do with the size of the inner collection:

scala> Some(Seq(1)).flatten
<console>:8: error: Cannot prove that Seq[Int] <:< Option[B].
              Some(Seq(1)).flatten
                           ^
This used to return an Iterable[Int] containing List(1) in 2.9.2

This is sad...

-- 
-- Antoine

[Jason Zaugg]

On Wed, Aug 22, 2012 at 9:12 AM, Chris Marshall <oxbow...@hotmail.com> wrote:
> It looks like the signature of flatten has changed when used from Option
> (it's been added to Option, rather than being inferred from the conversion
> to Iterable). It looks like this was made by Paul a year ago in this commit:
> https://github.com/scala/scala/commit/11ebee09918841b5b4175b60399af4e04fade833.
> It breaks existing code (and commonly-used code at that) - was it meant to?

I also noticed this recently [1]. Paul commented, "I wonder whether
changes to implicits and full source compatibility are not completely
intractable."

Option(Seq(1)).toSeq.flatten will work in both 2.9.x and 2.10.x.

-jason

[1] https://github.com/scala/scala/pull/1026#issuecomment-7577936

[Dave]

imo this is correct since flatten is a monadic natural transformation mu

The inner type constructor must be the same as the outer type constructor otherwise you cannot do a flatten.

It is afaik actually unboxing from boxes with equal type constructors.

 

e.g. this works

 

scala> Seq(Seq(1)).flatten
res0: Seq[Int] = List(1)

scala> Some(Some(1)).flatten
res1: Option[Int] = Some(1)

 

I would say that

scala> Seq(Some(1)).flatten
res2: Seq[Int] = List(1)

is an error. It should raise a compile time error.

But I don't know why this is permitted by scala.

 

It looks like scala has different notions of flatten:

in one meaning it means just unboxing,

in the other meaning it means unboxing from boxes with the same type constructors.

 

Op woensdag 22 augustus 2012 09:22:43 UTC+2 schreef Antoine Gourlay het volgende:

[Jason Zaugg]

On Wed, Aug 22, 2012 at 1:33 PM, Dave <dave.mah...@hotmail.com> wrote:
> imo this is correct since flatten is a monadic natural transformation mu
> The inner type constructor must be the same as the outer type constructor
> otherwise you cannot do a flatten.
> It is afaik actually unboxing from boxes with equal type constructors.
>
> e.g. this works
>
> scala> Seq(Seq(1)).flatten
> res0: Seq[Int] = List(1)
> scala> Some(Some(1)).flatten
> res1: Option[Int] = Some(1)
>
> I would say that
> scala> Seq(Some(1)).flatten
> res2: Seq[Int] = List(1)
> is an error. It should raise a compile time error.
> But I don't know why this is permitted by scala.
>
> It looks like scala has different notions of flatten:
> in one meaning it means just unboxing,
> in the other meaning it means unboxing from boxes with the same type
> constructors.

The Scala interpretation of flatten is more general; you can flatten
Collection[X] if X is implicitly convertible to TraversableOnce[B].

Here's the signature:

def flatten[B](implicit asTraversable: A => /*<:<!!!*/
TraversableOnce[B]): Traversable[B]

The commented out code hints at how to implement the less permissive
version, that would require X to be a subtype of TraversableOnce[B],
rather than just convertible this type.

-jason

[Matthew Pocock]

On 22 August 2012 12:39, Jason Zaugg <jza...@gmail.com> wrote:

The Scala interpretation of flatten is more general; you can flatten

Collection[X] if X is implicitly convertible to TraversableOnce[B].

Here's the signature:

  def flatten[B](implicit asTraversable: A => /*<:<!!!*/
TraversableOnce[B]): Traversable[B]

The commented out code hints at how to implement the less permissive
version, that would require X to be a subtype of TraversableOnce[B],
rather than just convertible this type.

What's the contract? Does flatten respect the semantics of the outer or inner container? I have to admit that nearly every time I reach for flatten, it's because I've either done something very wrong, or I have C[C[X]] (sometimes both at the same time).

Matthew

 

-jason

--

Dr Matthew Pocock

Integrative Bioinformatics Group, School of Computing Science, Newcastle University

mailto: turingate...@gmail.com

gchat: turingate...@gmail.com

msn: matthew...@yahoo.co.uk

irc.freenode.net: drdozer

skype: matthew.pocock

tel: (0191) 2566550

mob: +447535664143

[Jason Zaugg]

On Wed, Aug 22, 2012 at 4:10 PM, Matthew Pocock
<turingate...@gmail.com> wrote:
>
>
> On 22 August 2012 12:39, Jason Zaugg <jza...@gmail.com> wrote:
>>
>> The Scala interpretation of flatten is more general; you can flatten
>> Collection[X] if X is implicitly convertible to TraversableOnce[B].
>>
>> Here's the signature:
>>
>> def flatten[B](implicit asTraversable: A => /*<:<!!!*/
>> TraversableOnce[B]): Traversable[B]
>>
>> The commented out code hints at how to implement the less permissive
>> version, that would require X to be a subtype of TraversableOnce[B],
>> rather than just convertible this type.
>
>
> What's the contract? Does flatten respect the semantics of the outer or
> inner container? I have to admit that nearly every time I reach for flatten,
> it's because I've either done something very wrong, or I have C[C[X]]
> (sometimes both at the same time).

The outer.

xs.flatten is just xs.flatMap(x => x)

-jason