Migration from 6 to 7

[Runar Bjarnason]

Migration from Scalaz 6 to Scalaz 7 has been difficult. Here are some of my notes, with suggestions on how to improve. I have already made a lot of improvements, not covered here. Have a read through these and think about whether you think these are worth the effort for users. Some of them probably are, others not.

General note: I’m spending a lot of time hunting through the scalaz source to find out how to import a specific instance or function.

My biggest gripe is the flipping of State. It used to be S => (S, A) and now it's S => (A, S). Worth the effort to migrate? I think not.

Import issues:
* IterV no longer exists in the core project. Needed to drop in and import the iterv project specifically.
* Need to import Show._ and Equal._ in order to get instance constructors equalA and showFromToString (which used to be called showA). Rename all calls to showA.
* Various imports are no longer default in the Scalaz object. For example, Equal.equalA.
* To get a Foldable[Iterable] instance, you need to import std.iterable. Be careful with this because it will conflict with Foldable for subtypes like Stream and List.
* scalaz.effects has been renamed to scalaz.effect. Simple search/replace.

Missing/removed functionality:
* OrderOrdering no longer exists. Use the toScalaOrdering method on Order.
* NewType no longer exists. Needed to manually convert all newtypes to tagged types.
* PimpedType no longer exists. Needed to replace PimpedTypes with implicit unwrapper.
* foldMapDefault was missing. Globally replace with foldMap.
* .asMA no longer exists. Added concatenate instead of asMA.sum

Changed or moved functionality, or different types:
* “nel” has changed. It used to be a constructor that took varargs. That constructor is now NonEmptyList.apply, so a global replace is required. However, there was also a "nel" that did the same as the current "nel" does (construct a NonEmptyList out of a head and a tail), so replace carefully.
* Equal[Set[A]] no longer requires Equal[A], but Order[A]. So we have to implement Order for things where Equal was previously sufficient.
* foldl and foldr have been curried. Universally replace with foldLeft and foldRight when migrating from 6.
* Foldable.stream and .list have been renamed to toStream and toList.
* The return type of State has been flipped. Anything that uses “state” to construct a State needs to flip its return type.
* The lowercase state(s => (s, a)) constructor no longer exists. Need to convert all references to uppercase State(s => (a, s))
* Instead of liftKleisli(f) with a lot of type annotations, the new hotness is f.liftKleisli. This requires importing syntax.kleisli._. This requires considerably less type annotation. There is, however a conflict with IdOps.liftKleisli, which is disambiguated by using liftReaderT instead.
* Const no longer exists. This used to provide a phantom applicative functor for a monoid. This was helpful when traversing with a monoid like F[A], where F is already applicative, but you want the monoid action instead of the applicative action. The solution is to traverse with type λ[α]=F[A].
* The ! method no longer exists on State. Needs to be universally replaced with “eval”.
* The type of State.modify has been changed. It no longer returns Unit in the state monad, but S. Results in a lot of type errors if you modify a lot. This one was pretty annoying. I'm inclined to say that the correct return type of "modify" should be Unit.
* The constructors of Lens have completely changed. The “Lens” constructor now takes a Costate. There is a “lensu” constructor that takes the same arguments as the old Lens constructor, only flipped. So to migrate from 6 to 7 you have to substitute "lensu" for "Lens" and flip the arguments everywhere.
* The Reducer constructor has changed. It now takes (curried!) arguments unit, cons, and snoc. There is a UnitReducer constructor that takes the same arguments as the old Reducer constructor. We should consider flipping these, and we should in fact consider getting rid of UnitReducer. What is it for?


Runar

[Jason Zaugg]

On Mon, May 14, 2012 at 11:26 PM, Runar Bjarnason <runar...@gmail.com> wrote:
> Migration from Scalaz 6 to Scalaz 7 has been difficult. Here are some of my
> notes, with suggestions on how to improve. I have already made a lot of
> improvements, not covered here. Have a read through these and think about
> whether you think these are worth the effort for users. Some of them
> probably are, others not.

Thanks for the detailed account, it will help to refine things.

We can certainly do a better job to avoid incidental breakage.

I was keenly aware of the difficulty of naming functions related to
data types. In Scalaz6, there was essentially a global namespace for
these, as they were mixed into object Scalaz. I don't believe we
should continue with this as the number of functions increases.
Instead, I prefer to use the companion objects to scope the functions.

To recap the basic organization of these:

class DS
trait DSInstances
trait DSFunctions
object DS extends DSInstances with DSFunctions {
def apply() = ...
// and other functions with names that don't stand alone, e.g. fromString
}

For *some* of the XxxFunction traits, I have sat on the fence and
mixed them into object Scalaz.

object Scalaz extends StateTFunctions with ...

We could mix a few more into object Scalaz. Perhaps better, we could
create object ScalazMigrate with a reconstruction of the old API,
adorned with @deprecation messages that point to the new homes.

object ScalazMigrate {
@deprecated("use Equal.equalFromToString", "7.0")
val equalA[A](...) = Equal.equalFromToString(...)
}

Regarding State, you made a compelling argument in favour of the
flipped type arguments [1]. I'm not too bothered either way.

PimpedType/NewType could be resurrected if they are used outside of
the library. OTOH, they are trivial enough for users to implement in
their own codebase.

The changes to Reducer should be reviewed. They came from Tony's scalaz7 branch.

I just removed IdOps#{liftKleisli, liftReader}, non-ambiguously named
versions of these were already installed in KleisliIdOps.

The jury is still out on Lens (should it be a transformer, what
constructors are provided.)

Requiring an explicit importing instances for Iterable is by design,
for the reasons you mention. Needs to be documented better, though.

Equal[Set[A]] takes an Ordering rather than Equal to provide better
performance. You could provide a slow version that only requires an
Equal[A], and put it behind an explicit import.

-jason

[1] https://groups.google.com/d/msg/scalaz/-sNi3SpI3Pc/bzUbYPcyAIIJ

[Runar Bjarnason]

For *some* of the XxxFunction traits, I have sat on the fence and
mixed them into object Scalaz.

 object Scalaz extends StateTFunctions with ...

I was on the fence on this one as well, but I ended up adding it to the Scalaz object. I don't really have a problem with adding another import to get State functions, but I was able to get rid of a few hundred compiler errors this way.

 

We could mix a few more into object Scalaz. Perhaps better, we could
create object ScalazMigrate with a reconstruction of the old API,
adorned with @deprecation messages that point to the new homes.

 object ScalazMigrate {
     @deprecated("use Equal.equalFromToString", "7.0")
     val equalA[A](...) = Equal.equalFromToString(...)
 }

This is a splendid idea.

Regarding State, you made a compelling argument in favour of the
flipped type arguments [1]. I'm not too bothered either way.

Must have been some other idiot named Runar :)

OK, so those are pretty good arguments, but I didn't have strong feelings about it either way. There are good arguments for flipping back as well:

1. Tuple2 is right-biased. It's a good idea for Writer to be consistent with this.
2. It requires a heck of a lot less migration from 6 to 7.

If anybody has strong opinions on this, please pipe up.
 

Requiring an explicit importing instances for Iterable is by design,
for the reasons you mention. Needs to be documented better, though.

I also added instances for Vector.

[Heiko Seeberger]

On May 15, 2012, at 9:34 PM, Runar Bjarnason wrote:

> Must have been some other idiot named Runar :)

Who's that?

> OK, so those are pretty good arguments, but I didn't have strong feelings about it either way. There are good arguments for flipping back as well:
>
> 1. Tuple2 is right-biased. It's a good idea for Writer to be consistent with this.
> 2. It requires a heck of a lot less migration from 6 to 7.
>
> If anybody has strong opinions on this, please pipe up.

Flip back, please (just an opinion, not a strong one).

Heiko

[Runar Bjarnason]

> 1. Tuple2 is right-biased. It's a good idea for Writer to be consistent with this.

Alternatively, we could flip the bias of Tuple2 as well.

[Chris Marshall]

That's a joke. Right?

[Runar Bjarnason]

On Wed, May 16, 2012 at 9:12 AM, Chris Marshall <oxbow...@gmail.com> wrote:

That's a joke. Right?

No. Why?
 

[Chris Marshall]

val howManySharesInTypesafeIJustBought = 100

val myCurrentHoldings = findMyHoldings // a Pair (Int, Int) representing on the left, shares I have short and on the right, shares I own

val (_, newLongPosition) = myCurrentHoldings map (_ +  howManySharesInTypesafeIJustBought)

Sadness will ensue

--
You received this message because you are subscribed to the Google Groups "scalaz" group.
To post to this group, send email to sca...@googlegroups.com.
To unsubscribe from this group, send email to scalaz+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scalaz?hl=en.

[Runar Bjarnason]

The exact same sadness will ensue if you use State, because that has flipped. You seem to be making a strong argument for flipping it back.

[Chris Marshall]

Well, I don't use much State, because teh stackz overflowz. Of course, this will change shortly :-)

[Tony Morris]

Costate took on some variance annotations recently -- why is this?

-- 
Tony Morris
http://tmorris.net/

[Matthew Pocock]

On 16 May 2012 14:18, Chris Marshall <oxbow...@gmail.com> wrote:

val howManySharesInTypesafeIJustBought = 100

val myCurrentHoldings = findMyHoldings // a Pair (Int, Int) representing on the left, shares I have short and on the right, shares I own

val (_, newLongPosition) = myCurrentHoldings map (_ +  howManySharesInTypesafeIJustBought)

Sadness will ensue

Yeah, much sadness. But then you're using a semantics-free structure (Int, Int) to capture domain-specific semantically-rich information type Position = (SharesShort, SharesOwned) and then expecting map to do sensible things with this without even giving it a hope in hell of spotting if you make a mistake. I don't think the root cause here is the left-or-right biassed nature of (A, B) ;) How about some tagging with domain model types?

val howManySharesInTypesafeIJustBought = 100.@@[SharesOwned] // I have a @@ pimp to tag things

val myCurrentHoldings = findMyHoldings // type Position = (Int @@ SharesShort, Int @@ SharesOwned)

val (_, newLongPosition) = myCurrentHoldings mapSnd (_ + howManySharesInTypesafeIJustBought)

I don't really get why you would want (in the general case) to be able to apply map to a pair, and expect it to always operate on either fst or snd, and why it doesn't make sense to instead expose mapFst, mapSnd. If you really want bias, introduce @@LHSBiassed and @@RHSBiassed with the associated type instances - everything is explicitly in clearly named type annotations then, and nobody need get hurt.

val howManySharesInTypesafeIJustBought = 100.@@[SharesOwned] // I have a @@ pimp to tag things

val myCurrentHoldings = findMyHoldings // type Position = (Int @@ SharesShort, Int @@ SharesOwned) @@ RHSBiassed

val (_, newLongPosition) = myCurrentHoldings map (_ + howManySharesInTypesafeIJustBought)

It occurs to me that LHSBiassed and RHSBiassed could be used for both bias in products and sums. So, both (A, B) and Either[A, B] could be essentially symmetrical, and biassed operations dropped in where it makes sense by tagging. As an implementation detail, I think the instances would need to be over types like (A, B)@@C where C <:< RHSBiassed, so that the user can still tag the tuple with other domain-specific information.

Matthew

On Wed, May 16, 2012 at 2:13 PM, Runar Bjarnason <runar...@gmail.com> wrote:

On Wed, May 16, 2012 at 9:12 AM, Chris Marshall <oxbow...@gmail.com> wrote:

That's a joke. Right?

No. Why?
 

--
You received this message because you are subscribed to the Google Groups "scalaz" group.
To post to this group, send email to sca...@googlegroups.com.
To unsubscribe from this group, send email to scalaz+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scalaz?hl=en.

--
You received this message because you are subscribed to the Google Groups "scalaz" group.
To post to this group, send email to sca...@googlegroups.com.
To unsubscribe from this group, send email to scalaz+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scalaz?hl=en.

--

Dr Matthew Pocock

Integrative Bioinformatics Group, School of Computing Science, Newcastle University

mailto: turingate...@gmail.com

gchat: turingate...@gmail.com

msn: matthew...@yahoo.co.uk

irc.freenode.net: drdozer

skype: matthew.pocock

tel: (0191) 2566550

mob: +447535664143

[Runar Oli]

Because LensT did. And that did because StateT did. And I did that to get better type inference.

[Runar Oli]

On May 17, 2012, at 6:35, Matthew Pocock <turingate...@gmail.com> wrote:

> I don't really get why you would want (in the general case) to be able to apply map to a pair, and expect it to always operate on either fst or snd,

Because it's a perfectly good functor. For example, State and Costate are composed of that functor and another. Having the map function there makes working with these adjunctions that much nicer.

Runar

[Chris Marshall]

My point was based solely on the following logic:

• scalaz released a library with a right-biased Functor on Tuple2
• This was *long* before tagged types came into the frame
• People will have used it
• Switching biased-ness of the functor will result in horribly-broken, yet happily compiling code. You know, the stuff that crashed space ships are made from.

Chris

[Matthew Pocock]

Sure, but whether to choose left or right bias is arbitrary, and in different contexts different alternatives are the natural one. Even with state/costate, the two fields of the pair have distinct roles above and beyond simply filling the first and second slot of a pair - they have distinct meanings. In the days before tagged types, at the cost of an implicit or a constructor, we could have introduced a case class that gives these two roles meaningfully documented names. Anyway, as Chris has pointed out, the issue is now a practical one because of existing code, and different pressures are in play than if you where designing this stuff from scratch. I agree with everything he says about the dangers of switching existing behaviour now, if it is possible for code to compile wrongly.

Matthew

Runar

--
You received this message because you are subscribed to the Google Groups "scalaz" group.
To post to this group, send email to sca...@googlegroups.com.
To unsubscribe from this group, send email to scalaz+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scalaz?hl=en.

[Runar Bjarnason]

Sure, but whether to choose left or right bias is arbitrary,

 
It isn't completely arbitrary. Consider this type:

((A, B) => C) => A => B => C

This is just "Curry". But it has a generalization:

(F[A] => C) => A => G[C]

And it goes the other way as well (uncurry):

(A => B => C) => (A, B) => C
i.e.
(A => G[C]) => F[A] => C

F and G here form an adjunction (F is left-adjoint to G). If you think of G as the exponentiation functor, then F is the product functor. It's not arbitrary which side of the pair is fixed. It depends on the order of type arguments to =>.

G[F[_]] forms a monad which we call State. And F[G[_]] forms a comonad which we call Costate.

Of course these exist as well:
((B, A) => C) => A => B => C
(A => B => C) => (B, A) => C

But they involve an additional flip. So the left-biased pair is more fundamental in the sense that the dual of a functor involving Function1 will have a Tuple2 such that the left argument of the pair lines up with the right argument of the arrow (without flipping and without appealing to commutativity). The same argument can be applied to right-biasing Either.

So left-biasing a pair is pretty well supported, not just by arbitrary whim, but by the very nature of how products participate in categories.

[Chris Marshall]

*Sheepishly* - I have used it

For example, it's incredibly useful if I have a pair of values (a, b), a function B => Validation[X, C], and I want to get a  Validation[X, (A, C)]

scala> val v = ("a", "1") map (_.parseInt)

v: (java.lang.String, scalaz.Validation[NumberFormatException,Int]) = (a,Success(1))

scala> v.sequence[({type l[a]=Validation[Exception, a]})#l, Int]

res1: scalaz.Validation[java.lang.Exception,(java.lang.String, Int)] = Success((a,1))

Or, more succinctly, of course:

scala> ("a", "1").traverse[({type l[a]=Validation[Exception, a]})#l, Int](_.parseInt)

res11: scalaz.Validation[java.lang.Exception,(java.lang.String, Int)] = Success((a,1))

Chris

[Tony Morris]

It's giving me grief. When did variance annotations become a good idea?

[Jason Zaugg]

On Tue, May 22, 2012 at 8:25 AM, Tony Morris <tonym...@gmail.com> wrote:
> It's giving me grief. When did variance annotations become a good idea?
>
>
>
> On 17/05/12 20:49, Runar Oli wrote:
>
> Because LensT did. And that did because StateT did. And I did that to get
> better type inference.

These discussions will probably go in circles if we don't commit test
cases to back up our claims of better/worse inference.

-jason

[Tony Morris]

True indeed.

As it stands, everything on top of scalaz must now also change to use variance annotations. I am unconvinced this is a good idea.

[Runar Bjarnason]

Variance annotations are not optional. Scala requires you to annotate one of: covariant, contravariant, invariant.

Runar

[Michael Pilquist]

On Monday, May 14, 2012 5:26:06 PM UTC-4, Runar Bjarnason wrote:

Changed or moved functionality, or different types:

Adding a few more to this list:

 - Validation#liftFailNel replaced with Validation#toValidationNel 

 - Validation#lift replaced with Validation#pointSuccess

 - Validation#FailProjection#lift replaced with Validation#FailProject#point

These were all obvious and intuitive once I looked at the source for Validation, but searches for removal of liftFailNel didn't yield anything.  I figured others will run in to this while porting.

Regards,

Michael

[Tony Morris]

I typically provide no annotation, defaulting to invariance, then
annotate with covariance using "instance Functor" and contravariance
using "instance Contravariant."

Are we going to change this? Is the little bit of type inference for
acknowledging Scala's inheritance really worth the adverse consequences?

On 23/05/12 00:26, Runar Bjarnason wrote:
> Variance annotations are not optional. Scala requires you to annotate one
> of: covariant, contravariant, invariant.
>
> Runar
>
>
>
>
>
> On Tue, May 22, 2012 at 2:25 AM, Tony Morris <tonym...@gmail.com> wrote:
>

>> **

>> Tony Morrishttp://tmorris.net/

>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "scalaz" group.
>> To post to this group, send email to sca...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> scalaz+un...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/scalaz?hl=en.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "scalaz" group.
>> To post to this group, send email to sca...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> scalaz+un...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/scalaz?hl=en.
>>
>>
>>
>> --

>> Tony Morrishttp://tmorris.net/

>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "scalaz" group.
>> To post to this group, send email to sca...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> scalaz+un...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/scalaz?hl=en.
>>


--

Tony Morris
http://tmorris.net/

[Runar Bjarnason]

Yes, I think the adverse consequences can be dealt with. They make things a bit more difficult for us and a lot easier for the user.

For me the gain is mostly in type inference. E.g. with a covariant State I can do this without annotation:

10.state

instead of:

10.pure[({type f[x] = State[S, x]})#f]

[Tony Morris]

OK, taking your word for it for now.

Can we move the covariant/contravariant annotations off scalaz-seven as a patch for discussion? I mean, I would dearly love to learn that Scala's explicit variance annotations really are actually worth it, but I dismissed them long ago and if you are right, then I think it would be good to have more support and also comment from other peers.

Supposing we agree to go with explicit variance annotation, we'd update the developer's guide and a few other data structures, etc.

What say you?

[Runar Bjarnason]

I'd love to have two branches with differing annotations. It took quite some effort to make all the variances as they are now, so I am not sure I could commit to going through and making things invariant again. Those changes are intertwined with some others but maybe there's a way to branch before I made that change somehow. I defer to people with better git-fu than mine.

Now, I agree with you in spirit and I hate subtyping with the burning passion of a thousand suns. But let me be clear about this: All type constructors in Scala have explicit variance annotation. This is not optional. When you say trait Foo[A] you are explicitly annotating that Foo is invariant. It would be really really nice if Scala had better kind inference so that we could simply omit the annotations, but this is not the case.

Having said that, it's not strictly true that a type variable like F[_] means "invariant F of kind * -> *". Scala will happily let you unify that with a * -> * of any variance at all. This is what lets us declare things like StateSyntax without the variance annotation on the type constructor variable F. And we can say Monad[M[_]] without caring about subtyping. Strictly speaking, it should be Monad[M[+_]], but I believe that would not be worth the effort. So I am proposing that we use the correct variance annotation for data types (where they are not optional) but omit the annotations where we can, e.g. in type classes. I believe that this provides a nice "out" for us, a clean separation.

But, as lenient as Scala is with kind checking of type constructors, it is really a tyrant when it comes to applying them. For example, if you say StateT[F[_], S, +A], Scala will flip its shit when you apply F[A]. Because in that case it takes F[_] to be explicitly annotated as being invariant. This is not consistent. I'm even inclined to say that it is a bug in Scala's typer. And it's in this case that we need to say StateT[F[+_], S, +A].

Sorry if this is confusing. We can talk over IRC if that's better.