Questions about \/ and Validation

[Bill Venners]

Hi All,

Tony Morris suggested we take a discussion off Twitter. We decided to move it here because it involves Scalaz's \/, Validation, ValidationNEL, for which I believe Tony is the primary author, and ScalaUtils' Or, for which I am the author. Essentially I have some specific questions for Tony to help me understand why \/ and Valdation are designed the way they are, and I wanted to explain why I designed Or the way I did. Twitter is not the right forum for that. So here goes.

One difference between \/ and Or is that \/ is right-biased whereas Or is left-biased. My first question is why is right bias the tradition? \/ and Validation are right-biased outright; Either suggests putting "good" results on the right, error results on the left. The consistency is good, but where did it come from in the first place?

The reason I made Or left-biased, which is inconsistent with tradition, is that I think it reads better textually to have the "valid" result type first when types are written in infix notation:

def parseName(input: String): String Or ErrorMessage = ???

The other main question I have is why are there two types, \/ and Validation? ScalaUtils just has one type, Or, which can be used to accumulate errors if the "bad" type is an Every (which like NonEmptyList, is a sequence containing one to many elements). Having one type is is think simpler for users. It avoids user confusion about which one to use, such as Travis Brown's question here:

http://stackoverflow.com/questions/20065853/validation-versus-disjunction/20134523

I imagine there's a good reason to have both \/ and Validation, and I'd like to understand what it is. My guess is it has to do with the reason flatMap is deprecated in Validation, something to do with a conflict between monad and applicative laws making it impossible to be both at once.  If so I may need some hand-holding as I don't feel I understand applicatives very well yet, but I'd like to understand from the applicative perspective why I was able to do this in one type. Does the Or/Every combination perhaps violate an applicative law, something you would not want to do, and that's why I was able to do this in one type?

Those are my main two questions for Tony. There are other differences between \//Validation and Or, we can discuss that later if it is useful.

Bill

[Mark Hibberd]

On Fri, Jan 3, 2014 at 4:42 PM, Bill Venners <bi...@artima.com> wrote:
> Hi All,
>
> Tony Morris suggested we take a discussion off Twitter. We decided to move
> it here because it involves Scalaz's \/, Validation, ValidationNEL, for
> which I believe Tony is the primary author, and ScalaUtils' Or, for which I
> am the author. Essentially I have some specific questions for Tony to help
> me understand why \/ and Valdation are designed the way they are, and I
> wanted to explain why I designed Or the way I did. Twitter is not the right
> forum for that. So here goes.
>
> One difference between \/ and Or is that \/ is right-biased whereas Or is
> left-biased. My first question is why is right bias the tradition? \/ and
> Validation are right-biased outright; Either suggests putting "good" results
> on the right, error results on the left. The consistency is good, but where
> did it come from in the first place?

It because it is how the type parameters should be partially applied.
In scala-like terms you want the functor to be:
Either[A, _]
not
Either[_, B].

> The reason I made Or left-biased, which is inconsistent with tradition, is
> that I think it reads better textually to have the "valid" result type first
> when types are written in infix notation:
>
> def parseName(input: String): String Or ErrorMessage = ???

I don't think discussing it in terms of syntax is useful, I will just
say I strongly disagree with the above being a positive.

> The other main question I have is why are there two types, \/ and
> Validation? ScalaUtils just has one type, Or, which can be used to
> accumulate errors if the "bad" type is an Every (which like NonEmptyList, is
> a sequence containing one to many elements). Having one type is is think
> simpler for users. It avoids user confusion about which one to use, such as
> Travis Brown's question here:
>
> http://stackoverflow.com/questions/20065853/validation-versus-disjunction/20134523
>
> I imagine there's a good reason to have both \/ and Validation, and I'd like
> to understand what it is. My guess is it has to do with the reason flatMap
> is deprecated in Validation, something to do with a conflict between monad
> and applicative laws making it impossible to be both at once. If so I may
> need some hand-holding as I don't feel I understand applicatives very well
> yet, but I'd like to understand from the applicative perspective why I was
> able to do this in one type. Does the Or/Every combination perhaps violate
> an applicative law, something you would not want to do, and that's why I was
> able to do this in one type?

If you put both behaviours in one type any code that depends on the
abstract concepts (applicative/monad) is not predictable. Will this
function accumulate errors or will it not?

The reason both exist is simply because sometimes you want the non
accumulating version and sometimes you want the accumulating version.
This is dependent on the context not the types, i.e. just because the
left is a list doesn't mean you want to accumulate. You should be
explicit, I want this behaviour, so I want this type.

I would suggest that there are either edge conditions which are
confusing/non-predictable or there is actually no code exploiting the
more general concepts with respect to Or/Every.

Validation is scalaz is almost always used incorrectly because of the
existence of the flatMap method. I have seen very few uses of
validation that I would not consider a bug - they do exist - but
people leap to it because of the pretty name which some what
ridiculous, and then they have a Nel that always has one and only one
value in it.

>
> Those are my main two questions for Tony. There are other differences
> between \//Validation and Or, we can discuss that later if it is useful.

Don't think I have addressed everything, but you can probably take
some info from above.

[Bill Venners]

Hi Mark,

Thanks for your reply.

On Thursday, January 2, 2014 9:53:54 PM UTC-8, Mark Hibberd wrote:

On Fri, Jan 3, 2014 at 4:42 PM, Bill Venners <bi...@artima.com> wrote:
> Hi All,
>
> Tony Morris suggested we take a discussion off Twitter. We decided to move
> it here because it involves Scalaz's \/, Validation, ValidationNEL, for
> which I believe Tony is the primary author, and ScalaUtils' Or, for which I
> am the author. Essentially I have some specific questions for Tony to help
> me understand why \/ and Valdation are designed the way they are, and I
> wanted to explain why I designed Or the way I did. Twitter is not the right
> forum for that. So here goes.
>
> One difference between \/ and Or is that \/ is right-biased whereas Or is
> left-biased. My first question is why is right bias the tradition? \/ and
> Validation are right-biased outright; Either suggests putting "good" results
> on the right, error results on the left. The consistency is good, but where
> did it come from in the first place?

It because it is how the type parameters should be partially applied.
In scala-like terms you want the functor to be:
  Either[A, _]
not
  Either[_, B].

Interesting, but I could use a bit more elaboration. Can you give me a specific example of why I might want to partially apply one of the two types in an Either? Which one of the two types would I normally know and want to "fill in" leaving a type that just has one type parameter. And what would I be trying to accomplish by that?
 

> The reason I made Or left-biased, which is inconsistent with tradition, is
> that I think it reads better textually to have the "valid" result type first
> when types are written in infix notation:
>
> def parseName(input: String): String Or ErrorMessage = ???

I don't think discussing it in terms of syntax is useful, I will just
say I strongly disagree with the above being a positive.

Fair enough.
 

> The other main question I have is why are there two types, \/ and
> Validation? ScalaUtils just has one type, Or, which can be used to
> accumulate errors if the "bad" type is an Every (which like NonEmptyList, is
> a sequence containing one to many elements). Having one type is is think
> simpler for users. It avoids user confusion about which one to use, such as
> Travis Brown's question here:
>
> http://stackoverflow.com/questions/20065853/validation-versus-disjunction/20134523
>
> I imagine there's a good reason to have both \/ and Validation, and I'd like
> to understand what it is. My guess is it has to do with the reason flatMap
> is deprecated in Validation, something to do with a conflict between monad
> and applicative laws making it impossible to be both at once.  If so I may
> need some hand-holding as I don't feel I understand applicatives very well
> yet, but I'd like to understand from the applicative perspective why I was
> able to do this in one type. Does the Or/Every combination perhaps violate
> an applicative law, something you would not want to do, and that's why I was
> able to do this in one type?

If you put both behaviours in one type any code that depends on the
abstract concepts (applicative/monad) is not predictable. Will this
function accumulate errors or will it not?

I think it is predictable in the Or/Every case. If the "bad" type is an Every, it will accumulate. Otherwise it will not.

The reason both exist is simply because sometimes you want the non
accumulating version and sometimes you want the accumulating version.
This is dependent on the context not the types, i.e. just because the
left is a list doesn't mean you want to accumulate. You should be
explicit, I want this behaviour, so I want this type.

I am not sure I see the difference here. If you want accumulation, you make the "bad" type an Every. Otherwise you don't. I believe you can still pick the type that has the behavior you want.
 

I would suggest that there are either edge conditions which are
confusing/non-predictable or there is actually no code exploiting the
more general concepts with respect to Or/Every.
 

I don't think there is any non-predictable behavior. If the bad type is an Every is the "switch" that enables accumulation. Of which "more general concepts" do you refer?
 

Validation is scalaz is almost always used incorrectly because of the
existence of the flatMap method. I have seen very few uses of
validation that I would not consider a bug - they do exist - but
people leap to it because of the pretty name which some what
ridiculous, and then they have a Nel that always has one and only one
value in it.

I see. you mean they use Validation when they don't want to accumulate errors, and because it has flatMap they can make pretty for comprehensions with Validation that short circuit at the first sign of trouble? That's interesting. You can do the same thing with an accumulating Or, but I don't see that as a bug. It is just one way to use an Or. Accumulating or not, if you use an Or in a for-expression, it will short circuit at the first failure.

Bill
 

[Mark Hibberd]

On Friday, January 3, 2014, Bill Venners wrote:

It because it is how the type parameters should be partially applied.
In scala-like terms you want the functor to be:
  Either[A, _]
not
  Either[_, B].

Interesting, but I could use a bit more elaboration. Can you give me a specific example of why I might want to partially apply one of the two types in an Either? Which one of the two types would I normally know and want to "fill in" leaving a type that just has one type parameter. And what would I be trying to accomplish by that?

 

The most basic case is `trait Functor[F[_]] { def map[A, B](a: F[A])(f: A => B): F[B] }` for dealing generally with things that have map. 

In non-scala languages that have higher kinds, there are elegant ways to partially apply type parameters, to translate into an imaginary scala-like syntax, to get a conforming 'F' in this example for Either, you would just write something like Either[A]. In scala we have to do type lambdas and things get horrible pretty quickly, but you can exploit the same ordering to get some help in scala. In fact scalaz 6 used to provide a large number of types for doing just that. So running with the example you would normally write something like:

  def EitherFunctor[A] = new Functor[[({type l[a] = Either[A, a]})#l] { ... }

But by exploiting the ordering of the type arguments we can come up with general types to fit. In this case something like:

  

  class OneOfTwo[F[_, _], A] { type l[a] = F[A, a] }

And use it (in a non convincing way because of the somewhat trivial nature of Either):

  def EitherFunctor[A] = new Functor[OneOfTwo[Either, A]#l] { ... }

For just either this seems kind of neither here nor their, but as you build up larger monad transformer stacks you _really_ want there to be tools for dealing with these things rather than having to do the type lambda dance every time. So there is a benefit to the ordering even in scala (particularly with respect to consistency of other data structures like Reader, Writer and State, and it is worth nothing these all have there arguments the same order as Either in that the right most type parameter is the 'A' that you map across, and they have no concept of the "failure" case so the ordering of either based on failure the success is not an accurate reflection, it is simply about partially applying for Functor).

The "more general concepts" are the key here. I mean methods which exploit the structure and laws of Monad/Applicative/Functor and friends. An example might be `traverse` or `sequence` from scalaz. These can be implemented in terms of Monad via flatMap or Applicative via ap, but it would be very unpleasant if that choice changed the behaviour of traverse or sequence. And this is what would happen if you were to define Monad and Applicative for Validation (and Or). This is amplified because dealing with the concrete types is often not necessary and very often undesirable, we want to be able to write really general code and be able to reason about it a sensible way. Either, Validation, \/, Or are all somewhat boring, I just want to pick one that has the right properties at the right time, and the rest of my code just deals with the fact that is an applicative or what ever.

An example might be some parser like functions that maybe can handle failures in different ways (might be useful in streaming IO vs  in memory or something). So as an arbitrary function I might want to do something like:

  def all[F[_]: Applicative, A](s: String, parsers: List[String => F[A]]): F[A] = 

    parsers.traverse(p => p(s))  

I can use this with \/ or Validation and get the behaviour I want. This is desirable.

So Or may be predictable on its own, but only because you are not exploiting the fact that it could be an applicative or a monad. This may not be a bad thing for what you are trying to achieve, but I get a lot out of being able to write the general code above, but to do so I need to ensure that all the laws of these structures are closely followed. So what at first sounds like people just being pedantic about this must do this, or not this or whatever, is really just knowing that if we follow these few rules, we can get better code reuse, and better mechanisms to reason about the code. The benefits of these things can disappear quite quickly if the laws are violated, this is why Tony gets flustered, and why we really want to differentiate \/ vs Validation. 

Educating on the nuance of this is important, but also very difficult, because most people don't exploit these abstract things, discussion often gets wrapped up in specifics of Or vs \/ vs Validation, but on their own none of these structures are very interesting, slightly different trade-offs, slightly different syntax but the important thing is what larger concepts can they be mapped to, and more importantly do we get any benefit from exploiting those concepts (which in the case of Applicative and Monad is pretty compelling).

 

Validation is scalaz is almost always used incorrectly because of the
existence of the flatMap method. I have seen very few uses of
validation that I would not consider a bug - they do exist - but
people leap to it because of the pretty name which some what
ridiculous, and then they have a Nel that always has one and only one
value in it.

I see. you mean they use Validation when they don't want to accumulate errors, and because it has flatMap they can make pretty for comprehensions with Validation that short circuit at the first sign of trouble? That's interesting. You can do the same thing with an accumulating Or, but I don't see that as a bug. It is just one way to use an Or. Accumulating or not, if you use an Or in a for-expression, it will short circuit at the first failure.

I feel it is a bug because they will have to handle non-possible cases at the end. For example given a Validation[List[String], Int], if i can only ever have one error, I still have to handle the empty list case and the more than one case, what should I do, throw an exception? promise it won't happen?

Scalaz \/ and Validation are designed to work together, not necessarily one or the other. There are operations on both for switching between and back, and work has been done to ensure the libraries are almost identical, so in the situation you want to accumulate you can switch to validation and then back. This makes any behaviour change explicit and recored in the type, so you get what you want every time (at least after flatMap is removed off of validation) and importantly you can work with abstract examples I was discussing above.

Hope that helps with understanding the rationale.

[Mark Hibberd]

> An example might be some parser like functions that maybe can handle
> failures in different ways (might be useful in streaming IO vs in memory or
> something). So as an arbitrary function I might want to do something like:
>
> def all[F[_]: Applicative, A](s: String, parsers: List[String => F[A]]):
> F[A] =
> parsers.traverse(p => p(s))
>

I realised there was a type error in my above code after I posted
sorry, should have been:

def all[F[_]: Applicative, A](s: String, parsers: List[String =>
F[A]]): F[A] =
parsers.traverse(p => p(s))

To further clarify I used this to run up a small example here (sorry
about the type annotations, they can be removed but too much effort
for an email :) --

https://gist.github.com/markhibberd/8235979

I was going to try and do it with Or, but I don't see anyway to write
it. If I understand the code correctly, `all` itself would have to
decide to be accumulating or not, which is what we are trying to
avoid. I may be missing something about how the Or/Every thing works,
but I think this gets to the crux of the difference and importance of
the formumaltion of Validation and \/.

[Bill Venners]

Hi Mark,

Thanks for the detailed reply.

On Friday, January 3, 2014 2:11:30 AM UTC-8, Mark Hibberd wrote:

On Friday, January 3, 2014, Bill Venners wrote:

It because it is how the type parameters should be partially applied.
In scala-like terms you want the functor to be:
  Either[A, _]
not
  Either[_, B].

Interesting, but I could use a bit more elaboration. Can you give me a specific example of why I might want to partially apply one of the two types in an Either? Which one of the two types would I normally know and want to "fill in" leaving a type that just has one type parameter. And what would I be trying to accomplish by that?

 

The most basic case is `trait Functor[F[_]] { def map[A, B](a: F[A])(f: A => B): F[B] }` for dealing generally with things that have map. 

In non-scala languages that have higher kinds, there are elegant ways to partially apply type parameters, to translate into an imaginary scala-like syntax, to get a conforming 'F' in this example for Either, you would just write something like Either[A]. In scala we have to do type lambdas and things get horrible pretty quickly, but you can exploit the same ordering to get some help in scala. In fact scalaz 6 used to provide a large number of types for doing just that. So running with the example you would normally write something like:

  def EitherFunctor[A] = new Functor[[({type l[a] = Either[A, a]})#l] { ... }

But by exploiting the ordering of the type arguments we can come up with general types to fit. In this case something like:

  

  class OneOfTwo[F[_, _], A] { type l[a] = F[A, a] }

And use it (in a non convincing way because of the somewhat trivial nature of Either):

  def EitherFunctor[A] = new Functor[OneOfTwo[Either, A]#l] { ... }

For just either this seems kind of neither here nor their, but as you build up larger monad transformer stacks you _really_ want there to be tools for dealing with these things rather than having to do the type lambda dance every time. So there is a benefit to the ordering even in scala (particularly with respect to consistency of other data structures like Reader, Writer and State, and it is worth nothing these all have there arguments the same order as Either in that the right most type parameter is the 'A' that you map across, and they have no concept of the "failure" case so the ordering of either based on failure the success is not an accurate reflection, it is simply about partially applying for Functor).

That makes sense. If I understand correctly, in languages that have nice syntax for partially applying types, syntax like "Either[String]" would fix the left-hand type, and result in a type constructor that takes one type parameter, which is the right-hand type. And in the case of Either, because you would want to map across the "valid" or "good" type, it makes sense to put that as the right-hand type. In Scala it looks like that's less useful, because it makes you do a type alias to partially apply a type. This doesn't look much different than your either example:

trait OrFunctor[A] extends Functor[({type l[a] = a Or A})#l]

But I see that the value of always putting the desired-to-be-mapped-over type parameter at the end, because it is consistent with other similar situations, such as State, etc., and also lets you use things like OneOfTwo to avoid the the structural type lambdas, which can make things hard to read if you need to use too many.
 

OK. That also makes sense to me. I'm all for being able to get better code reuse and better ways to reason about code. So I would really like to see if it is possible to have Or[G, B] to pass the monad laws while Or[G, Every[B]] passes the applicative laws. I am pretty sure plain old Or should be a law-abiding monad, but I'm not sure about applicative because I don't quite understand that one.
 

 

Validation is scalaz is almost always used incorrectly because of the
existence of the flatMap method. I have seen very few uses of
validation that I would not consider a bug - they do exist - but
people leap to it because of the pretty name which some what
ridiculous, and then they have a Nel that always has one and only one
value in it.

I see. you mean they use Validation when they don't want to accumulate errors, and because it has flatMap they can make pretty for comprehensions with Validation that short circuit at the first sign of trouble? That's interesting. You can do the same thing with an accumulating Or, but I don't see that as a bug. It is just one way to use an Or. Accumulating or not, if you use an Or in a for-expression, it will short circuit at the first failure.

I feel it is a bug because they will have to handle non-possible cases at the end. For example given a Validation[List[String], Int], if i can only ever have one error, I still have to handle the empty list case and the more than one case, what should I do, throw an exception? promise it won't happen?

OK, that does make sense, and it is something you can't say with Or. You can say Int Or List[String], but that doesn't accumulate errors because the right hand side isn't an Every. If you say Int Or Every[String], then it can accumulate errors, but you can never get an empty Every.
 

Scalaz \/ and Validation are designed to work together, not necessarily one or the other. There are operations on both for switching between and back, and work has been done to ensure the libraries are almost identical, so in the situation you want to accumulate you can switch to validation and then back. This makes any behaviour change explicit and recored in the type, so you get what you want every time (at least after flatMap is removed off of validation) and importantly you can work with abstract examples I was discussing above.

OK. That also makes sense. I also have a way to switch from a "regular" or (like a \/) to an "accumulating" or (like a ValidationNEL), which is to just call accumulating:

scala> val x = Good(1).orBad[ErrorMessage]
x: org.scalautils.Good[Int,org.scalautils.ErrorMessage] = Good(1)

scala> x.accumulating
res0: org.scalautils.Or[Int,org.scalautils.One[org.scalautils.ErrorMessage]] = Good(1)

But I don't have a simple way to go back in the other direction. It hit me when I wrote this that perhaps the key difference is that Or only gives you a \/ and a ValidationNEL sort of. You can't just have a Validation of any random error type. It needs to be Every, which is a non-empty sequence. I think that's why it doesn't have that potential Validation[List[String], Int] pitfall you mentioned.
 

Hope that helps with understanding the rationale.

It certainly does. Thanks. Main thing I'd like to see now is if accumulating Ors can actually fulfill the applicative contract.

Bill

[Bill Venners]

Hi Mark,

That's a good example. I'll need to study it later when I have a bit more time, but it seems that the crux of the question is can I provide a valid (law-abiding) Applicative instance for Or[G, Every[B]]. If so I don't see why it wouldn't work the same.

Bill

[Runar Bjarnason]

can I provide a valid (law-abiding) Applicative instance for Or[G, Every[B]]

It seems like you could, but since every monad is also an Applicative, it might be difficult for Scala to infer which you want, the Or monad or the Or.Every Applicative.

—
Sent from Mailbox for iPhone

--
You received this message because you are subscribed to the Google Groups "scalaz" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalaz+un...@googlegroups.com.
To post to this group, send email to sca...@googlegroups.com.
Visit this group at http://groups.google.com/group/scalaz.
For more options, visit https://groups.google.com/groups/opt_out.

[Runar Bjarnason]

> and that's why I was able to do this in one type?

It's absolutely not true that you were able to do this with one type. You have two types: 

_ Or E

And...

_ Or Every[E]

The composition of Or and Every is not the same type as Or by itself.

—
Sent from Mailbox for iPhone

--

[Runar Bjarnason]

> The other main question I have is why are there two types, \/ and Validation? ScalaUtils just has one type, Or, which can be used to accumulate errors if the "bad" type is an Every (which like NonEmptyList, is a sequence containing one to many elements).

We can ask the same question another way. Why does the Scala standard library have two types, `Option` and `List`? We could have just one type, `Option`, which can be used to accumulate answers if the "inner" type is an `Every`. I mean, having one type is simpler for users, right?

[Dave Stevens]

can I provide a valid (law-abiding) Applicative instance for Or[G, Every[B]]

It seems like you could, but since every monad is also an Applicative, it might be difficult for Scala to infer which you want, the Or monad or the Or.Every Applicative.

This was an issue with the Validation.Monad definition in scalaz 6.0.x. Importer beware!

https://github.com/scalaz/scalaz/blob/5e9dca0a4e59c14777925cd17390d665f3c4947f/core/src/main/scala/scalaz/Validation.scala#L139

> and that's why I was able to do this in one type?

It's absolutely not true that you were able to do this with one type. You have two types: 

_ Or E

And...

_ Or Every[E]

The composition of Or and Every is not the same type as Or by itself.

An example is the Applicative definition for Validation.

implicit def ValidationApplicative[L: Semigroup]: Applicative[({type l[a] = Validation[L, a]})#l]

https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/scalaz/Validation.scala#L451

[Bill Venners]

Hi All,

Thanks for all the pointers. I made a first attempt here:

https://gist.github.com/bvenners/8242014

One thing missing from Or, which has already been requested by a user, is a "badMap" method similar \/'s leftMap. I'm planning to add that in the next release, but I made an implicit class to add that for now. I can write an applicative instance, but I didn't test that it obeys the laws nor did I ensure it works sensibly for subtypes of Every (One and Many). But it does seem to work with Mark's all method when accumulation is desired.

My next step will be to attempt to write a Monad instance for plain-old Or. I'm guessing that scalaz has it set up such that if there's a Monad instance you can get an Applicative automatically? If so then I'm hoping the compiler would be able to pick the most specific Applicative. If there's an Every in the Bad type, it would pick the Applicative that accumulates. If there isn't, it would pick the Applicative provided through Monad. Let me know if I'm surmising wrongly, but regardless I'll give Monad a try to make sure that works.

Bill

[Bill Venners]

Hi All,

In the end the compiler was not able to chose between the Applicative inherited by Monad and the one I provided for Every, so it gave me an ambiguous implicit error when I tried to use the Every one. To solve that I pulled the Monad instance provider into a trait inherited by an object that provides the Every one, then imported its members. That made the accumulating Applicative personality the preferred one. If the Bad type is an Every, then it picks the Applicative personality that accumulates. Otherwise it picks the Applicative personality that makes sense for Monad. Gist is here:

https://gist.github.com/bvenners/8242691

It seems to work for Mark's example. I haven't written any tests (unlike my usual personality), including making sure laws aren't broken. So I don't know that it actually works generally. But so far so good.

Bill

[Runar Bjarnason]

Bill,

The Applicative trait has the laws on it. See `Applicative.applicativeLaw`. Pass that to ScalaCheck as a Property and Bob's your uncle. You will need to make a Gen for Or, which is trivial to do.

Runar

[Bill Venners]

Hi Runar,

OK. I'll try Applicative.applicativeLaw with Or a bit later. This was a fun way to spend half a morning, but other duties are calling.

I just updated the gist because it had a one-character typo, then I accidentally deleted half the gist trying to fix one-character typo, then I put the whole thing back. It should be correct now.

https://gist.github.com/bvenners/8242691

Bill

[Stephen Compall]

On Fri, 2014-01-03 at 06:53 -0800, Bill Venners wrote:
> But I see that the value of always putting the
> desired-to-be-mapped-over type parameter at the end, because it is
> consistent with other similar situations, such as State, etc., and
> also lets you use things like OneOfTwo to avoid the the structural
> type lambdas, which can make things hard to read if you need to use
> too many.

There's even more going on here, arising mainly from SI-2712
<https://issues.scala-lang.org/browse/SI-2712>.

Scalaz 7 implements a guided version of the technique Miles Sabin
describes there, `Unapply'. The available `Unapply' instances encode
the conventions you wish to support; the more you have, the more likely
you are to conflict. For example, one reason scalaz does not provide
both `Functor`s over \/ is that it would trigger the [a]a\/X and [a]X\/a
implicits, leading to conflict. Since scalaz provides both unapply
conventions, it must not provide both functors. One way or another,
convention must be embraced, setting aside the other consistency
benefits Mark Hibberd mentioned.

Secondly, supposing that currying is at some time implemented in the way
Paul Chiusano describes in 2712, it would behoove us to have the
functorial parameter at the end to take advantage.


--
Stephen Compall
"^aCollection allSatisfy: [:each|aCondition]": less is better

[Stephen Compall]

On Fri, 2014-01-03 at 09:54 -0800, Bill Venners wrote:
> In the end the compiler was not able to chose between the Applicative
> inherited by Monad and the one I provided for Every, so it gave me an
> ambiguous implicit error when I tried to use the Every one. To solve
> that I pulled the Monad instance provider into a trait inherited by an
> object that provides the Every one, then imported its members. That
> made the accumulating Applicative personality the preferred one. If
> the Bad type is an Every, then it picks the Applicative personality
> that accumulates. Otherwise it picks the Applicative personality that
> makes sense for Monad.

This shares something in common with the Monad for Validation. For
example, this function:

def blah[F[_]:Applicative,A](fa: F[A]): Z

changes behavior when I increase the constraint to Monad, supposing `fa'
was an Or, when it should only be shrinking the function's domain. In a
sense, it is as if the necessary information to choose a semigroup for
the applicative was lost, and a default semigroup (First) which is
defined for all A was chosen instead.

The law tests won't spot this, since they operate on one piece of
evidence at a time. Strictly speaking, they are lawful instances; they
merely overlap in an interesting way. Your examples look lawful to me,
without testing, anyway. In this way they are different from the old
validation monad in scalaz, which was not.

[Bill Venners]

Hi All,

I was thinking along those lines, that even if the laws hold, something seems odd about it. It turns out that both regular and accumulating Ors seem to pass the laws as specified with the properties provided by scalaz:

https://github.com/bvenners/orEveryLaws/blob/master/src/test/scala/itsnotjustagoodidea/OrLawsSpec.scala

and:

https://github.com/bvenners/orEveryLaws/blob/master/src/test/scala/itsnotjustagoodidea/AccumulatingOrLawsSpec.scala

All these tests pass, even though some of the implicit instances are not implemented. So the "validation" provided by these passing tests is limited, but it suggests these probably are law abiding.

I would like to better understand the relationship between Monad and Applicative. The answer to my original question about why \/ and Validation are two different types in scalaz seems to be that the Applicative provided with Monad is different from the Applicative that accumulates. So an Applicative that accumulates can't be paired with Monad. Can someone elaborate on exactly what that is, and why? Is the Applicative that comes out of the box with Monad required by the math principles, or just provided by scalaz because it is usually convenient.

Thanks.

Bill 
----

[Lars Hupel]

> I would like to better understand the relationship between Monad and
> Applicative. The answer to my original question about why \/ and Validation
> are two different types in scalaz seems to be that the Applicative provided
> with Monad is different from the Applicative that accumulates. So an
> Applicative that accumulates can't be paired with Monad. Can someone
> elaborate on exactly what that is, and why? Is the Applicative that comes
> out of the box with Monad required by the math principles, or just provided
> by scalaz because it is usually convenient.

For each Monad, there is an implementation of Applicative which follows
from the implementation of `bind` and `return`. The relevant code in
scalaz is this:

override def ap[A, B](fa: => F[A])(f: => F[A => B]): F[B] = {
lazy val fa0 = fa
bind(f)(map(fa0))
}

This default implementation gives rise to a lawful Applicative.

(More generally speaking, a valid Bind gives rise to a valid Apply, and
`Bind.scala` is in fact the place where this snippet appears in the
scalaz sources.)

Now, in the `Monad[Validation[E, _]]` case, there is an Applicative
which accumulates errors. Additionally, there can be no Monad which
accumulates errors (try to write it � it's impossible). On the other
hand, there is a Monad with fail-fast behaviour. Now, that Monad gives
rise to an Applicative instance (via the default implementation given
above), but it's an entirely different one.

Now, the question is, why does a Monad give rise to an Applicative?
Well, because you can implement a law-abiding `ap` based on a
law-abiding `bind`. It's as simple as that; one could even say that it's
inevitable.

[Bill Venners]

Hi Lars,

On Saturday, January 4, 2014 2:08:33 AM UTC-8, Lars Hupel wrote:

> I would like to better understand the relationship between Monad and
> Applicative. The answer to my original question about why \/ and Validation
> are two different types in scalaz seems to be that the Applicative provided
> with Monad is different from the Applicative that accumulates. So an
> Applicative that accumulates can't be paired with Monad. Can someone
> elaborate on exactly what that is, and why? Is the Applicative that comes
> out of the box with Monad required by the math principles, or just provided
> by scalaz because it is usually convenient.

For each Monad, there is an implementation of Applicative which follows
from the implementation of `bind` and `return`. The relevant code in
scalaz is this:

  override def ap[A, B](fa: => F[A])(f: => F[A => B]): F[B] = {
    lazy val fa0 = fa
    bind(f)(map(fa0))
  }

This default implementation gives rise to a lawful Applicative.

(More generally speaking, a valid Bind gives rise to a valid Apply, and
`Bind.scala` is in fact the place where this snippet appears in the
scalaz sources.)

Yes, I found it, and it makes sense. If my understanding is correct, because monad offers flatMap or bind, I can implement the ap method in terms of that, which in an Either/Or/\//Validation would short circuit in the monad way and not accumulate errors. And you provide that by default in scalaz because it is a reasonable default that is convenient for users.
 

Now, in the `Monad[Validation[E, _]]` case, there is an Applicative
which accumulates errors. Additionally, there can be no Monad which
accumulates errors (try to write it � it's impossible). On the other
hand, there is a Monad with fail-fast behaviour. Now, that Monad gives
rise to an Applicative instance (via the default implementation given
above), but it's an entirely different one.

Now, the question is, why does a Monad give rise to an Applicative?
Well, because you can implement a law-abiding `ap` based on a
law-abiding `bind`. It's as simple as that; one could even say that it's
inevitable.

But I guess my question could be phrased: is it *inevitable* that every monad will be associated with an Applicative that implements ap as the reasonable default provided in Bind.scala. From what I've seen so far, nothing about monad-hood requires that. Moreover I've observed scalaz users worried about flatMap going away in Validation. They want to use Validation in a monadic way in for expressions, even though it short circuits when they use it that way. Could you not go further and offer a Monad[Validation...] that overrides ap (using the wonders of OO and subtyping) to do accumulation? Wouldn't that make Validation even more generally useful?

That's essentially how accumulating Or's work. You can use, say, an Int Or Every[ErrorMessage] in a for expression, and it will short circuit in the monad style, which I think is OK because they are using its monad-nature when using it in a for expression. If they want error accumuilation, they use a different set of methods than the ones to which for expressions are desugared.

Bill

[Mark Hibberd]

On Sunday, January 5, 2014, Bill Venners wrote:

Now, in the `Monad[Validation[E, _]]` case, there is an Applicative
which accumulates errors. Additionally, there can be no Monad which
accumulates errors (try to write it � it's impossible). On the other
hand, there is a Monad with fail-fast behaviour. Now, that Monad gives
rise to an Applicative instance (via the default implementation given
above), but it's an entirely different one.

Now, the question is, why does a Monad give rise to an Applicative?
Well, because you can implement a law-abiding `ap` based on a
law-abiding `bind`. It's as simple as that; one could even say that it's
inevitable.

But I guess my question could be phrased: is it *inevitable* that every monad will be associated with an Applicative that implements ap as the reasonable default provided in Bind.scala. From what I've seen so far, nothing about monad-hood requires that. Moreover I've observed scalaz users worried about flatMap going away in Validation. They want to use Validation in a monadic way in for expressions, even though it short circuits when they use it that way. Could you not go further and offer a Monad[Validation...] that overrides ap (using the wonders of OO and subtyping) to do accumulation? Wouldn't that make Validation even more generally useful?

No. ap / flatMap consistency is very important. It would make it harder to understand any code that has a monad constraint. It could use ap and get one behaviour or bind and get another. In order to write and work with this type of code I should be able to implement either way and I should expect the same result.

That's essentially how accumulating Or's work. You can use, say, an Int Or Every[ErrorMessage] in a for expression, and it will short circuit in the monad style, which I think is OK because they are using its monad-nature when using it in a for expression. If they want error accumuilation, they use a different set of methods than the ones to which for expressions are desugared.

It is not clear to me what you gain by this. Why a different set of methods vs a different type.

If you look again at the all example, I would argue that different types provides significant benefit, and with a complete api like Validation you can still do what you want. If you want to use a for comprehension just add .either and make your requirement explicit. The insistence to use validation like an either really is misguided due to the issues I mentioned earlier and the fact that you can do exactly what you want in a syntactically clean way by using validation and disjunction as a pair. Runar's comparison to option vs list is quite accurate. I am really not sure what makes this case different, if people want to use validation in a short circuiting fashion they can, it is just that the descion is encoded in the type, this seems somewhat desirable rather then something that should be tried to work around. 

As a thought experiment, If validation was called AccumulatingOr and disjunction called Or, would the desire to combine them persist? If not it seems like more of a documentation / education issue on how to use validation and disjunction for best effect.

Cheers

Mark

[Runar Bjarnason]

But I guess my question could be phrased: is it *inevitable* that every monad will be associated with an Applicative that implements ap as the reasonable default provided in Bind.scala. From what I've seen so far, nothing about monad-hood requires that.

You could certainly implement `ap` in a way that is inconsistent with the monad. But then what about `liftM2`, `liftM3`, etc? Which behavior should they exhibit? You can special-case everything, but it's inconsistent and confusing. Why not have one type for one behavior and a different (but isomorphic) type for the other? Is there a type famine that I'm not aware of?

[Mark Hibberd]

On Sunday, January 5, 2014, Bill Venners wrote. 

But I guess my question could be phrased: is it *inevitable* that every monad will be associated with an Applicative that implements ap as the reasonable default provided in Bind.scala. From what I've seen so far, nothing about monad-hood requires that. Moreover I've observed scalaz users worried about flatMap going away in Validation. They want to use Validation in a monadic way in for expressions, even though it short circuits when they use it that way. Could you not go further and offer a Monad[Validation...] that overrides ap (using the wonders of OO and subtyping) to do accumulation? Wouldn't that make Validation even more generally useful?

Also worth pointing out that it would require a semigroup constraint on the error, so the monad definition would not be as general as that of disjunction. This really emphasizes the mismatch in wanting to use validation in a for expression. You have to say *I guarantee you that I can accumulate errors* at the same time as saying *I guarantee you that I won't actually accumulate errors*. This contradiction directly leads to code paths which can not possibly happen and should be avoided if at all possible.

[Mark Hibberd]

On Sunday, January 5, 2014, Runar Bjarnason wrote:

But I guess my question could be phrased: is it *inevitable* that every monad will be associated with an Applicative that implements ap as the reasonable default provided in Bind.scala. From what I've seen so far, nothing about monad-hood requires that.

You could certainly implement `ap` in a way that is inconsistent with the monad. But then what about `liftM2`, `liftM3`, etc? Which behavior should they exhibit? You can special-case everything, but it's inconsistent and confusing. 

This is exactly what bothers me. There is no way I see that doesn't lead to these introducing confusion and bugs unless the behaviours are encoded in two independent types. 

[Bill Venners]

Hi Runar,

On Saturday, January 4, 2014 12:58:05 PM UTC-8, Runar Bjarnason wrote:

But I guess my question could be phrased: is it *inevitable* that every monad will be associated with an Applicative that implements ap as the reasonable default provided in Bind.scala. From what I've seen so far, nothing about monad-hood requires that.

You could certainly implement `ap` in a way that is inconsistent with the monad. But then what about `liftM2`, `liftM3`, etc? Which behavior should they exhibit? You can special-case everything, but it's inconsistent and confusing. Why not have one type for one behavior and a different (but isomorphic) type for the other? Is there a type famine that I'm not aware of?

No type famine. I just want to make sure I understand the rationale. Where are liftM2 and liftM3 defined? I couldn't find any such names in the 7.0.5 codebase.

Bill

[Bill Venners]

Hi Mark,

Actually in my question about Validation I wasn't suggesting you could merge them into one type, just trying to understand why some of you think Validation shouldn't also be a monad. It sounds to me like the answer is that even though it wouldn't strictly violate any law of monadic nature, it would be inconsistent with other monads provided by scalaz and therefore possibly surprising to users in a way that leads to bugs. Is that accurate?

If so that makes sense to me, as well as the desire to rename flatMap to something else on Validation so that people can't use it monadically in a for expression. That would be a hint that they should use \/ instead.

Bill

[Bill Venners]

Hi Mark,

True, a Validation monad would need to constrain its error type to a semigroup, which I think would be fine when looked at independently. The problem is more that it would be inconsistent with other monads in scalaz.

The approach I took is to constrain the error type even further than "any semigroup" to just one and only one type, Every. So accumulating Or is less general than Validation, on purpose. I felt Validation was a bit over-general if the use case was just to allow people to accumulate errors. If you want a different type with an Or, say some semigroup, you'd need to use Every to accumulate, then you could transform that result to whatever else you wanted at the end.

Bill
 

[Runar Bjarnason]

Actually in my question about Validation I wasn't suggesting you could merge them into one type, just trying to understand why some of you think Validation shouldn't also be a monad. It sounds to me like the answer is that even though it wouldn't strictly violate any law of monadic nature, it would be inconsistent with other monads provided by scalaz and therefore possibly surprising to users in a way that leads to bugs. Is that accurate?

Almost. It would be a monad that is necessarily inconsistent with the Validation applicative functor. Or if you will, it would be a monad that is exactly the same as \/ (which is a totally different Applicative).

[Runar Bjarnason]

No type famine. I just want to make sure I understand the rationale.

The rationale for having two types `Validation` and `\/` is the same as your rationale for having two types `Or` and `Or * Every`.

 

Where are liftM2 and liftM3 defined? I couldn't find any such names in the 7.0.5 codebase.

def liftM2[A,B,C](fa: F[A], fb: F[B])(f: (A, B) => C): F[C]

def liftM3[A,B,C,D](fa: F[A], fb: F[B], fc: F[C])(f: (A, B, C) => D): F[D]

etc.

If you see these signatures and F[_] = Validation[E, _], then you can reasonably expect that the errors are accumulated. If F[_] = E \/ _, then you know for sure that they are not (because it's not possible).

[Mark Hibberd]

> Actually in my question about Validation I wasn't suggesting you could merge
> them into one type, just trying to understand why some of you think
> Validation shouldn't also be a monad. It sounds to me like the answer is
> that even though it wouldn't strictly violate any law of monadic nature, it
> would be inconsistent with other monads provided by scalaz and therefore
> possibly surprising to users in a way that leads to bugs. Is that accurate?

The problem is that it is inconsistent with itself (as opposed to with
other monads). This means then general code such as lift2M (which is
not provided by scalaz because we assert that lift2A and lift2M should
be identical so just have lift2) would be unpredictable given a Monad
constraint.

[Runar Bjarnason]

If you see these signatures and F[_] = Validation[E, _], then you can reasonably expect that the errors are accumulated. If F[_] = E \/ _, then you know for sure that they are not (because it's not possible).

Sorry, that should be stronger. It's not just that you can "reasonably expect" the errors to be accumulated. It is impossible that they aren't. And the signatures should be:

def liftM2[F[_]:Applicative,A,B,C](fa: F[A], fb: F[B])(f: (A, B) => C): F[C]

def liftM3[F[_]:Applicative,A,B,C,D](fa: F[A], fb: F[B], fc: F[C])(f: (A, B, C) => D): F[D]

[Tony Morris]

// Questions about \/ and Validation
object scalaz {

/*
This explanation might help. This is a compilable source file with revisions
available at https://gist.github.com/tonymorris/8263051

Fact: All monads are applicative functors. As has been seen we can witness the
`Applicative` that arises from the `Monad` primitives. Let's illustrate this:

*/

trait Applicative[F[_]] {
  def pure[A](a: A): F[A]
  def ap[A, B](f: F[A => B], a: F[A]): F[B]
}

trait Monad[F[_]] extends Applicative[F] {
  def unit[A](a: A): F[A]
  def bind[A, B](f: A => F[B], a: F[A]): F[B]

  override def pure[A](a: A): F[A] =
    unit(a)
  override def ap[A, B](f: F[A => B], a: F[A]): F[B] =
    bind((ff: A => B) => bind((aa: A) => pure(ff(aa)), a), f)
}

/*
We see that the Applicative primitives `ap` and `pure` are derived from the
monad primitives, `bind` and `unit`. This applicative functor that arises is
guaranteed to be lawful. Ergo, we can confidently say, "all monads are
applicative."

However, not all applicative functors are monads. Some are, but not all. This
discussion centres around two different applicative functors. One of those gives
rise to a monad, while the other doesn't. Furthermore, one of the applicative
functors (the one that is not a monad) requires a constraint on the functor
itself.

That constraint is called a semigroup and is closed binary operation that is
associative.
*/

trait Semigroup[A] {
  def op(a1: A, a2: A): A
}

/*
Let's look at these two applicative functors by first using `scala.Either`
*/

// From hereon, we will call this "the \/ applicative"
def Applicative_\/[X]: Applicative[({type λ[α]=X Either α})#λ] =
  new Applicative[({type λ[α]=X Either α})#λ] {
    def pure[A](a: A) =
      Right(a)
    def ap[A, B](f: X Either (A => B), a: X Either A) =
      for {
        ff <- f.right
        aa <- a.right
      } yield ff(aa)
}

// From hereon, we will call this "the Validation applicative"
// Note the Semigroup constraint, which is required.
def Applicative_Validation[X](s: Semigroup[X]): Applicative[({type λ[α]=X Either α})#λ] =
  new Applicative[({type λ[α]=X Either α})#λ] {
    def pure[A](a: A) =
      Right(a)
    def ap[A, B](f: X Either (A => B), a: X Either A) =
      f match {
        case Left(x1) =>
          a match {
            case Left(x2) =>
              Left(s.op(x1, x2))
            case Right(_) =>
              Left(x1)
          }
        case Right(ff) =>
          a match {
            case Left(x2) =>
              Left(x2)
            case Right(aa) =>
              Right(ff(aa))
          }
      }
}

/*
OK, so that was a bit of a mouthful. Bladdy Scala.

It is important to observe that we could have written the \/ applicative by
implementing the Monad trait and automatically inheriting the behaviour that
was written out.

Like so:
*/
def Monad_\/[X]: Monad[({type λ[α]=X Either α})#λ] =
  new Monad[({type λ[α]=X Either α})#λ] {
    def unit[A](a: A) =
      Right(a)
    def bind[A, B](f: A => X Either B, a: X Either A) =
      a.right flatMap f
}

/*
Before going further, convince yourself that `Monad_\/#ap` and
`Applicative_\/#ap` both exhibit exactly the same behaviour. They are the same
function.

OK, so what about `Applicative_Validation`? Can we just implement the `Monad`
trait instead?

No. This is not possible. What we have here is an applicative functor without a
corresponding monad.

In summary, we have two applicative functors, one of which has a corresponding
monad, while the other doesn't.
*/

/*
OK, how would we exploit all of these lovely libraries in
practice? It might be argued, let's implement all the different types of
behaviour in a single data type like so:
*/

case class Or[A, B](run: A Either B) {
  // the \/ monad
  def flatMap[X](f: B => A Or X): A Or X = 
    Or(run.right.flatMap(f(_).run))

  // the \/ applicative
  def ap_\/[X](f: A Or (B => X)): A Or X =
    Or(
        for {
          ff <- f.run.right
          aa <- run.right
        } yield ff(aa)
    )

  // the validation applicative
  def ap_Validation[X](f: A Or (B => X))(implicit s: Semigroup[A]): A Or X =
    Or(
        f.run match {
          case Left(x1) =>
            run match {
              case Left(x2) =>
                Left(s.op(x1, x2))
              case Right(_) =>
                Left(x1)
            }
          case Right(ff) =>
            run match {
              case Left(x2) =>
                Left(x2)
              case Right(aa) =>
                Right(ff(aa))
            }
      }
    )
}

/*
OK, so we have a data type that has all the different possibilities. We may or
may not accumulate errors using an applicative operation or we might using the
only possible monad instance with `flatMap`. We are done!

Not so fast. This has a problem. Actually, it has a *huge* problem.

Let's go back to basics for a minute. What is the entire point of generalising
to `Applicative` or `Monad`? Why have such interfaces to begin with? The answer
is the same reason we write interfaces all the time as responsible programmers:
to abstract the commonalities to give rise to operations common across both.

OK, what is one such operation that arises with the interfaces we are discussing
here? Here is one of a possible many:
*/
def sequence[F[_], A](x: List[F[A]])(implicit F: Applicative[F]): F[List[A]] =
  x.foldRight[F[List[A]]](F.pure(Nil))((a, b) =>
      F.ap(F.ap(F.pure((a: A) => (as: List[A]) => a :: as), a), b))

/*
Right, so if we have a list of F[A] we can turn it into a F of list of A, as
long as we have an Applicative for F. This means we could take a list through
either the \/ applicative or the validation applicative and get very different
results.

So which applicative instance should we implement for `Or`? We are stuck, we
have to pick one! OK, let's pick the \/ applicative implementation, which has no
accumulation and a corresponding monad.
*/
implicit def Monad_Or[X]: Monad[({type λ[α]=X Or α})#λ] =
  new Monad[({type λ[α]=X Or α})#λ] {
    def unit[A](a: A) =
      Or(Right(a))
    def bind[A, B](f: A => X Or B, a: X Or A) =
      a flatMap f
}

/*
Great, now we can use `sequence` with our new (derived) applicative functor.

(Please excuse that ugly annotation to help out the inferencer).
*/
val list =
  List(Left(List(7)), Left(List(8)), Right("abc"), Right("def"))

def hithere = 
  sequence[({type λ[α]=List[Int] Or α})#λ, String](
    list map (Or(_))
  )

/*
OK, but what if we want to sequence a list of Or values, accumulating with our
other applicative functor?

We are completely boned, that's what.

Just kidding, we write a new data type that is isomorphic to `Or` and provides
the appropriate Applicative instance (but not a Monad instance).

Let's call it ... oh I don't know ... hmm how about Validation?
*/
case class Validation[A, B](run: A Either B)

implicit def Applicative_Validation_for_realz[X](implicit s: Semigroup[X]): Applicative[({type λ[α]=X Validation α})#λ] =
  new Applicative[({type λ[α]=X Validation α})#λ] {
    def pure[A](a: A) =
      Validation(Right(a))
    def ap[A, B](f: X Validation (A => B), a: X Validation A) =
      Validation(
        f.run match {
          case Left(x1) =>
            a.run match {
              case Left(x2) =>
                Left(s.op(x1, x2))
              case Right(_) =>
                Left(x1)
            }
          case Right(ff) =>
            a.run match {
              case Left(x2) =>
                Left(x2)
              case Right(aa) =>
                Right(ff(aa))
            }
          }
      )
}

/*
Now we have the best of both worlds. We can sequence with our earlier
applicative or with the accumulating applicative. We'll just need a `Semigroup`
for whatever we are accumulating. Easy enough:
*/
implicit def ListSemigroup[A]: Semigroup[List[A]] =
  new Semigroup[List[A]] {
    def op(a1: List[A], a2: List[A]) =
      a1 ::: a2
  }

/*
And now we can sequence in our other applicative functor:
*/
def hithereagain = 
  sequence[({type λ[α]=List[Int] Validation α})#λ, String](
    list map (Validation(_))
  )

/*
Great, now we truly do have the best of all worlds. Our strategy applies to the
entire monad and applicative library, not just sequence. This is the practical
point of monads and applicative functors after all!

We have two data types, `Or` and `Validation`, each denoting the different types
of applicative functor behaviour and one of which has a corresponding monad. It
is important to know why you are using one or the other. They do not exist in
redundancy. Nobody thought, "let's write two data types that overlap!" No no,
repeating useless library code is for ... well ... it's very much not in the
scalaz handbook.

There is a very important difference between these two data types. If you want
accumulation, you use `Validation`. If you do not want accumulation, you use
`Or`. If it makes no difference, pick either and choose wisely.

As for naming, `Or` is actually called `\/` in scalaz. I am so pleased that this
discussion started off in the degenerate pits of the merits of identifier names
and has managed to crawl out to a world of utility. Let's keep it that way!
*/

def main(args: Array[String]) {
  println(hithere)
  println(hithereagain)

--
You received this message because you are subscribed to the Google Groups "scalaz" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalaz+un...@googlegroups.com.
To post to this group, send email to sca...@googlegroups.com.
Visit this group at http://groups.google.com/group/scalaz.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
Tony Morris
http://tmorris.net/

[Bill Venners]

// Questions about \/ and Validation
object scalaz {
 
/*

Thanks for the great explanation, Tony. This explanation as well as the answers
that came back from Runar and Mark nudged my thinking in a different direction.
The feeling I get is that there's a monad inside of Validation trying to get out.
It looks like Validation would be monadic just from its shape, and users seem to
want to use them in for expressions. I now see that a Validation monad would need
to have an applicative personality that is consistent with monad, and I think it
could be done by offering a lower priority Monad instance for Validation (complete
with the default, inherited Applicative behavior that short-circuits on the first
error) as well as a higher priority Applicative instance that accumulates errors.
That way if a method abstracts over Monad, you could pass a Validation to it, but
it would short-circuit if they sequenced it or did anything else that accessed
its Applicative nature. If a method abstracts over Applicative, you could (as you can
now) pass a Validation to it, and it would accumulate.
 
The reason you might not want to do that is it may be more confusing than any
benefit it provides. It might be simpler for user if there's one type, \/, that
is a monad with consistent applicative behavior and similar type Validation that
is not a monad (including not having a flatMap method so it can't be used monadically
in a for expression) and has an applicative that accumulates.
 
The reason this works for Or, I think, is that Or isn't tied into scalaz-like type
classes. It is just a functional/OO design that has an API that allows it to
be treated monadically, and another API when the bad type is an Every that allows
accumulation.
 
But just for fun, I decided to try it and I got it to work. It was a bit tricky to
get the Scala compiler to understand the implicit priorities I wanted. I improved
my understanding of how that works in the process, and will explain that below.

 
This is a compilable source file with revisions

available at https://gist.github.com/bvenners/8273112
 
That's a fork of Tony's gist. I have removed his comments and replaced
them with my own.
*/
 
/*
All of the code until my next comment is unchanged from Tony's original gist, except
with Tony's comments removed.

 */
trait Applicative[F[_]] {
  def pure[A](a: A): F[A]
  def ap[A, B](f: F[A => B], a: F[A]): F[B]
}
 
trait Monad[F[_]] extends Applicative[F] {
  def unit[A](a: A): F[A]
  def bind[A, B](f: A => F[B], a: F[A]): F[B]
 
  override def pure[A](a: A): F[A] =
    unit(a)
  override def ap[A, B](f: F[A => B], a: F[A]): F[B] =
    bind((ff: A => B) => bind((aa: A) => pure(ff(aa)), a), f)
}
 

trait Semigroup[A] {
  def op(a1: A, a2: A): A
}
 

def Applicative_\/[X]: Applicative[({type λ[α]=X Either α})#λ] =
  new Applicative[({type λ[α]=X Either α})#λ] {
    def pure[A](a: A) =
      Right(a)
    def ap[A, B](f: X Either (A => B), a: X Either A) =
      for {
        ff <- f.right
        aa <- a.right
      } yield ff(aa)
}
 

def Applicative_Validation[X](s: Semigroup[X]): Applicative[({type λ[α]=X Either α})#λ] =
  new Applicative[({type λ[α]=X Either α})#λ] {
    def pure[A](a: A) =
      Right(a)
    def ap[A, B](f: X Either (A => B), a: X Either A) =
      f match {
        case Left(x1) =>
          a match {
            case Left(x2) =>
              Left(s.op(x1, x2))
            case Right(_) =>
              Left(x1)
          }
        case Right(ff) =>
          a match {
            case Left(x2) =>
              Left(x2)
            case Right(aa) =>
              Right(ff(aa))
          }
      }
}
 

def Monad_\/[X]: Monad[({type λ[α]=X Either α})#λ] =
  new Monad[({type λ[α]=X Either α})#λ] {
    def unit[A](a: A) =
      Right(a)
    def bind[A, B](f: A => X Either B, a: X Either A) =
      a.right flatMap f
}
 

def sequence[F[_], A](x: List[F[A]])(implicit F: Applicative[F]): F[List[A]] =
  x.foldRight[F[List[A]]](F.pure(Nil))((a, b) =>
      F.ap(F.ap(F.pure((a: A) => (as: List[A]) => a :: as), a), b))
 

implicit def Monad_Or[X]: Monad[({type λ[α]=X Or α})#λ] =
  new Monad[({type λ[α]=X Or α})#λ] {
    def unit[A](a: A) =
      Or(Right(a))
    def bind[A, B](f: A => X Or B, a: X Or A) =
      a flatMap f
}
 

val list =
  List(Left(List(7)), Left(List(8)), Right("abc"), Right("def"))
 
def hithere = 
  sequence[({type λ[α]=List[Int] Or α})#λ, String](
    list map (Or(_))
  )
 

case class Validation[A, B](run: A Either B) {
  def flatMap[X](f: B => A Validation X): A Validation X =
    Validation(run.right.flatMap(f(_).run))
}
 
/*
Here's where the code first departs from Tony's original. I actually
define a Monad instance for Validation, but I put it in a trait.
Note that as a Monad, Validation requires no Semigroup constraint.
*/
trait MonadicValidation {
  implicit def Monad_Validation[X]: Monad[({type λ[α]=X Validation α})#λ] =
    new Monad[({type λ[α]=X Validation α})#λ] {

      def unit[A](a: A) =

        Validation(Right(a))
      def bind[A, B](f: A => X Validation B, a: X Validation A) =
        a flatMap f
    }
}
 
/*
I then put Tony's method unchanged providing the accumulating Applicative for
Validation in Validation's companion object, and made that extend the trait
that holds the implicit providing the monad instance for Validation. Here is
where I ran into an unexpected ambiguous implicit compiler error. I didn't fully
understand the point system that the compiler uses to select implicits. Turns out you
get points for being more specific as well as points for being in subtypes, and
these points are added together.
 
Monad is more specific than Applicative because Monad is a subtype of
Appicative. So if placed in the same trait Monad_Validation scores one
more point than Applicative_Validation_for_realz when the compiler is looking
for an Applicative. The compiler then awards Applicative_Validation_for_realz a point
for being in a sub-object, and that evens the score, so the compile fails with an
ambiguity error. What I ended up doing to solve it is making this clunky subtrait
of Applicative, MoreSpecificApplicative, that does nothing more than cause the compiler
to give it a point for being more specific. That evens that score, so the compiler will
pick the accumulating one if X is a Semigroup and an Applicative is asked for, because
it is declared in a subobject.
*/
object Validation extends MonadicValidation {
 
  trait MoreSpecificApplicative[F[_]] extends Applicative[F]
 
  implicit def Applicative_Validation_for_realz[X](implicit s: Semigroup[X]): MoreSpecificApplicative[({type λ[α]=X Validation α})#λ] =
    new MoreSpecificApplicative[({type λ[α]=X Validation α})#λ] {

      def pure[A](a: A) =
        Validation(Right(a))
      def ap[A, B](f: X Validation (A => B), a: X Validation A) =
        Validation(
          f.run match {
            case Left(x1) =>
              a.run match {
                case Left(x2) =>
                  Left(s.op(x1, x2))
                case Right(_) =>
                  Left(x1)
              }
            case Right(ff) =>
              a.run match {
                case Left(x2) =>
                  Left(x2)
                case Right(aa) =>
                  Right(ff(aa))
              }
            }
        )
  }
}
 

implicit def ListSemigroup[A]: Semigroup[List[A]] =
  new Semigroup[List[A]] {
    def op(a1: List[A], a2: List[A]) =
      a1 ::: a2
  }
 
/*

This works the same as before: the compiler picks the higher priority
impicit and you get accumulation. The result is: Validation(Left(List(7, 8)))

 */
def hithereagain =
  sequence[({type λ[α]=List[Int] Validation α})#λ, String](
    list map (Validation(_))
  )
 
/*

To demonstrate what happens when a Validation is treated as a monad, I made
this sequenceAsMonad method. It is the exact same implementation as Tony's
original sequence, but it requires a Monad instead of an Applicative.
 */
def sequenceAsMonad[F[_], A](x: List[F[A]])(implicit F: Monad[F]): F[List[A]] =

  x.foldRight[F[List[A]]](F.pure(Nil))((a, b) =>
    F.ap(F.ap(F.pure((a: A) => (as: List[A]) => a :: as), a), b))
 
/*

When I call this method, it passes in the Monad instance, and uses the
Applicative inherited by monad, so it short-circuits. The result is: Validation(Left(List(7)))
 */
def hithereyetagain =
  sequenceAsMonad[({type λ[α]=List[Int] Validation α})#λ, String](

    list map (Validation(_))
  )
 
/*

One way this makes Validation more general (and possibly also more confusing for users)
is that it will still work when an Applicative is asked for but the Left type is not
a Semigroup. In this case, the higher priority Applicative_Validation_for_realz does
not apply at all, because X is not a Semigroup, so it picks the Monad one, which
doesn't accumulate. Although Vector is a semigroup in essence, it isn't one here because
we haven't defined a Semigroup instance for Vector. So a Vector will work to demonstrate:
*/
val listOfNonSemigroups =
  List(Left(Vector(7)), Left(Vector(8)), Right("abc"), Right("def"))
 
/*
Even though we are passing a Validation to a method that abstracts
over Applicative, because no Semigroup is available for Vector, it
picks the Applicative that makes sense more generally, which is the
one supplied by Monad. This results in: Validation(Left(Vector(7)))
 */
def hithereonelasttime =
  sequence[({type λ[α]=Vector[Int] Validation α})#λ, String](
    listOfNonSemigroups map (Validation(_))
)
 
/*
This prints:
 
Or(Left(List(7)))
Validation(Left(List(7, 8)))
Validation(Left(List(7)))
Validation(Left(Vector(7)))

*/
def main(args: Array[String]) {
  println(hithere)
  println(hithereagain)

  println(hithereyetagain)
  println(hithereonelasttime)

[Kris Nuttycombe]

On Sunday, January 5, 2014 2:02:03 PM UTC-7, Bill Venners wrote:

 
The reason you might not want to do that is it may be more confusing than any
benefit it provides. It might be simpler for user if there's one type, \/, that
is a monad with consistent applicative behavior and similar type Validation that
is not a monad (including not having a flatMap method so it can't be used monadically
in a for expression) and has an applicative that accumulates.

I dislike the argument that having one type with two different behaviors is less confusing that two types with two different (but consistent) behaviors. The fundamental problem is that the behavior of my code can change dramatically if I accidentally change the error type to something that has a semigroup - namely, I go from abort-on-first-error semantics to abort-if-one-fails semantics. This kind of hidden information (not evident at the call site, because all the relevant typeclasses are fetched implicitly) is really dangerous. 

I'd like to try to convince you that having one type represent two behaviors is insidious with a more obvious example. Consider strings.

In the current codebase that I'm rehabilitating, strings containing values that are semantically dates occur in many locations. Now, the original author *did* use joda DateTime values in some places, but not every place. The problem is additionally complicated by the fact that the strings are not universally formatted consistently, so what he'd do is pass the strings along through a number of calls, then (because he knew how they were formatted) he'd parse them to dates just at the final use site where he needed to do some date-based computation. I'm sure you can imagine what a nightmare this situation is to deal with; without the knowledge of how a string is formatted tied to the value, it's exceptionally difficult to reason about what a particular String-Date means. 

Now, suppose that I decided to pass around a tuple of the date string, and the format string that could be used to parse it to a DateTime, inferring the format back through the call stacks from where I have the format information available. By doing this, I'd make it possible to handle these values consistently wherever they appear. But, some of these values are bare dates, and some others have time information; the format string distinguishes the two. Suddenly I have another problem; if they're semantically bare dates, I shouldn't add precision by arbitrarily choosing some hour, minute, and second values to go along with them. To have correct semantics, I have to treat bare dates, and date/time values separately - which means I need different types.

Do you see the parallel? Using separate types to distinguish between halt-on-first-error and accumulate-all-errors makes it very easy to reason about how a given value of a type can be used anywhere that it appears; furthermore, it is trivial to convert between them at need when you need to include a value with one set of semantics in a computation that requires the other. Contrarily, including both sets of semantics in the same type, with some potentially hidden information, is like using DateTime for both bare dates and values with more precision - it means that you have hidden state to track and consider when you're using a value... and it's likely that you'll forget.

Kris

[Tony Morris]

On 07/01/14 07:19, Kris Nuttycombe wrote:
> I'd like to try to convince you that having one type represent two
> behaviors is insidious with a more obvious example.

One more
https://www.ruby-lang.org/en/

[Bill Venners]

Hi Kris,

On Monday, January 6, 2014 1:19:29 PM UTC-8, Kris Nuttycombe wrote:

On Sunday, January 5, 2014 2:02:03 PM UTC-7, Bill Venners wrote:

 
The reason you might not want to do that is it may be more confusing than any
benefit it provides. It might be simpler for user if there's one type, \/, that
is a monad with consistent applicative behavior and similar type Validation that
is not a monad (including not having a flatMap method so it can't be used monadically
in a for expression) and has an applicative that accumulates.

I dislike the argument that having one type with two different behaviors is less confusing that two types with two different (but consistent) behaviors. The fundamental problem is that the behavior of my code can change dramatically if I accidentally change the error type to something that has a semigroup - namely, I go from abort-on-first-error semantics to abort-if-one-fails semantics. This kind of hidden information (not evident at the call site, because all the relevant typeclasses are fetched implicitly) is really dangerous. 

I'd like to try to convince you that having one type represent two behaviors is insidious with a more obvious example. Consider strings.

Actually my opinion was already leaning in that direction, so I'll be easy to convince. My example was interesting to work out, but the end result is a rather slippery Validation type, which could transform itself to fit its context in ways that may surprise its user. I do see that.

I posted here originally because wanted to understand why the design of scalaz's \/ and Validation came out differently than scalautils' Or. I expected there were good reasons, and I think I have figured out what those reasons are. Hopefully I won't ignite a z-explosion, but I'd like to post my conclusion to see what you all think.

I believe the main difference that drives the scalaz and scalautils designs for these concepts in different directions is that scalaz wants to use Applicative to accumulate errors, and scalautils does not. And the issue for scalaz is that, for example, an Applicative[Validation[List[Int], String]]'s ap method could either accumulate errors or not. Both accumulation and non-accumulation are valid implementations of the Applicative.ap method. Thus I believe Applicative[Validation[List[Int], String]] is an example of, to quote Kris, "having one type represent two behaviors."
 
Now, I believe that's the point of Applicative. It is an abstraction defined by methods of a certain shape that obey certain laws, which allows many different concepts to be represented and treated uniformly. It allows you to write generic code for those different concepts, reducing code duplication, and in ways that still I'd like to understand better, helps you reason about that code. But because Applicative.ap can mean two different things in this case, you want to have two different types, \/ and Validation, to make it more obvious which Applicative behavior is going to happen.

Or doesn't have that same pull to its design, because it doesn't use Applicative. I also, like you, want it to be obvious to readers and writers of code whether an Or is accumulating or short-circuiting. And it is clear, even though I just use one type.

Or by itself just behaves like a monad, very much like \/. You'd can access its monad-hood through for expressions, for example. And it always works the same. It's flatMap methods can be chained and will short circuit at the first sign of trouble. Always. No matter what. Even if the bad type is an Every, which makes it an "accumulating Or", flatMap will still short circuit.

If the bad type is an Every, though, you can import the members of object Accumulation, and a *different* API will become available (though implicits) that allows you to ask that Or to accumulate errors. If you use that accumulation API and it compiles, it will accumulate, always. If you try and use the accumulating API on an Or that doesn't have an Every bad type, then it just won't compile because it can't accumulate. So if and only if it compiles, the accumulating API will accumulate, always.

In short, I think the underlying reason the scalautils design works on one type is because I don't have any method like ap that can mean two different things. The reason the scalaz design works better with two types, is you do have that ap method to contend with.

Bill

[Mark Hibberd]

That all seem fairly reasonable. There are just a few follow on
ramifications from the "different" API that should be noted,
specifically the inability to write code that may or may not
accumulate depending on what is past in, which for me is critical.

[Bill Venners]

Hi Mark,

On Monday, January 6, 2014 10:31:45 PM UTC-6, Mark Hibberd wrote:

<snipped history>

That all seem fairly reasonable. There are just a few follow on
ramifications from the "different" API that should be noted,
specifically the inability to write code that may or may not
accumulate depending on what is past in, which for me is critical.

Yes, that's a good next puzzle for me. One good way to do that of course would be to provide Applicatives for Or. I did a hasty attempt here a couple days back:

https://github.com/bvenners/orEveryLaws/blob/master/src/main/scala/itsnotjustagoodidea/OrInstances.scala

But I think it may have that "slippery" type problem we saw when trying to associate both a monad-derived applicative and an accumulating applicative to Validation. I'll think about it, though as an exercise. I didn't figure scalaz folks would want to use Or, since they already have good alternatives in scalaz. Or was intended for non-scalazers. Nevertheless I won't be able to stop my brain from thinking about it. I'll post here if I come up with anything interesting.

Bill

[Kris Nuttycombe]

On Monday, January 6, 2014 9:00:09 PM UTC-7, Bill Venners wrote:

Hi Kris,

On Monday, January 6, 2014 1:19:29 PM UTC-8, Kris Nuttycombe wrote:

On Sunday, January 5, 2014 2:02:03 PM UTC-7, Bill Venners wrote:

 
The reason you might not want to do that is it may be more confusing than any
benefit it provides. It might be simpler for user if there's one type, \/, that
is a monad with consistent applicative behavior and similar type Validation that
is not a monad (including not having a flatMap method so it can't be used monadically
in a for expression) and has an applicative that accumulates.

I dislike the argument that having one type with two different behaviors is less confusing that two types with two different (but consistent) behaviors. The fundamental problem is that the behavior of my code can change dramatically if I accidentally change the error type to something that has a semigroup - namely, I go from abort-on-first-error semantics to abort-if-one-fails semantics. This kind of hidden information (not evident at the call site, because all the relevant typeclasses are fetched implicitly) is really dangerous. 

I'd like to try to convince you that having one type represent two behaviors is insidious with a more obvious example. Consider strings.

Actually my opinion was already leaning in that direction, so I'll be easy to convince. My example was interesting to work out, but the end result is a rather slippery Validation type, which could transform itself to fit its context in ways that may surprise its user. I do see that.

I posted here originally because wanted to understand why the design of scalaz's \/ and Validation came out differently than scalautils' Or. I expected there were good reasons, and I think I have figured out what those reasons are. Hopefully I won't ignite a z-explosion, but I'd like to post my conclusion to see what you all think.

I believe the main difference that drives the scalaz and scalautils designs for these concepts in different directions is that scalaz wants to use Applicative to accumulate errors, and scalautils does not. And the issue for scalaz is that, for example, an Applicative[Validation[List[Int], String]]'s ap method could either accumulate errors or not. Both accumulation and non-accumulation are valid implementations of the Applicative.ap method. Thus I believe Applicative[Validation[List[Int], String]] is an example of, to quote Kris, "having one type represent two behaviors."
 
Now, I believe that's the point of Applicative. It is an abstraction defined by methods of a certain shape that obey certain laws, which allows many different concepts to be represented and treated uniformly. It allows you to write generic code for those different concepts, reducing code duplication, and in ways that still I'd like to understand better, helps you reason about that code. But because Applicative.ap can mean two different things in this case, you want to have two different types, \/ and Validation, to make it more obvious which Applicative behavior is going to happen.

Or doesn't have that same pull to its design, because it doesn't use Applicative. I also, like you, want it to be obvious to readers and writers of code whether an Or is accumulating or short-circuiting. And it is clear, even though I just use one type.

Or by itself just behaves like a monad, very much like \/. You'd can access its monad-hood through for expressions, for example. And it always works the same. It's flatMap methods can be chained and will short circuit at the first sign of trouble. Always. No matter what. Even if the bad type is an Every, which makes it an "accumulating Or", flatMap will still short circuit.

If the bad type is an Every, though, you can import the members of object Accumulation, and a *different* API will become available (though implicits) that allows you to ask that Or to accumulate errors. If you use that accumulation API and it compiles, it will accumulate, always. If you try and use the accumulating API on an Or that doesn't have an Every bad type, then it just won't compile because it can't accumulate. So if and only if it compiles, the accumulating API will accumulate, always.

In short, I think the underlying reason the scalautils design works on one type is because I don't have any method like ap that can mean two different things. The reason the scalaz design works better with two types, is you do have that ap method to contend with.

Bill

Given that Or is specifically intended to deal with success vs. failure scenarios, I think this isn't unreasonable; you're limiting abstractive power in exchange for semantic certainty, which is something I personally appreciate. In fact, I've frequently thought that Validation was poorly named, because it's a more general construct - a biased disjunctive type which accumulates values in the "secondary" type during applicative composition - I've found reasons to use Validation in places that are completely unrelated to success and failure, and have always aliased it in those circumstances because otherwise the name is misleading. 

Something that might have been missed in this discussion is why we frequently want to write in terms of the applicative functor instead of a specific type that has an applicative functor (like Validation) is that, since applicative functors compose, it's frequently useful to "stack" their effects (and the same is true of monads, though of course not all monads compose) and then use those values at call sites where the only constraint is provided by the applicative typeclass. In my code this is usually a retrograde discovery - I start working with the specific types, then realize that I only use a subset of the functionality provided for those types and so narrow such that only that subset is required by requesting the typeclass that provides it. The more I do this, though, the more frequently I'm able to think in terms of choosing the correct typeclasses "up front" and so the code ends up being more explicit about *exactly* what semantics are required for it to function. I think this is called the "principle of least power" or something like that - only ask for what you need, and the caller of the function will have more flexibility.

Kris

[Bill Venners]

Hi Kris,

That's interesting, and makes sense about the principal of least power. Make your parameter types as abstract as possible.

I have done some thinking about how to deal with Or's multiple applicative personalities and it is not easy. I got to wondering if scalaz has evolved some "recommended" way to deal with such multiple personality situations. For example I can think of two Monoids for Int, one for addition and another for multiplication. I tried this and discovered (as I expected) that the default would be addition:

scala> import scalaz._

import scalaz._

scala> import Scalaz._

import Scalaz._

scala> val default = implicitly[Monoid[Int]]

default: scalaz.Monoid[Int] = scalaz.std.AnyValInstances$$anon$5@7c8985ea

scala> default.zero

res1: Int = 0

scala> default.append(5, 6)

res2: Int = 11

Is there some recommended way to say I actually want to use a Monoid[Int] that does multiplication, such that append(5, 6) would give me 30? I would guess that you just either pass a different Monoid in explicitly where desired, or if desired a lot, define it as an implicit in scope such that the default one (which I'd expect is in the companion object for Monoid) is "overpowered." Is that the recommended approach in scalaz? Is there any other techniques that evolved to deal with multiple personalities?

Thanks.

Bill

 

Kris

[Tom Switzer]

The Scalaz way is to use type tags:

Monoid[Int @@ Tags.Multiplication]

I will plug Spire here too, since I work on it...

Spire makes the distinction by not providing a default Monoid for Int. Instead, Spire has AdditiveMonoid and MultiplicativeMonoid. These types of semigroups/monoids/groups are inherited by other type classes, such as Ring and Fractional. You can get a bare Monoid for addition/multiplication by using Monoid.additive[Int] or Monoid.multiplicative[Int]

--

[Runar Bjarnason]

The recommended way is to use different types for different monoids.

e.g.

Monoid[Int @@ Multiplication]

Monoid[Int @@ Addition]

Also, you keep insisting that you're using only one type `Or` for both of your "personalities". That's simply not true.

Runar

On Tue, Jan 7, 2014 at 4:09 PM, Bill Venners <bi...@artima.com> wrote:

--

[Bill Venners]

Hi Runar,

On Tuesday, January 7, 2014 3:30:04 PM UTC-6, Runar Bjarnason wrote:

The recommended way is to use different types for different monoids.

e.g.

Monoid[Int @@ Multiplication]

Monoid[Int @@ Addition]

Also, you keep insisting that you're using only one type `Or` for both of your "personalities". That's simply not true.

Sorry, I wasn't being precise. What I mean by multiple personalities is potential for multiple, desireable Applicative instances. I think Or, \/, and Validation are all type constructors that give rise a family of types, some of whose members (the ones where the Bad/-\/, or Failure types are a Semigroup) have the potential for multiple, desirable Applicative instances.

Int Or String, for example, is a type that has that multiple personalities, because you could conceivably want to either short circuit or accumulate error strings. Although outside of Scalaz, Or provides an API to accumulate in Every. With Scalaz it could be used to accumulate any Semigroup, like Validation (or \/ for that matter).

I whipped up some examples of turning various personalities on or off here:

https://github.com/bvenners/orEveryLaws/blob/master/src/test/scala/itsnotjustagoodidea/OrExampleSpec.scala

I did it by just disabling the monad personality and enabling the accumulating applicative one, or vice versa. It is a bit ugly because I needed to actually hide the monad instance in some cases, but if the monad instance were actually provided in the companion object that wouldn't be necessary. I don't do that here because that would require scalautils to have a dependency on scalaz so it could provide a method that produces scalaz Monad instances in Or's companion. But were you to do something like this with \/, it could be made to accumulate by just the one line of code that brings in an accumulating implicit. That would overpower the monadic one in the \/ companion object.

Anyway, what I demo is that given the way I set up the implicits for Or, an short-circuiting, monadic Applicative personality is provided for any Or type except Every. For Every the default is an accumulating Applicative. But that can be changed. I show you can turn off the accumulating Applicative if the bad type is an Every, and just get the monadic personality. I also show that you can turn on an accumulating Applicative for any Semigroup. I demo accumulating into a single String and List[String].

The last bit in that test class just demos that you can accumulate in Every, then at the end of the day "badMap" the accumulated errors into any type you want. So really accumulating into a Semigroup could possibly be lived without. It isn't my party, and there's a lot of existing code using Validation, but one other approach is that \/ could be used to accumulate just NonEmptyList in this manner, and you wouldn't need Validation at all.

Lastly, I tried using type tags, because I thought it might be nice to just turn accumulation on with an @@ Acc annotation. I didn't quite get it to work yet, but already had to do lots of casting. Seems the compiler doesn't infer phantom-type-hood.

Bill

[Grzegorz Balcerek]

Bill,

 

Lastly, I tried using type tags, because I thought it might be nice to just turn accumulation on with an @@ Acc annotation. I didn't quite get it to work yet, but already had to do lots of casting. Seems the compiler doesn't infer phantom-type-hood.

A possible alternative approach of using different types for different monoids is wrapping your values:

case class Product(value: Int) extends AnyVal

trait Monoid[T] {

  def mempty: T

  def mappend(a: T, b: T): T

}

object Monoid {

  implicit object MonoidInt extends Monoid[Int] {

    override def mempty = 0

    override def mappend(a: Int, b: Int) = a + b

  }

  implicit object MonoidProduct extends Monoid[Product] {

    override def mempty = Product(1)

    override def mappend(a: Product, b: Product) = Product(a.value * b.value)

  }

  def msum[T](list: T*)(implicit m: Monoid[T]): T =

    list.foldLeft(m.mempty)(m.mappend)

}

scala> Monoid.msum(2, 5, 4)

res0: Int = 11

scala> Monoid.msum(Product(2), Product(5), Product(4))

res1: Product = Product(40)

scala> Monoid.msum(Product(2), Product(5), Product(4)).value

res2: Int = 40

Regards,

Grzegorz Balcerek

 

[Bill Venners]

Hi Grzegorz,

Thanks for the idea. You're right, and the code is prettier than tags (assuming I'm correct that the tags require a lot of casting in practice). My brain kept thinking about this, though, and I ultimately decided (yes, it's my final answer) that something like Or or \/, or \/ or Or if you prefer, should just default to monadic personality period. No special, schizophrenic switching of accumulating behavior. But you could with a simple implicit definition switch that accumulating behavior on. I think that is easy and obvious for users. I update my Or examples:

https://github.com/bvenners/orEveryLaws/blob/master/src/test/scala/itsnotjustagoodidea/OrExampleSpec.scala

And created a similar one for \/:

https://github.com/bvenners/orEveryLaws/blob/master/src/test/scala/itsnotjustagoodidea/DisjunctionExampleSpec.scala
 
I think I'm done. Thanks for all your input.

Bill