Deprecated flatMap in Validation on Scalaz 7.1

[Edmondo Porcu]

Dear all,

I can see in Scalaz 7.1 source code the following:

  /** Bind through the success of this validation. */
  @deprecated("flatMap does not accumulate errors, use `\\/` instead", "7.1")
  def flatMap[EE >: E, B](f: A => Validation[EE, B]): Validation[EE, B] =
    this match {
      case Success(a) => f(a)
      case e @ Failure(_) => e
    }

Since I do large usage of this feature, I am interested in understanding more, but I am not able to find \\/ method.

Where is it defined? 

Best

Edmondo

[Kenji Yoshida]

Hi.

`\\/` means scalaz.\/ not method.

https://github.com/scalaz/scalaz/blob/v7.1.0-M4/core/src/main/scala/scalaz/Either.scala

[Edmondo Porcu]

oops, one is just an escape character. Thanks, it was taking me forever.

but:

Validation:

/** Bind through the success of this validation. */
  @deprecated("flatMap does not accumulate errors, use `\\/` instead", "7.1")
  def flatMap[EE >: E, B](f: A => Validation[EE, B]): Validation[EE, B] =
    this match {
      case Success(a) => f(a)
      case e @ Failure(_) => e
    }

Either:
  /** Bind through the right of this disjunction. */
  def flatMap[AA >: A, D](g: B => (AA \/ D)): (AA \/ D) =
    this match {
      case a @ -\/(_) => a
      case \/-(b) => g(b)
    }

Could you please explain me the difference?

[Paul Chiusano]

This is kind of strange. I can understand why Validation is not a monad but there's no reason not to keep the flatMap method around.

--
You received this message because you are subscribed to the Google Groups "scalaz" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalaz+un...@googlegroups.com.
To post to this group, send email to sca...@googlegroups.com.
Visit this group at http://groups.google.com/group/scalaz.
For more options, visit https://groups.google.com/groups/opt_out.

[Chris Marshall]

Totally agree. Pretty sure Jason does too!

[Tony Morris]

So that it cannot be used in a for comprehension.

[Chris Marshall]

But why is that a good reason?

[Tony Morris]

It's not an excellent reason, but what is lost by not calling it flatMap?

-- 
Tony Morris
http://tmorris.net/

[Chris Marshall]

You can't use it in a for-comprehension (aka. code that used to compile, no longer does)

[Tony Morris]

This is a desirable thing. Therefore, it is a net gain so far.

Are there any desirable things that are lost.

[Devon Miller]

I am just starting to use scalaz (new to scala as well) and my entry point was the Validation capability.

If Validation should not be used in for-comprehensions, which I use them in a lot, what is the recommended usage then? 

I'm okay changing but it would be helpful to understand the recommended pattern.

My frequent usage is in chaining together several "gets" where each line in the for-comprehension returns a raw object, an Option or a scalar value and for most uses, each line is dependent on the previous one. The for-comprehension allows me to sequence them. The overall function should return a Validation[String, MyObjectType] if one of the "gets" fails.

[Stephen Compall]

On Sun, 2013-12-29 at 08:26 -0800, Devon Miller wrote:

My frequent usage is in chaining together several "gets" where each line in the for-comprehension returns a raw object, an Option or a scalar value and for most uses, each line is dependent on the previous one. The for-comprehension allows me to sequence them. The overall function should return a Validation[String, MyObjectType] if one of the "gets" fails.

This function should probably return String \/ MyObjectType instead; this structure has the behavior you are describing.

Validation is only good for you insofar as its behavior gives you the results you expect.  Consider your "error message", String.  If I have two failures from your code, s1 and s2, is (s1 + s2) also an "error message"?

-- Stephen Compall "^aCollection allSatisfy: [:each|aCondition]": less is better

[Devon Miller]

This function should probably return String \/ MyObjectType instead; this structure has the behavior you are describing.

Validation is only good for you insofar as its behavior gives you the results you expect.  Consider your "error message", String.  If I have two failures from your code, s1 and s2, is (s1 + s2) also an "error message"?

Yes, in many cases, but not all, s1+s2 (as in list concatenation) is valuable and needed. I am finding that in some (not all require this level of error reporting) of these functions (backend database repository service and DAO layer processing) that I need to gather up the messages and there are potential multiple messages. In other cases, I need to return on the first failure because of the nature of the dependencies in the calling sequence.

Given the comment about \/, I played around with using \/ on some of my layers and since these disjunctions are isomorphic, it was easy to convert between them when needed. Where I found it harder was around readability. If I use \/ to allow me to use for-comprehensions nicely, but some of the methods needed to return ValidationNel for multiple messages, that I have a mix of Validation, ValidationNel and \/ within related sets of classes/functions. That seemed confusing reading the code. This data layer is for data import processing so the "user" oriented messages are important. But perhaps I am confusing things such as "user" oriented messages and "logging" in that I am overusing this feature for the wrong purpose. I don't think I am confusing those things, but I'll check it again.

 

After playing around, I think what is now, and probably always, was recommended was that if I need to concatenate messages together and that parts of my calling sequence are independent, I need to use the |@| operator (I think that's the applicative functor) to get my messages to "add" together in a Nel since the string/left/failure part does not add in a for-comprehension since for-comprehensions fail fast.

Perhaps service layers should have a tendency to return ValidationNels and the DAOs more \/. Internal to the service layer, its mostly \/ in for-comprehensions and some applicative functors. Whatever it is, I'll try to be consistent as scalaz has already helped me reduce errors and code complexity while increasing readability already.

[Dave Stevens]

My suggestion is to always use \/ and convert to Validation for the purposes of collecting failures only, then back to \/. It is easy to convert from Nel[A] \/ B to ValidationNel[A, B] and back. 

From my experience, forcing the use of ValidationNel across the board makes code harder to read, not easier. If a function/method can never return more than one error, defining the return type as ValidationNel[A, B] is not accurate and forces you to handle the multiple A when there can be only one. If your A is string and you are not really doing anything with the errors besides logging/reporting them to a "user", this might not be as apparent. When using more accurate types to describe the errors, this accuracy issue becomes much more obvious.

Travis Brown has an unanswered question on stackoverflow on the same topic - http://stackoverflow.com/questions/20065853/validation-versus-disjunction/20134523

[Tony Morris]

Validation and \/ have methods on them to run functions as if they are the other.

[Lars Hupel]

> You can't use it in a for-comprehension (aka. code that used to compile, no longer does)

Let me just add that no, your code will still compile. We've adopted a
deprecation policy for scalaz, and the `flatMap` method will be included
up to 7.1.x (which will be supported at least for the whole of 2014).

[Tony Morris]

:(

[Senia Popugaev]

Could someone, please, explain me all that stuff about `Validation#flatMap`  vs `\/#flatMap`?

Let's suppose I have 2 functions: `ab: A => ValisdationNel[String, B]` and `bc: B => ValidationNel[String, C]`. So it seems natural to me  that I should use `flatMap` to get `ac: A => ValidationNel[String, C]`. But `flatMap` is deprecated. So I guess I'm using it wrong. And I should not use `Validation` as drop-in replacement for exceptions.

But `\/` doesn't seems like a semantically correct instrument at all. `-\/` and `\/-`. Which one is correct result? When I see `a.success` or `b.failNel` it's clear to me with one is correct. But `a.left` and `b.right`... It seems like a pair of completely equivalent expressions. Why one is correct and other is fail? And `map` and `flatMap`. I can't be sure with part is transformed.

But `Validation#flatMap` is deprecated, so I guess I just can't understand something very simple. So could someone, please, explain me what am I missing here?



пятница, 27 декабря 2013 г., 14:12:28 UTC+4 пользователь Kenji Yoshida написал:

[Chris Marshall]

As has been stated, you can run a function on Validation as if it were an either: the method is called "disjunctioned". In your case above, you must flatMap through a disjunction. That is:

  lazy val c = (ab(a).disjunction flatMap (b => bc(b).disjunction)).validation

Chris

[Tony Morris]

Keep calling flatMap. It just needs to be renamed so as not to be used in a for-comprehension.

[Paul Chiusano]

Why is it bad if validation can be used in a for comprehension? Just because you prefer that syntax be used for types that are monads?

Validation is isomorphic to Either, and if it it didn't mess with implicit lookup, we could define both a monad and an applicative (with different meanings) for the same type. This overlap happens a lot - streams have a 'zippy' applicative and a sequencing monad with different meanings.

So, I'd like to hear an actual argument. The existing flatMap is convenient at times. Getting rid of flatMap will break existing code, and what are you going to call that function if not flatMap?

[Runar Bjarnason]

Yeah, I actually disagree slightly with deprecating flatMap on Validation. The reasoning seems to be that people might use it incorrectly. The solution to that is to not use it incorrectly. After all, any old wanker can write their own flatMap and then proceed to use it.

—
Sent from Mailbox for iPhone

[Bill Venners]

Hi Paul,

On Monday, January 6, 2014 7:19:26 AM UTC-8, Paul Chiusano wrote:

Why is it bad if validation can be used in a for comprehension? Just because you prefer that syntax be used for types that are monads?

Validation is isomorphic to Either, and if it it didn't mess with implicit lookup, we could define both a monad and an applicative (with different meanings) for the same type. This overlap happens a lot - streams have a 'zippy' applicative and a sequencing monad with different meanings.

I believe I worked out a way yesterday to arrange the implicits so that Validation can have both monad and applicative personalities:

https://groups.google.com/d/msg/scalaz/R-YYRoqzSXk/TCNQ7RjFsP8J

Not sure the result is desirable, or if other problems may pop up when trying to bring it to real code (I did it for the simple Validation example Tony created for the discussion), but it does do what I think you described: defines both a monad and an applicative (with different meanings) for the same Validation type.
 

So, I'd like to hear an actual argument. The existing flatMap is convenient at times. Getting rid of flatMap will break existing code, and what are you going to call that

function if not flatMap?

Bill
 

[Paul Chiusano]

Yes it is possible to have two different instances in scope, with different implicit priority. I don't think we should do this though. I'd rather we just undeprecate flatMap.

[Tony Morris]

I think, not strongly, that it is a reasonable expectation that these two programs are the same:

for { // p1

  a <- aa

  b <- bb

} yield (a, b)

(aa |@| bb)((a, b) => (a, b)) // p2

[Runar Bjarnason]

I think that is only a reasonable expectation for applicatives that are also monads.

[Tony Morris]

So if I were to look at a for-comprehension, I should determine in which Applicative I am running, and whether it has a Monad?

[Runar Bjarnason]

If you're writing a for-comprehension, you should require Monad.

[Alec Zorab]

I think that is an unreasonable expectation. The second one compiling does not imply that the first is valid. 

I also think that, given that a scala for comprehension is clearly not a monad comprehension (cf Set, Try, the entire collections library etc) that any assumption based on the predictable behaviour of a for comprehension is not necessarily sound.

[Tony Morris]

I agree with this. I think the type system should also enforce the requirement of Monad. Therefore, Validation in a for-comprehension should yield a type error.

[Tony Morris]

I am happy to accept this unprincipled premise on its own. It's just that I would never have a practical use for a for-comprehension.

On this premise and in the absence of a better solution to what is now "for-comprehensions don't exist", I think let's just save whatever utility is available.

[Runar Bjarnason]

Well, for-comprehension is 100% syntax in Scala, so I see deprecating flatMap as a bit like a gun manufacturer trying to make their product misfire if the customer points it at their foot.

[Michael Pilquist]

I don't have much to add besides a big +1 for removing the deprecation. I've never been confused by the short circuiting behavior of flatMap on Validation and I don't look forward to refactoring existing code over the syntax disagreement.

[Stephen Compall]

On Tue, 2014-01-07 at 06:50 +1000, Tony Morris wrote:

I think, not strongly, that it is a reasonable expectation that these two programs are the same: <for-comprehension/applicative versions elided>

I think this as well, albeit somewhat more strongly, because I regularly transform between applicative and bind syntax when it seems convenient to do so.  I'd rather doing so be predictable as is required by the bind laws.

 I think the type system should also enforce the requirement of Monad. Therefore, Validation in a for-comprehension should yield a type error.

Agreed.  I also think it's reasonable to expect that, where you see a flatMap of the appropriate type, it should be one that gives rise to a lawful Bind instance, particularly as the compiler doesn't tell you that it used an incidentally-named method when you expected a BindSyntax method.  Accordingly, a method that doesn't give rise to a lawful Bind instance shouldn't be called flatMap.  (In scalaz 7.1 ap/bind consistency is upgraded to a Bind law from a Monad law).

With respect to scala collections: fixing scala-library to follow the above rule is harder than following it in scalaz, but the difficulty of the former should not be an enemy of the latter.

[Dave Stevens]

Being introduced to the concepts of monad and applicative through Scalaz, my experiences support Tony and Stephen's arguments and counter most of Bill's arguments in the other thread regarding new people. I expect most people like me first experimented with Scalaz through Validation. I was confused, not by the behavior of flatMap, but by the behavior of applicative because of the behavior of flatMap. I would also suspect that it is not uncommon to first try and use Validation[String, A]. What is this damn NEL anyways, amirite? String is a semigroup and the type-checker will not complain if you attempt to use Validation[String, A] as an Applicative. It is unlikely that the results will be desirable though. I used this for many months before realizing errors were being accumulated via the semigroup for String. It type-checked and I didn't write enough tests.

Lesson learned, don't use strings to represent errors. Lesson learned, RTFM. Way back when, had \/ existed and had Validation.flatMap not, I doubt I would have been able to shoot myself in the foot though.

I know I am not the only person who was confused by Validation and Monads.

https://groups.google.com/forum/#!topic/scalaz/IWuHC0nlVws

http://stackoverflow.com/questions/12211776/why-isnt-validation-a-monad-scalaz7

I remember being at a meetup where Runar corrected a presenter on this topic as well.

With all that said, I probably would not be too happy if I had to do this refactor on a large project.

[Tony Morris]

If a for-comprehension is 100% syntax and flatMap should be allowed, then you also accept that the two earlier programs are not the same. I don't necessarily feel strongly that they definitely should, but I admit to suspecting they probably should. I could easily be persuaded either way. However, I want to be clear that this is what we would be accepting. I don't see the benefit though.

PS: I also agree that listening to advice from beginners on how to coach beginners is almost always a red herring.

[Chris Marshall]

Tony has persuaded me that we should remove flatMap. I was previously strongly against it, if anyone places any value on my opinions.

Chris

[Senia Popugaev]

Maybe `flatMap` should be renamed to prevent `for-comprehensions` on `Validation`?

`flatMap` on `Validation` is useful. I guess without convenient replacement some developers will add `flatMap` to `Validation` using `implicit class`. `(ab(a).disjunction flatMap (b => bc(b).disjunction)).validation` is not a convenient replacement, it's a workaround for one time.

вторник, 7 января 2014 г., 13:27:32 UTC+4 пользователь oxbow_lakes написал:

[Runar Bjarnason]

> If a for-comprehension is 100% syntax and flatMap should be allowed, then you also accept that the two earlier programs are not the same.

They are not the same. Let's transliterate into Haskell. What is the type of this program?

p aa bb = do

  a <- aa

  b <- bb

  return (a, b)

Its type is: (Monad m) => m a -> m b -> m (a, b)

What about this one?

p aa bb = (,) <$> aa <*> bb

Its type is (Applicative f) => f a -> f b -> f (a, b)

These are clearly not the same. But it is a reasonable expectation that for a given type constructor f, instantiating the former at f should be the same as instantiating the latter at f. In Scala, the equivalent of the former cannot be instantiated at Validation at all:

def p[M[_]: Monad,A,B](aa: M[A], bb: M[B]): M[(A,B)] = for {

  a <- aa

  b <- bb

} yield (a, b)

Because there is no Monad[Validation], nor should there be. But you could also write it with a different type:

def p[A, B, M[_] <: Validation[_]](aa: M[A], bb: M[B]): M[(A, B)]

Or any of a zillion other possibilities. I'm not saying that's good practice, but I believe that trying to get in the way of such programs is a red herring.

In Haskell, do notation is allowed only on Monads, which makes a lot of sense. But in Scala, for-comprehension is not based on a type class unless the author explicitly says that it is. Looking at a for-comprehension by itself doesn't allow you to infer the type of that expression.

In practice, I often call flatMap on Validation. I see it as shorthand for:

  1. Convert to disjunction

  2. Bind across the disjunction

  3. Convert back to Validation

If it were removed, I would find myself adding it back with an implicit class anyway.

Rúnar

[Tony Morris]

I cannot conceive a well-principled argument in either case, because the strongest positions for/against arrive at making excuses for Scala's poor-behaviour. I think I will tap out of this one.

To be clear on a point of Monad instance for Validation, this definitely should not exist.

-- 
Tony Morris
http://tmorris.net/