questioning FP
Ittay Dror
Hi All,
(I welcome any sort of replies, even containing foul language, as
long as there's also information inside.)
I'm about to give a lecture on FP. Now, some concepts in FP I love, but for some I have a hard time finding a good angle. Specifically, the IO monad.
Consider the following imperative code:
service.fetchData
service.closeObviously, both method calls create side effects.
Now, (discarding iteratees), lets consider making this with the IO monad:
for (data <- service.fetchData;
_ <- service.close)
yield data
where all methods are changed to return IO[Data], IO[Unit] and also the calling code must now return an IO[X] for some X.
The question is, what have we gained?
Instead of sequencing by line order, we're explicitly sequencing in the for comprehension. Not much of a gain here, since any mistake that causes code to be written in the first order in the imperative case can also be done in the functional case.
I also don't see anything that makes the code less prone to other mistakes. Such as forgetting to close or calling close twise. Obviously, there are ways to fix this such as service.withData(f: Data => X), but again, the more imperative style of silently calling close inside withData doesn't loose to the functional style that must return IO[X].
What am I missing? I'd appreciate any other examples which show how the use of IO is superior by making code that is more robust.
Regards,
Ittay
martin odersky
The IO monad was a neat trick for combining side effects with lazy
evaluation. But your title seems to be wrong. Questioning FP? FP is
much broader than lazy evaluation. In fact, there is only one lazily
evaluated language in wide usage today and even its creators have said
that laziness was probably a mistake.
Strict languages don't need the IO monad, and generally don't have it,
even though they could. Bob Harper's posts in his "existential type"
series are a good explanation on why not.
An interesting quote from Simon Peyton Jones is that "laziness has
kept us pure". That is, it has motivated the use of monads to
represent effects.
I agree that a static representation of effects is very useful, but my
gut feeling is that there must be a more lightweight and polymorphic
way to get effect checking than what monads provide.
-- Martin
Steven Obua
Rex Kerr
I also have yet to see any point to IO monads in the vast majority of cases. They seem to allow one to pay a tiny little bit less attention after everything is set up in exchange for requiring heaps more abstraction* and boilerplate.
For what it's worth, one of the best discussions I've seen of IO monads--with something of a positive spin on them--is from James Iry:
http://james-iry.blogspot.com/2007/11/monads-are-elephants-part-4.html
Anyway, I find that adhering to four principles yields robust IO.
(1) Input that may fail should be wrapped in an option or the like, and manipulated functionally (i.e. maps and such on the option).
(2) Output that may fail should take a lambda whose job it is to deal with the consequences of failure if it cannot be ignored.
(3) If multiple steps must happen in sequence, access them only through a method that ensures that all steps happen in sequence. If this is not possible, create a single entity (class, actor, whatever) whose job it is to supervise all the input or output from one channel.
I _do_ have to keep these things in mind. The framework does not force me to do it this way. But this doesn't at all diminish the importance of functional programming approaches; it just uses them in a different way than the IO monad.
For example, my personal library contains things like
object Print {
def toFile(f: File)(out: PrintWriter => Unit) {
val p = new java.io.PrintWriter(f)
try { out(p) } finally { p.close() }
}
}
def toFileSafely(f: File)(err: (File,IOException) => Unit)(out: PrintWriter => Unit) {
// Same deal, but catch exceptions and have err handle them
}
}
which make it almost impossible to mess up sending text to a file:
val xs = List(1,3,5,7,9)
Print.toFile(new File("test.txt")) { pr => xs.map(pr.println) }
For cases where resources need to be cleaned up, I also have a variant of the forward pipe that looks something like:
class TidyPipe[A](a: A) {
def tidy[Z](f: A => Z)(g: A => Any) = { val z = f(a); g(a); z }
}
which with you can then (given implicit conversion in scope):
input tidy { parser } { _.close() }
and a well-written parser(in: Input) method should emit an Option[ParsedResult] of some sort, so then this is manipulated with stuff like:
input tidy { parser } { _.close() } map { result => some; complicated; thing(result) }
These sorts of things make it so easy to do the right thing that I really don't miss IO monads; I already spend almost no time thinking about IO (aside from complex things like parsing, and that is because parsing is inherently complex), and it comes out correct essentially all the time (and the mistakes are _not_ because I lost track of what the state of the world was supposed to be). Adding more of a framework around things would not help.
--Rex
* Abstraction is great when the essence of your problem remains behind, and irrelevant but distracting details disappear. However, with IO I find that abstraction makes the essence of the problem disappear while leaving behind necessary but trivial details. For example, maybe I need to know it's a byte buffer of size 8192 in order to do my parsing!
Jesper Nordenberg
> Strict languages don't need the IO monad, and generally don't have it,
> even though they could. Bob Harper's posts in his "existential type"
> series are a good explanation on why not.
>
> An interesting quote from Simon Peyton Jones is that "laziness has
> kept us pure". That is, it has motivated the use of monads to
> represent effects.
> I agree that a static representation of effects is very useful, but my
> gut feeling is that there must be a more lightweight and polymorphic
> way to get effect checking than what monads provide.
Well, the IO monad (or rather the IO type) is a simple and
straightforward way to keep the language pure without introducing an
additional effects system. So there are certainly benefits to using it
regardless of if the language is lazy or strict.
/Jesper Nordenberg
Meredith Gregory
L.G. Meredith
Managing Partner
Biosimilarity LLC
7329 39th Ave SW
Ittay Dror
martin odersky wrote:
The IO monad was a neat trick for combining side effects with lazy evaluation. But your title seems to be wrong. Questioning FP? FP is much broader than lazy evaluation. In fact, there is only one lazily evaluated language in wide usage today and even its creators have said that laziness was probably a mistake. Strict languages don't need the IO monad, and generally don't have it, even though they could. Bob Harper's posts in his "existential type" series are a good explanation on why not. An interesting quote from Simon Peyton Jones is that "laziness has kept us pure". That is, it has motivated the use of monads to represent effects. I agree that a static representation of effects is very useful, but my gut feeling is that there must be a more lightweight and polymorphic way to get effect checking than what monads provide. -- Martin
My questioning is specifically about making functions pure. And the hoops one need to jump through to get that (at least the way I know which is the IO monad). More specifically:
1. what does it buy me? in theory, pure functions are nice since they behave better, but i want to find a use case that appeals to everyday java developers. That is, a case where making a function pure removes pitfalls (while the function remains basically the same)
2. what are the alternatives to IO monad? an implicit World argument?
Thank you,
Ittay
Lars Hupel
You can also make the "World" argument not implicit; but in any case,
the type system has to ensure that you don't reuse an "old" world. This
is called "linear types", and there is a language which does something
similar ("uniqueness types"): <http://wiki.clean.cs.ru.nl/Clean>
Roland Kuhn
How can this ever work in a distributed concurrent setting? I saw “Concurrent Clean” mentioned on that page, but couldn’t (quickly) find further mention. Where can I dig deeper?
Roland Kuhn
Typesafe – Enterprise-Grade Scala from the Experts
twitter: @rolandkuhn
Jesper Nordenberg
> The title is probably too much of a troll. Sorry about that.
>
> My questioning is specifically about making functions pure. And the
> hoops one need to jump through to get that (at least the way I know
> which is the IO monad). More specifically:
> 1. what does it buy me? in theory, pure functions are nice since they
> behave better, but i want to find a use case that appeals to everyday
> java developers. That is, a case where making a function pure removes
> pitfalls (while the function remains basically the same)
That's basically the same question as "what does a type system buy me?".
It enables automatic detection of programming errors at compile time
rather than at runtime.
It also opens up opportunities for performance optimizations and makes
your code a whole lot easier to test.
> 2. what are the alternatives to IO monad? an implicit World argument?
The ones I know are uniqueness typing (for example Clean) and effects
tracking (for example D, DDC, upcoming Scala effects system). So, either
you track effects using the type system, or you build a separate system
for it.
/Jesper Nordenberg
martin odersky
program fragment has to be rewritten completely once the first effect
is introduced, even if that effect is just a print statement. I don't
find that very easy to use, and would like to shoot for something more
polymorphic.
Cheers
-- Martin
martin odersky
that it's impure.
An explicit world argument would make the function technically pure,
but again, doing so
is dubious in practice.
If I'd search for good examples where purity is useful, I'd look
elsewhere. For instance, parallel collections allow automatic speedups
in exchange for all computations being pure.
Cheers
-- Martin
Ittay Dror
Jesper Nordenberg wrote:
Ittay Dror skrev 2011-10-10 07:16:
The title is probably too much of a troll. Sorry about that.
My questioning is specifically about making functions pure. And the
hoops one need to jump through to get that (at least the way I know
which is the IO monad). More specifically:
1. what does it buy me? in theory, pure functions are nice since they
behave better, but i want to find a use case that appeals to everyday
java developers. That is, a case where making a function pure removes
pitfalls (while the function remains basically the same)
That's basically the same question as "what does a type system buy me?". It enables automatic detection of programming errors at compile time rather than at runtime.
It also opens up opportunities for performance optimizations and makes your code a whole lot easier to test.
How is code easier to test? I initially thought that when I have a pure function, it will behave the same every time. So once I test it I'm sure it works the same in production. But then I thought: Ok, lets assume that I have a function that closes a socket. Further, assume that closing it a second time creates an error. If I have code that does 'for (_ <- close(socket); _ <- close(socket))', it will fail just the same as the (simpler) imperative code "socket.close; socket.close", wouldn't it? Side effects are not gone. They are just abstracted over, the question is whether the abstraction helps.
Again, would love some code examples.
Adriaan Moors
It's intriguing to contemplate the possibility of something lighter weight than a three method api!
I say, monads are starting to sound like a regular straightjacket by now (well, this is a feature, not a bug -- to some degree)
Lars Hupel
> that it's impure.
I have to disagree here, or at least ask why you think so. Let's
consider the function `putStrLn` from Haskell with the type `String ->
IO ()`. The definition of "purity" I am used to basically requires that
`putStrLn "foo"` does not execute a side-effect (true) and does not
depend on (hidden) state such that subsequent executions yield exactly
the same value (also true). The side-effect is executed by the
underlying system only if it is somehow included in `main`.
Kevin Wright
On Mon, Oct 10, 2011 at 2:47 AM, Meredith Gregory <lgreg.m...@gmail.com> wrote:
It's intriguing to contemplate the possibility of something lighter weight than a three method api!(I know you're aware of all this, Greg, just pointing it out for the sake of the "debate" part of this mailing list)so, let's see...something with few methods may sound like it's lightweight, but that's like saying programming straight in binary is even lighterweight: look! there's only ones and zeroes!then again, don't forget about the monad laws, which, implicit in your program though they may be, are an important constraint on the interface (2 downsides: they are implicit, they add more constraints)the real deal breaker, I'd say, is that monads don't compose well -- even simply mixing Option (I mean, Maybe) and List in a for (pardon me, do) comprehension is cumbersome! (not so much in Scala, thanks to CanBuildFrom and breakOut). For a recent proposal to solve monad composition, see "the monad zipper" (http://lambda-the-ultimate.org/node/3929),
I say, monads are starting to sound like a regular straightjacket by now (well, this is a feature, not a bug -- to some degree)Personally, I think we (as in, a strict language with subtyping, objects,...) are much better off with a (lightweight...) polymorphic effect system (being developed for Scala by Lukas Rytz -- see also Disciple (http://disciple.ouroborus.net/) for something more Haskelly)cheersadriaan
Kevin Wright
mail: kevin....@scalatechnology.com
gtalk / msn : kev.lee...@gmail.com
martin odersky
functions just "assemble" a computation which then gets executed only
by calling main. So instead of Monadic bind, Scala has
val x = a; b
On the type level, every type T would be interpreted as IO[T]. Every
function application is a bind. And voila! No side effects!
So the only thing IO provides is that code *not* using it is certified
to be pure, whereas in Scala everything is in the IO monad.
Cheers
-- Martin
Rex Kerr
Ittay Dror skrev 2011-10-10 07:16:That's basically the same question as "what does a type system buy me?". It enables automatic detection of programming errors at compile time rather than at runtime.
The title is probably too much of a troll. Sorry about that.
My questioning is specifically about making functions pure. And the
hoops one need to jump through to get that (at least the way I know
which is the IO monad). More specifically:
1. what does it buy me? in theory, pure functions are nice since they
behave better, but i want to find a use case that appeals to everyday
java developers. That is, a case where making a function pure removes
pitfalls (while the function remains basically the same)
It enables automatic detection of _some_ errors; one still would like to know how big of a hassle it is (much more in Java than Scala, to the point of motivating dynamically typed languages!) and how frequent those errors are (not very, when it comes to IO, from what I've seen).
It also opens up opportunities for performance optimizations and makes your code a whole lot easier to test.
I could buy the part about testing, but performance is usually better when you pay _less_ attention, not more, to what is allowed and not. In particular, writing wrappers that are instantiated with real objects in addition to the actual IO you need to do is not a good way to have higher performance.
The ones I know are uniqueness typing (for example Clean) and effects tracking (for example D, DDC, upcoming Scala effects system). So, either you track effects using the type system, or you build a separate system for it.2. what are the alternatives to IO monad? an implicit World argument?
I'm quite skeptical of the value of _any_ of these approaches since the world outside your program does not have type information. Thus, you end up having to watch for unexpected effects and do run-time identification of your types regardless. That your type system will catch you if you, say, fail to have a file handle closed at a certain point is usually not a big deal (and that has lighter-weight solutions anyway for the large majority of cases).
--Rex
Jesper Nordenberg
> I'd love to see a code example where the introduction of IO[_] makes
> more errors detectable. In the code example I used in the first
> message, I couldn't find where it helps.
Simple:
foo : Int => Int
If your language is impure (like Scala), foo can format your HD, store and
retrieve stuff in a DB etc. If your language is pure, foo will always return the
same value given the same argument and it will do nothing else, thus it's much
easier to verify it's behavior.
You are thinking backwards, it's not the functions that perform side effects
that are easy to test, it's the ones that have no side effects. Problem in a
language without effects tracking is that you have no idea or control over which
functions perform side effects.
Also, knowing that a function is pure gives the compiler more options for
optimization, it can memoize the return value, it can complete remove calls if
the result is not used, it can parallelize calls etc.
So, how do you keep the language pure and still allow side effects? The IO monad
is a quite natural solution to that.
> If I have
> code that does 'for (_ <- close(socket); _ <- close(socket))',
> it will fail just the same as the (simpler) imperative code
> "socket.close; socket.close", wouldn't it? Side effects are not
> gone. They are just abstracted over, the question is whether the
> abstraction helps.
You have just discovered that ';' is shorthand for flatMap. It's the same
abstraction.
/Jesper Nordenberg
Patrik Andersson
√iktor Ҡlang
ok.foo: Int => IO[Int]Are you now somehow safe from having your HD reformatted?
Yes, until you call unsafePerformIO
--
Viktor Klang
Akka Tech Lead
Patrik Andersson
Ittay Dror
Replies inline. Please note I'm replying from the perspective of
an every-day developer, not from the theoretical point of view. I
want to find a killer argument for using IO.
Jesper Nordenberg wrote:
Ittay Dror <ittay.dror <at> gmail.com> writes:I'd love to see a code example where the introduction of IO[_] makes more errors detectable. In the code example I used in the first message, I couldn't find where it helps.Simple: foo : Int => Int If your language is impure (like Scala), foo can format your HD, store and retrieve stuff in a DB etc. If your language is pure, foo will always return the same value given the same argument and it will do nothing else, thus it's much easier to verify it's behavior.
For the same reasons, a function 'printMsg: String => IO[Unit]' can format my HD just as well
If IO[_] is just a tag to mean a function is dangerous, then I'm fine with this approach (it is similar to how Option is an alternative to document the possible return of null), but then why go through the hassle of a monad? I can have:
case class Dangerous[A](value: A)
You are thinking backwards, it's not the functions that perform side effects that are easy to test, it's the ones that have no side effects. Problem in a language without effects tracking is that you have no idea or control over which functions perform side effects.
Also, knowing that a function is pure gives the compiler more options for optimization, it can memoize the return value, it can complete remove calls if the result is not used, it can parallelize calls etc.
Jesper Nordenberg
> So you're not safe then. If calling foo is not optional then neither is having
your HD be reformatted. IO buys us nothing in this - admittedly dumb - example.
I don't understand your objection. It's the guaranteed lack of side effects
that buys us something. "Int => IO[Int]" is the same as
"Int => World => (Int, World)" so obviously it's still a pure function.
Compared to "Int => Int" though it's much harder to verify its behaviour.
/Jesper Nordenberg
Lars Hupel
> when I use it. People don't name their functions 'foo', they name them
> 'formatHD', 'readFromDB' etc. I can't imagine a situation where I'd write a
> function named 'factorial' and inside have code to format the HD.
Sorry, but that doesn't work. I have stumbled upon so many seemingly non
side-effecting methods with "innocent" names which did some really
fundamental state changes that I came to the conclusion that anything
which is not "documented" in the types harms development.
> For the same reasons, a function 'printMsg: String => IO[Unit]' can format my HD
> just as well
You are mixing two problems. Admittedly, putting so many different side
effects into `IO` is not a good idea. IMHO it would be good to separate
different effects into `Filesystem` or `Network` etc.
> If IO[_] is just a tag to mean a function is dangerous, then I'm fine with this
> approach (it is similar to how Option is an alternative to document the possible
> return of null), but then why go through the hassle of a monad? I can have:
> case class Dangerous[A](value: A)
This smells like you want to re-invent monads, as `Dangerous` can be
treated as a monad. I challenge you to write an API using that class and
I bet that you will use some monadic patterns with or without knowing it.
Jesper Nordenberg
> This is true in theory, but in practice, I usually know what a
> function does when I use it. People don't name their functions
> 'foo', they name them 'formatHD', 'readFromDB' etc. I can't imagine
> a situation where I'd write a function named 'factorial' and inside
> have code to format the HD.
> For the same reasons, a function 'printMsg: String => IO[Unit]'
> can format my HD just as well
Fine, you obviously don't need and value compile time checking. Personally,
I value it very high. The IO monad may not be most precise and user friendly
way to encode side effects, but it's a simple solution and compared to having
no effects tracking I would take it any day.
> If IO[_] is just a tag to mean a function is dangerous, then I'm
> fine with this approach (it is similar to how Option is an
> alternative to document the possible return of null), but then why
> go through the hassle of a monad? I can have:
> case class Dangerous[A](value: A)
What's stopping you from doing: def makeSafe[A](d : Dangerous[A]) : A =
d.value? You do realize that many side effects are transitive?
> But when I test I know the function I am testing. It's not a random
> thing. Testing is a very big thing in the Java world, people manage
> to do it fine. Sometime they manage the difference between a test
> environment and a production environment with DI (so a fetchUser
> function will work with a real database in production, but be tested
> with a mock database)
Really? So you never test an abstract interface method without knowing the
details of the implementation?
That people in the Java world manage to do proper testing is a huge
overstatement. I would rather say that the opposite is true.
> True, but this can cause the reverse effect, like large memory
> consumption, thrashing of the cpu etc. In practice, I'd rather have
> code that works plainly and then optimize the bottlenecks I find,
> controlling how I do it. Of course it is easier to optimize pure
> code (e.g., scala's parallel collections). Btw, when the code is not
> pure I might need to optimize it manually anyway, so I need to use a
> profiler anyway.
So, your argument is that because you can't optimize some parts of the code
you might as well not do it for any part? Or maybe you're totally against
compiler optimizations? The JVM is not the right platform for you then.
/Jesper Nordenberg
Jesper Nordenberg
> So the only thing IO provides is that code *not* using it is certified
> to be pure, whereas in Scala everything is in the IO monad.
That's quite a huge thing, don't you agree?
/Jesper Nordenberg
Ittay Dror
Lars Hupel wrote:
This is true in theory, but in practice, I usually know what a function does when I use it. People don't name their functions 'foo', they name them 'formatHD', 'readFromDB' etc. I can't imagine a situation where I'd write a function named 'factorial' and inside have code to format the HD.Sorry, but that doesn't work. I have stumbled upon so many seemingly non side-effecting methods with "innocent" names which did some really fundamental state changes that I came to the conclusion that anything which is not "documented" in the types harms development.
For the same reasons, a function 'printMsg: String => IO[Unit]' can format my HD just as wellYou are mixing two problems. Admittedly, putting so many different side effects into `IO` is not a good idea. IMHO it would be good to separate different effects into `Filesystem` or `Network` etc.If IO[_] is just a tag to mean a function is dangerous, then I'm fine with this approach (it is similar to how Option is an alternative to document the possible return of null), but then why go through the hassle of a monad? I can have: case class Dangerous[A](value: A)This smells like you want to re-invent monads, as `Dangerous` can be treated as a monad. I challenge you to write an API using that class and I bet that you will use some monadic patterns with or without knowing it.
println(bar * 2) // doesn't compile, bar is of type Dangerous[Int]
// an ahha moment, realizing foo is dangerous. check what foo actually does and then, after making sure all is well:
println(bar.value * 2)
So I need to jump through a hoop to use foo, but it is a very small one. I don't need to use bind, or propogate Dangerous up the call trace or anything like that.
Ittay Dror
Jesper Nordenberg wrote:
Ittay Dror <ittay.dror <at> gmail.com> writes:This is true in theory, but in practice, I usually know what a function does when I use it. People don't name their functions 'foo', they name them 'formatHD', 'readFromDB' etc. I can't imagine a situation where I'd write a function named 'factorial' and inside have code to format the HD. For the same reasons, a function 'printMsg: String => IO[Unit]' can format my HD just as wellFine, you obviously don't need and value compile time checking. Personally, I value it very high. The IO monad may not be most precise and user friendly way to encode side effects, but it's a simple solution and compared to having no effects tracking I would take it any day.
If IO[_] is just a tag to mean a function is dangerous, then I'm fine with this approach (it is similar to how Option is an alternative to document the possible return of null), but then why go through the hassle of a monad? I can have: case class Dangerous[A](value: A)What's stopping you from doing: def makeSafe[A](d : Dangerous[A]) : A = d.value? You do realize that many side effects are transitive?
But when I test I know the function I am testing. It's not a random thing. Testing is a very big thing in the Java world, people manage to do it fine. Sometime they manage the difference between a test environment and a production environment with DI (so a fetchUser function will work with a real database in production, but be tested with a mock database)Really? So you never test an abstract interface method without knowing the details of the implementation?
as a "pragmatic" example, the command-query segregation principle is a nice one: if a function returns a value it must be pure, if it returns Unit, it means it has side effects. This is of course another way of "tagging".
That people in the Java world manage to do proper testing is a huge overstatement. I would rather say that the opposite is true.True, but this can cause the reverse effect, like large memory consumption, thrashing of the cpu etc. In practice, I'd rather have code that works plainly and then optimize the bottlenecks I find, controlling how I do it. Of course it is easier to optimize pure code (e.g., scala's parallel collections). Btw, when the code is not pure I might need to optimize it manually anyway, so I need to use a profiler anyway.So, your argument is that because you can't optimize some parts of the code you might as well not do it for any part? Or maybe you're totally against compiler optimizations? The JVM is not the right platform for you then.
Anyways, this is getting into what I call a theological argument, with fuzzy arguments, so lets stop here. If you can show me a piece of code
/Jesper Nordenberg
martin odersky
Sure. But I think there will at some point be better ways to track
effects than monads. Monads are too
rigid. Adding a single println somewhere means you have to rewrite
your whole program.
-- Martin
Tony Morris
This thread is epic.
Jesper Nordenberg
> This thread is epic.
Tony, I was just wondering when you would chime in. Care to enlighten us with
your views on the epicness of this thread and the IO monad?
/Jesper Nordenberg
Patrik Andersson
Lars Hupel
> println(bar * 2) // doesn't compile, bar is of type Dangerous[Int]
> // an ahha moment, realizing foo is dangerous. check what foo actually does and
> then, after making sure all is well:
> println(bar.value * 2)
>
> So I need to jump through a hoop to use foo, but it is a very small one. I don't
> need to use bind, or propogate Dangerous up the call trace or anything like that.
Same argument would apply to `Option`. As a careful programmer, you
would always check `x.isDefined` before calling `x.get`, wouldn't you?
Actually, that would be the same as using `null`. Instead, we have that
higher level abstraction called "catamorphism" (or use the pattern
matching instead if you wish) which allows the compiler to check what
you are doing. Same thing for side effects.
Dean Wampler
> Ittay Dror skrev 2011-10-10 07:16:
>>
> That's basically the same question as "what does a type system buy me?". It
> enables automatic detection of programming errors at compile time rather
> than at runtime.
>
> It also opens up opportunities for performance optimizations and makes your
> code a whole lot easier to test.
>
Types also document aspects of the program for the reader.
--
Dean Wampler
"Functional Programming for Java Developers" and
"Programming Scala" (O'Reilly)
twitter: @deanwampler, @chicagoscala
http://polyglotprogramming.com
Lars Hupel
> buy protection against serious (side-effecting) bugs. How easy it is to
> locate such bugs is going to come down to the usual suspects such as: Did
> the person separate, for example, IO stuff into a single module which can be
> debugged? If not then it happens all over the place and the whole program
> runs within the IO monad.
The usual approach is to write as many parts of your program as pure
functions and as few functions as possible using IO. That turns out to
work quite well.
Lars Hupel
Documentation which cannot get out of sync with the code and can be
reasoned about. The latter might seem like a totally academic idea, but
is extremely useful.
Lars Hupel
> functions just "assemble" a computation which then gets executed only
> by calling main. So instead of Monadic bind, Scala has
>
> val x = a; b
>
> On the type level, every type T would be interpreted as IO[T]. Every
> function application is a bind. And voila! No side effects!
But you agree that this interpretation is not useful at all? Having no
possibility to choose whether to use IO or not does not make an effect
system.
martin odersky
Of course.
-- Martin
Ittay Dror
With IO, I must propagate it up. Not only is this inconvenient, it also hurts abstraction. I know formatting the HD is a horrific thought, but I never encounter this. I do encounter reading from files and maybe creating them maybe even deletion. When I do that, I don't want this implementation detail to propagate up my whole program. e.g., If I have a fetchUser(userName) method, I wish for it to return User, regardless if it is a constant function (e.g. for testing), a partially applied one with a list of Users, or one that reads from the DB or from the filesystem. I would be happy if the compiler warned me before reading from the DB, so I'd make sure I'm doing it right, but then get out of my way once I've made sure my code is sound.
Daniel Sobral
>> This is true in theory, but in practice, I usually know what a function does
>> when I use it. People don't name their functions 'foo', they name them
>> 'formatHD', 'readFromDB' etc. I can't imagine a situation where I'd write a
>> function named 'factorial' and inside have code to format the HD.
>
> Sorry, but that doesn't work. I have stumbled upon so many seemingly non
> side-effecting methods with "innocent" names which did some really
> fundamental state changes that I came to the conclusion that anything
> which is not "documented" in the types harms development.
Word.
def isScalaLang(url: java.net.URL): Boolean = url == new
java.net.URL("http://www.scala-lang.org")
--
Daniel C. Sobral
I travel to the future all the time.
Lars Hupel
> some low level function returns Option, it doesn't mean every client of it needs
> to also return Option.
That's only true because `Option` is not "opaque" in its definition. If
`IO` had a catamorphism you also had a way out.
> With IO, I must propagate it up. Not only is this inconvenient, it also hurts
> abstraction. I know formatting the HD is a horrific thought, but I never
> encounter this. I do encounter reading from files and maybe creating them maybe
> even deletion. When I do that, I don't want this implementation detail to
> propagate up my whole program. e.g., If I have a fetchUser(userName) method, I
> wish for it to return User, regardless if it is a constant function (e.g. for
> testing), a partially applied one with a list of Users, or one that reads from
> the DB or from the filesystem. I would be happy if the compiler warned me before
> reading from the DB, so I'd make sure I'm doing it right, but then get out of my
> way once I've made sure my code is sound.
That's not entirely true. If you want to both side-effecting and not
side-effecting functions uniformly, you could simply lift the latter to
the former.
However, I see a point (also what Martin said) that Monads are too
"rigid", i. e. you have to propagate the use, but as long as there are
no other formalisms which doesn't suffer from that "problem" (if you
wish to call it so), `IO` is a reasonable solution.
As I said, the usual approach is that you strictly separate both types
of functions and transform the side effects with your pure functions.
`bind` and `fmap` are there for that.
Ittay Dror
Lars Hupel wrote:
With Option, I have a way out. I can write 'someOption getOrElse default'. So if some low level function returns Option, it doesn't mean every client of it needs to also return Option.That's only true because `Option` is not "opaque" in its definition. If `IO` had a catamorphism you also had a way out.
With IO, I must propagate it up. Not only is this inconvenient, it also hurts abstraction. I know formatting the HD is a horrific thought, but I never encounter this. I do encounter reading from files and maybe creating them maybe even deletion. When I do that, I don't want this implementation detail to propagate up my whole program. e.g., If I have a fetchUser(userName) method, I wish for it to return User, regardless if it is a constant function (e.g. for testing), a partially applied one with a list of Users, or one that reads from the DB or from the filesystem. I would be happy if the compiler warned me before reading from the DB, so I'd make sure I'm doing it right, but then get out of my way once I've made sure my code is sound.That's not entirely true. If you want to both side-effecting and not side-effecting functions uniformly, you could simply lift the latter to the former. However, I see a point (also what Martin said) that Monads are too "rigid", i. e. you have to propagate the use, but as long as there are no other formalisms which doesn't suffer from that "problem" (if you wish to call it so), `IO` is a reasonable solution.
Daniel Sobral
Better to propagate up than down.
To see the difference, imagine that all IO methods in Java had
something like a @throws clause (let's call it @io), except that it
could not be caught. One can easily see that it propagates up. Now
consider a Function class -- how would the apply method be annotated?
If it doesn't have an @io annotation, it can call anything annotated
with @io. If it has @io, then everything calling it will have to be
annotated with @io. Consider what this would do to a collections
library... What is happening is that this tagging is propagating
*down* to methods that, otherwise, would not need to know anything
about it.
With an IO Monad, I can pass the whole monad to any method, and it
won't care less. So while it has to propagate "up" to the "main", it
does not need to propagate "down" to all methods.
Lukas Rytz
Better to propagate up than down.
To see the difference, imagine that all IO methods in Java had
something like a @throws clause (let's call it @io), except that it
could not be caught. One can easily see that it propagates up. Now
consider a Function class -- how would the apply method be annotated?
If it doesn't have an @io annotation, it can call anything annotated
with @io. If it has @io, then everything calling it will have to be
annotated with @io. Consider what this would do to a collections
library... What is happening is that this tagging is propagating
*down* to methods that, otherwise, would not need to know anything
about it.
With an IO Monad, I can pass the whole monad to any method, and it
won't care less. So while it has to propagate "up" to the "main", it
does not need to propagate "down" to all methods.
Lars Hupel
> see `map` and `mapM` in haskell
See documentation: `mapM` is just `sequence . map`. `sequence` exists
precisely for that.
Daniel Sobral
>
>> With an IO Monad, I can pass the whole monad to any method, and it
>> won't care less. So while it has to propagate "up" to the "main", it
>> does not need to propagate "down" to all methods.
>
> But you need to duplicate methods to work in both pure and monadic world,
> see `map` and `mapM` in haskell
> (http://stackoverflow.com/questions/932639/haskell-cant-use-map-putstrln)
Let the Haskellites correct me if needed, but all I saw there was a
simple type mismatch.
Runar Bjarnason
You gain two things.
First, you gain modularity by separating concerns. In the case of the IO data type, you are separating the concern of what to do from how to do it and when to do it. When you say service.fetchData, you're simply constructing a value of the IO data type. Think of this in terms of a more familiar data type: List. When you construct the list List(1,2), observe what you're not doing. You're not mutating a variable to first be 1 and then be 2, with some intervening logic. No, you have separated concerns. You're saying that there may be a computation later that will receive first the value 1 and then the value 2, and it will combine them in some yet-to-be-defined way. In exactly the same sense, a value of type IO[String] is just data that will be consumed by some computation (e.g. unsafePerformIO) later. That computation might write to files and launch missiles, but then again it might not. It all depends on the implementation of the IO data type, which you don't need to care about.
Secondly, you gain compositionality because your IO actions (which are just ordinary data) can be composed with other actions in a completely predictable way. And I don't mean that you can copy and paste. I mean that since your IO actions are just first-class values, they can be composed arbitrarily to form more complex programs. What's more, such composition can be understood by local reasoning alone. You don't need to have an understanding of your entire program in order to understand what happens when you compose one IO action with another. To illustrate, this let's go back to the familiar List. A value of type List[IO[String]] can be turned into a value of type IO[List[String]] by composing all the actions in the list:
ios.foldRight(io(Nil))((a, b) => for { t <- b; h <- a } yield h :: t)
Note that we only needed to know how to compose two IO actions, and that was enough to compose the entire list.
This brings us back to modularity. There is nothing particularly "IO" about this code. So we can abstract that part out and re-use it for, not just IO, but all monads:
def sequence[M[_]:Monad, A](xs: List[M[A]]): M[List[A]] =
xs.foldRight(Nil.pure[M])((a, b) => for { t <- b; h <- a } yield h :: t)
Of course, you could argue that it's somehow "simpler" to reinvent this functionality, in different ways, all over your code. It's a prevalent fallacy that it is simpler to not abstract. But you are in possession of a language, Scala, that makes abstraction easy. Why not take advantage of it?
"Abstract ideas are conceptual integrations which subsume an incalculable number of concretes—and without abstract ideas you would not be able to deal with concrete, particular, real-life problems. You would be in the position of a newborn infant, to whom every object is a unique, unprecedented phenomenon."
Runar Bjarnason
Meredith Gregory
The fact that it's possible to defeat an effect system (for example, in the presence of unsafePerformIO) does not mean that it's not useful. Just like the type system remains useful even though it's possible to defeat that (e.g. in the presence of asInstanceOf).
L.G. Meredith
Managing Partner
Biosimilarity LLC
7329 39th Ave SW
nicola...@gmail.com
The fact that it's possible to defeat an effect system (for example, in the presence of unsafePerformIO) does not mean that it's not useful. Just like the type system remains useful even though it's possible to defeat that (e.g. in the presence of asInstanceOf).
Rex Kerr
Thanks for the detailed reply--I don't know how Ittay feels about the response, but it leaves me just as profoundly unconvinced that IO monads per se are worthwhile in almost any typical case (see below).
First, you gain modularity by separating concerns. In the case of the IO data type, you are separating the concern of what to do from how to do it and when to do it. When you say service.fetchData, you're simply constructing a value of the IO data type.
But there isn't one thing which is "an IO data type". You might have text output or binary output or Base64 encoded output or some other weird thing. It might be for a log file or a blog post or a database record and these might be the same data type but formatted differently. The key is transforming from your data type to/from text or binary or Base64 or whatnot (and not mixing your output types/formats up); the IO container adds nothing beyond a flag that states that in this particular case, you intend the data for IO instead of something else. You could mark any part of your program with types in this way, whether it's eventually going to be side-effecting or not, with equal (dis)utility. Foldable data type transformers are useful, but that has nothing to do with IO specifically. Thus, this position for separation does not seem to me to be at an overwhelmingly useful place.
Secondly, you gain compositionality because your IO actions (which are just ordinary data) can be composed with other actions in a completely predictable way. And I don't mean that you can copy and paste. I mean that since your IO actions are just first-class values, they can be composed arbitrarily to form more complex programs. What's more, such composition can be understood by local reasoning alone. You don't need to have an understanding of your entire program in order to understand what happens when you compose one IO action with another.
You know, you also don't need to have this understanding if you add an extra println. You _do_ need to know in what order things happen, but you _also_ need to know in what order you've built your IO operations. You gain nothing in simple cases. In complex cases, such as with reading and overwriting records on disk using seeks, you _could_ use the extra level of abstraction to sneak in a layer of caching that avoids too much disk churn...but you could just as well if you gave people a println-style interface and a flush(). Only if all IO computations are lazy do you have potential for benefit: you can reorder large amounts of data without having to cache large amounts of data in memory. But again, this is a pretty specialized use-case.
To illustrate, this let's go back to the familiar List. A value of type List[IO[String]] can be turned into a value of type IO[List[String]] by composing all the actions in the list:
ios.foldRight(io(Nil))((a, b) => for { t <- b; h <- a } yield h :: t)
Note that we only needed to know how to compose two IO actions, and that was enough to compose the entire list.
This brings us back to modularity. There is nothing particularly "IO" about this code. So we can abstract that part out and re-use it for, not just IO, but all monads:
def sequence[M[_]:Monad, A](xs: List[M[A]]): M[List[A]] =
xs.foldRight(Nil.pure[M])((a, b) => for { t <- b; h <- a } yield h :: t)
Of course, you could argue that it's somehow "simpler" to reinvent this functionality, in different ways, all over your code. It's a prevalent fallacy that it is simpler to not abstract. But you are in possession of a language, Scala, that makes abstraction easy. Why not take advantage of it?
You should take advantage of it, but it has nothing to do with IO. You can take advantage of this for data transformation operations without any IO-specific anything, and even with having inverse-sequence be full of side-effecting IO operations.
--Rex
Ittay Dror
+1
martin odersky
The argument boils down to: It's very useful to have a way to compose
actions. But it's not specific to IO in any way. You can just as well
compose closures in a language without the IO monad. What's specific
about IO is that it marks the underlying action as having a particular
kind of effect.
I argue there are better, more polymorphic, ways to do this than monads
Cheers
-- Martin
Ittay Dror
To add to what Rex had to say:
* monads are useful. they provide common abstraction for use case of a value in a context (btw, in that respect, sequence is my favorite example since without the use of applicative, it is hard to imagine how one can turn a List[X[A]] to X[List[A]]). IO is a special case in that it is pervasive.
* unlike other monads, it is hard for me to find what additional important information IO adds. Option[_] for example documents the fact that an equivalent of null may be returned. IO[_] documents that some side effect can happen, so what? The CPU may heat up and more IO instances take more memory, do we document these effects?
* say I have two programs, one touches some configuration file, deep inside its logic, another opens a socket to read some data, again, deep inside. Both return IO of something. Now what does this IO tell me? nothing. I can't tell what side effect they have made. I only know they have made something. But won't any sufficiently large app do some kind of side effect? to me, IO is like those Java programs that deal with a function that throws an exception, then catch this exception and throw an Exception instance (otherwise, they break encapsulation), so now i have 2 functions that document they may throw Exception. What can I do with this? nothing, just call them one after the other, hope for the best and maybe wrap in a 'catch'.
* I can compose functions that hide the side effect by just composing them
* I don't mind having IO to tag that a low-level function is doing side-effect. Just as long as I can escape it when I want to (like I can use catamorphism on Option or List). But then is it so useful?
Ittay
Razvan Cojocaru
Thanks,
Ittay Dror
Razvan Cojocaru wrote:
Gents, isn't the STM monad a better fit for this discussion than the IO? There might be less questions as to it's intinsic value and more focus on the monadic constructs and alternatives.
Martin, you stated that possibility a few times already - I'm sure we're all really curious now about what you have in mind :)
Also, since monads don't compose etc, why hasn't the discussion mutate to Applicative?
Ittay
Razvan Cojocaru
Thanks,
Runar Bjarnason
* unlike other monads, it is hard for me to find what additional important information IO adds. Option[_] for example documents the fact that an equivalent of null may be returned. IO[_] documents that some side effect can happen, so what? The CPU may heat up and more IO instances take more memory, do we document these effects?
Maybe, maybe not. A value of type IO[A] is just a program that produces a value of type A, given some IO subsystem that knows how to run it. The type doesn't tell you what the program does, only what it is.
That's not to say that you couldn't inspect an IO program. You could have an IO data type with a finite set of instructions, and a function that tells you what such a program does without running it. There's nothing about the IO type that tells you how it's implemented, and this is a good thing. That is an implementation detail over which we want to abstract.
* say I have two programs, one touches some configuration file, deep inside its logic, another opens a socket to read some data, again, deep inside. Both return IO of something. Now what does this IO tell me? nothing. I can't tell what side effect they have made.
That might be important if you want to be able to replace one with the other in your program. Also note that when you're in IO, you have effects but not side-effects. This is an important distinction.
List[Int] does not tell you which numbers are in your list. It only tells you that you have zero or more integers. But the fact that List doesn't give us all the information we might want doesn't mean that we should never use List.
* I can compose functions that hide the side effect by just composing them
For very trivial cases, maybe. But there will come a time when you want to use a function and you cannot, because it has a hidden dependency on JDBC, or a logging framework, or some other I/O having occurred before it. Your reasoning is basically: "I can write code that isn't modular but works just fine". Sure. But if you can easily separate concerns, why wouldn't you?
* I don't mind having IO to tag that a low-level function is doing side-effect. Just as long as I can escape it when I want to (like I can use catamorphism on Option or List). But then is it so useful?
Jim Powers
I'd love to see a code example where the introduction of IO[_] makes more errors detectable. In the code example I used in the first message, I couldn't find where it helps.
- The evaluation of A.foo(f) cannot possibly effect the results of A.bar(b) and therefore can trivially be eliminated (optimization!) as dead code
- The evaluation of A.bar(b) cannot possibly alter the result returned bu A.foo(f) otherwise A#foo type signature would indicate that
Sébastien Bocq
Sure. But I think there will at some point be better ways to trackOn Mon, Oct 10, 2011 at 12:56 PM, Jesper Nordenberg <mega...@yahoo.com> wrote:
> martin odersky <martin.odersky <at> epfl.ch> writes:
>> So the only thing IO provides is that code *not* using it is certified
>> to be pure, whereas in Scala everything is in the IO monad.
>
> That's quite a huge thing, don't you agree?
effects than monads. Monads are too
rigid. Adding a single println somewhere means you have to rewrite
your whole program.
-- Martin
Your earlier example:
val x = a; b
What can't be expressed by users at language level will inevitably end up in annotations, or built-in the compiler or in a runtime imposed by the compiler. Please don't overlook monad and functional friends. Good support for these is also a measure of how expressive Scala really is.
Kind regards,
Sébastien
P.S. You're right for println, I find it also really useful for debug.
Jim Powers
effects than monads. Monads are tooSure. But I think there will at some point be better ways to track
rigid. Adding a single println somewhere means you have to rewrite
your whole program.
- Most of the programming world either cannot cope with the likes of the IO monad and therefore ignore it at their peril. OR
- Some languages embrace models that work based on decades of research that bring useful tools to bear on reasoning about programs with interacting with "the real world" (IO/effects).
Ittay Dror
Thank you for this excellent example.
All your points are valid, and very true. The question is not
whether we gain something by using the IO monad. It is whether we
gain enough. Since using it comes with its own price of trickier
code, and mental hoops to jump through, spending brain CPU on how
to deal with the monad, rather than the business problem.
Normally, one does not work with functions without knowing what they do. When I know what a function is supposed to do, I usually know what side effects it is doing, or, I don't care because they are an implementation detail. If they were not, then all I know is that something is going on under there, but can't do anything about it.
Ittay
Rex Kerr
In particular, you have not given an example that shows why it is a bad idea to inject new state with IO but deal with that state functionally thereafter, or to output existing state, treated functionally prior to that point, with IO. Just leave off the IO marker and all your points remain; you can do all the same things with unwrapped and/or Option-wrapped types (depending on whether or not the result could fail to exist).
The minimal example where the IO monad is actually better than the _most sensible alternative_ (not a mythically messy imperative situation) seems to me to require an awfully complicated setup, if it even exists at all. (I suspect based on the logic of the arguments that it does exist, but I have yet to see an example that would not have been simpler without the IO monad specifically.)
--Rex
Jim Powers
Thank you for this excellent example.
All your points are valid, and very true. The question is not whether we gain something by using the IO monad. It is whether we gain enough. Since using it comes with its own price of trickier code, and mental hoops to jump through, spending brain CPU on how to deal with the monad, rather than the business problem.
Normally, one does not work with functions without knowing what they do. When I know what a function is supposed to do, I usually know what side effects it is doing, or, I don't care because they are an implementation detail. If they were not, then all I know is that something is going on under there, but can't do anything about it.
Runar Bjarnason
We are using an IO-like monad to isolate database effects where I work. We're also using a separate State-like monad to compose computations that have state (instead of using mutable state). We find this to be a huge benefit. Especially when we can re-use the same code for both things that seemingly have nothing to do with each other. The code that is IO-specific is very short and confined to one or two files.
In practice, once you have factored out this concern, almost all of your code is pure. I.e. You don't see "IO" in very many places. It's not at all true that most code has to have side-effects.
The argument I'm hearing is that IO is somehow special. A "cross-cutting concern". Bullshit. It's just another DSL.
I'm kind of done with "oh, but don't you want to convince me?!" No, I really don't. The best you can do is try it both ways, know what you're talking about first hand, and then decide for yourself. There's no magic in monads. It's just separation of concerns and combination of components according to a simple interface.
Jim Powers
Thanks for the examples, but...you didn't actually do anything IO-specific.
You made more or less the same point three times, and independently of those points decorated some types with IO.
In particular, you have not given an example that shows why it is a bad idea to inject new state with IO but deal with that state functionally thereafter, or to output existing state, treated functionally prior to that point, with IO. Just leave off the IO marker and all your points remain; you can do all the same things with unwrapped and/or Option-wrapped types (depending on whether or not the result could fail to exist).
The minimal example where the IO monad is actually better than the _most sensible alternative_ (not a mythically messy imperative situation) seems to me to require an awfully complicated setup, if it even exists at all. (I suspect based on the logic of the arguments that it does exist, but I have yet to see an example that would not have been simpler without the IO monad specifically.)
Runar Bjarnason
Adriaan Moors
martin odersky
> I'm not aware of a "sensible alternative", I'd be happy to hear of one.
> Martin is claiming such a thing exists, it would be nice if is work was
> fruitful. I would suspect that if he is he would be ushering in an whole
> new branch of mathematics :-)
>
It's called research.
-- Martin
--
Martin Odersky
Prof., EPFL and Chairman, Typesafe
PSED, 1015 Lausanne, Switzerland
Tel. EPFL: +41 21 693 6863
Tel. Typesafe: +41 21 691 4967
martin odersky
> I want to know more about these other, more polymorphic ways. Do you have some papers handy that I could read?
>
Start with "The Marriage of effects and monads". Then, add
polymorphism to the effect system. This is still too complicated for
my taste, so we have to find simpler solutions. It's research, which
comes before published papers.
-- Martin
Ittay Dror
Runar Bjarnason wrote:
Well, that's true with anything, that you have to decide for yourself if it's worth the trouble to understand. Before you can say "this is beneficial" you have to ask "beneficial for whom and for what purpose?" We are using an IO-like monad to isolate database effects where I work. We're also using a separate State-like monad to compose computations that have state (instead of using mutable state). We find this to be a huge benefit. Especially when we can re-use the same code for both things that seemingly have nothing to do with each other. The code that is IO-specific is very short and confined to one or two files.
Runar Bjarnason
Interesting. Maybe I missed something. I thought that when you use IO you need to propagate it up to 'main' (or 'run'). So one function that does IO (a println) pollutes your whole program. Where am I wrong?
In practise, you compose effectful things in two ways:
for {
x <- effectsHere
y <- effectsHereToo(x)
} yield noEffectsHere(y)
or...
(effectsHere |@| effectsHereToo)((a, b) => noEffectsHere(a, b))
You'll find that most of your code is in "noEffectsHere". It doesn't need to be aware that it can be lifted into IO. This is what separation of concerns is all about.
Derek Williams
On Oct 11, 2011 7:56 AM, "Ittay Dror" <ittay...@gmail.com> wrote:
>
>
>
> Runar Bjarnason wrote:
>>
>> Well, that's true with anything, that you have to decide for yourself if it's worth the trouble to understand. Before you can say "this is beneficial" you have to ask "beneficial for whom and for what purpose?"
>>
>> We are using an IO-like monad to isolate database effects where I work. We're also using a separate State-like monad to compose computations that have state (instead of using mutable state). We find this to be a huge benefit. Especially when we can re-use the same code for both things that seemingly have nothing to do with each other. The code that is IO-specific is very short and confined to one or two files.
>
> Interesting. Maybe I missed something. I thought that when you use IO you need to propagate it up to 'main' (or 'run'). So one function that does IO (a println) pollutes your whole program. Where am I wrong?
As someone who hasn't gotten much further then LYAH, I'll see if I can explain (as much for my benefit as anything else):
Just like the fact that you can use the value stored within an Option using map or flatMap, the value within IO can be passed to pure functions as well.
If you consider the previous example of adding 2 numbers that are retrieved by some IO, you do not need '+' to be aware of the source of these numbers. Ideally you would keep the IO type as close to your 'main' method as possible, using pure functions to actually do something with the results of the IO.
This results in a program where you naturally tend to separate your IO from the functions that process that data, hopefully leaving the majority of your program IO free.
--
Derek
Ittay Dror
Runar Bjarnason wrote:
On Tue, Oct 11, 2011 at 9:55 AM, Ittay Dror <ittay...@gmail.com> wrote:Right there.
Interesting. Maybe I missed something. I thought that when you use IO you need to propagate it up to 'main' (or 'run'). So one function that does IO (a println) pollutes your whole program. Where am I wrong?
In practise, you compose effectful things in two ways:
for {
x <- effectsHere
y <- effectsHereToo(x)
} yield noEffectsHere(y)
Runar Bjarnason
On Monday, October 10, 2011 3:54:05 PM UTC-4, Ittay Dror wrote:
Because someone has to generate the applicative. Once you have 2 functions that takes normal arguments and produce an Applicative, that applicative needs to be a monad for the functions to compose. AFAIU, monads don't compose in that M[_] and N[_] cannot be composed to M[N[_]], which is why all side effects (reading a file, opening a socket, querying a DB) all use IO and not a more specific monad (like FileIO, SocketAction etc.), if they did, and you needed to use to such functions with different monads you'd need to compose them, but you would not be able to
This is not true. It's entirely possible for you to have a more granular DSL for your IO actions than simply the opaque "IO". And they can be monads individually, and they can commute with each other easily. For example, look at the MonadIO type class from the Haskell libraries. It's specifically for this purpose, to allow "io-like" monads to commute.
Don't take it to mean more than it does when people say that "monads don't compose". What that really means is this: If M[_] is a monad, and N[_] is a monad, then M[N[_]] may not be a monad. But it often is, and it depends on what M and N are. The implementation of the M[N[_]] monad then depends on the nature of one or the other monad, either M or N.
Applicative functors, on the other hand, always compose. If M[_] is Applicative, and N[_] is Applicative, then M[N[_]] is Applicative too, in a completely mechanical way that in no way depends on what M and N are. Note that all monads are applicative functors, but not all applicative functors are monads. So you can say that all monads M and N compose to form an applicative functor M[N[_]].
What is the difference between applicative functors and monads then? It's very simple. If you have two "effectful" functions, f:A=>M[B], and g:B=>M[C], then if you compose them, the "effect" produced by g depends on the value returned by f. So M has to be a monad if you want to compose them. But if you have two "effectful" values a:M[A] and b:M[B], then you can compose them with a "pure" function p:(A,B)=>C to form a result of type M[C]. In that case you only need an applicative functor because there is no dependency between the effects.
Runar Bjarnason
On Tuesday, October 11, 2011 10:20:42 AM UTC-4, Ittay Dror wrote:
Lets take a more concrete example. I have a function fetchUser: String => IO[User]. What you're suggesting is that this method needs to be used at 'main', to get the User instance and pass it to the rest of my application ('noEffectsHere'), right? So I do all my IO upfront, and then call the rest of my code?
No, that's not being suggested at all, and I'm not sure what gives you that idea. Consider:
fetchUser(s) map f
Both s and f can be pure and neither one of them needs to know anything at all about IO. The vast majority of code in real applications falls in this category. In fact, I'll bet that almost none of the code invoked by "fetchUser" has anything at all to do with IO either.
Dennis Haupt
you'll pollute the hierarchy because of that single println deep down there or you have to make a dirty cut at some point.
doing all sideeffect-stuff before or after all pure stuff is not always possible, and most likely very ugly for complex operations. i tend to just ignore side effects that "shouldn't matter much"
> -------- Original-Nachricht --------
> Datum: Tue, 11 Oct 2011 16:20:42 +0200
> Von: Ittay Dror <ittay...@gmail.com>
> An: Runar Bjarnason <runar...@gmail.com>
> CC: scala-...@googlegroups.com
> Betreff: Re: [scala-debate] Re: questioning FP
Runar Bjarnason
On Monday, October 10, 2011 3:25:11 PM UTC-4, martin odersky wrote:
You can just as well
compose closures in a language without the IO monad. What's specific
about IO is that it marks the underlying action as having a particular
kind of effect.
type IO[A] = Function0[A]
I'm on board with better typing though. Uniqueness types or "effect tagging" provide more granular types for effects. But don't seek comfort in that these are not monads, because they are. They can all be described as monadic region calculi, or delimited continuations, i.e. indexed monads. And it would be bad if we had these without the ability to exploit their monadicity in code.
Ittay Dror
Ittay Dror
Derek Williams wrote:
On Oct 11, 2011 7:56 AM, "Ittay Dror" <ittay...@gmail.com> wrote:
>
>
>
> Runar Bjarnason wrote:
>>
>> Well, that's true with anything, that you have to decide for yourself if it's worth the trouble to understand. Before you can say "this is beneficial" you have to ask "beneficial for whom and for what purpose?"
>>
>> We are using an IO-like monad to isolate database effects where I work. We're also using a separate State-like monad to compose computations that have state (instead of using mutable state). We find this to be a huge benefit. Especially when we can re-use the same code for both things that seemingly have nothing to do with each other. The code that is IO-specific is very short and confined to one or two files.
>
> Interesting. Maybe I missed something. I thought that when you use IO you need to propagate it up to 'main' (or 'run'). So one function that does IO (a println) pollutes your whole program. Where am I wrong?As someone who hasn't gotten much further then LYAH, I'll see if I can explain (as much for my benefit as anything else):
Just like the fact that you can use the value stored within an Option using map or flatMap, the value within IO can be passed to pure functions as well.
The difference is that for Option I have a catamorphism. So I can
always "get out" of using Option. With IO, once I use it, for no
matter how small an issue, I need to use it in all the call
hierarchy and change everything to work through for comprehension.
If you consider the previous example of adding 2 numbers that are retrieved by some IO, you do not need '+' to be aware of the source of these numbers. Ideally you would keep the IO type as close to your 'main' method as possible, using pure functions to actually do something with the results of the IO.
This results in a program where you naturally tend to separate your IO from the functions that process that data, hopefully leaving the majority of your program IO free.
A sane developer will do that anyway (a bad developer will muck any codebase).
Returning to my original question, using Options for example (and
monads in general) is easy to explain: you have an alternative,
more semantically true, to returning null, so your code becomes
safer (a function that returns nothing is clearly documented with
the type), and remains clean since the use of Option is usually
local (at some point you have a default value, and use getOrElse
to reduce the Option to a value). With IO, it is trickier to
explain:
First, because IO is not necessarily dangerous (unlike trying to invoke a method on null).
Second, because the IO monad does not prevent me from doing a dangerous action, just wraps it (if I try to read a non existent file, my app will fail, with or without IO[_]).
Third, it is pervasive (no catamorphism)
So the question is, what clear gain can I show?
Runar Bjarnason
On Tuesday, October 11, 2011 11:26:43 AM UTC-4, Ittay Dror wrote:
Sure, then fetchUser is mostly non-IO, but needs to return IO[User]. Similarly, the functions that use fetchUser will also need to return IO[Something]. So a small implementation detail in a function can change its signature and forces it to be used differently (inside a for comprehension).
It sounds like your apprehension is entirely syntactic. And the fact that a function requires IO is no small implementation detail. It's a very real dependency, a cause of nondeterminism, and a barrier to composition. But go ahead and use null instead of Option, exceptions instead of Either, mutable iterators instead of lists, and identity instead of IO. It's super dynamic and pragmatic.
martin odersky
-- Martin
Ittay Dror
1. With Option and Either I have catamorphism, so using them in a function does not necessarily escape the function. And they avoid the alternative (meaning, if I use Option, i will never get an NPE)
2. Using the IO monad does not prevent non-determinism. The actions will run just the same (meaning if I use IO[_] i will still get FileNotFoundException). It only adds a tagging
3. The tagging is cumbersome to use.
Ittay Dror
-- Martin
Rex Kerr
object B {def run(f:String,b:String):Int =for {a <- A.foo(f)b <- A.bar(b)} yield a+b}Is the exact same program as:object B {def run(f:String,b:String):Int =for {b <- A.bar(b)a <- A.foo(f)} yield a+b}Because in the case of a purely functional language I know that I can always commute evaluation order. Further, the computer knows this too! Substitute in the IO monad for the Option and the guarantees about commutativity go out the window.
Granted, but your earlier code didn't do anything to guarantee order, so it wasn't of any use except as a tag. And it's not a very useful tag because the tag is at the _declaration site_ not the _use site_. At least if you had to call something like Console.readLine it would be clear where the action was. If you mark inputs as being IO[String] and then unwrap them somewhere buried in the code, you don't have to know which sort of thing you're unwrapping (maybe it was an Option or some other sort of wrapper).
Granted that using functions in an IO context like above where I didn't take steps to eliminate the possibility of a bug by forcing the correct order (say through the use of a Reader or State monad) is possible.
Indeed. But if one does things this way, it is a huge hassle (of the major-rewrite-to-add-a-println variety). For specialized cases, encoding a finite state machine into your types (for instance) can be exactly what you need to keep things straight. But from what I've seen, this is a niche case when it comes to IO.
The minimal example where the IO monad is actually better than the _most sensible alternative_ (not a mythically messy imperative situation) seems to me to require an awfully complicated setup, if it even exists at all. (I suspect based on the logic of the arguments that it does exist, but I have yet to see an example that would not have been simpler without the IO monad specifically.)
I'm not aware of a "sensible alternative", I'd be happy to hear of one.
The sensible alternative is untagged side-effectful code that immediately moves data into an appropriate typesafe and perhaps immutable form.
--Rex
Runar Bjarnason
Second, because the IO monad does not prevent me from doing a dangerous action, just wraps it (if I try to read a non existent file, my app will fail, with or without IO[_]).
That having been said, I am working on getting regional IO into Scalaz's IO monad. This will guarantee, in the type system, that you can never read, write, or close a file that isn't open. Having kind polymorphism in Scala would make this much easier than it is (Martin, Adriaan, et al, take note).
Ittay Dror
Runar Bjarnason wrote:
On Tue, Oct 11, 2011 at 11:36 AM, Ittay Dror <ittay...@gmail.com> wrote:And if I take the head of a List, my app will fail too. If I divide an Int by zero, my app will fail. This doesn't mean List and Int are useless types.Second, because the IO monad does not prevent me from doing a dangerous action, just wraps it (if I try to read a non existent file, my app will fail, with or without IO[_]).
That having been said, I am working on getting regional IO into Scalaz's IO monad. This will guarantee, in the type system, that you can never read, write, or close a file that isn't open. Having kind polymorphism in Scala would make this much easier than it is (Martin, Adriaan, et al, take note).
Runar Bjarnason
I thought that similar to how Option guards us from nulls, IO guards us from side-effects (Int is not supposed to guard us from division.). If it doesn't, then again, what is the point? Just to tag a function as doing IO? And then tag all its callers? And make their code a bit awkward (for comprehension)? Isn't this like checked exceptions?
All I can say is "try it". I don't think you will run into all the problems you think you might, and it's not as awkward as all that (certainly not as awkward as, say, trying to retrofit to pure code a function that assumes the database is configured and open and tries to read from it). You don't ever need to "get out" of IO. You can always take a function of type A => B and turn it into a function of type IO[A] => IO[B].
Jim Powers
As someone who hasn't gotten much further then LYAH, I'll see if I can explain (as much for my benefit as anything else):
Just like the fact that you can use the value stored within an Option using map or flatMap, the value within IO can be passed to pure functions as well.
The difference is that for Option I have a catamorphism. So I can always "get out" of using Option. With IO, once I use it, for no matter how small an issue, I need to use it in all the call hierarchy and change everything to work through for comprehension.
If you consider the previous example of adding 2 numbers that are retrieved by some IO, you do not need '+' to be aware of the source of these numbers. Ideally you would keep the IO type as close to your 'main' method as possible, using pure functions to actually do something with the results of the IO.
This results in a program where you naturally tend to separate your IO from the functions that process that data, hopefully leaving the majority of your program IO free.
A sane developer will do that anyway (a bad developer will muck any codebase).
Returning to my original question, using Options for example (and monads in general) is easy to explain: you have an alternative, more semantically true, to returning null, so your code becomes safer (a function that returns nothing is clearly documented with the type), and remains clean since the use of Option is usually local (at some point you have a default value, and use getOrElse to reduce the Option to a value). With IO, it is trickier to explain:
First, because IO is not necessarily dangerous (unlike trying to invoke a method on null).
Second, because the IO monad does not prevent me from doing a dangerous action, just wraps it (if I try to read a non existent file, my app will fail, with or without IO[_]).
Third, it is pervasive (no catamorphism)
So the question is, what clear gain can I show?
Razvan Cojocaru
Cool - It is the first time someone points out that this is like using null or exceptions… BUT, isn’t the point that this kind of IO[] wrapping creates the same kind of headaches that a checked exception does?
Runar Bjarnason
On Tuesday, October 11, 2011 3:44:12 PM UTC-4, Razvan (Pub) Cojocaru wrote:
Cool - It is the first time someone points out that this is like using null or exceptions… BUT, isn’t the point that this kind of IO[] wrapping creates the same kind of headaches that a checked exception does?
No, not at all. See here for replacing checked exceptions with a monadic data type:
http://apocalisp.wordpress.com/2008/05/16/thrower-functor/
Runar Bjarnason
On Tuesday, October 11, 2011 3:44:12 PM UTC-4, Razvan (Pub) Cojocaru wrote:
Cool - It is the first time someone points out that this is like using null or exceptions… BUT, isn’t the point that this kind of IO[] wrapping creates the same kind of headaches that a checked exception does?
martin odersky
> Cool - It is the first time someone points out that this is like using null
> or exceptions… BUT, isn’t the point that this kind of IO[] wrapping creates
> the same kind of headaches that a checked exception does?
Very good observation. That's why you want polymorphism.
-- Martin
Josh Suereth
Well, that's true with anything, that you have to decide for yourself if it's worth the trouble to understand. Before you can say "this is beneficial" you have to ask "beneficial for whom and for what purpose?"
We are using an IO-like monad to isolate database effects where I work. We're also using a separate State-like monad to compose computations that have state (instead of using mutable state). We find this to be a huge benefit. Especially when we can re-use the same code for both things that seemingly have nothing to do with each other. The code that is IO-specific is very short and confined to one or two files.
Jesper Nordenberg
> Very good observation. That's why you want polymorphism.
Can you give an example of the polymorphism you want that the IO monad
cannot provide?
/Jesper Nordenberg
martin odersky
should be effectful if f is.
-- Martin
Runar Bjarnason
xs map f
should be effectful if f is.
http://scalaz.github.com/scalaz/scalaz-2.9.1-6.0.2/doc.sxr/scalaz/Traverse.scala.html
Jesper Nordenberg
I'm not sure I agree with that. If f doesn't have side effects the map
operation can for example be performed in parallel. If f has side
effects the map operation must be performed sequentially.
So, the lack of side effects gives you less implementation restrictions.
/Jesper Nordenberg
martin odersky
I believe on parallel collections map might require its argument to be
side-effect free. But the only practical solution I see for the
foreseeable future would be to do that as part of an optional type
system. On sequential collections map should certainly be polymorphic.
Cheers
-- Martin