Can I PR a @pure annotation?

181 views
Skip to first unread message

David Barri

unread,
May 24, 2016, 12:37:14 AM5/24/16
to scala-internals
Hello. If I get the go-ahead here, I'd like to create a Scala PR.

Scala has a very helpful feature in that it issues a warning when a pure value is discarded / converted to Unit:


  > scala
  Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_92).
  Type in expressions for evaluation. Or try :help.

  scala> def blah(): Unit = 3
  <console>:11: warning: a pure expression does nothing in statement position; you may be omitting necessary parentheses
         def blah(): Unit = 3
                          ^
  blah: ()Unit


The problem is that it only seems to work for built-in types. I'd like the same warning issued for user-defined types too.
Here is an example of the problem:


  scala> def printHello = Effect(() => println("Hello"))
  printHello: Effect[Unit]

  scala> def blah(): Unit = printHello
  blah: ()Unit


A simple solution would be to be able to annotate user-defined classes as @pure (similar), and then have the compiler warn when thus-annotated types are discarded.

In one of my libraries, the same question seems to arise daily because users are accidentally having an effect type cast to Unit without realising.
If I could annotate my effect type and have the compiler issues warnings, it would greatly reduce the problems that people are having and reduce the learning curve wrt typed effects.

If room approves, I'll get to work on a PR ASAP.

Cheers,
David


Naftoli Gugenheim

unread,
May 24, 2016, 1:13:27 AM5/24/16
to scala-internals
+99

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Lukas Rytz

unread,
May 24, 2016, 3:47:14 AM5/24/16
to scala-i...@googlegroups.com

Intuitively, I think it would make more sense to annotate methods (and constructors) @pure rather than classes. In your example, how do you know that the printHello method does not perform any side-effects?

David Barri

unread,
May 24, 2016, 4:31:29 AM5/24/16
to scala-internals
On Tuesday, 24 May 2016 17:47:14 UTC+10, Lukas Rytz wrote:

Intuitively, I think it would make more sense to annotate methods (and constructors) @pure rather than classes. In your example, how do you know that the printHello method does not perform any side-effects?


That would be a different goal in that it would be establishing the purity of an entire graph. While awesome, I imagine that a pretty big task and quite noisy to code. Automatic, mechanical AST analysis would likely be better suited.

Instead, what I'm hoping to accomplish here is to mark user-defined types so that a warning is emitted when they are cast to Unit. There won't be any proof that the user created the value without side-effects, and that can be an acceptable caveat.
In fact, @pure is probably misleading in that purity isn't what is important here; @warnOnDiscard is probably better.

Lukas Rytz

unread,
May 24, 2016, 4:40:55 AM5/24/16
to scala-i...@googlegroups.com

OK. But note that is not how the existing warning works: it really checks purity
of the expression that is discarded. Adapting your example to a built-in type:

scala> def printHello = 3
printHello: Int

scala> def blah(): Unit = printHello
blah: ()Unit

There’s no warning here.





Simon Schäfer

unread,
May 24, 2016, 4:50:44 AM5/24/16
to scala-i...@googlegroups.com
There is already -Ywarn-value-discard, which would help in your use case. It is only broken for non return values:

scala> def f = 0
f: Int

scala> def x(): Unit = f
<console>:12: warning: discarded non-Unit value
       def x(): Unit = f
                       ^
x: ()Unit

scala> def x(): Unit = {f;()}
x: ()Unit

The warning should also appear in the last example.

David Barri

unread,
May 24, 2016, 5:11:35 AM5/24/16
to scala-internals
Lukas: Oh. That's an oversight on my part. I didn't realise that it doesn't emit warnings on defs.

Simon: Thanks. The problem with -Ywarn-value-discard is that most people won't use it can result in a lot of noise.

I guess I'd like to see behaviour like -Ywarn-value-discard on selected types and change it so that it also bleeps anywhere the discard occurs. Is this getting larger already?

Btw, I'm of course open to alternative solutions. The problem is that a lot of problems go unsolved, causing pain because we don't have the resources to achieve perfection. I'm hoping to find something small that we can have soon that will alleviate things until the long-term solutions appear (which are likely years away) which is why I was thinking of a "trust-me"-type manual annotation.

Also, just remembered: this could have overlap with https://github.com/scala-js/scala-js/issues/2227.

Viktor Klang

unread,
May 24, 2016, 5:18:19 AM5/24/16
to scala-i...@googlegroups.com
Can we first #define pure?

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Cheers,

Lukas Rytz

unread,
May 24, 2016, 5:31:23 AM5/24/16
to scala-i...@googlegroups.com
No change in semantics if a pure expression in statement position is not evaluated.
Or, pure expressions can be evaluated in arbitrary order with no change in semantics.
We can include non-termination or not, according to your taste :)

Viktor Klang

unread,
May 24, 2016, 5:37:49 AM5/24/16
to scala-i...@googlegroups.com
StackOverflowError if evaluated but no StackOverflowError if not evaluated == pure?

Naftoli Gugenheim

unread,
May 24, 2016, 5:39:29 AM5/24/16
to scala-i...@googlegroups.com

I think there are two different but overlapping concerns. One is forgetting you need to do something with a value. The only time I can think of that it's a big problem is, there's a confusing assumption that an effect should take place, but actually you have to pass the captured effect along for it to run. Another is coercion to Unit (by inserting () thus resulting in value discarding. This can be annoying (why didn't the compiler catch my typo) in much more places, but on its own isn't as catastrophic.

If a method returns a String or JButton or whatever and I call the method and discard it, no big deal. If you realize you need a String or whatever later, you'll figure out what to do. And if you don't feel a need to use a String, no harm done (except for some wasted CPU cycles).

There's a much more specific and catastrophic case, which is a type that captures effects, like IO and all its cousins. Because there, there's lots of grounds for confusion. You might need the value later (so it gets evaluates) and not realize it. You thought you already ran some effect. So that's much more dangerous. And that's pretty strongly tied to the type, not particular methods. For example any method that returns an IO has this potential for confusion.

This is completely unrelated to whether it should be a type mismatch to assign an Int to Unit.

Naftoli Gugenheim

unread,
May 24, 2016, 5:42:57 AM5/24/16
to scala-i...@googlegroups.com

I would call the annotation @noDiscard and make it a compiler error to discard it, actually. For those use cases, there's basically zero reason to ever discard it. And if there might be, the library author (or other) can provide a .discard combinator.

Lukas Rytz

unread,
May 24, 2016, 5:45:46 AM5/24/16
to scala-i...@googlegroups.com

I agree that -Ywarn-value-discard is probably too noise to be useful in most projects.

However, I don’t think that a type-based approach is correct in this context. As a library
author you can write a class and annotate it @warnIfDiscarded. However, anybody
using that class may write a side-effecting method that returns a value of this type.

For example, should we warn about discarding an Int or not? For example:

class Counter {
  private var v: Int
  def inc(): Int = { v += 1; v }
}

This is a sensible definition: inc returns the new value. There are situations when a
client wants to use the returned value, but also situations when it makes sense to just
discard it.

So my point is that you cannot decide on the class-author-side if it makes sense to
annotate warnIfDiscarded or not.

Lukas

On Tue, May 24, 2016 at 11:11 AM, David Barri <japg...@gmail.com> wrote:

--

Viktor Klang

unread,
May 24, 2016, 5:51:45 AM5/24/16
to scala-i...@googlegroups.com
Would be nice to annotate the type rather than the method.

def foo: Int with Importance = …
--
Cheers,

Lukas Rytz

unread,
May 24, 2016, 5:55:22 AM5/24/16
to scala-i...@googlegroups.com
StackOverflowError is related to termination. How about OOME? Also spawning a thread is an interesting case, cannot be discarded, but delayed / re-ordered without changing semantics. So, as usual, it depends.

Lukas Rytz

unread,
May 24, 2016, 5:57:53 AM5/24/16
to scala-i...@googlegroups.com
On Tue, May 24, 2016 at 11:51 AM, Viktor Klang <viktor...@gmail.com> wrote:
Would be nice to annotate the type rather than the method.

def foo: Int with Importance = …

Depends what you're after - types propagate differently than effects:

def bar = {
  foo
  33
}

is bar: Int with Importance or not?

Viktor Klang

unread,
May 24, 2016, 6:03:40 AM5/24/16
to scala-i...@googlegroups.com
Consider:
val exprA = {
  val exprB = recursiveCallWhichSOEs()
  val exprC = tailrecursiveCallWhichNeverTerminates()
  s"$exprB + $exprC"
}

Question: Which of the expressions above are considered @pure?

Kevin Wright

unread,
May 24, 2016, 6:15:53 AM5/24/16
to scala-internals
In line with `NoStackTrace`, perhaps this would be better phrased as a negative:

def foo: Int with DontForsake = …

Lukas Rytz

unread,
May 24, 2016, 6:16:35 AM5/24/16
to scala-i...@googlegroups.com
I think this cannot be answered generally, only for specific situations. ("It depends").

Kevin Wright

unread,
May 24, 2016, 6:21:34 AM5/24/16
to scala-internals
Message has been deleted

Scott Carey

unread,
May 25, 2016, 6:14:26 PM5/25/16
to scala-internals

On Tuesday, May 24, 2016 at 2:42:57 AM UTC-7, nafg wrote:

I would call the annotation @noDiscard and make it a compiler error to discard it, actually. For those use cases, there's basically zero reason to ever discard it. And if there might be, the library author (or other) can provide a .discard combinator.


Re-posting with the intended markdown formatting:

I agree with the @noDiscard idea. This is not about purity or termination and many messages here seem to have wandered off into the wilderness.

It does not apply to the ‘Int or Int with Importance’ example. @noDiscard would tag a type that the author knows represents effect capture or lazy execution. User libraries that are IO like would all happily add this if it lead to smart warnings.

I’m not sure if any basic scala type is a candidate for this. Any function type that returns Unit might be a candidate for effect capture, but such an annotation would apply to all FunctionX, not only those with Unit type params.

toy example:

def foo: () => String = () => { println("its happening!"); "blah" }
def bar(s: String): String => Unit = println

def blah(): Unit = {
  foo // not what the author intended
}

def blargh(): Unit = {
  bar("oops")
}

def nowWhat(): Unit = {
  foo
  bar("oops")
  println("done")
}

The first two above are caught by -Ywarn-value-discard , which I generally use and put up with manually adding () but some do not. The third is not, even though we discarded two suspicious values.

The first value is impossible to prove that is has an effect.
The second value is most likely an effect — A unit returning function.
In both cases, a function was created and discarded, which is suspect whether or not there is an effect. Is there a -Ywarn-function-discard ?

Below is a scalaz.concurrent.Task example that has nothing to do with converting to (), but is related to discarding a type that you really don’t want to discard since it represents lazy evaluation or effect capture

import scalaz.syntax.monad._

def debugPrint(msg: => String): Task[Unit] = Task.delay(logger.debug(msg))

def takeOffShoe: Task[Shoe] = ???

def throwShoe(shoe: Shoe): Task[Unit] = ???

def randomTask(): Task[Unit] = {
  takeOffShoe >> debugPrint("shoe taken off") flatMap { shoe =>
    throwShoe(shoe)
  }                                   // oops discarded the value here by mistake, forgot the >>  and scalac doesn't help!
  debugPrint("shoe thrown")
}

I don’t know if there is any answer to this example, but I thought it was relevant, as Task is an example of an IO like type and would most certainly work well with an @noDiscard.

An example that is in line with the original post:


def debugLog(msg: => String): Task[Unit] = Task.delay(strictLogger.debug(msg))

def blah(): Unit = {
  debugLog("doing stuff") // oops, user didn't realize the tool they are  using is capturing an effect, not running it, or its a typo.  This can  not be caught by -Ywarn-value-discard
  doStuff()
  doMoreStuff()
  debugLog("stuff done")  // oops, user didn't realize the tool they are  using is capturing an effect, not running it, or a typo  this one can be  caught by -Ywarn-value-discard
}

Which gets me back to the agreement that this seems to be about types and their intended use, not purity in general. It is also not strictly about coercion toUnit.

IMO, if Task was annoated with @noDiscard, then there would be two warnings above since a value of that type was produced but not used or stored or returned.

Lukas Wegmann

unread,
May 26, 2016, 4:33:39 AM5/26/16
to scala-internals


On Thursday, May 26, 2016 at 12:14:26 AM UTC+2, Scott Carey wrote:

I’m not sure if any basic scala type is a candidate for this. 

@noDiscard could be useful on `Try` and `Future`:

def f(): Try[Int] = Try { throw new Exception() }

def foo(): Try[Unit] = Try{
  f
() // silently discards the exception thrown
  f
() // also discards the exception but is caught by -Ywarn-value-discard
}


Lukas Rytz

unread,
Jun 30, 2016, 11:33:45 AM6/30/16
to scala-i...@googlegroups.com
For reference, here's a related thing in the Java language spec:

"Optimizing transformations of a program can be designed that reduce the number of objects that are reachable to be less than those which would naively be considered reachable. For example, a Java compiler or code generator may choose to set a variable or parameter that will no longer be used to null to cause the storage for such an object to be potentially reclaimable sooner."

This can be exploited to make a program OOME under `-Xint` but terminate when
the JVM optimizer is enabled:

import java.lang.ref.WeakReference;
import java.util.*;

public class A {
  public static void main(final String[] args) throws InterruptedException {
    Object obj = new Object();
    WeakReference<Object> ref = new WeakReference<Object>(obj);

    List<byte[]> filler = new LinkedList<byte[]>();
    while (ref.get() != null) {
        filler.add(new byte[1000]);
    }
    System.out.println("Filler size " + filler.size());
  }
}

$ java -Xint A
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at A.main(A.java:11)
$ java A
Filler size 126974





Reply all
Reply to author
Forward
0 new messages