Akka doesn't build with range positions

[Lex Spoon]

I've spent three work days trying to build Akka with range positions
turned on, and I can't justify spending more time on that right now.
This email is a summary for anyone who picks it up from where I got.

If you try, be aware that you need to get a very recent build of SBT;
until yesterday, due to the way the RangePositions trait works,
-Yrangepos wouldn't do anything.

Using such a recent build of SBT, I got one crasher in Akka that turns
out to already be fixed in Scala's master. Here's that patch:

https://github.com/scala/scala/commit/120e14fadf30b4c39f953832108d19b736dc6f2d

If it's not too late, the above patch might be a good candidate for
cherry picking into 2.10.0. It's a small and safe patch, and it
prevents a compiler crash.

That patch is not enough to build Akka, however. With that patch in, I
still get two more crashers, both of them involving synthetic methods
of case classes: one for readResolve, one for unapply. Those two still
need to be fixed, and possibly other things as well.

Going forward, I would propose consideration of phasing out -Yrangepos
and developing a single unified position implementation that does
ranges. Despite the -Y on the option, it's not really optional any
more, is it? The IDE can't be crashing! Yet, as things stand, range
positions don't get exercised as much as they should, and so they are
eternally fragile. In general, options are bad; whenever you can make
a non-optional version that is 90% as good, it's usually a good
tradeoff. Options mean an ongoing complexity in the tool and an
ongoing burden for regression testing.

My fellow Scalists, we can do this. Our compiler can do the cake
pattern, dependent methods, user-supplied pattern matchers, and
type-directed macro expansion.... but ask it to compute range
positions, and it collapses into a gibbering transporter accident?
I've got to think that a unified position implementation would be
straightforward to get working and get stable. Moreover, there are a
number of ways to improve the memory usage of range positions, and
some of them are easier with a unified implementation.

Just a thought. Anyway I thought folks would like to know what I found.

Lex Spoon

[Jason Zaugg]

I'd actually fixed this problem independentally and submitted a pull reqest to 2.10.x [1], in which I asked the same question. We'll discuss the possibility of inclusion in 2.10.0 today.

In the past [2], there was consensus the standardizing on range positions would be a good thing assuming performance doesn't suffer.

-jason

[1] https://github.com/scala/scala/pull/1701

[2] https://groups.google.com/forum/?fromgroups#!topic/scala-internals/H-ROUV5jg4I

[Paul Phillips]

On Tue, Dec 4, 2012 at 6:38 AM, Jason Zaugg <jza...@gmail.com> wrote:

In the past [2], there was consensus the standardizing on range positions would be a good thing assuming performance doesn't suffer.

The reason that dried up on the vine is that performance did suffer, at least a little. This branch should be a fine starting point, or given the presumed degree of merge drift, at least a reference point.

  https://github.com/paulp/scala/tree/topic/positions

[martin odersky]

I would assume that performance will suffer. Trees account for a lot of the memory pressure of the compiler. That's why, initially, they had the position embedded as in Int. We had to change that when we made the compiler embeddable in IDEs. With range positions, now instead of one tree node you mostly end up with two: the node proper and its position. Offset positions alleviate this to some degree because many trees share the same offset position, so there can be fewer positions than trees.

On the other hand, maybe we can use a value class (with a single long) to encode a range? With the current API, we need three ints (start, end, point) and a source reference. As the source reference is the same for all positions in a tree there should be a smarter way to encode it than put it in all positions. So that leaves three ints. We could drop point in IDE mode, then the remaining two ints would fit, but we'd again create a mode.

Cheers

 - Martin

--
Martin Odersky
Prof., EPFL and Chairman, Typesafe
PSED, 1015 Lausanne, Switzerland
Tel. EPFL: +41 21 693 6863
Tel. Typesafe: +41 21 691 4967

[Paul Phillips]

On Wed, Dec 5, 2012 at 12:30 AM, martin odersky <martin....@epfl.ch> wrote:

On the other hand, maybe we can use a value class (with a single long) to encode a range? With the current API, we need three ints (start, end, point) and a source reference. As the source reference is the same for all positions in a tree there should be a smarter way to encode it than put it in all positions. So that leaves three ints. We could drop point in IDE mode, then the remaining two ints would fit, but we'd again create a mode.

I'm not sure where the point usually lands relative to the range start/end (I assume at least "inside it") but with a 32-bit start and a 20-bit offset from that for end, there can be a megabyte of text and there's still 12 bits left for the point. If the point needs total freedom, then 32/16/16 will do it but 64K isn't enough to cover the longest classes. 28/18/18 might work, or even 24/20/20 (16 Mb per file, 1 Mb max range - I hate to imagine what people are doing which requires more than this...)

One way or another, if the point and end are encoded as offsets against start, it should be possible to fit all three into a long without anyone noticing new limitations.

[martin odersky]

Point is always between start and end, and usually reasonably close to start, so reducing bits for representing point looks promising. 

On the other hand, I cam not aware of any application that uses both start/end and point. Do people know of any? 

Cheers

 - Martin

[Mirko Stocker]

On Wednesday 05 December 2012 10:45:53 martin odersky wrote:
> Do people know of any?

Me! That is, the refactoring library. I use start/end mostly but sometimes
also need the point, for example to find the exact position of a name. Maybe if
names were proper trees, that wouldn't be necessary anymore.

Cheers

Mirko

--
Mirko Stocker | m...@misto.ch
Work: http://ifs.hsr.ch | http://infoq.com
Personal: http://misto.ch | http://twitter.com/m_st

[Mirko Stocker]

On Wednesday 05 December 2012 00:56:57 Paul Phillips wrote:
> I'm not sure where the point usually lands relative to the range start/end
> (I assume at least "inside it")

It's not always inside:

new Some("")

here the point points to new and start to Some.. maybe that's a bug though?

[martin odersky]

Yes, I think we can declare it as a bug. I do not see what start should not also point to new in this case.

Cheers

 - Martin

[Mirko Stocker]

On Wednesday 05 December 2012 11:13:29 martin odersky wrote:
> Yes, I think we can declare it as a bug.

Ok, here it is: https://issues.scala-lang.org/browse/SI-6768