merging scala-virtualized into trunk, hidden behind -Yvirtualize

[Adriaan Moors]

Hi,

I'd like to give everyone a chance to review this patch: https://github.com/adriaanm/scala/compare/master...topic%2Fvirt before I submit a pull request to master (tonight).

It brings virtualization to master, but scalac's behaviour should be unmodified unless -Yvirtualize is specified (except for some additions to manifests that are always on: SourceLocation/SourceContext, RefinedManifest)

Please ignore the build.xml, src/build/pack.xml changes; everything else is good to go:

  - I've tested that lockers built from a) master and b) topic/virt produce identical bytecode for quick.bin

  - the test suite passes as well of course, and it builds a dist

the functionality in this commit that's unleashed by -Xexperimental:

  - more refined applyDynamic/selectDynamic/updateDynamic

the following functionality is behind -Yvirtualize:

  - virtualization of if/then/else, while, return, mutable variables, new Row{...} (to be renamed to Struct)

  - external methods (bar.foo(...) --> infix_foo(bar, ...))

  - statically typed applyDynamic/... for rows

cheers

adriaan

[Tiark Rompf]

I looked over the patch and it seems good to me. Thanks for putting this together!

My main worry at this point is that we will have a lot more external DSL users very soon and we need to provide them with a reasonably stable environment. Following trunk may be too much of a moving target for them.

But since we have a patch now, it should be easy to apply it on top of the 2.9.x branch, too, right? Anybody who uses 2.9.2-SNAPSHOT would get 2.9.1 + epsilon + virtualization. That sounds highly desirable to me.

Is it possible to use Eclipse with 2.9.2-SNAPSHOT, too?

For trunk, I think it will be important to have Jenkins do additional nightly build + test cycles with -Yvirtualize enabled everywhere to make sure we catch any strange interactions with new commits.

Cheers,

- Tiark

[Daniel Sobral]

On Wed, Jan 18, 2012 at 06:15, Adriaan Moors <adriaa...@epfl.ch> wrote:
> Hi,

>
>   - external methods (bar.foo(...) --> infix_foo(bar, ...))

I couldn't find (or identify) any test case for the infix stuff. Is it
missing or did *I* miss it?

--
Daniel C. Sobral

I travel to the future all the time.

[iulian dragos]

On Wed, Jan 18, 2012 at 1:17 PM, Tiark Rompf <tiark...@epfl.ch> wrote:

I looked over the patch and it seems good to me. Thanks for putting this together!

My main worry at this point is that we will have a lot more external DSL users very soon and we need to provide them with a reasonably stable environment. Following trunk may be too much of a moving target for them.

But since we have a patch now, it should be easy to apply it on top of the 2.9.x branch, too, right? Anybody who uses 2.9.2-SNAPSHOT would get 2.9.1 + epsilon + virtualization. That sounds highly desirable to me.

Is it possible to use Eclipse with 2.9.2-SNAPSHOT, too?

Eclipse is built using the 2.9.2-SNAPSHOT *with the 2.9.1 library*. I see that Adriaan's patch contains changes to the library as well. Vlad set up a build for scala-virtualized. Will that be obsolete once this lands in trunk? We could replace it by a build off scala-virtualized-29, if you will.

iulian

 

For trunk, I think it will be important to have Jenkins do additional nightly build + test cycles with -Yvirtualize enabled everywhere to make sure we catch any strange interactions with new commits.

Cheers,

- Tiark

On Jan 18, 2012, at 9:15 AM, Adriaan Moors wrote:

Hi,

I'd like to give everyone a chance to review this patch: https://github.com/adriaanm/scala/compare/master...topic%2Fvirt before I submit a pull request to master (tonight).

It brings virtualization to master, but scalac's behaviour should be unmodified unless -Yvirtualize is specified (except for some additions to manifests that are always on: SourceLocation/SourceContext, RefinedManifest)

Please ignore the build.xml, src/build/pack.xml changes; everything else is good to go:

  - I've tested that lockers built from a) master and b) topic/virt produce identical bytecode for quick.bin

  - the test suite passes as well of course, and it builds a dist

the functionality in this commit that's unleashed by -Xexperimental:

  - more refined applyDynamic/selectDynamic/updateDynamic

the following functionality is behind -Yvirtualize:

  - virtualization of if/then/else, while, return, mutable variables, new Row{...} (to be renamed to Struct)

  - external methods (bar.foo(...) --> infix_foo(bar, ...))

  - statically typed applyDynamic/... for rows

cheers

adriaan

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

[Adriaan Moors]

You didn't miss anything. Infix methods are unlikely to become an official feature; they should be superseded by inlined implicit classes.

Similarly, dsl scope methods and __new (structs) are likely to be replaced by macros and some more generic compiler support, as soon as those are in place.

The uncontroversial bits of this patch will land in trunk, but the rest will mature in topic/virt for a while longer.

--adriaan

[Tiark Rompf]

On Jan 18, 2012, at 7:04 PM, Adriaan Moors wrote:

> You didn't miss anything. Infix methods are unlikely to become an official feature; they should be superseded by inlined implicit classes.

implicit conversions (and by extension inline implicit classes) can not replace infix_ methods:

1) they don't allow you to override-pimp methods (like clone or toString on Rep[T])
2) they don't expose methods as top-level traits members, so cake-style composition requires an indirection
3) they have different overloading semantics: infix_+(a,b) behaves like plus(a,b), not like plus(a)(b)

> Similarly, dsl scope methods and __new (structs) are likely to be replaced by macros and some more generic compiler support, as soon as those are in place.
>
> The uncontroversial bits of this patch will land in trunk, but the rest will mature in topic/virt for a while longer.

I'm mystified. Are we going to have a -Yvirtualize flag or not? Keeping a separate scala-virtualized repository as before is totally fine with me but we should avoid fragmentation. So I think it should be either or.

- Tiark

[Tiark Rompf]

Just wanted to add that infix_ methods will be immensely powerful as macros because they can retroactively macro-fy methods of existing classes (in the current scope):

object DownWithForeach {
macro def infix_foreach[T](xs: Iterable[T], f: T => Unit) = // expand as while loop + iterator
}

import DownWithForeach._

HashMap(....) foreach { x => ... } // will compile as while loop + iterator
List(...) foreach { x => ... } // same here

cheers,
- Tiark

[martin odersky]

Tiark,

I think the current policy should be that trunk will only accept
contributions that make sense in trunk or that are very easy to
support. We want to avoid large swathes of code that make only sense
for virtualized, because they impose a tax on everyone maintaining
trunk. And, of course, we should try to avoid forking the language, so
if it is at all possible that we get the necessary behavior of
virtualized without infix methods we should try that.

Cheers

-- Martin

--
Martin Odersky
Prof., EPFL and Chairman, Typesafe
PSED, 1015 Lausanne, Switzerland
Tel. EPFL: +41 21 693 6863
Tel. Typesafe: +41 21 691 4967

[Erik Osheim]

On Wed, Jan 18, 2012 at 07:04:33PM +0100, Adriaan Moors wrote:
> You didn't miss anything. Infix methods are unlikely to become an
> official feature; they should be superseded by inlined implicit
> classes.

Is there documentation about how inlined implicit classes will work?
I'm hoping it means something like:

a + b --> new GenericOps(a).+(b) --> GenericOps.+(a, b)

Would inlined implicit classes avoid creating the anonymous GenericOps
object (assuming it didn't do any work in its constructor)?

-- Erik

[Adriaan Moors]

nothing is set in stone, so this is the perfect time to send us your wishlist :-)

[Tiark Rompf]

Yes, I think we all agree on the policy. The merge proposal caught me by surprise and I assumed you had signed off on it.
- Tiark

[Erik Osheim]

On Wed, Jan 18, 2012 at 10:16:38PM +0100, Adriaan Moors wrote:
> nothing is set in stone, so this is the perfect time to send us your
> wishlist :-)

Awesome. :)

The issues I face (and the infix operator inlining solution I used) are
described in the paper I'm submitting to ScalaDays. I bet many others
are in the same situation.

Thanks,

-- Erik

[martin odersky]

On Wed, Jan 18, 2012 at 10:24 PM, Tiark Rompf <tiark...@epfl.ch> wrote:
> Yes, I think we all agree on the policy. The merge proposal caught me by surprise and I assumed you had signed off on it.

No, it caught me by surprise as well. But it triggered a very
productive afternoon with Adriaan where we went over all the diffs
between trunk and virtualized.

Cheers

-- Martin

[Adriaan Moors]

I apologise for my unorthodox ways, but -- as Martin said -- I think we made progress. --a

[Grzegorz Kossakowski]

On 19 January 2012 09:51, Adriaan Moors <adriaa...@epfl.ch> wrote:

I apologise for my unorthodox ways, but -- as Martin said -- I think we made progress. --a

I applaud the fact that this discussion happened in open. --g

[Ismael Juma]

On Thu, Jan 19, 2012 at 10:10 AM, Grzegorz Kossakowski <grzegorz.k...@gmail.com> wrote:

I applaud the fact that this discussion happened in open. --g

Yes, this is great indeed.

Best,

Ismael