@exported import pre-sip: some considerations

[Daniel Sobral]

On Sun, Sep 9, 2012 at 7:58 AM, Ruslan Shevchenko <ruslan.s....@gmail.com> wrote:

Good day, colleagues.

It's about possible 'implicit import' feature: here is 'pre-sip:' https://docs.google.com/document/d/1dlT6NgB9610jqLscCJW2LRB7TapDh3q4d2S3YA_q5zs/edit 

I think to submit one as SIP after release of scala-2.10,  on absence of big concerns against.

Note, that future is implemented [ https://github.com/rssh/scala/tree/implicit-import ]  but current implementation is just an experiment to estimate value of changes.

Currently it's not optimal and has code duplication -- so I want to discuss the main implementation points in scala-internals after 2.10 release,  if  this pre-sip is go on.

I *knew* I'd get to it someday, and I just did. :-)

I made two comments in the document, which I'm also bringing here for discussion.

First, and less important, is that there's no discussion on what happens when an exported import imports something that contains exported imports, or how cycles are dealt with when it happens.

Most importantly, I saw no discussion at all on how this gets documented. I'd strongly oppose any such feature that did not come with all changes required on scaladoc to make it clear to the user what's going to happen when he or she imports something! Just look at the damage that undocumented implicits has been dealing to Scala since it ever arrived, and we have just started doing something about it.

I'd rather have a mock scaladoc page than with the existing PoC implementation.

Other than that, and that I prefer to keep my namespace clean, which is going to be made more difficult with this, I like it.

--
Daniel C. Sobral

I travel to the future all the time.

[L.W.]

Another interesting question is: Which namespace are the members of the exported import shown to be in?
Suppose you've got this:

// DeclarationSite.scala
package object original_package {
  class A
}
package object importing_package {
  @exported import original_package._
}

// UsageSite.scala
import importing_package._

val a: A
// ...

So, what namespace is class A residing in (conceptually)?
That is, which namespace should be shown e.g. by IDE helpers or similar tools?

Possiblity 1: Let @exported mean that importing_package declares type aliases for all classes exported by original_package.
Then the above code could be rewritten to:

// DeclarationSite.scala
package object original_package { /* same as before */ }
package object importing_package {
  type A = original_package.A
}
// UsageSite.scala: Same as before

In this setting, importing_package.A would be a type alias fo original_package.A.

Possibility 2: @exported has no direct effect at the declaration site.
Instead, when importing importing_package._ at the usage site, the compiler behaves as if you also had imported original_package._.
So the initial code snippet would be equal to:

// DeclarationSite.scala:
package object original_package { /* again, same as before */ }
package object importing_package {
  import original_package._
}

// UsageSite.scala:
import importing_package._ // Same as above
import original_package._

So there would be no type called importing_package.A, but only the type orginial_package.A.
In this setting, the scaladoc of importing_package could just contain a linked named "This package also exports(/imports) original_package._" - which, when clicked, leads you to the scaladoc page of 'original_package'.

Personally, I'd strongly opt for the second approach, but that is up to discussion.
Variant 1 might be more 'transparent' to the user, regarding which types he can use in his code (he doesn't has to care what package the type 'A' is originally coming from).
Variant 2 is, in my opinion, is more easily to grasp regarding "what's really going on". The transformation done by the compiler can also be done "by hand" by the libraries end-user.

It is also, IMO, more closely to the motivation stated in the introduction of the pre-sip:
Replacing the long, error-prone import sequences at the start of an application's scala files with one, compact import (or, equivalently, a few, well defined import),
while maintaing the modularity possible with packages.

(Old question: Do we allow only @exported import of packages (easy), or also of objects & vals (possibly hard)?
In the case of packages, handling recursive (and possibly circular) @exported imports is relatively easy - because import order doesnt matter (correct me if I'm wrong):

When parsing "import A._" (at UsageSite.scala, for example), all @exported imports of A get inserted after the "import A._" statement, and then the @exported imports of the imports, and so, recursively, until:
All packages we are currently looking at:
1) Contain no @exported imports at all, or
2) Contain no @exported imports no already included at the usage site.

So when parsing "import A._", and package A imports package B, which, in turn, imports A, the last import needn't be followed, because package A is already being imported at the usage site.

Greetings.

[L.W.]

Addendum: (Disclaimer: I didn't have a look at the source code yet, so apologies if I'm fighting windmills here. Also applies to the post before.)
With the second approach, you can keep your namespace clear, because nothing is ever imported into the namespace itself.
It's more like some note attached to the package, saying: "Please also import package XYZ".

Another question: How should imports with aliases be handled? Like, for example:

@exported import scala.util.continuations.{reset => delimit, shift => capture}

Just literally including them at the usage site, where the package declaring the @exported import above, is imported,
would open up interesting possibilities.

[Ruslan Shevchenko]

On Thu, Nov 15, 2012 at 6:51 PM, Daniel Sobral <dcso...@gmail.com> wrote:

On Sun, Sep 9, 2012 at 7:58 AM, Ruslan Shevchenko <ruslan.s....@gmail.com> wrote:

Good day, colleagues.

It's about possible 'implicit import' feature: here is 'pre-sip:' https://docs.google.com/document/d/1dlT6NgB9610jqLscCJW2LRB7TapDh3q4d2S3YA_q5zs/edit 

I think to submit one as SIP after release of scala-2.10,  on absence of big concerns against.

Note, that future is implemented [ https://github.com/rssh/scala/tree/implicit-import ]  but current implementation is just an experiment to estimate value of changes.

Currently it's not optimal and has code duplication -- so I want to discuss the main implementation points in scala-internals after 2.10 release,  if  this pre-sip is go on.

I *knew* I'd get to it someday, and I just did. :-)

I made two comments in the document, which I'm also bringing here for discussion.

First, and less important, is that there's no discussion on what happens when an exported import imports something that contains exported imports, or how cycles are dealt with when it happens.

Situation must be exactly the same as with 'usual'  import:   if A  import B and B import A,  than  A visible from B and vice-verse.

with @exprorted import:   'import clienst'  of both A and B  will see import from A and B   (where 'import clients of X' -- scopes which directly or indirectly import  X._  (i.e. wildcard import of X) )

current implementation contains test for this:  https://github.com/rssh/scala/blob/annotated-import/test/files/pos/exported-import-dual.scala

 

Most importantly, I saw no discussion at all on how this gets documented. I'd strongly oppose any such feature that did not come with all changes required on scaladoc to make it clear to the user what's going to happen when he or she imports something! Just look at the damage that undocumented implicits has been dealing to Scala since it ever arrived, and we have just started doing something about it.

I'd rather have a mock scaladoc page than with the existing PoC implementation.

Yes,  

  I.e. scaladoc must contains sections for exported imports if they are present.

// section about scaladoc changes must be added to pre-SIP,   also will try to grock scaladoc sources and do something with this.

 

Other than that, and that I prefer to keep my namespace clean, which is going to be made more difficult with this, I like it.

btw, the rules for keeping namespaces clean are the same as without exports:  just does not use wildcard imports.

I.e. since exported imports have no own names,  they can be imported to scope only with wildcard import.  So, if module does not contains wildcard import, then it can't contains indirect exported imports.

[Ruslan Shevchenko]

Another interesting question is: Which namespace are the members of the exported import shown to be in?
Suppose you've got this:

// DeclarationSite.scala
package object original_package {
  class A
}
package object importing_package {
  @exported import original_package._
}

// UsageSite.scala
import importing_package._

val a: A
// ...

So, what namespace is class A residing in (conceptually)?
That is, which namespace should be shown e.g. by IDE helpers or similar tools?

  Currently original_package  (i.e. variant #2).

why

a)  belive annotated import must  not change semantics of 'import'.  

b)  if we add aliases, than we must do one available for non-scala clients  [i.e. java parts, and so on].  This is nontrivial and can be source of errors.

 ............................    

 

In this setting, the scaladoc of importing_package could just contain a linked named "This package also exports(/imports) original_package._" - which, when clicked, leads you to the scaladoc page of 'original_package'.

Yes, I think right approach exactly the same.

 

(Old question: Do we allow only @exported import of packages (easy), or also of objects & vals (possibly hard)?

second

 

In the case of packages, handling recursive (and possibly circular) @exported imports is relatively easy - because import order doesnt matter (correct me if I'm wrong):

When parsing "import A._" (at UsageSite.scala, for example), all @exported imports of A get inserted after the "import A._" statement, and then the @exported imports of the imports, and so, recursively, until:
All packages we are currently looking at:
1) Contain no @exported imports at all, or
2) Contain no @exported imports no already included at the usage site.

So when parsing "import A._", and package A imports package B, which, in turn, imports A, the last import needn't be followed, because package A is already being imported at the usage site.

True. 

Currently now exported imports implemented as changes in symbol search algorithm  (not as transformation of sources by inserting import statements).  But usage of such transformation is potentially possible for implementation, I guess result will be the same.

[Ruslan Shevchenko]

Another question: How should imports with aliases be handled? Like, for example:

@exported import scala.util.continuations.{reset => delimit, shift => capture}

The same as with usual imports -- i.e. must work  

 

Just literally including them at the usage site, where the package declaring the @exported import above, is imported,
would open up interesting possibilities.

.

[Ruslan Shevchenko]

Short information about 'exported import pre-sip'  [(see https://docs.google.com/document/d/1dlT6NgB9610jqLscCJW2LRB7TapDh3q4d2S3YA_q5zs/edit  

 ]  status and implementation:

 

1.  Scaladoc generation is implemented, example is available at  https://github.com/rssh/scala-annotated-import-example

(src -- sources of example application, generated scaladoc: http://rssh.github.com/scala-annotated-import-example/api/index.html#package ) 

It is also possible to build patched compiler, than fork scala-annotated-import-example and play with @exported import using supplied scripts for compilation and generation of scaladoc  

(as  macro-paradise by Eugene)

2.  Also added note, that defining of  @exported imports should be enabled by  language feature flag, since it is possible to shoot yourself in the foot while @exporting imports.

3.   What' next (may be this is topic for separate discussion)

 I guess introduction of exported imports solve one part of problem (composition),  but exists other part (cleanup).

Problem - that we have no way to remove (or hide) import from scope.

I.e. if we work with normal variables, than {  val x = "A" { val x = "B"; ...  }  } allows us to use own value of x in internal scope, 

but with import:  { import a.x { import b.x } }  does not allow using of x in internal scope.  

So, it it possible to think about something like @override import annotation and in more far perspective -- some mechanism for opening imports by inheritance, 

which I see during code review, often emulated by defining names in traits instead package objects.

   

But this is next step, which, I guess, must come after @exported import presip as separate feature.