Limitations of package objects (WAS: Re: [scala-user] criticism of scala article)

[Jason Zaugg]

On Wed, Dec 11, 2013 at 9:46 PM, Haoyi Li <haoy...@gmail.com> wrote:

> The limitations of package objects come from the demands of separate compilation. We can't mandate that you alway compile all files in a given package at once.

 

How does it work now when you separately compile multiple package objects of the same package and then try to use the code together? Presumably the same logic that now works for that case would work for any auto-generated package objects created by top-level definitions?

Package objects are just syntactic sugar for:

   object `package` { ... }

The compiler looks one of these in every package, by looking for `package.class` in the classpath, and by looking for any package object definitions in the current batch of source files.

Just like regular classes/objects, nothing prevents you from accidentally defining the same named top-level entity in multiple files, and compiling them separately. If the output directory is the same, the last one will overwrite the previous one.Depending on how you do things, the last one might win (overwrite the previous one). Otherwise, they could both end up on your classpath, and the first one that the classloader finds will win.

If you compile them together, scalac will be able to error out.

So, that's just something that people are used to and in practice it rarely causes problems. The file naming convention that javac strongly enforces is probably designed with this problem in mind, now that I think about it.

But, if we just allowed top level definitions from the current set of files to define `package.class`, things would be really hard to reason about. People usually don't care about the order of compilation, and leave that to SBT or their IDE. So you really would need some way to merge results of multiple compilation batches into a package object. This isn't something we can really specify and implement.

-jason

[Naftoli Gugenheim]

On Wed, Dec 11, 2013 at 3:59 PM, Jason Zaugg <jza...@gmail.com> wrote:

On Wed, Dec 11, 2013 at 9:46 PM, Haoyi Li <haoy...@gmail.com> wrote:

> The limitations of package objects come from the demands of separate compilation. We can't mandate that you alway compile all files in a given package at once.

 

How does it work now when you separately compile multiple package objects of the same package and then try to use the code together? Presumably the same logic that now works for that case would work for any auto-generated package objects created by top-level definitions?

Package objects are just syntactic sugar for:

   object `package` { ... }

That's interesting, not sure if I realized that. Do you mean you can actually skip the "package outer; package object inner { ... }" syntax and just do "package outer.inner; object `package` { ... }"?

If yes, just thinking aloud, there was some recent discussion about simplifying the language...

The compiler looks one of these in every package, by looking for `package.class` in the classpath, and by looking for any package object definitions in the current batch of source files.

Just like regular classes/objects, nothing prevents you from accidentally defining the same named top-level entity in multiple files, and compiling them separately. If the output directory is the same, the last one will overwrite the previous one.Depending on how you do things, the last one might win (overwrite the previous one). Otherwise, they could both end up on your classpath, and the first one that the classloader finds will win.

If you compile them together, scalac will be able to error out.

So, that's just something that people are used to and in practice it rarely causes problems. The file naming convention that javac strongly enforces is probably designed with this problem in mind, now that I think about it.

But, if we just allowed top level definitions from the current set of files to define `package.class`, things would be really hard to reason about. People usually don't care about the order of compilation, and leave that to SBT or their IDE. So you really would need some way to merge results of multiple compilation batches into a package object. This isn't something we can really specify and implement.

-jason

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Jon Pretty]

On Wednesday, 11 December 2013 20:59:59 UTC, Jason Zaugg wrote:

So, that's just something that people are used to and in practice it rarely causes problems. The file naming convention that javac strongly enforces is probably designed with this problem in mind, now that I think about it.

It happens often enough that I've seen it happen. Large codebases have a tendency for code to migrate between projects quite frequently (a project being a single run of the compiler), and it frequently migrates without changing package. This results in a single package being distributed across multiple projects, and it only takes two package objects for the same package to cause a problem...
 

But, if we just allowed top level definitions from the current set of files to define `package.class`, things would be really hard to reason about. People usually don't care about the order of compilation, and leave that to SBT or their IDE. So you really would need some way to merge results of multiple compilation batches into a package object. This isn't something we can really specify and implement.

It's a real shame package objects are so complicated! Having the same support for all kinds of class "members" at the top-level would be an obvious candidate for reducing the language spec size. Or, alternatively, some general way to remove the distinction between a package and an object. Unfortunately, I don't think this will be possible.

Incidentally, I discovered SI-6225 (https://issues.scala-lang.org/browse/SI-6225) a while ago as a more obvious leak in the package object abstraction. You (Jason) kindly fixed that, though prioritization of implicits in package objects doesn't seem to work as it does with objects. I've not been able to find an open issue for this. Is anyone aware of one? If not, I'll file one.

Cheers,
Jon

[Jason Zaugg]

On Wed, Dec 11, 2013 at 10:46 PM, Naftoli Gugenheim <nafto...@gmail.com> wrote:

Package objects are just syntactic sugar for:

   object `package` { ... }

That's interesting, not sure if I realized that. Do you mean you can actually skip the "package outer; package object inner { ... }" syntax and just do "package outer.inner; object `package` { ... }"?

If yes, just thinking aloud, there was some recent discussion about simplifying the language...

Yes, you can write it either way.

But I recommend to uniformly use the sugar, and also to put your package object as the only definition in a file named `package.scala`.

-jason

[Jason Zaugg]

On Wed, Dec 11, 2013 at 10:49 PM, Jon Pretty <googl...@accounts.propensive.com> wrote:

On Wednesday, 11 December 2013 20:59:59 UTC, Jason Zaugg wrote:

So, that's just something that people are used to and in practice it rarely causes problems. The file naming convention that javac strongly enforces is probably designed with this problem in mind, now that I think about it.

It happens often enough that I've seen it happen. Large codebases have a tendency for code to migrate between projects quite frequently (a project being a single run of the compiler), and it frequently migrates without changing package. This results in a single package being distributed across multiple projects, and it only takes two package objects for the same package to cause a problem...

No that you mention it, I also saw this recently when helping out some fellow typesafers troubleshoot a problem. My point was really that forcing you to put all the definitions in one file makes it easier to explain this gotcha in terms of a gotcha that people might already know about.

 

 

But, if we just allowed top level definitions from the current set of files to define `package.class`, things would be really hard to reason about. People usually don't care about the order of compilation, and leave that to SBT or their IDE. So you really would need some way to merge results of multiple compilation batches into a package object. This isn't something we can really specify and implement.

It's a real shame package objects are so complicated! Having the same support for all kinds of class "members" at the top-level would be an obvious candidate for reducing the language spec size. Or, alternatively, some general way to remove the distinction between a package and an object. Unfortunately, I don't think this will be possible.

Packages are open with regards to separate compilation, that what fundamentally differentiates them from objects, and imposes limits on what one can do with them.

 

Incidentally, I discovered SI-6225 (https://issues.scala-lang.org/browse/SI-6225) a while ago as a more obvious leak in the package object abstraction. You (Jason) kindly fixed that, though prioritization of implicits in package objects doesn't seem to work as it does with objects. I've not been able to find an open issue for this. Is anyone aware of one? If not, I'll file one.

I don't know of that one. But, as a wild guess, perhaps you're experience a little known factoid about implicits and package objects: they contribute to the implicit scope.

So, if an implicit search for `p1.p2.C[p3.D]` fails to find an in-scope implicit, the package objects `p{1,2,3}` are consulted, along with the companions for `C` and `D`, and the companions of their base classes.

I say this as stern warning against putting implicits in package objects. It is just *so* easy to end up with disasters like:

// look ma, no imports!

scala> Nil ifParSeq ((x: Any) => true) otherwise false

res2: Boolean = false

Exercise for the reader: how does that extension method come into play? (Answer: SI-8072)

-jason

[Som Snytt]

The use case for backticked package is the object for the package which shall not be named, viz, the empty package.

--

[Naftoli Gugenheim]

Well it's good to follow the convention. But I would have preferred if the new syntax was not introduced. The convention would be identical in the regards besides the syntax. One difference is that you can say that all files in a/b/c begin with package a.b.c (or package a;package b;package c). I see "package object c" as "package" being a modifier on "object" rather than "object" a modifier on "package."

 

-jason

 

[Naftoli Gugenheim]

On Wed, Dec 11, 2013 at 5:26 PM, Som Snytt <som....@gmail.com> wrote:

The use case for backticked package is the object for the package which shall not be named, viz, the empty package.

What?
 

[Som Snytt]

You can't do this in the empty package with package object syntax:

scala> :pa -raw
// Entering paste mode (ctrl-D to finish)

object `package` { implicit class X(val i: Int) { def empty = 0 } }

// Exiting paste mode, now interpreting.


scala> :pa -raw
// Entering paste mode (ctrl-D to finish)

object Test { Console println 3.empty }

// Exiting paste mode, now interpreting.


scala> Test
0
res1: Test.type = Test$@603f99fa

[Jon Pretty]

Hi Jason,

On Wednesday, 11 December 2013 22:16:07 UTC, Jason Zaugg wrote: 

I don't know of that one. But, as a wild guess, perhaps you're experience a little known factoid about implicits and package objects: they contribute to the implicit scope.

Oh, another little factoid about them!? And just when I was feeling starved of arcane implicit rules! ;)
 

So, if an implicit search for `p1.p2.C[p3.D]` fails to find an in-scope implicit, the package objects `p{1,2,3}` are consulted, along with the companions for `C` and `D`, and the companions of their base classes.

I'm not sure how that transforms my mental picture of things yet... I'll have to ponder this a while longer before I work out whether the strange behaviour I saw before was a bug, or just my misinterpretation. Unfortunately, my simple test-case attempt didn't reproduce the problem when I tried it, and I've long-since refactored the real-world broken example into something which compiles.

I'll spend some more time on it some time when it's not 4am. ;)
 
Thanks,
Jon

[Naftoli Gugenheim]

On Thu, Dec 12, 2013 at 10:59 PM, Jon Pretty <googl...@accounts.propensive.com> wrote:

Hi Jason,

On Wednesday, 11 December 2013 22:16:07 UTC, Jason Zaugg wrote: 

I don't know of that one. But, as a wild guess, perhaps you're experience a little known factoid about implicits and package objects: they contribute to the implicit scope.

Oh, another little factoid about them!? And just when I was feeling starved of arcane implicit rules! ;)

How is that arcane? It seems to me to be quite consistent. A package object is a way to put definitions directly in the package. The rule for implicits is basically that any container referenced in the type is searched. So for example if you expect a p1.p2.O.C[p3.D], by the same token that O is searched, why shouldn't p2 be?

 

 

So, if an implicit search for `p1.p2.C[p3.D]` fails to find an in-scope implicit, the package objects `p{1,2,3}` are consulted, along with the companions for `C` and `D`, and the companions of their base classes.

I'm not sure how that transforms my mental picture of things yet... I'll have to ponder this a while longer before I work out whether the strange behaviour I saw before was a bug, or just my misinterpretation. Unfortunately, my simple test-case attempt didn't reproduce the problem when I tried it, and I've long-since refactored the real-world broken example into something which compiles.

I'll spend some more time on it some time when it's not 4am. ;)
 
Thanks,
Jon

[Naftoli Gugenheim]

Ah, so another reason in favor of the more general syntax.

[Jon Pretty]

On Friday, 13 December 2013 08:06:22 UTC, nafg wrote:

Oh, another little factoid about them!? And just when I was feeling starved of arcane implicit rules! ;)

How is that arcane? It seems to me to be quite consistent. A package object is a way to put definitions directly in the package. The rule for implicits is basically that any container referenced in the type is searched. So for example if you expect a p1.p2.O.C[p3.D], by the same token that O is searched, why shouldn't p2 be?

That was more a quip about how many rules there are in total. This isn't one of the arcane ones...

Jon