How much reflection is leaking?

[Grzegorz Kossakowski]

Hi,

I wanted to have a better idea when do we leak in reflection API. I heard that the only way to leak is to have reflective calls against unbounded number of types (e.g. to have unbounded number of TypeTags). Is this true?

--
Grzegorz Kossakowski

[Eugene Burmako]

Here's the report on what's going to leak: http://groups.google.com/group/scala-internals/browse_thread/thread/eabcf3d406dab8b2.

So far I only came up with two scenarios that exploit these weaknesses: 1) calculating a glb of refinement types, 2) typechecking stuff that involves refinement types and type inference. You could take a closer look in my ticket/6214_wip

[Grzegorz Kossakowski]

On 24 September 2012 13:23, Eugene Burmako <eugene....@epfl.ch> wrote:

Here's the report on what's going to leak: http://groups.google.com/group/scala-internals/browse_thread/thread/eabcf3d406dab8b2.

So far I only came up with two scenarios that exploit these weaknesses: 1) calculating a glb of refinement types, 2) typechecking stuff that involves refinement types and type inference. You could take a closer look in my ticket/6214_wip

I see. If refinement types are the problem then what if we just put this into release notes and fix after 2.10.0?

The point is that if we can have a sense that for most common cases reflection is not leaking then I'd mark odd cases not critical and move forward with the release.

WDYT?

--
Grzegorz Kossakowski

[Eugene Burmako]

I'm preparing a build that fixes a low-hanging leakage for Nikita (the original submitter of the bug).

If it helps with his problem, then I think we can proceed with the release and mark the other issues as todo for 2.10.1.

[Grzegorz Kossakowski]

On 24 September 2012 13:36, Eugene Burmako <eugene....@epfl.ch> wrote:

I'm preparing a build that fixes a low-hanging leakage for Nikita (the original submitter of the bug).

Ok. I just don't want yet another round of last-minute refactorings because they inevitably introduce other bugs. Is this fix small enough?

 

If it helps with his problem, then I think we can proceed with the release and mark the other issues as todo for 2.10.1.

Sounds good. Would be great if we could flesh this out by tomorrow's Scala meeting.

--
Grzegorz Kossakowski

[Eugene Burmako]

Sure I don't want drastic stuff either. The fix only affects runtime reflection. It doesn't touch scalac in any way.

[Mark Harrah]

On Mon, 24 Sep 2012 13:36:27 +0200
Eugene Burmako <eugene....@epfl.ch> wrote:

> I'm preparing a build that fixes a low-hanging leakage for Nikita (the
> original submitter of the bug).
>
> If it helps with his problem, then I think we can proceed with the release
> and mark the other issues as todo for 2.10.1.

Sorry if these were explicitly addressed already, but the following questions aren't clear to me:

1) Are there reproducible test cases that demonstrate leaks?
2) Are there known sources of leaks (identified as fixing the above test cases) or likely sources of leaks (identified from just inspecting the code)?

-Mark

[Eugene Burmako]

1) Yes there are.
2) The tests from #1 exploit the sources of leaks found by inspecting the code. However, not all sources of leaks are currently fixed, only the most blatant ones.

[Mark Harrah]

On Mon, 24 Sep 2012 17:33:40 +0200
Eugene Burmako <eugene....@epfl.ch> wrote:

> 1) Yes there are.
> 2) The tests from #1 exploit the sources of leaks found by inspecting the
> code. However, not all sources of leaks are currently fixed, only the most
> blatant ones.

Thanks, Eugene. I asked because in my experience, the hard part about memory leaks is the process of reproducing and identifying them. So, I think it would be a mistake to ignore known memory leaks even if they were only discovered after publishing RC1. Memory leaks in Scala itself show up in downstream projects and those projects or their users will have to duplicate the time consuming process of reproducing and identifying the leak only to find it isn't in their code and it was already known before Scala's release.

[Eugene Burmako]

Luckily for us (and me personally), reflection will be experimental in 2.10.0, so we'll have an official list of known issues, which will include yet unfixed leaks.

I would love to fix them all, but we don't have enough time for that :(

[Ismael Juma]

On Mon, Sep 24, 2012 at 4:49 PM, Mark Harrah <dmha...@gmail.com> wrote:

Thanks, Eugene.  I asked because in my experience, the hard part about memory leaks is the process of reproducing and identifying them.  So, I think it would be a mistake to ignore known memory leaks even if they were only discovered after publishing RC1.  Memory leaks in Scala itself show up in downstream projects and those projects or their users will have to duplicate the time consuming process of reproducing and identifying the leak only to find it isn't in their code and it was already known before Scala's release.

Good point Mark.

Best,

Ismael

[Mark Harrah]

On Mon, 24 Sep 2012 17:54:52 +0200
Eugene Burmako <eugene....@epfl.ch> wrote:

> Luckily for us (and me personally), reflection will be experimental in
> 2.10.0, so we'll have an official list of known issues, which will include
> yet unfixed leaks.

I was under the impression that reflection code could get called from the compiler itself even if the code being compiled doesn't use anything non-experimental. I could see a completely experimental feature having known memory leaks, but it looks to me (from the outside) that reflection is too intertwined with normal compilation.

* ClassManifest/ClassTag? ClassTag is in scala-library.jar and is used for constructing arrays. Will only classes in scala-reflect.jar be behind language.experimental? Or, is the reflection code causing the memory leaks not actually involved in these?

* Manifests are deprecated in favor of TypeTags, but TypeTags are now experimental and only in scala-reflect.jar, right?

* Will normal users never cause macros to run? For example, are there no macros exposed by the standard library that users might invoke?

> I would love to fix them all, but we don't have enough time for that :(

I understand that all bugs won't be fixed (for example, I know I'll have to wait until at least 2.10.1 for macros), but I think memory leaks are especially bad. If it is clearly the case that the memory leaks can only occur by some obvious, experimental reflection method not used by the compiler, ok.

[Paul Phillips]

On Mon, Sep 24, 2012 at 9:20 AM, Mark Harrah <dmha...@gmail.com> wrote:

* Will normal users never cause macros to run?  For example, are there no macros exposed by the standard library that users might invoke?

There's at least this, but maybe if it's in "fast track" it doesn't count as a macro.

scala> val bippy = 1.0 / 3; f"$bippy%.3f"

bippy: Double = 0.3333333333333333

res0: String = 0.333

scala> val bippy = 1.0 / 3; f"$bippy%.3d"

<console>:10: error: type mismatch;

 found   : Double

 required: Int

        f"$bippy%.3d"

           ^

[Eugene Burmako]

Reflection leaks because its mode of operation is different from the compiler.

Roughly speaking, compiler = reflection core + compiler infrastructure, whereas reflection = reflection core + runtime reflection infrastructure. Unfortunately, some cleanup to caches from reflection core is done only in compiler infrastructure. To the best of my knowledge, this is the only source of leaks in reflection, which means that the compiler should be immune (or at least not worse than usual).

Class tags won't be experimental, and they shouldn't leak.

Type tags will be experimental, right. Manifests are deprecated in favor of type tags, that's also right. Josh, could you please clarify?

Macros are hosted within the compiler, so they shouldn't have problems with leaks.

[Eugene Burmako]

Tag generation is also done by macros (even class tag generation!).

But as mentioned above, macros are hosted within the compiler, so they shouldn't leak. Since macros are just functions called by the compiler, they are no different from, say, methods in Typers.scala or Implicits.scala.

[Josh Suereth]

On Mon, Sep 24, 2012 at 12:40 PM, Eugene Burmako <eugene....@epfl.ch> wrote:

Reflection leaks because its mode of operation is different from the compiler.

Roughly speaking, compiler = reflection core + compiler infrastructure, whereas reflection = reflection core + runtime reflection infrastructure. Unfortunately, some cleanup to caches from reflection core is done only in compiler infrastructure. To the best of my knowledge, this is the only source of leaks in reflection, which means that the compiler should be immune (or at least not worse than usual).

Class tags won't be experimental, and they shouldn't leak.

Type tags will be experimental, right. Manifests are deprecated in favor of type tags, that's also right. Josh, could you please clarify?

Manifests were deprecated in favor of TypeTags, but I'm not 100% certain we want to start the policy of deprecating something in favor of something else that's experimental (Yes, we may have done it before, but not without ramifications).  Considering we don't think TypeTags "stable" yet, I'm not positive we should deprecate Manifest unless we would have done so in the *absence* of TypeTags.   At this point, I'm on the fence here, so I'm going to list out points I've used to reach my conclussion (listed below) and see if everyone agrees or if I'm missing something vital:

• A *lot* of use cases of Manifest were solely for Arrays or erasure, in which case ClassTag is a working alternative that's superior.
• There were still valid use cases of Manifest
• Disambiguate runtime erasure  (`typeParameters`  method, not obtainable through a ClassTag)
• toString, oddly enough:  https://issues.scala-lang.org/browse/SI-6053
• Manifests implementation of <:< is an approximation and hack, not to be relied on for true <:< behavior.
• While we don't think many relied on this feature, it's worth noting that we want to move them off of it.
• Note: == on Manifests uses <:<
• TypeTags require a dependency on scala-reflect, that comes with no binary compatibility guarantees
• Libraries that must use TypeTags *in public APIs* therefore cannot offer strict binary compatibility guarantees (although note that this means library + scala reflect version are tied tightly).

SO, my current thinking is the following:

• Un-deprecate Manifest
• Deprecate `<:<` on Manifests with warning to use TypeTags for correct behavior.  Note:  I think == should be ok
• Note: There's already a nice warning in the Scaladoc, so this is just to give *extra* warning to those who don't read scaladocs.
• When reflection goes stable, deprecate Manifest fully 
• This may be in 2.10.1/2.10.2.  If see, see follow up email on deprecation and expected removal.  Deprecated in a point release *DOES NOT MEAN* we can remove in the next major release.  Otherwise, let's release Scala 2.9.3 with nothing but deprecations.

Does this sound reasonable?  If so I can update the @deprecated annotations.

[Eugene Burmako]

Thanks for reminding about SI-6053. It only affected scala.reflect.base, so I've closed it :)

As to deprecation. I agree, this makes sense. <:< is already deprecated, so you'll only have to remove deprecation annotations from Manifest[T] and manifest[T].

Also back then I've written a bidirectional typetag <-> manifest converter. If a typetag is in scope, then its value will be used to create a manifest and vice versa. Originally this was designed for backward compatibility, but I think it's useful in itself, so I suggest we don't touch it.

[Josh Suereth]

Sounds good.