What's the recommended way to recognize a value class at runtime?

[Simon Ochsenreither]

Hi,

I'm currently looking into unboxed value arrays and I'm wondering what's the recommended way to detect whether a class is a value class in the first place.
For testing, I just check for a special class name, like className.endsWith("ValueClass"), but that's just an ugly and a more general, supported way would be preferable.

Is there a reason why "AnyVal" does not end up in the class file's superclass list (or any other maker type)?

Thanks and bye,

Simon

[Jason Zaugg]

scala> :power

** Power User mode enabled - BEEP WHIR GYVE **

** :phase has been set to 'typer'.          **

** scala.tools.nsc._ has been imported      **

** global._, definitions._ also imported    **

** Try  :help, :vals, power.<tab>           **

scala> class A(val a: Int) extends AnyVal

defined class A

scala> typeOf[A].typeSymbol.isDerivedValueClass

res0: Boolean = true 

-jason

[Simon Ochsenreither]

I don't think it is acceptable that the VM depends on language-specific reflection libraries ... in addition to hardly being a language-agnostic solution, I guess.

Is there a reason why the superclass gets removed in the first place? (isInstanceOf[AnyVal] is forbidden any way ...)

[Jason Zaugg]

On Tue, Mar 19, 2013 at 10:31 PM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

I don't think it is acceptable that the VM depends on language-specific reflection libraries ... in addition to hardly being a language-agnostic solution, I guess.

Is there a reason why the superclass gets removed in the first place? (isInstanceOf[AnyVal] is forbidden any way ...)

Sorry, I thought you were trying to figure that out in the compiler. After that, I don't know of any trace of this other than in the pickled signatures, which means you need to use Scala reflection.

-jason

[Rex Kerr]

With vanilla reflection in Java you have to look for the $extension methods in the companion object.  It's a huge pain.  I advise against it.  When would you ever have a value class and not be able to use the language-specific reflection library for that language that generated the value class?

  --Rex

On Tue, Mar 19, 2013 at 5:31 PM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

I don't think it is acceptable that the VM depends on language-specific reflection libraries ... in addition to hardly being a language-agnostic solution, I guess.

Is there a reason why the superclass gets removed in the first place? (isInstanceOf[AnyVal] is forbidden any way ...)

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Simon Ochsenreither]

When would you ever have a value class and not be able to use the language-specific reflection library for that language that generated the value class?

As mentioned, runtime support for unboxed value arrays.

[Rex Kerr]

The array is unboxed, but the value-class-with-unboxed array is itself in a box, right?  You can't just take an Array[Long] and miraculously know that it is a range with two ints instead of a float in vector form packed in there.

And if you have that, then I don't quite see why Scala's reflection abilities would be unavailable to you.  Are you wanting something in ClassTag or somesuch?

  --Rex

On Tue, Mar 19, 2013 at 6:33 PM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

When would you ever have a value class and not be able to use the language-specific reflection library for that language that generated the value class?

As mentioned, runtime support for unboxed value arrays.

--

[Simon Ochsenreither]

The array is unboxed, but the value-class-with-unboxed array is itself in a box, right?  You can't just take an Array[Long] and miraculously know that it is a range with two ints instead of a float in vector form packed in there.

?
I'm not sure I follow. Instances of value classes are currently boxed before they are stored in an array. Otherwise, I wouldn't look into not boxing them in the first place.

 

And if you have that, then I don't quite see why Scala's reflection abilities would be unavailable to you.  Are you wanting something in ClassTag or somesuch?

Why should a VM bundle some random language's reflection library? I think it would be a very bad idea to hard-code Scala-specific stuff into a language-agnostic runtime. Additionally, I don't think it makes sense to execute huge amounts of Scala reflection stuff for basically every unvisited arrayload/arraystore instruction.

[Rex Kerr]

Maybe I don't understand the context in which you mean you are "looking into unboxed value arrays".  If you are wondering what VM support would really help out, it would be if the array could store the signature of the box instead of the signature of the primitive.  Java wouldn't know what the heck to do with these weird things, but Scala would have a ball.  But it wouldn't be a JVM.

If you are looking into how to store primitive arrays and use them usefully in Scala, well, then you have all of Scala's libraries at your disposal.

If you're doing something else, please explain, because at least to me it is far from clear.

  --Rex

--

[Simon Ochsenreither]

Maybe I don't understand the context in which you mean you are "looking into unboxed value arrays".

https://groups.google.com/forum/?fromgroups=#!topic/avian/0DODhYok8Kk

If you are wondering what VM support would really help out, it would be if the array could store the signature of the box instead of the signature of the primitive.  Java wouldn't know what the heck to do with these weird things, but Scala would have a ball.  But it wouldn't be a JVM.

Well, that's already working.

Even Java could potentially benefit from it, because it is just a simple optimization done by the VM if certain conditions are met. But as a start, it's far easier to let scalac enforce the invariants (for AnyVal) instead of checking it at class-load time. That's why I'm concentrating on Scala's value classes currently.

Consider this code snippet for the aastore/aaload/alength/anewarray byte codes:

if (isValueClass(class_))
  // Create/load a specialized array representation/emit appropriate instructions for the operation
else
  // Emit standard instructions for reference arrays

isValueClass does the following currently (simplified):

bool isValueClass(object class_)
  return endsWith("ValueClass", className(class_)));


The question I'm having is simply how to clean up the "isValueClass" method, which currently checks whether the class name ends with "ValueClass" and emits the appropriate instructions for the JIT compiler based on that.
Ideally, checking the superclass list should be enough. (Which doesn't work because "extends AnyVal" doesn't end up in the class file for unknown reasons.)

 

If you're doing something else, please explain, because at least to me it is far from clear.

I just want to know what's the cheap, easy, library-independent way to check for value classes (and if there isn't one, why not?).


Simon

[Joel Dice]

On Tue, 19 Mar 2013, Simon Ochsenreither wrote:

> I just want to know what's the cheap, easy, library-independent way to check
> for value classes (and if there isn't one, why not?).

Why not just look for final classes which have only final fields such that
any reference fields have types which also satisfy those constraints?
Then add a reasonable size limit so you don't end up copying around huge
objects on the stack. Done!

[Simon Ochsenreither]

Why not just look for final classes which have only final fields such that
any reference fields have types which also satisfy those constraints?
Then add a reasonable size limit so you don't end up copying around huge
objects on the stack.  Done!

How to infer non-nullability?

[Joel Dice]

On Wed, 20 Mar 2013, Simon Ochsenreither wrote:

> How to infer non-nullability?

Good question. I'd be inclined to treat null as equivalent to an instance
initialized with all zeros. That ends up breaking strict Java
compatibility, but we already have to do that for other reasons (e.g.
reference equality won't mean the same thing when there aren't any
references involved). You could certainly add an annotation to the type
to indicate that it's "non-nullable", but we still have to define what the
VM should do if it encounters bytecode which misbehaves with respect to
that directive.

Anyway, I do agree that we'll need some way for the language to tell the
VM that it promises to follow certain rules with respect to a type (e.g.
no nulls, no reference comparisons) if we want to avoid breaking code that
uses e.g. java.lang.Integer in non-referentially-transparent ways. Even
so, it would be nice to allow the user to pass a flag to the VM that says,
"treat all types containing only final fields (and which only reference
other such types) as referentially-transparent; I promise my program will
not do any referentially-opaque things with such types, and if it does
you can throw a UserIsABigFatLiarError."

[Simon Ochsenreither]

Hi!

Good question.  I'd be inclined to treat null as equivalent to an instance
initialized with all zeros.  That ends up breaking strict Java
compatibility, but we already have to do that for other reasons (e.g.
reference equality won't mean the same thing when there aren't any
references involved).

As far as I remember, both things are well-defined, in the Scala space at least. E. g. the zero instance of a value type is an instance with all fields initialized to the their respective zero value (which is pretty much equivalent to "everything zeroed out" in practice.
And in a recent discussion people discussed that eq (Scala's identity method, == in Java) could be defined across all reference and value types as "identical bits".

 

You could certainly add an annotation to the type
to indicate that it's "non-nullable", but we still have to define what the
VM should do if it encounters bytecode which misbehaves with respect to
that directive.

Would it be possible to add some additional bytecode verification and just reject such usages at class-load time?

Anyway, I do agree that we'll need some way for the language to tell the
VM that it promises to follow certain rules with respect to a type (e.g.
no nulls, no reference comparisons) if we want to avoid breaking code that
uses e.g. java.lang.Integer in non-referentially-transparent ways.

Even if the Java JEP talks about it, I think the chance to fix Java's wrapper classes is long gone. There is no way to put the brokenness back into the bottle.

 

Even so, it would be nice to allow the user to pass a flag to the VM that says,
"treat all types containing only final fields (and which only reference
other such types) as referentially-transparent; I promise my program will
not do any referentially-opaque things with such types, and if it does
you can throw a UserIsABigFatLiarError."

I'm not sure whether this is practical, but maybe some whitelist would be safer?

Thanks and bye,

Simon

PS: Question to the Scala hackers: Should I file some "value classes miss AnyVal parent in class files" issue?

[Paul Phillips]

On Wed, Mar 20, 2013 at 4:55 PM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

PS: Question to the Scala hackers: Should I file some "value classes miss AnyVal parent in class files" issue?

Sure.

[Joel Dice]

On Wed, 20 Mar 2013, Simon Ochsenreither wrote:

> Would it be possible to add some additional bytecode verification and just
> reject such usages at class-load time?

Yes.

[Simon Ochsenreither]

https://issues.scala-lang.org/browse/SI-7281

[Simon Ochsenreither]

I have looked into it, but I think I haven't found the right place in the compiler to keep/add the AnyVal type as a parent.

Paul, could you have a look or suggest where the appropriate place would be to do that?

[Paul Phillips]

On Sun, May 19, 2013 at 9:45 AM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

I have looked into it, but I think I haven't found the right place in the compiler to keep/add the AnyVal type as a parent.

Paul, could you have a look or suggest where the appropriate place would be to do that?

The bad news is I already tried to do it and it was inordinately complicated.

You have primitive types like Int, which have AnyVal as a parent, and which erase to themselves and survive in Arrays.

You have user-defined value classes like Bippy(val x: Int), which have AnyVal as a parent, and which sometimes erase to Int and sometimes erase to Bippy.

You have AnyVal itself, which erases to Object.

You have all kinds of boxing, unboxing, casting, and erasing, which traverses two phases (erasure and posterasure) and failure is not an option since it's VerifyError or ClassCastException time, both of which I enjoyed while trying to tweak this.

And you have Arrays, and I can't say anything about that except good luck to you.

Screwing around with AnyVal puts you right at the nexus of all the worst bugs in the compiler. I decided it was out of reach until the erasure/boxing/casting/etc logic is brought down to a fraction of its current complexity. Here's some of the test I was working with:

  https://github.com/paulp/scala/tree/issue/7281-tests

This doesn't even begin to hit the real composition quicksand, and I didn't get this much working.

[Simon Ochsenreither]

Hi Paul,

The bad news is I already tried to do it and it was inordinately complicated.

Ok, thanks a lot! I wouldn't have expected that having an additional supertype in the class would create such an issue. (Note that this is the only thing required, no runtime/value support in any form is needed.)

You have primitive types like Int, which have AnyVal as a parent, and which erase to themselves and survive in Arrays.

You have user-defined value classes like Bippy(val x: Int), which have AnyVal as a parent, and which sometimes erase to Int and sometimes erase to Bippy.

You have AnyVal itself, which erases to Object.

You have all kinds of boxing, unboxing, casting, and erasing, which traverses two phases (erasure and posterasure) and failure is not an option since it's VerifyError or ClassCastException time, both of which I enjoyed while trying to tweak this.

And you have Arrays, and I can't say anything about that except good luck to you.

I'm not sure, but I think none of this is required for the use-case in question. The only thing it needs to support is having a class and asking it whether AnyVal is a parent, whether the actual values are boxed/erased/put in arrays ... who cares? Considering that AnyVal doesn't have any "real" members, it should just work like any other marker trait. But if you say it's hard, I completely trust you.

The nearest alternative which makes sense in my opinion would be to have some @ValueType annotation. One benefit of that would be that it wouldn't be restricted to single-field value types and could be used for all the stuff which should be a value type, but is not currently supported by the compiler.

Opinions on that?

Thanks and bye,

Simon

[Paul Phillips]

On Sun, May 19, 2013 at 1:48 PM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

I'm not sure, but I think none of this is required for the use-case in question. The only thing it needs to support is having a class and asking it whether AnyVal is a parent, whether the actual values are boxed/erased/put in arrays ... who cares? Considering that AnyVal doesn't have any "real" members, it should just work like any other marker trait. But if you say it's hard, I completely trust you.

I'm never able to do justice to the whirlwind of incidental complexity which accompanies such efforts as this with my after-the-fact summaries. Suffice to say you're correct to trust me.

Not sure about an annotation. I'm still kind of hoping to get the compiler under control. Who knows, this could be my decade.

[Simon Ochsenreither]

Not sure about an annotation. I'm still kind of hoping to get the compiler under control. Who knows, this could be my decade.

Assuming that this decade will last another 7 years, what about a scala.annotation.unchecked.uncheckedValueType in the meantime? (Automatically added to AnyVal classes, manually addable to "possible" value types?)

[Simon Ochsenreither]

Assuming that this decade will last another 7 years, what about a scala.annotation.unchecked.uncheckedValueType in the meantime? (Automatically added to AnyVal classes, manually addable to "possible" value types?)

Sorry to bump this again, but with Avian 0.7 out (which includes all fixes to make the test suite green) I would really like start looking deeper into runtime support for value classes.
Any chance to get this proposal (or anything else) going, so that it ships with 2.11?

[Grzegorz Kossakowski]

I reread the last few messages in this thread and I'm wondering why anything in Scala is needed to start experimenting with this?

If I understand correctly, the annotation would serve as hint to Avian's JIT to work harder on eliminating boxing and unboxing operations, right? If so, the hint can be sent in many ways like:

• an annotation defined outside of scala namespace
• a file listing all classes considered to be value classes

The second bullet addresses the problem of annotating classes in std. library. You can generate that file using Scala reflection that would process all types declared in standard library.

There are many ways to experiment with value classes support in Avian without getting Scala compiler involved. Once we learn from those experiments we can revisit this discussion.

--
Grzegorz Kossakowski

Scalac hacker at Typesafe

twitter: @gkossakowski

github: @gkossakowski

[Simon Ochsenreither]

Hi!

• an annotation defined outside of scala namespace
• a file listing all classes considered to be value classes

The second bullet addresses the problem of annotating classes in std. library. You can generate that file using Scala reflection that would process all types declared in standard library.

Yes, these things can be done, but they are pretty much no improvement over the current “manual” approach. I was hoping for a more seamless approach which can ship with the next release so that no further action is required by users.
 

There are many ways to experiment with value classes support in Avian without getting Scala compiler involved.

I just want to make sure that it doesn't miss the feature freeze for 2.11. Having to wait for 2.12 would suck tremendously.

Once we learn from those experiments we can revisit this discussion.

It's only experimental in the sense of “when will it arrive?”. Support will ship, sooner or later (e. g. as soon as I have a reliable grasp of the particular C++ code style of Avian).

Bye,

Simon

[Grzegorz Kossakowski]

On 27 July 2013 16:30, Simon Ochsenreither <simon.och...@gmail.com> wrote:

The second bullet addresses the problem of annotating classes in std. library. You can generate that file using Scala reflection that would process all types declared in standard library.

Yes, these things can be done, but they are pretty much no improvement over the current “manual” approach. I was hoping for a more seamless approach which can ship with the next release so that no further action is required by users.

I still don't see what do we need to improve specifically. Maybe you explain it somewhere in this thread but I think I lost that context.

 

Once we learn from those experiments we can revisit this discussion.

It's only experimental in the sense of “when will it arrive?”. Support will ship, sooner or later (e. g. as soon as I have a reliable grasp of the particular C++ code style of Avian).

I don't know the scope of the work you are planning to do on Avian side but in general, value classes are very tricky to get even at runtime.

Also, I find it confusing that you want to promote to runtime a property (extends AnyVal) which is meant to mean something statically only.

Moreover, if you are working on runtime support for value classes then probably you'd like to support Option[T] as a canonical example. However, Option won't be marked as a value class in Scala sense (optimized statically) any time soon due to problems discussed in SI-7685.

If we had any JVM that supports value classes at JIT level then we would probably need to rethink Scala support for value classes from scratch. The current, static approach can get you only that far.

[Simon Ochsenreither]

I still don't see what do we need to improve specifically. Maybe you explain it somewhere in this thread but I think I lost that context.

The compiler and developers need a reliable, fast way to mark a certain type as “this should really be a value type”, regardless of the current, limited support for value types in the compiler (only single-field classes, no unboxed array representation, poor support for specialization, issues with clashing members, etc.)

 

 I don't know the scope of the work you are planning to do on Avian side but in general, value classes are very tricky to get even at runtime.

As mentioned, I will first be looking into unboxed value types in arrays. The scope is limited and the potential issues well-known. After that, I will be looking into dropping more and more restrictions.

Also, I find it confusing that you want to promote to runtime a property (extends AnyVal) which is meant to mean something statically only.

In what sense are super types a static property only? It transports an important piece of information, just like other marker interfaces. It seems hard to emit the information to the class file for unrelated reasons (see Paul's efforts). That's why I was proposing to introduce some sort of annotation instead, which doesn't interfere with the LUB business.

 

Moreover, if you are working on runtime support for value classes then probably you'd like to support Option[T] as a canonical example. However, Option won't be marked as a value class in Scala sense (optimized statically) any time soon due to problems discussed in SI-7685.

Sigh, that's exactly why I want that annotation: To mark types which should be value classes, but can't be handled by the compiler currently! If the runtime knows the intention, it can have a shot at it, too (and might probably be more successful than the compiler currently is).
As soon as value types would be supported reliably at runtime¹, I would even add a switch to the compiler to make it stay away from messing with value classes completely.

¹ That's further down the road because support means a lot more than just unboxed arrays, but some parts should be pretty easy to implement. (E. g. replacing all virtual calls with static ones).

If we had any JVM that supports value classes at JIT level then we would probably need to rethink Scala support for value classes from scratch. The current, static approach can get you only that far.

That's exactly what I'm doing here.

[Grzegorz Kossakowski]

On 28 July 2013 06:21, Simon Ochsenreither <simon.och...@gmail.com> wrote:

I still don't see what do we need to improve specifically. Maybe you explain it somewhere in this thread but I think I lost that context.

The compiler and developers need a reliable, fast way to mark a certain type as “this should really be a value type”, regardless of the current, limited support for value types in the compiler (only single-field classes, no unboxed array representation, poor support for specialization, issues with clashing members, etc.)

I'd like to know what marking a given class to be a value class means before we start sprinkle it on any classes in scala/scala repo.

Do we know already? 

 

 I don't know the scope of the work you are planning to do on Avian side but in general, value classes are very tricky to get even at runtime.

As mentioned, I will first be looking into unboxed value types in arrays. The scope is limited and the potential issues well-known. After that, I will be looking into dropping more and more restrictions.

I'd like to hear about results of those experiments. I'd expect arrays to be actually very tricky to get right but I might be missing something.

 

Also, I find it confusing that you want to promote to runtime a property (extends AnyVal) which is meant to mean something statically only.

In what sense are super types a static property only? It transports an important piece of information, just like other marker interfaces. It seems hard to emit the information to the class file for unrelated reasons (see Paul's efforts). That's why I was proposing to introduce some sort of annotation instead, which doesn't interfere with the LUB business.

My point is that AnyVal is not like any other marker interface. It has a very special treatment in Scala compiler and translates to bytecode in a special way.

 

 

Moreover, if you are working on runtime support for value classes then probably you'd like to support Option[T] as a canonical example. However, Option won't be marked as a value class in Scala sense (optimized statically) any time soon due to problems discussed in SI-7685.

Sigh, that's exactly why I want that annotation: To mark types which should be value classes, but can't be handled by the compiler currently! If the runtime knows the intention, it can have a shot at it, too (and might probably be more successful than the compiler currently is).
As soon as value types would be supported reliably at runtime¹, I would even add a switch to the compiler to make it stay away from messing with value classes completely.

¹ That's further down the road because support means a lot more than just unboxed arrays, but some parts should be pretty easy to implement. (E. g. replacing all virtual calls with static ones).

As I mentioned before we don't even know what this annotation would mean apart from vague 'JIT should try harder to optimize'. This kind of vaguely defined concepts should stay in experimental land and not go into scala/scala until the concept is fleshed out and has at least one working implementation that supports that concept.

Don't get me wrong. I'm all for supporting your experiments with Avian. I'm all for competition in JVM space. Scala can only benefit from that. However, I don't see much value in putting an annotation into std lib that doesn't have a single implementation supporting it.

[Paul Phillips]

On Sun, Jul 28, 2013 at 4:59 PM, Grzegorz Kossakowski <grzegorz.k...@gmail.com> wrote:

I'd like to know what marking a given class to be a value class means before we start sprinkle it on any classes in scala/scala repo.

Do we know already? 

I think it's give the box marker interface scala.AnyVal (or failing that, any marker at all) as a parent.

 

My point is that AnyVal is not like any other marker interface. It has a very special treatment in Scala compiler and translates to bytecode in a special way.

For the user, it is like any other marker interface. You say "Foo extends Bar" and something happens because of it. You reasonably expect to find Bar in the parents of Foo, because you wrote "Foo extends Bar". You don't expect this:

final class A(val x: Int) extends AnyVal

% javap A

public final class A extends java.lang.Object{

[Simon Ochsenreither]

I'd like to know what marking a given class to be a value class means before we start sprinkle it on any classes in scala/scala repo.

It doesn't involve sprinkling it on scala/scala. The compiler is supposed to add them to the class file when it encounters "extends AnyVal". It's a replacement for not being able to (easily) check whether a class extends AnyVal. The possibility to use it for additional types manually is just a nice benefit.
 

Do we know already?

Yes, the semantics are pretty well-defined. .NET figured it out years ago already, too.
 

I'd like to hear about results of those experiments. I'd expect arrays to be actually very tricky to get right but I might be missing something.

Can you explain?
One way to do it is to always store it unboxed and rebox it when loading it again. Probably not the best approach in terms of performance, but pretty simple.
There are of course a few concurrency-related issues like shearing, but in the end it's mainly about making a decision about what the given guarantees are.
 

My point is that AnyVal is not like any other marker interface. It has a very special treatment in Scala compiler and translates to bytecode in a special way.

Sure, but I still think that it is very reasonable that one wants to know whether some class is a value class.
 

As I mentioned before we don't even know what this annotation would mean apart from vague 'JIT should try harder to optimize'. This kind of vaguely defined concepts should stay in experimental land and not go into scala/scala until the concept is fleshed out and has at least one working implementation that supports that concept.

???
Value types are a pretty well-defined concept.
 

Don't get me wrong. I'm all for supporting your experiments with Avian. I'm all for competition in JVM space. Scala can only benefit from that. However, I don't see much value in putting an annotation into std lib that doesn't have a single implementation supporting it.

If you can make AnyVal work, even better.

[Rex Kerr]

On Sun, Jul 28, 2013 at 7:59 PM, Grzegorz Kossakowski <grzegorz.k...@gmail.com> wrote:

On 28 July 2013 06:21, Simon Ochsenreither <simon.och...@gmail.com> wrote:

 

 I don't know the scope of the work you are planning to do on Avian side but in general, value classes are very tricky to get even at runtime.

As mentioned, I will first be looking into unboxed value types in arrays. The scope is limited and the potential issues well-known. After that, I will be looking into dropping more and more restrictions.

I'd like to hear about results of those experiments. I'd expect arrays to be actually very tricky to get right but I might be missing something.

No, no.  Arrays are really easy to get right.  They're just not easy to get right with the standard set of JVMs because they don't understand the concept of a primitive array with decorated type.  The rest of the machinery (invariance, etc.) is all in place; they just don't know who they are unless you encode it as an array of boxed types.

If you know that these boxes are _just_ boxes (hence the annotation!), then every time you see

  box
  stuff-into-array

you replace it with

  stuff-into-array

and every time you see

  stuff-existing-box-into-array

you replace it with

  unbox
  stuff-into-array

and the opposite on the way out.

And you do all your array-reasoning just like you do with Array[String] vs. Array[Object].

The reason arrays are hard to get right has everything to do with the underlying VM and almost nothing to do with solving the computational problem (since it's already solved for objects!).  Even if you can't change the bytecode, with a hint that it's wise to apply the above transformation, it's pretty simple as far as JIT transformations go.

  --Rex

 

[Paul Phillips]

On Mon, Jul 29, 2013 at 7:14 AM, Rex Kerr <ich...@gmail.com> wrote:

And you do all your array-reasoning just like you do with Array[String] vs. Array[Object].

Except the runtime can distinguish an Array[String] from an Array[Object]. You can't tell your Array[Int with Tag] from Array[Int] if you unboxed and stuffed them as described. I realize this doesn't matter for a lot of people and a lot of uses, but unless a certain someone changes their tune, you'll be performing your own stuffing and unstuffing.

[Rex Kerr]

My whole point (Simon's also, I assume?) is that your runtime needs to distinguish Array[Int with Tag] from Array[Int], and it is not at all hard for a runtime to do given that it distinguishes Array[String] from Array[Object].  The only challenge is getting a runtime *cough* Avian *cough* to _actually_ do it.

I'd bet that having Avian do these things is the best way to get support in the more widely-used JVMs.  (It's also probably the best way for Avian to get hit with a lawsuit.)

  --Rex

 

[Paul Phillips]

On Mon, Jul 29, 2013 at 8:00 AM, Rex Kerr <ich...@gmail.com> wrote:

it is not at all hard for a runtime to do given that it distinguishes Array[String] from Array[Object]

It could be hard for implementation-accidental reasons. That's how things usually turn out in scalac: shouldn't be hard, but are anyway.

But yes, I guess I'd lost the thread of this conversation when it went on hiatus. Clearly there is no fundamental reason the runtime can't carry around a type tag on its primitive arrays. Unfortunately my gut is telling me (stupid gut) I'll be driving a flying car first.

[Simon Ochsenreither]

It could be hard for implementation-accidental reasons. That's how things usually turn out in scalac: shouldn't be hard, but are anyway.

The good things is that scalac is (or shouldn't be) involved here. That's why I'm asking for a way to distinguish classes which are supposed to be value types from the rest: As soon as the runtime is able to figure out the intent (“Is this type supposed to be a value type, yes or no?”), the runtime can start handling the rest. If Int with Tag is a value type I don't see how it would be any harder to have an unboxed array representation than for some Point(x: Int, y: Int), Vector3f(x: Float, y: Float: z: Float) or Decimal(low: Long, high: Long).
 

But yes, I guess I'd lost the thread of this conversation when it went on hiatus. Clearly there is no fundamental reason the runtime can't carry around a type tag on its primitive arrays. Unfortunately my gut is telling me (stupid gut) I'll be driving a flying car first.

I already synthesized unboxed value type arrays at runtime a few months ago.

[Rex Kerr]

Well, if you're motivated, you could probably test-fly one of these:

  http://www.terrafugia.com/aircraft/transitionR

so, yeah, flying car could well be first.

  --Rex

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Simon Ochsenreither]

s/scalac is/scala isn't/

[Paul Phillips]

On Mon, Jul 29, 2013 at 10:11 AM, Rex Kerr <ich...@gmail.com> wrote:

so, yeah, flying car could well be first.

HACKER DIES WHEN FLYING CAR ENCOUNTERS PIGEON FLOCK 

"All he wanted were tagged primitive arrays," says grief-stricken widow.

[Simon Ochsenreither]

I implemented the workaround with an annotation here: https://github.com/scala/scala/pull/3206

While testing, I found some weird issue:

scala> val aa = classOf[ArrowAssoc[_]]
aa: Class[ArrowAssoc[_]] = class java.lang.Object

scala> val re = classOf[RichException]
re: Class[RichException] = class scala.Predef$RichException

Both are declared as AnyVals. Why does one end up as object but one doesn't?
The only difference seems to be the type arguments.

[Paul Phillips]

On Fri, Nov 29, 2013 at 10:31 AM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

scala> val aa = classOf[ArrowAssoc[_]]
aa: Class[ArrowAssoc[_]] = class java.lang.Object

scala> val re = classOf[RichException]
re: Class[RichException] = class scala.Predef$RichException

Both are declared as AnyVals. Why does one end up as object but one doesn't?
The only difference seems to be the type arguments.

This will either clarify or further confuse.

scala> classOf[ArrowAssoc[Throwable]]

res0: Class[ArrowAssoc[Throwable]] = class scala.Predef$ArrowAssoc

scala> classOf[ArrowAssoc[_]]

res1: Class[ArrowAssoc[_]] = class java.lang.Object