Is it just me, or is VType[] values wrong?

19 views
Skip to first unread message

tsaloranta

unread,
Jan 10, 2012, 9:21:46 PM1/10/12
to High Performance Primitive Collections for Java
One issue I noticed when converting code from Colt to HPPC (yay!) is
that assignment using 'map.values' tends to fail with
ClassCastException (for generic-valued maps). And thinking it through,
it makes sense: since no Class object is passed to map constructor, I
don't see how code could construct actual type-safe array. So code
probably allocates an Object[] and cast it to V[] (haven't verified
this, but based on exception this makes sense).

If so, it seems wrong to declare entries as "VType[] values" since
while this may work for indexed access, it won't work for assignments
to concrete array types.
So for external access it would seem like one would have to expose it
as plain old Object[], or, alternatively, require passing of value
type (Class) in constructor (or pre-allocated array to get type out
of) to allow allocation of properly typed storage array.

In my case, I just use assignment to Object[], do cast on iteration,
and things work as expected.

Is above correct, or am I missing something obvious?

-+ Tatu +-

Dawid Weiss

unread,
Jan 11, 2012, 2:21:46 AM1/11/12
to java-high-performance...@googlegroups.com
Your observation is correct, Tatu.

map.values is an Object[] because this is in most cases faster than
creating a typed array using reflection (and this is what would have
to happen if you passed a concrete type). Also, map.values is exposed
publicly but is a rather internal detail which you should use only if
striving for maximum performance (or if you don't want to reallocate
by a call to toArray(java.lang.Class<? super KType> clazz)).

Finally, this is kind of explained in the JavaDoc of this field --
http://labs.carrotsearch.com/download/hppc/0.4.1/api/com/carrotsearch/hppc/ObjectArrayList.html#buffer

Hope this helps,
Dawid

Tatu Saloranta

unread,
Jan 11, 2012, 2:04:22 PM1/11/12
to java-high-performance...@googlegroups.com
On Tue, Jan 10, 2012 at 11:21 PM, Dawid Weiss <dawid...@gmail.com> wrote:
> Your observation is correct, Tatu.
>
> map.values is an Object[] because this is in most cases faster than
> creating a typed array using reflection (and this is what would have
> to happen if you passed a concrete type). Also, map.values is exposed

Right, I understand reasoning for this.

> publicly but is a rather internal detail which you should use only if
> striving for maximum performance (or if you don't want to reallocate
> by a call to toArray(java.lang.Class<? super KType> clazz)).

Yes, in this case I do want maximum iteration speed, to go over
relatively large aggregated dataset.
In other cases I do create result array as it is more convenient.

> Finally, this is kind of explained in the JavaDoc of this field --
> http://labs.carrotsearch.com/download/hppc/0.4.1/api/com/carrotsearch/hppc/ObjectArrayList.html#buffer

Ah ok. I did miss that.

I will add casts as necessary and it'll work; just wanted to ping to
see if this is a known issue/caveat, and sounds like it is.

Thank you for quick answer!

-+ Tatu +-

Dawid Weiss

unread,
Jan 11, 2012, 2:11:53 PM1/11/12
to java-high-performance...@googlegroups.com
> Yes, in this case I do want maximum iteration speed, to go over
> relatively large aggregated dataset.
> In other cases I do create result array as it is more convenient.

You may want to try with pseudo-closures; if your code isn't too
complex they inline very well and provide excellent performance and
maybe a little bit cleaner code (depending on a particular use case
results may vary :).

http://labs.carrotsearch.com/download/hppc/0.4.1/api/com/carrotsearch/hppc/ObjectArrayList.html#forEach(T)

> I will add casts as necessary and it'll work; just wanted to ping to
> see if this is a known issue/caveat, and sounds like it is.

Sure, thanks. I agree it is clumsy but out of several options that
I've tried (including actual typing of that array, generic Object[]
signature and explicit casts, etc.) this turned out to be least
annoying.

Dawid

Tatu Saloranta

unread,
Jan 11, 2012, 3:34:34 PM1/11/12
to java-high-performance...@googlegroups.com
On Wed, Jan 11, 2012 at 11:11 AM, Dawid Weiss <dawid...@gmail.com> wrote:
>> Yes, in this case I do want maximum iteration speed, to go over
>> relatively large aggregated dataset.
>> In other cases I do create result array as it is more convenient.
>
> You may want to try with pseudo-closures; if your code isn't too
> complex they inline very well and provide excellent performance and
> maybe a little bit cleaner code (depending on a particular use case
> results may vary :).
>
> http://labs.carrotsearch.com/download/hppc/0.4.1/api/com/carrotsearch/hppc/ObjectArrayList.html#forEach(T)

Ok, I'll try that -- I did see a reference but for some reason went
with direct access.
One nice thing would be elimination of having to check the status flag.

>> I will add casts as necessary and it'll work; just wanted to ping to
>> see if this is a known issue/caveat, and sounds like it is.
>
> Sure, thanks. I agree it is clumsy but out of several options that
> I've tried (including actual typing of that array, generic Object[]
> signature and explicit casts, etc.) this turned out to be least
> annoying.

Yeah I know, generics are a PITA often, and I had to deal with
complexities of array allocation (for Jackson/JSON). While type-safety
of arrays is sometimes nice (compared to type-erased fakeness of
generics collections), it can also be a problem.

-+ Tatu +-

Reply all
Reply to author
Forward
0 new messages