support for Big numeric types in Reflector

42 views
Skip to first unread message

trptcolin

unread,
Mar 31, 2011, 12:51:37 AM3/31/11
to Clojure Dev
I started looking into adding support for Big* numeric types to
Reflector - CLJ-666 (gasp!) - (http://dev.clojure.org/jira/browse/
CLJ-666). It's relatively straightforward to add support for these in
clojure.lang.Reflector.paramArgTypeMatch, but it causes a difference
in bit-shift-left's behavior, breaking a test:

;; current master
user=> (bit-shift-left 1 10000)
65536
user=> (bit-shift-left 1N 10000)
19950631168807583848837421626835....etc.
user=> (bit-shift-left 1N 10000N)
19950631168807583848837421626835....etc.

;; after adding Reflector support for BigInt, BigInteger, and
BigDecimal
user=> (bit-shift-left 1 10000)
65536
user=> (bit-shift-left 1N 10000)
65536
user=> (bit-shift-left 1N 10000N)
19950631168807583848837421626835....etc.

So upon adding reflection for BigInt => int, bit-shift-left picks the
primitive version of Numbers.shiftLeft(long, int).

So it seems like there are 3 questions here:

1) Should bit-shift-left overflow silently for primitive math? If not,
might an overflow guard for this primitive version of shiftLeft be a
way forward, as a symmetry with the way addition and multiplication
work in 1.3?
2) Is there a problem with (BigInt, Long) dispatching to (long, int)
here instead of (Object, Object)?
3) I caught this potential problem because it was the one test case
that failed, so I'm concerned there might be other changes exposed as
well that just aren't in the test suite. Are there other places to
look out for?

Thoughts? I have a patch with tests for the initial change, which
might clarify anything I haven't explained well, but wanted to get
feedback on the bit-shift-left stuff before submitting. I'll
certainly link any discussion here to the issue.

Thanks!

- Colin

ataggart

unread,
Mar 31, 2011, 1:41:40 AM3/31/11
to Clojure Dev
CLJ-666 will be fixed via the latest patch on CLJ-445 (if TPTB ever
get around to applying it).

Regarding overflow, I'm disinclined to view bit-twiddling ops as
precision math ops. Shifting bits on a fixed-length integer means
pushing them off one end or the other.

Rich Hickey

unread,
Mar 31, 2011, 12:01:45 PM3/31/11
to cloju...@googlegroups.com
I'm definitely having second thoughts about the bit shift ops for non-
primitives, and the negative shift support.

I'd like to remove both - does anyone care?

Rich

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure Dev" group.
> To post to this group, send email to cloju...@googlegroups.com.
> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com
> .
> For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en
> .
>

ataggart

unread,
Apr 1, 2011, 12:27:00 AM4/1/11
to Clojure Dev
It wouldn't negatively impact any of the bit-twiddling code I have.

How should this fit with "contagious bigints" policy? E.g., should
(bit-shift-left 1N 1) return 2N or 2?

Should all bit ops be limited to a 64-bit long?

Rich Hickey

unread,
Apr 1, 2011, 8:59:57 AM4/1/11
to cloju...@googlegroups.com

On Apr 1, 2011, at 12:27 AM, ataggart wrote:

> It wouldn't negatively impact any of the bit-twiddling code I have.
>
> How should this fit with "contagious bigints" policy? E.g., should
> (bit-shift-left 1N 1) return 2N or 2?
>
> Should all bit ops be limited to a 64-bit long?
>


Shift ops would be limited to primitives only. Default versions 64-
bit, but explicit shift-int versions for 32 bits.

Rich

ataggart

unread,
Apr 4, 2011, 1:28:17 AM4/4/11
to Clojure Dev
Regarding the negative shift support, removing it might have less
desirable behaviour. Since only the bottom 5-bits of the shift
distance are used, negative numbers behave oddly, e.g., 1 << -2 is
processed as 1 << 30. Leaving it in, or erroring on negatives might
be preferable.

Also, do you want primitives-only for all bit operations, e.g., bit-
and, test-bit.

Rich Hickey

unread,
Apr 4, 2011, 7:36:36 AM4/4/11
to cloju...@googlegroups.com

On Apr 4, 2011, at 1:28 AM, ataggart wrote:

> Regarding the negative shift support, removing it might have less
> desirable behaviour. Since only the bottom 5-bits of the shift
> distance are used, negative numbers behave oddly, e.g., 1 << -2 is
> processed as 1 << 30. Leaving it in, or erroring on negatives might
> be preferable.
>

The goal is to get rid of the conditional. People that are using
shifts as nature intended are doing so for the utmost performance, and
any conditional test kills it. I'm fine with following:

http://java.sun.com/docs/books/jls/third_edition/html/expressions.html#15.19

People looking for mathematical purity should avoid bit ops, they are
mechanical.

> Also, do you want primitives-only for all bit operations, e.g., bit-
> and, test-bit.
>

I'd like primitive-only to be the default, but BigInteger has and/or
etc and there's no reason to hide them. Perhaps we need prime versions
for that?

Rich

ataggart

unread,
Apr 4, 2011, 4:01:25 PM4/4/11
to Clojure Dev
Apologies for going round and round on this.

On Apr 4, 4:36 am, Rich Hickey <richhic...@gmail.com> wrote:
> I'd like primitive-only to be the default, but BigInteger has and/or  
> etc and there's no reason to hide them. Perhaps we need prime versions  
> for that?

BigInteger also has shiftLeft and shiftRight, and its negative-shift
behaviour is the same as ours currently. Assuming we don't want to
hide that as well, is it acceptable to have BigIntegers behave
differently from other Numbers and primitives?

Aside: was there a reason the clojure.unchecked namespace [1] was
abandoned? Seems like that would be a good place for all these
changes.

[1] http://dev.clojure.org/jira/browse/CLJ-441

ataggart

unread,
Apr 7, 2011, 3:13:31 AM4/7/11
to Clojure Dev
Rich, please see http://dev.clojure.org/jira/browse/CLJ-767


On Apr 4, 4:36 am, Rich Hickey <richhic...@gmail.com> wrote:
> On Apr 4, 2011, at 1:28 AM, ataggart wrote:
>
> > Regarding the negative shift support, removing it might have less
> > desirable behaviour.  Since only the bottom 5-bits of the shift
> > distance are used, negative numbers behave oddly, e.g., 1 << -2 is
> > processed as 1 << 30.  Leaving it in, or erroring on negatives might
> > be preferable.
>
> The goal is to get rid of the conditional. People that are using  
> shifts as nature intended are doing so for the utmost performance, and  
> any conditional test kills it. I'm fine with following:
>
> http://java.sun.com/docs/books/jls/third_edition/html/expressions.htm...
Reply all
Reply to author
Forward
0 new messages