Problem with modular arithmetic

[Stefan Baumann]

Hello, I'm getting wrong results when calculating the following modular sum:

+m.1234567891/<.(%1+i.@%:)10^14

281233455

1234567891&|@+/<.(%1+i.@%:)10^14

281233455

Using extended integers gives the correct result:

1234567891|+/<.(%1+i.@%:)10^14x

281233407

Am I missing something? Thanks. Stefan.

[Andrea Piseri]

Hi, the problem seems to be with the division rather than the modular arithmetic.

   a =. <.(%i.@%:)10^14
   b =. <.(%i.@%:)10^14x
   +/a ~: b

48

the first occurrence of the rounding error is with 50000000000000r3=(10^14x)%6. Curiously the log 2 of the relative error required to make the failing cases round incorrectly is:

    2^.(%>.-]) (10^14x)% I. a ~: b

less than 46 bits of precision for all of them, which makes sense because 46.507=2^.10^14. Much less than the 52 bits that IEEE754 should get you. maybe I'm missing something too.

To unsubscribe from this group and stop receiving emails from it, send an email to forum+un...@jsoftware.com.

[Stefan Baumann]

Many thanks, Andrea, you're right, the problem is the rounding.

What's interesting is that when I pull the floor into the hook I get the correct result, but only when I start with an integer (1e14) not a floating point (10^14).

3!:0 ] 1e14

4

+m.1234567891/(<.@%1+i.@%:)1e14

281233407

3!:0 ] 10^14

8

+m.1234567891/(<.@%1+i.@%:)10^14

281233455

Ciao. Stefan.

[Henry Rich]

<.@% is on integers is vestigially supported as modular arithmetic.

<. is tolerant by default.

    +m.1234567891/<.!.0(%1+i.@%:) 10^14
281233407

Henry Rich

[Stefan Baumann]

Thanks a lot for clarifying!