Extension of Julia to support decimal floating-point arithmetic?

[Eric Diaz]

So apparently the ISO group managing the C language is considering adding support for decimal Floating-point arithmetic.

• http://www.open-std.org/jtc1/sc22/wg14/www/projects
  
• http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1242.pdf
  
• http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1312.pdf
  

So, I looked up Julia support of decimal floating point arithmetic and came up with...

• http://docs.julialang.org/en/release-0.4/manual/integers-and-floating-point-numbers/
  
• https://github.com/tinybike/Decimals.jl
  
• https://github.com/stevengj/DecFP.jl
  

Perhaps, Julia should consider building decimal arithmetic into its core numeric libraries. Any thoughts?

[Yichao Yu]

It should stay in a package unless it is needed for the language core.

[Eric Diaz]

I'm not sure how you determine what is needed in the language core.

Perhaps it is just my naivety on the subject, however from my perspective, if the C language developers have published a rationale and proposal scheme for adding it to the core parts of the C language, then it seems like it might be a good idea.

Have their been any discussions about this in the Julia community?

Also, I wanted to add a link to this website which goes into great detail about the General Decimal Arithmetic

http://speleotrove.com/decimal/

[Erik Schnetter]

On Sat, Apr 2, 2016 at 7:19 PM, Eric Diaz <eric...@gmail.com> wrote:

Packages are very easy to use, and there are discussions of bundling
certain packages with Julia. In practical terms that would mean they
could be in the Base library, but would not be in the Core library.
The core would then be a very small number of functions, mostly those
that require explicit support from the compiler (e.g. arrays, tuples).

By using the `llvmcall` functionality, it is possible to make use of
very low-level LLVM features from a package.

-erik

--
Erik Schnetter <schn...@gmail.com>
http://www.perimeterinstitute.ca/personal/eschnetter/

[Yichao Yu]

On Sat, Apr 2, 2016 at 7:46 PM, Eric Diaz <eric...@gmail.com> wrote:
> I'm not sure how you determine what is needed in the language core.
>
> Perhaps it is just my naivety on the subject, however from my perspective,
> if the C language developers have published a rationale and proposal scheme
> for adding it to the core parts of the C language, then it seems like it
> might be a good idea.
>
> Have their been any discussions about this in the Julia community?

In general, if it can be implemented in a package and is not needed by
the compiler, it should be kept in a package and move out of the base.

[Eric Diaz]

@Yu. I'm very naive about how the people developing the core determine what is needed by the compiler.  In following the logic, I guess I don't understand the rationale or decision making process went into including the current floating-point types: Float16, Float32, Float64. Are these "needed" by the compiler, or were they just included in the core out of tradition?

Additionally, full support for Complex and Rational Numbers is built on top of these primitive numeric types. All numeric types interoperate naturally without explicit casting, thanks to a flexible, user-extensible type promotion system.

@Schnetter. It sounds like you are saying that the additional numeric types can be compiled with special features using "llvmcall".

[Yichao Yu]

On Sat, Apr 2, 2016 at 8:01 PM, Eric Diaz <eric...@gmail.com> wrote:
> @Yu. I'm very naive about how the people developing the core determine what
> is needed by the compiler. In following the logic, I guess I don't
> understand the rationale or decision making process went into including the
> current floating-point types: Float16, Float32, Float64. Are these "needed"
> by the compiler, or were they just included in the core out of tradition?

For machine floating point types (this also includes complex floating
point btw), they are necessary to interface correctly with c code. It
is also necessary to support them in the parser so that we can have
floating point literals.

BigInt, BigFloat, Rationals, Linalgs, OTOH, are indeed candidates for
moving out of the base. Some of them involves calling external C
libraries which is still a non-trivial problem to get those libraries
installed on all platforms by the package manager if we move them into
packages.

For most of the existing pieces in Base, they need to be considered
and discussed on a case-by-case basis but for a new function that can
be implemented in a package, it is always perferred to implement and
keep it in a package.

[Stefan Karpinski]

I would point out that in C, to have convenient support for things like decimal floating-point arithmetic, you need to add support for it at the language level, which is not the case at all in Julia, so the C standards committee has an entirely different set of constraints and motivations to work with.

[Scott Jones]

On Sunday, April 3, 2016 at 7:39:15 AM UTC-4, Stefan Karpinski wrote:

I would point out that in C, to have convenient support for things like decimal floating-point arithmetic, you need to add support for it at the language level, which is not the case at all in Julia, so the C standards committee has an entirely different set of constraints and motivations to work with.

That's true for the most part, however, one thing that makes it hard to deal with decimal floating-point easily in Julia is that floating point literals are always converted to Float64 in the femto-lisp interpreter, so that it is impossible to do things like have a macro that treats all floating point literals as some type of decimal floating point,

you are forced to use rather inconvenient string macros. These documents for adding decimal point floats to C describe what they did for a similar issue, so that the interpretation of a numeric literal is delayed until it is actually needed.

Doing that in Julia would be quite useful for dealing with Unums and BigFloats as well.

Most of the support can be done in packages, however the 6 defined IEEE types really should be added to Base, along with some generic function definitions, so that no matter which decimal float point package you wish to use, your code would be the same.

I think that is similar to the current situation where one can use different libraries, such as MKL, with Julia, without having to rewrite your Julia code.

Support for an IEEE standard, which even has hardware support on some platforms (on IBM POWER architectures), would be very useful for Julia for certain applications dealing with currency, for example.

Finally, people might be surprised that using the `Dec128` type from `DecFP.jl` is actually faster than using a 128-bit BigFloat [which is because of the use of the gmp library, and vectors to store the values], and takes many many times less space (`Dec128` is a bitstype), and that's without any hardware support for the `Dec128` type.

(It would be interesting to see if we could at some point use the hardware support on POWER architecture machines and do some comparisons).

[Stefan Karpinski]

Custom parsing of numeric literals might be a plausible future feature.

[Yichao Yu]

On Sun, Apr 3, 2016 at 8:27 AM, Stefan Karpinski <ste...@karpinski.org> wrote:
> Custom parsing of numeric literals might be a plausible future feature.

Custom parsing is what string macros are for.

[Tomas Lycken]

Sure. But you have to agree that it’s pretty clear that one of these two reads better than the other:

# decimal literals with a string macro
x = dec"10.3" + dec"5.8" * dec"0.15"

# decimal literals with built-in support for parsing
# m is the suffix for decimal in e.g. C#, works similar to e.g. Julia's 0.15f0 for Float32
x = 10.3m + 5.8m * 0.15m

Adding support for a few more of these kinds of literals - or even an extension mechanism to enable registering your own type and suffix - is definitely useful, even if the equivalent program can already be written with today’s features.

// T

[Yichao Yu]

On Sun, Apr 3, 2016 at 9:26 AM, Tomas Lycken <tomas....@gmail.com> wrote:
> Sure. But you have to agree that it’s pretty clear that one of these two
> reads better than the other:
>
> # decimal literals with a string macro
> x = dec"10.3" + dec"5.8" * dec"0.15"
>
> # decimal literals with built-in support for parsing
> # m is the suffix for decimal in e.g. C#, works similar to e.g. Julia's
> 0.15f0 for Float32
> x = 10.3m + 5.8m * 0.15m

The `f` for floating point is not a suffix. `0.15f` is `0.15 * f` not `0.15f0`.
The syntax above only works if we don't have juxtapose multiplication
syntax. There's not even much space left for custom numerical syntax.
The only one we can use without too much breakage is 0.*[0-9]

https://github.com/JuliaLang/julia/issues/15731

[Yichao Yu]

On Sun, Apr 3, 2016 at 9:37 AM, Yichao Yu <yyc...@gmail.com> wrote:
> On Sun, Apr 3, 2016 at 9:26 AM, Tomas Lycken <tomas....@gmail.com> wrote:
>> Sure. But you have to agree that it’s pretty clear that one of these two
>> reads better than the other:
>>
>> # decimal literals with a string macro
>> x = dec"10.3" + dec"5.8" * dec"0.15"
>>
>> # decimal literals with built-in support for parsing
>> # m is the suffix for decimal in e.g. C#, works similar to e.g. Julia's
>> 0.15f0 for Float32
>> x = 10.3m + 5.8m * 0.15m
>
> The `f` for floating point is not a suffix. `0.15f` is `0.15 * f` not `0.15f0`.
> The syntax above only works if we don't have juxtapose multiplication
> syntax. There's not even much space left for custom numerical syntax.
> The only one we can use without too much breakage is 0.*[0-9]
>
> https://github.com/JuliaLang/julia/issues/15731
>

In another word, without giving up juxtapose multiplication, the
syntax comparison is between

dec"0.1" vs 0dec0.1

or

m"0.1" vs 0m0.1

which differs only by one character now.

[Tamas Papp]

I am unfamiliar with the application domain (banking etc), but I wonder
whether they need to have a lot of literals in the code --- my first
guess would be that values mostly come from a database.

Of course literals are useful for testing etc, but data on how prevalent
they are in code would be informative. Maybe there is an open-source
application for which one could do a quick grep?

Best,

Tamas

On Sun, Apr 03 2016, Tomas Lycken wrote:

> Sure. But you have to agree that it’s pretty clear that one of these two
> reads better than the other:
>
> # decimal literals with a string macro
> x = dec"10.3" + dec"5.8" * dec"0.15"
>

> # decimal literals with built-in support for parsing# m is the suffix for decimal in e.g. C#, works similar to e.g. Julia's 0.15f0 for Float32

> x = 10.3m + 5.8m * 0.15m
>
> Adding support for a few more of these kinds of literals - or even an
> extension mechanism to enable registering your own type and suffix - is
> definitely useful, even if the equivalent program can already be written
> with today’s features.
>
> // T
>
> On Sunday, April 3, 2016 at 2:49:53 PM UTC+2, Yichao Yu wrote:
>
> On Sun, Apr 3, 2016 at 8:27 AM, Stefan Karpinski <ste...@karpinski.org
>> <javascript:>> wrote:
>> > Custom parsing of numeric literals might be a plausible future feature.
>>
>> Custom parsing is what string macros are for.
>>
>> >
>> > On Sun, Apr 3, 2016 at 8:23 AM, Scott Jones <scott.pa...@gmail.com

>> <javascript:>>

[Scott Jones]

I'm not saying that a decimal literal syntax needs to be added right away to the language, but rather, change the way the femto-lisp parser converts literals to `Float64` before it is even seen by the rest of Julia, which means that it isn't possible to handle numeric literals specially in a macro at compile time.

It's an issue for any custom numeric type, BigFloat could also benefit from this change.

I don't think the default behavior should change at this point, or put much in Base (I'm maintaining a cut-down branch "Julia-lite" as it is), simply make the *minimal* changes in the parser and Base to allow more flexibility.

Once that change is done, it would be possible to make macros such as the following, which makes all floating point literals be decimal instead of binary:

@declits begin
     x = 0.70 * 0.05
    println(x)
end

[Eric Diaz]

I wish I could be of more help.

The only implementations that I have seen which comply with IEEE 754-2008 are

1. Intel® Decimal Floating-Point Math Library (https://software.intel.com/en-us/articles/intel-decimal-floating-point-math-library)
   
2. IBM Decimal number arithmetic module for the decNumber C Library (https://github.com/gcc-mirror/gcc/tree/master/libdecnumber)
   

The best summary so far of Decimal arithmetic has been in the dissertation:

"Algorithms and Hardware Designs for Decimal Multiplication" by Mark A. Erle in 2008
(http://speleotrove.com/decimal/erle2008-decimal-multipliers-dissertation-duplex.pdf)

My impression from the thesis is that this is a field of computation which has many opportunities for research and that the demand for DFP (Decimal Floating Point) may be something which the market will determine. 

[Tim Holy]

If' you're concerned that the existing packages don't meet these standards,
the best option would be to submit issues to the packages. You should submit
specific examples that don't work properly, rather than "please check that you
meet IEEE 754-2008" (which is basically a big homework assignment without
showing evidence that you care enough to bother testing things yourself). If
you end up writing a bunch of tests of things that aren't well-covered in the
package's test suite, then also please submit the tests (even if they are
things that already happen to work properly).

Echoing what others have said here, packages are not 2nd-class citizens in
julia---having functionality in base julia confers almost no advantage over
having it in a package, and indeed it sometimes makes development slower
because building julia is slower than building a package. Anytime you want new
functionality, your reflex should be (1) search for packages that already have
it, and if not (2) consider writing a package. Adding it to base julia should
only be your choice once you've already written a package, have it working,
but can identify a very specific reason why that functionality needs to move
into base julia. Even when I know (because I've been doing this a long time)
that something is going to have to be in base Julia, I invariably do most the
hard development in a "private package" before copying it into julia and
submitting it---it just makes the development oh-so-much easier.

Best,
--Tim

On Sunday, April 03, 2016 09:07:29 AM Eric Diaz wrote:
> I wish I could be of more help.
>
> The only implementations that I have seen which comply with IEEE 754-2008
> are
>

> 1. Intel® Decimal Floating-Point Math Library
>
> (https://software.intel.com/en-us/articles/intel-decimal-floating-point-mat
> h-library) 2. IBM Decimal number arithmetic module for the decNumber C

[Tomas Lycken]

Re:juxtaposition of multiplication, I'm well aware that e.g. 0.15m won't work with current Julia as it stands, and that it would be breaking to introduce something like it. However, there is already some special handling of this for 32-bit floats:

f0 = 2
@assert 0.5f0 == 0.5
@assert 0.5*f0 == 1

Somehow the parser knows that the first one isn't a case of juxtaposition - what I and Scott are suggesting is a way to, outside of Base, register literal syntaxes for other types.

I would prefer those to be suffixes, but prefixes would also be an improvement over the current situation. For example, even if the difference between 0m0.15 m"0.15" is only one character, syntax highlighting etc will work significantly better for the former.

// T

[Scott Jones]

Decimal arithmetic is as old as the hills, and the market uses it all the time.  It's simply in a rather different space than most of the people using Julia are used to.

Think COBOL (which used to be the most used language in the world), PL/1, MUMPS (huge in healthcare), Caché ObjectScript and other languages cater to decimal arithmetic,

as well as any common database (at least, "legacy" databases and relational databases)

IEEE 754-2008 just standardizes some decimal floating point formats (3 formats suited for software implementation, used by the Intel library [and the Julia wrapper package, DecFP.jl],

and another 3 better suited to hardware implementations [used by the IBM decnumber library, and also by the IBM POWER6 and up architectures, since 2008].

[Páll Haraldsson]

On Sunday, April 3, 2016 at 1:44:33 PM UTC, Tamas Papp wrote:

I am unfamiliar with the application domain (banking etc), but I wonder
whether they need to have a lot of literals in the code --- my first
guess would be that values mostly come from a database.

The last time I brought up the literal decimal issue, I dropped the issue, when I realized this to be the case (and the few literals actually integers..).

I assume division to also be rare, for decimals (and integers..), with:

dollars = cents / 100

to be the exception.

For banking, I assume division (by literal), only in that case, just before you display the result.

decimal / integer -> decimal (not Float64), should be possible, and ok.

[Andy Ferris]

Just to add my two cents, I can see two possibilities.

First the underscore `_` character has a special place in numerical literals already. Perhaps a trailing underscore could precede a numerical literal macro, like 123456789987654321_bigint or 1.024_dec (or _m). Then juxtaposition rules are only changed slightly for references beginning with _ (sort of like `1.0 * f0` would require a space in the juxtaposition).

Secondly, to get macros working nicely here it would be great if the expression parser left the numerical literals as Symbols, like `Symbol("1")` and `Symbol("1.0"). Perhaps they could convert them to constructor syntax :(1) == :(Int(Symbol("1"))) or similar. Or perhaps they can be a different expression object, NumericalSymbol or something, from which both the string/symbol and the value can be easily obtained. Perhaps inner constructors could also be defined specially for literals (and fully-literal expressions, like BigFloat(1/3)).

With these, the rest of the work could be done in the packages. As I see it, the bottleneck at the moment is with the parser.

Andy

[Páll Haraldsson]

On Wednesday, April 6, 2016 at 1:22:41 PM UTC, Andy Ferris wrote:

Just to add my two cents, I can see two possibilities.

I see zero possibilities! :) I mean Julia is fine as it is. 

First the underscore `_` character has a special place in numerical literals already. Perhaps a trailing underscore could precede a numerical literal macro, like 123456789987654321_bigint or 1.024_dec (or _m). Then juxtaposition rules are only changed slightly for references beginning with _ (sort of like `1.0 * f0` would require a space in the juxtaposition).

This has potential to make for syntax mistakes. I think adding anything to the parser is no needed, and can't be taken back. Do not do it unless, it's really needed.

I do not believe you'll see the number 1.024 in code (maybe not even often with out the decimal separator..), to it be important to support over 1024 // 1000. Using the REPL as a calculator and expecting non-binary decimal, is cool but maybe not needed..

julia> typeof(123456789987654321)

Int64

julia> typeof(12345678998765432100000)

Int128

julia> typeof(1234567899876543210000000000000000000000000000)

BigInt

@decimal function ..

..

  1.024

..

end

should also work, to get parser to read Float64 literal and change to Decimal.

[Páll Haraldsson]

On Wednesday, April 6, 2016 at 2:37:16 PM UTC, Páll Haraldsson wrote:

On Wednesday, April 6, 2016 at 1:22:41 PM UTC, Andy Ferris wrote:

Just to add my two cents, I can see two possibilities.

@decimal function ..

..

  1.024

..

end

A macro like this would do for me (or nothing at all), and the parsers doesn't need changeing. I did propose "switchable literals":

https://groups.google.com/forum/#!searchin/julia-dev/switchable$20literal$20types/julia-dev/YIfC1XL8Yd0/4mfG7J79HlcJ

but I see it doesn't need to be a command-line switch. It would be cool (I would like my literals "generic"), I'm not pressing for it, a macro should do.

[Kevin Squire]

The parser would need some small changes, because right now, there is no way to access the original non-parsed representation of a number (i.e., it's already a Float64 or Int64 by the time any macro sees it).  

Stephen suggested this could happen in the future earlier in this thread, in his comment about custom parsing of numeric literals.  Probably good to create an issue about this, if there isn't one already.

Cheers,

   Kevin