Negated exponent

[Jiří Činčura]

Hi *,

I'm looking at this https://github.com/FirebirdSQL/jaybird/blob/master/src/extern/org/firebirdsql/extern/decimal/DecimalCodec.java#L92 line. And wondering why the exponent is negated? With this, 0.10 will be C=10 and E=2, while I would expect E=-2. But maybe I'm confusing myself.

--
Mgr. Jiří Činčura
https://www.tabsoverspaces.com/

[Mark Rotteveel]

On 13-10-2020 13:45, Jiří Činčura wrote:
> Hi *,
>
> I'm looking at this https://github.com/FirebirdSQL/jaybird/blob/master/src/extern/org/firebirdsql/extern/decimal/DecimalCodec.java#L92 line. And wondering why the exponent is negated? With this, 0.10 will be C=10 and E=2, while I would expect E=-2. But maybe I'm confusing myself.
>

IEEE-754 Decimals use an exponent similar to the scientific notation (as
in 0.10 is 10E-2 or 10*10^-2), see also
http://speleotrove.com/decimal/dbspec.html. So for the encoding of an
IEEE-754 Decimal, your assumption is correct: C=10, E=-2.

However, java.math.BigDecimal uses a scale denoting the number of digits
after the decimal point (as in 0.10 is 10, scale=2), see also
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/math/BigDecimal.html.
To quote from the javadoc of java.math.BigDecimal:
"""
A BigDecimal consists of an arbitrary precision integer unscaled value
and a 32-bit integer scale. If zero or positive, the scale is the number
of digits to the right of the decimal point. If negative, the unscaled
value of the number is multiplied by ten to the power of the negation of
the scale. The value of the number represented by the BigDecimal is
therefore (unscaledValue × 10-scale)
"""
So to store the value in a java.math.BigDecimal, we need to use C=10,
scale=2, or scale=-E.

So, using the negative of the exponent converts the exponent as used by
the IEEE-754 Decimal encoding, to a scale for use with java.math.BigDecimal.

For readability it might have been better if I had written that line as:

```
int scale = -exponent;
return decimalFactory.createDecimal(signum, new BigDecimal(coefficient,
scale));
```

Mark
--
Mark Rotteveel

[Jiří Činčura]

> However, java.math.BigDecimal uses a scale denoting the number of
> digits
> after the decimal point (as in 0.10 is 10, scale=2), see also
> https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/math/BigDecimal.html.
> To quote from the javadoc of java.math.BigDecimal:

There's my explanation. Thanks Mark.