WTF Double Precision

[Zh id]

I want to enter the number 0.0000247542259. But the recorded numbers don't match. 

I've changed the data type to double precision. When I enter the number 0.0004 it works. But when I enter the number 0.00004 it records 4E-5. There is something wrong?

And here :

Why is the result like that? even though I have used double precision.

 

This will be a problem when you need to see a 0 or a number that is close to 0 .

[Mark Rotteveel]

On 19-01-2021 10:38, Zh id wrote:
> I want to enter the number 0.0000247542259. But the recorded numbers
> don't match.
> I've changed the data type to double precision. When I enter the number
> 0.0004 it works. But when I enter the number 0.00004 it records 4E-5.
> There is something wrong?

There is nothing wrong here. 4E-5 is the scientific notation of 0.00004.
This is a presentation matter, which depends on the tooling you use to
view a query result. Also keep in mind, in double precision, some
numbers cannot be represented exactly.

> And here :
>
> Why is the result like that? even though I have used double precision.

> 2.jpg1.jpg

The result is like that because you're using double precision, and your
query tool uses scientific notation for very small (and probably very
large) values.

> This will be a problem when you need to see a 0 or a number that is
> close to 0 .

If you want precise results, you'll need to use DECIMAL or NUMERIC, or -
in Firebird 4 - DECFLOAT.

--
Mark Rotteveel

[liviuslivius]

It is realy not good to start support question from "WTF" ;-) 

Good that you got answer from Mark at all ;-)

Regards,

Karol Bieniaszewski

-------- Oryginalna wiadomość --------

Od: Mark Rotteveel <ma...@lawinegevaar.nl>

Data: 19.01.2021 11:10 (GMT+01:00)

Do: firebird...@googlegroups.com

Temat: Re: [firebird-support] WTF Double Precision

--
You received this message because you are subscribed to the Google Groups "firebird-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-suppo...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/firebird-support/adc90470-559a-8d9a-520c-b944177e4173%40lawinegevaar.nl.

[Tim Crawford]

I am embarrassed to admit that I had long ago forgotten about how floats work
and came across a similar WTF moment recently.

See this: https://en.wikipedia.org/wiki/Floating-point_arithmetic

"In computing, floating-point arithmetic (FP) is arithmetic using formulaic representation of real numbers as an approximation to support a trade-off between range and precision"

IMHO, for any RDBMS, don't use float or double unless you are storing raw scientific data
and do not have to rely on reconciling to other calculated numbers.
Use Numeric/Decimal as a rule rather than exception

Note: in future you might also investigate the existance of the png file format.
Seriously, 1.4 MB is a bit much for two screen shots

--
You received this message because you are subscribed to the Google Groups "firebird-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-suppo...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/firebird-support/6e220653-cf0e-491b-b696-8fa1142c4f13n%40googlegroups.com.

[Zh id]

I am sorry.  Previously I just asked, then run and it doesn't work.

[Dimitry Sibiryakov]

19.01.2021 16:32, Tim Crawford wrote:
> Note: in future you might also investigate the existance of the png file format.
> Seriously, 1.4 MB is a bit much for two screen shots

Better yet would be to discover text copy/paste functions.

--
WBR, SD.

[Mark Rotteveel]

Or the fact that twitter message can be linked to, so we know the actual
context of that reply.

Mark
--
Mark Rotteveel

[Mark Rotteveel]

Although in this case it wouldn't have been helpful, because the TS
hides there tweets...
https://twitter.com/firebirdsql/status/1346470670583037955?s=20

--
Mark Rotteveel

[Zh id]

https://twitter.com/ProductiveStay/status/1345927069490823168?s=19

[Mark Rotteveel]

On 19-01-2021 16:56, Zh id wrote:
> Screenshot_2021-01-19-22-53-09-118_com.twitter.android_1.jpg
>
> I am sorry.  Previously I just asked, then run and it doesn't work.

The @firebirdsql Twitter account isn't meant as a support channel, and
because you did not include the original question (nor is it visible at
https://twitter.com/firebirdsql/status/1346470670583037955?s=20), we
don't know the context of that reply (i.e. what the original question was).

You may want to start again by presenting your actual problem, instead
of only saying WTF, and not really describing the problem, but having us
read between the lines.

Please also refrain from using further screenshots, when you can instead
post queries and result as **text**.

Mark
--
Mark Rotteveel

[Dimitry Sibiryakov]

19.01.2021 17:08, Zh id wrote:
> https://twitter.com/ProductiveStay/status/1345927069490823168?s=19

If you want to enter this number and get it back as is you must use NUMERIC data type
with precision high enough (and database dialect 3).

--
WBR, SD.

[Mark Rotteveel]

On 19-01-2021 17:08, Zh id wrote:
> https://twitter.com/ProductiveStay/status/1345927069490823168?s=19

Thanks, but that doesn't really help with understanding your actual
problem. This looks like an XY problem to me.

Given the number 0.0000247542259, DOUBLE PRECISION seems suitable, but
you could also have used DECIMAL(18,13), but again, without knowing the
actual problem you're trying to solve, that is just guessing.

Please describe the actual problem you're trying to solve, and also show
necessary DDL and sample data, etc.

Mark
--
Mark Rotteveel

[Bogdan Mugerli]

Hi

You are all very sweet

Regards
Bogdan Mugerli

---

From: firebird...@googlegroups.com <firebird...@googlegroups.com> on behalf of Zh id <zahir...@gmail.com>
Sent: Tuesday, January 19, 2021 5:08:53 PM
To: firebird-support <firebird...@googlegroups.com>
Subject: Re: [firebird-support] WTF Double Precision

 

--
You received this message because you are subscribed to the Google Groups "firebird-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-suppo...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/firebird-support/53aafe97-cd68-4204-a79b-36cd5670cc24n%40googlegroups.com.