Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

ADO Stream changes float values

12 views
Skip to first unread message

David Baldi

unread,
Sep 26, 2001, 8:28:02 PM9/26/01
to
I''m using the Recordset.Save method to save recordset
data to a Stream, and then Stream.ReadText to save the
stream to an XML DOM. Everything worked fine until I
tested Float values.

I have a column in the DB (SQL Server) defined as Float
with precision of 15. The data in the Recordset object
shows OK, but after using the stream, the value is changed
to fully use the precision. For example, the data in the
recordset may be 1.4, but after the Stream.ReadText
method, it becomes 1.399999999999996.

I've crawled MSDN and found nothing so far. Any help is
appreciated.

Andreas Walkenhorst

unread,
Sep 27, 2001, 8:45:16 AM9/27/01
to

"David Baldi" <dba...@hesinet.com> wrote in message
news:33cf01c146eb$44a0c3d0$3def2ecf@TKMSFTNGXA14...

> I''m using the Recordset.Save method to save recordset
> data to a Stream, and then Stream.ReadText to save the
> stream to an XML DOM. Everything worked fine until I
> tested Float values.
>
> I have a column in the DB (SQL Server) defined as Float
> with precision of 15. The data in the Recordset object
> shows OK, but after using the stream, the value is changed
> to fully use the precision. For example, the data in the
> recordset may be 1.4, but after the Stream.ReadText
> method, it becomes 1.399999999999996.

Hi David,
Somewhere in your stream an implicit type conversion must exist. If you used
an unbound rs, you just have to define the type of the fields (if you miss
that ado does a conversion from float to variant to float, which can be a
reason of the strage behaviour), if you used only bound rs, try to use cdbl
where you pass data from / to the rs (btw here it is useful too to check the
fields.type method)
If nothing of these 2 helps, you may need to write a round function (be
VERY!! careful with the Microsoft round function, better do not use it).

Andreas

Stephen Howe

unread,
Sep 27, 2001, 12:39:09 PM9/27/01
to

David Baldi <dba...@hesinet.com> wrote in message
news:33cf01c146eb$44a0c3d0$3def2ecf@TKMSFTNGXA14...
> I''m using the Recordset.Save method to save recordset
> data to a Stream, and then Stream.ReadText to save the
> stream to an XML DOM. Everything worked fine until I
> tested Float values.
>
> I have a column in the DB (SQL Server) defined as Float
> with precision of 15. The data in the Recordset object
> shows OK, but after using the stream, the value is changed
> to fully use the precision. For example, the data in the
> recordset may be 1.4, but after the Stream.ReadText
> method, it becomes 1.399999999999996.

You should know of a few things about float/doubles that are not apparent.

In decimal (base 10), the fraction 1/3 (one third) cannot be expressed to an
exact number of digits.
It is 0.3333333333333333... forever. It is said to "recur" (mathematical
jargon).
That means that no matter where you truncate the digits, be it 6 places or
15 places, it will never be exact.

Now floats/doubles are stored in a binary (base 2) format. In binary format,
all fractions recur except those that are powers of 1/2. So 1/2, 3/4, 5/8
can be represented exactly but not 1/10. So fractions like 0.1, 0.03 cannot
be stored exactly. So your number 1.4 cannot be represented exactly and that
means that when Stream.ReadText() reads your number in, it will choose the
nearest fraction that is some power of 1/2 (1.399999999999996), not the
nearest decimal fraction (1.4).

Most programmers go through shock when they realise that fp is like this,
then anger that they don't get the answers they want from fp, then despair
as to whether they can ever trust fp again. But it is possible to use fp
reasonably safely.

The fact is 1.4 cannot be stored exactly in a float/double anyway. You
cannot do it, it is impossible!!! What you have in the fp variable is the
nearest approximation. You should accept that and just make sure that when
you display, you display to 2 decimal places. At the point, you will be
displaying 1.40

Stephen Howe

.


David Baldi

unread,
Sep 27, 2001, 12:52:14 PM9/27/01
to
>The fact is 1.4 cannot be stored exactly in a
float/double anyway. You
>cannot do it, it is impossible!!! What you have in the fp
variable is the
>nearest approximation. You should accept that and just
make sure that when
>you display, you display to 2 decimal places. At the
point, you will be
>displaying 1.40

I fully appreciate the way float values are stored, but
that's not at issue. The question is, rather: why is 1.4
successfully returned by the Recordset, but mangled by the
Stream? The RS represents the real "storage" - the stream
is just an intermediate form. I see no reason why I should
have to control the decimal places in display when using
data from a Stream when I don't have to do that when
pulling directly from the RS.

0 new messages