Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Shortint calculations
134 views
Skip to first unread message
kvste...@web.de
May 27, 2013, 11:53:43 AM5/27/13
to
Hi out there,
I'm getting strange results with simple calculations with shortints. Consider this mini-test-app:
FUNCTION Start()
LOCAL nA, nB, nC AS SHORTINT
nA := 748
nB := 22
nC := 35
IF nA > ( nB - nC )
? "748 is greater than (22 - 35)"
ELSE
? "748 is not greater than (22 - 35). Oops!"
ENDIF
_Wait("")
Try it and tell me what happens here. When the three variables are declared as longint, everything seems ok. But when they are shortints, the result is obviously wrong. It seems to me that the results here are well within the scope of a shortint, so what am I missing? (Using VO 2.8 Sp2)
Thanks for input
Kay-Viktor
I'm getting strange results with simple calculations with shortints. Consider this mini-test-app:
FUNCTION Start()
LOCAL nA, nB, nC AS SHORTINT
nA := 748
nB := 22
nC := 35
IF nA > ( nB - nC )
? "748 is greater than (22 - 35)"
ELSE
? "748 is not greater than (22 - 35). Oops!"
ENDIF
_Wait("")
Try it and tell me what happens here. When the three variables are declared as longint, everything seems ok. But when they are shortints, the result is obviously wrong. It seems to me that the results here are well within the scope of a shortint, so what am I missing? (Using VO 2.8 Sp2)
Thanks for input
Kay-Viktor
Karl Faller
May 27, 2013, 12:09:08 PM5/27/13
to
Kay,
strange indeed - i see it with Sp 4 but not with VN2...
Karl
strange indeed - i see it with Sp 4 but not with VN2...
Karl
dlzc
May 27, 2013, 1:21:40 PM5/27/13
to
Dear kvste...:
On Monday, May 27, 2013 8:53:43 AM UTC-7, kvste...@web.de wrote:
...
> It seems to me that the results here are well
> within the scope of a shortint, so what am I missing?
> (Using VO 2.8 Sp2)
Shortint might be unsigned, so the amount that nC is greater than nB, is deducted from 65536.
David A. Smith
On Monday, May 27, 2013 8:53:43 AM UTC-7, kvste...@web.de wrote:
...
> LOCAL nA, nB, nC AS SHORTINT
>
> nA := 748
> nB := 22
> nC := 35
>
> IF nA > ( nB - nC )
> ? "748 is greater than (22 - 35)"
> ELSE
> ? "748 is not greater than (22 - 35). Oops!"
> ENDIF
...
>
> nA := 748
> nB := 22
> nC := 35
>
> IF nA > ( nB - nC )
> ? "748 is greater than (22 - 35)"
> ELSE
> ? "748 is not greater than (22 - 35). Oops!"
> ENDIF
>
> When the three variables are declared as longint,
> everything seems ok. But when they are shortints,
> the result is obviously wrong.
Are shortints signed? Can you print (nB - nC)?
> When the three variables are declared as longint,
> everything seems ok. But when they are shortints,
> the result is obviously wrong.
> It seems to me that the results here are well
> within the scope of a shortint, so what am I missing?
> (Using VO 2.8 Sp2)
David A. Smith
Karl Faller
May 27, 2013, 2:32:53 PM5/27/13
to
>Are shortints signed? Can you print (nB - nC)?
gives -13, as it should...
Karl
kvste...@web.de
May 27, 2013, 2:46:25 PM5/27/13
to
> Are shortints signed? Can you print (nB - nC)?
David,
shortints are signed and should have no problems with negative values in this range. I've had enough headaches with word/dword values within calculations therefore I preferred shortints, but now this ...
Kay-Viktor
Karl Faller
May 27, 2013, 5:40:58 PM5/27/13
to
Kay-Viktor,
really matters vs "regular" ints?
Not saying, that your results are pointing on some bug, which should
be corrected, but in this case i think the workaround relatively easy
;-)
Karl
>values within calculations therefore I preferred shortints, but now this ...
any reason to use shorts? Do you such number crunching that the size
really matters vs "regular" ints?
Not saying, that your results are pointing on some bug, which should
be corrected, but in this case i think the workaround relatively easy
;-)
Karl
Mathias
May 28, 2013, 1:39:08 AM5/28/13
to
This sample may be en explanation, but I still Think it's very confusing. It seems like if all the numbers are longints. But maybe only nb - nC is a longint and something goes wrong when comparing a shortint and a longint in nA > ( nB - nC )?
/Mathias
function Start()
local nA, nB, nC as shortint
nA := 748
nB := 22
nC := 35
? "Shortint?"
? UsualType(nB - nC) == shortint
? "Longint?"
? UsualType(nB - nC) == longint
? "Shortint?"
? UsualType(nA) == shortint
? "Longint?"
? UsualType(nA) == longint
? nB - nC
? nA
? nA > ( nB - nC )
? longint(nA) > ( nB - nC )
if longint(nA) > nB - nC
_Wait("")
return nil
/Mathias
function Start()
local nA, nB, nC as shortint
nA := 748
nB := 22
nC := 35
? UsualType(nB - nC) == shortint
? "Longint?"
? UsualType(nB - nC) == longint
? "Shortint?"
? UsualType(nA) == shortint
? "Longint?"
? UsualType(nA) == longint
? nB - nC
? nA
? nA > ( nB - nC )
? longint(nA) > ( nB - nC )
if longint(nA) > nB - nC
? "748 is greater than (22 - 35)"
else
? "748 is not greater than (22 - 35). Oops!"
endif
_Wait("")
return nil
Wolfgang Riedmann
May 28, 2013, 2:02:48 AM5/28/13
to
Hi Kay-Viktor,
> shortints are signed and should have no problems with negative values
> in this range. I've had enough headaches with word/dword values
> within calculations therefore I preferred shortints, but now this ...
on a 32 bit system it is better to user int and dword because they are
32 bit values. Shortint and word are 16 bit values and have a
performance penalty because they need to be aligned to 32 bit.
Wolfgang
P.S. I think that VO behaves erratically in your sample
--
> shortints are signed and should have no problems with negative values
> in this range. I've had enough headaches with word/dword values
> within calculations therefore I preferred shortints, but now this ...
32 bit values. Shortint and word are 16 bit values and have a
performance penalty because they need to be aligned to 32 bit.
Wolfgang
P.S. I think that VO behaves erratically in your sample
--
Karl Faller
May 28, 2013, 3:01:36 AM5/28/13
to
Mathias & all,
just had a look, my compiler options has set (as per default)
"Overflow checking", so i'd expect a compiletime error at nA := 748,
not a "silent" cast. Stranger still, why, if it does the casts, the
compare goes wrong. Strangest, that the same code in VN2 gives the
correct compare, but also no warning at the assignment of nA...
Karl
just had a look, my compiler options has set (as per default)
"Overflow checking", so i'd expect a compiletime error at nA := 748,
not a "silent" cast. Stranger still, why, if it does the casts, the
compare goes wrong. Strangest, that the same code in VN2 gives the
correct compare, but also no warning at the assignment of nA...
Karl
kvste...@web.de
May 28, 2013, 4:02:28 AM5/28/13
to
Karl,
why should there be any sort of error at nA := 748 ? A shortint should be perfectly capable of holding a value like 748. The range of shortints is -32767 to +32768, correct me if I'm wrong. Therefore all the mentioned operations should work perfectly.
To all: The original reason why I used shortints sometimes (in this case the whole thing is about window size calculations) was because some Windows API functions require shortint parameters. But I would never have suspected that shortint computations are broken in VO (after all those years ...). I will change them to longints and hope the headache goes away.
Have a nice day
Kay-Viktor
why should there be any sort of error at nA := 748 ? A shortint should be perfectly capable of holding a value like 748. The range of shortints is -32767 to +32768, correct me if I'm wrong. Therefore all the mentioned operations should work perfectly.
To all: The original reason why I used shortints sometimes (in this case the whole thing is about window size calculations) was because some Windows API functions require shortint parameters. But I would never have suspected that shortint computations are broken in VO (after all those years ...). I will change them to longints and hope the headache goes away.
Have a nice day
Kay-Viktor
Karl Faller
May 28, 2013, 4:32:43 AM5/28/13
to
Kay,
* delphibasics.com says: "ShortInt type is the smallest form of
signed integer, occupying 8 bits (1 byte) of storage. It supports only
the integers from -128 to 127. "
* wikipedia says: Short/short, Integer signed -32.768 32.767
* VO pre release manuals <g>, says 16bit int, as in Ansi C short
Good, that i never used them...
Karl
>why should there be any sort of error at nA := 748 ? A shortint should be perfectly capable of holding a value like 748. The range of shortints is -32767 to +32768, correct me if I'm wrong.
that was my thinking, too, but then i googled and found:
* delphibasics.com says: "ShortInt type is the smallest form of
signed integer, occupying 8 bits (1 byte) of storage. It supports only
the integers from -128 to 127. "
* wikipedia says: Short/short, Integer signed -32.768 32.767
* VO pre release manuals <g>, says 16bit int, as in Ansi C short
Good, that i never used them...
Karl
kvste...@web.de
May 28, 2013, 4:46:13 AM5/28/13
to
Am Dienstag, 28. Mai 2013 10:32:43 UTC+2 schrieb Karl Faller:
> that was my thinking, too, but then i googled and found:
>
> * delphibasics.com says: "ShortInt type is the smallest form of
>
> signed integer, occupying 8 bits (1 byte) of storage. It supports only
>
> the integers from -128 to 127. "
>
>
>
> * wikipedia says: Short/short, Integer signed -32.768 32.767
>
>
>
> * VO pre release manuals , says 16bit int, as in Ansi C short
> that was my thinking, too, but then i googled and found:
>
> * delphibasics.com says: "ShortInt type is the smallest form of
>
> signed integer, occupying 8 bits (1 byte) of storage. It supports only
>
> the integers from -128 to 127. "
>
>
>
> * wikipedia says: Short/short, Integer signed -32.768 32.767
>
>
>
Hi Karl,
VO shortints are 16 bit as stated in the manual, therefore there should be no problem storing the values. The bug must be in the operations somewhere. This is definitely a VO bug.
Kay-Viktor
Meinhard Schnoor-Matriciani
May 28, 2013, 5:45:31 AM5/28/13
to
Mathias,
this approach must lead to confusions, because UsualType would never
return something like Shortint ( see documentation ).
I am going to have a look at this.
Regards
Meinhard
--
Meinhard Schnoor-Matriciani
Grafx Development Team
this approach must lead to confusions, because UsualType would never
return something like Shortint ( see documentation ).
I am going to have a look at this.
Regards
Meinhard
Meinhard Schnoor-Matriciani
Grafx Development Team
dlzc
May 28, 2013, 10:15:04 AM5/28/13
to
> LOCAL nA, nB, nC AS SHORTINT
>
> nA := 748
> nB := 22
> nC := 35
>
> IF nA > ( nB - nC )
Here is the problem, the result of subtraction is NOT a shortint. It is (probably) a long integer. When compared to the shortint nA, the result of subtraction may be being truncated, rather than the result being demoted to a shortint, or nA being promoted to a long int.
>
> nA := 748
> nB := 22
> nC := 35
>
> IF nA > ( nB - nC )
> ? "748 is greater than (22 - 35)"
> ELSE
> ? "748 is not greater than (22 - 35). Oops!"
> ENDIF
David A. Smith
Mathias
May 29, 2013, 2:45:04 AM5/29/13
to
Hi Meinard,
I suspected that usuals don't carry shortints. However the whole thing is very confusing and it gets even more confusing if I modify the testprogram like below. It seems like it works if I calculate the subtraction before comparing.
function Start()
local nA, nB, nC, nD as shortint
nA := 748
nB := 22
nC := 35
nD := nB - nC
? nB - nC // -13
? nD // -13
? nA // 748
? nA > nB - nC // .F.
? nA > nD // .T.
? longint(nA) > ( nB - nC ) // .T.
if longint(nA) > nB - nC
? "748 is greater than (22 - 35)"
else
? "748 is not greater than (22 - 35). Oops!"
endif
_Wait("")
return nil
I suspected that usuals don't carry shortints. However the whole thing is very confusing and it gets even more confusing if I modify the testprogram like below. It seems like it works if I calculate the subtraction before comparing.
function Start()
local nA, nB, nC, nD as shortint
nA := 748
nB := 22
nC := 35
? nB - nC // -13
? nD // -13
? nA // 748
? nA > nB - nC // .F.
? nA > nD // .T.
? longint(nA) > ( nB - nC ) // .T.
if longint(nA) > nB - nC
? "748 is greater than (22 - 35)"
else
? "748 is not greater than (22 - 35). Oops!"
endif
_Wait("")
return nil
Mathias
May 29, 2013, 7:44:22 AM5/29/13
to
Hi David,
no it's more complicated than this...
? shortint(nA) > nD // .T.
No surprise, as nA > nD also was true. nA is allready a shortint so this is expected.
? shortint(nA) > nB - nC // .F.
Expected...
? nA > shortint(nD) // .F.
What? nD is a shortint. Converting to a shortint should have no effect as nD allready is a shortint.
? shortint(nA) > shortint(nD) // .F.
Strange ideed...
/MathiasH
no it's more complicated than this...
? shortint(nA) > nD // .T.
No surprise, as nA > nD also was true. nA is allready a shortint so this is expected.
? shortint(nA) > nB - nC // .F.
Expected...
? nA > shortint(nD) // .F.
What? nD is a shortint. Converting to a shortint should have no effect as nD allready is a shortint.
? shortint(nA) > shortint(nD) // .F.
Strange ideed...
/MathiasH
dlzc
May 29, 2013, 10:25:19 AM5/29/13
to
Dear Mathias:
On Wednesday, May 29, 2013 4:44:22 AM UTC-7, Mathias wrote:
...
Really not, but you did find the exact cause, I think.
> ? shortint(nA) > nB - nC // .F.
>
> Expected...
... since nA was already a shortint, and (nB - nC) is a long int (or worse)
> ? nA > shortint(nD) // .F.
>
> What? nD is a shortint. Converting to a shortint
> should have no effect as nD allready is a shortint.
The problem is sign extending in shortint conversion. Apparently conversion occurs even if the data type is already shortint.
> ? shortint(nA) > shortint(nD) // .F.
>
> Strange ideed...
The conversion fails miserably on negatives.
David A. Smith
On Wednesday, May 29, 2013 4:44:22 AM UTC-7, Mathias wrote:
...
> no it's more complicated than this...
...
Really not, but you did find the exact cause, I think.
> ? shortint(nA) > nB - nC // .F.
>
> Expected...
> ? nA > shortint(nD) // .F.
>
> What? nD is a shortint. Converting to a shortint
> should have no effect as nD allready is a shortint.
> ? shortint(nA) > shortint(nD) // .F.
>
> Strange ideed...
David A. Smith
Er!k \/!$$Er
May 29, 2013, 10:57:06 AM5/29/13
to
Mathias,
I have tried this code on VO 2.6, VO 2.7 build 2773 and VO 2.8 build 2837
and it works fine here.
Erik
-13
-13
748
.F.
.T.
.T.
news:018d375d-3c3f-4ba3...@googlegroups.com...
> > Den m�ndagen den 27:e maj 2013 kl. 17:53:43 UTC+2 skrev
--- news://freenews.netfront.net/ - complaints: ne...@netfront.net ---
I have tried this code on VO 2.6, VO 2.7 build 2773 and VO 2.8 build 2837
and it works fine here.
Erik
-13
-13
748
.F.
.T.
.T.
748 is greater than (22 - 35)
"Mathias" schreef in bericht
news:018d375d-3c3f-4ba3...@googlegroups.com...
dlzc
May 29, 2013, 12:33:47 PM5/29/13
to
Dear Er!k /!$$Er:
On Wednesday, May 29, 2013 7:57:06 AM UTC-7, Er!k /!$$Er wrote:
> Mathias,
>
> I have tried this code on VO 2.6, VO 2.7 build 2773
> and VO 2.8 build 2837 and it works fine here.
>
>
> -13
> -13
> 748
> .F.
This above says 748 is not greater than -13. Do you want to revise your "fine"?
> .T.
> .T.
> 748 is greater than (22 - 35)
...
I wonder what:
? shortint( longint( -13 )) // looks like?
David A. Smith
On Wednesday, May 29, 2013 7:57:06 AM UTC-7, Er!k /!$$Er wrote:
> Mathias,
>
> I have tried this code on VO 2.6, VO 2.7 build 2773
> and VO 2.8 build 2837 and it works fine here.
>
>
> -13
> 748
> .F.
This above says 748 is not greater than -13. Do you want to revise your "fine"?
> .T.
> .T.
> 748 is greater than (22 - 35)
> function Start()
>
> local nA, nB, nC, nD as shortint
>
> nA := 748
> nB := 22
> nC := 35
>
> nD := nB - nC
>
> ? nB - nC // -13
> ? nD // -13
> ? nA // 748
> ? nA > nB - nC // .F.
> ? nA > nD // .T.
> ? longint(nA) > ( nB - nC ) // .T.
>
> if longint(nA) > nB - nC
> ? "748 is greater than (22 - 35)"
> else
> ? "748 is not greater than (22 - 35). Oops!"
> endif
>
> _Wait("")
>
> return nil
Erik's results say that this has been broken for a long time. And it looks like a negative longint result is downcoverted incorrectly to a shortint.
>
> local nA, nB, nC, nD as shortint
>
> nA := 748
> nB := 22
> nC := 35
>
> nD := nB - nC
>
> ? nB - nC // -13
> ? nD // -13
> ? nA // 748
> ? nA > nB - nC // .F.
> ? nA > nD // .T.
> ? longint(nA) > ( nB - nC ) // .T.
>
> if longint(nA) > nB - nC
> ? "748 is greater than (22 - 35)"
> else
> ? "748 is not greater than (22 - 35). Oops!"
> endif
>
> _Wait("")
>
> return nil
I wonder what:
? shortint( longint( -13 )) // looks like?
David A. Smith
Jamal
May 29, 2013, 5:18:10 PM5/29/13
to
Hi Kay-Victor,
It seems to be a bug the > operator internal handling of Short INT.
If you reverse the test to use the < (less than operator):
IF ( nB - nC ) < nA
? "(22 - 35) is less than 748"
ENDIF
It works as expected. So try putting the complex expression part in left
hand of the comparison.
Jamal
wrote in message
news:aa22a440-8e92-444c...@googlegroups.com...
It seems to be a bug the > operator internal handling of Short INT.
If you reverse the test to use the < (less than operator):
IF ( nB - nC ) < nA
? "(22 - 35) is less than 748"
ENDIF
It works as expected. So try putting the complex expression part in left
hand of the comparison.
Jamal
wrote in message
news:aa22a440-8e92-444c...@googlegroups.com...
kvste...@web.de
May 30, 2013, 4:47:42 AM5/30/13
to
Am Mittwoch, 29. Mai 2013 23:18:10 UTC+2 schrieb Jamal:
> So try putting the complex expression part in left hand of the comparison.
Jamal,
> So try putting the complex expression part in left hand of the comparison.
there's really no more need of another workaround. If I want a workaround, I just use longints. The question is, where to use workarounds? What exactly is broken? Is it the > operator with shortints? Is it the evaluation of simple operations like subtractions with shortints? (Not really a "complex expression" ...) Is it an exotic combination of expressions?
I hope Meinhard finds out, so that we can spot the places where a workaround is needed. And then of course, a fix would be nice.
Jamal
May 30, 2013, 1:19:05 PM5/30/13
to
Kay-Victor,
Good luck waiting for a solution. The compiler people like Fru left the Dev
team since CA abandoned the product.
Jamal
wrote in message
news:a168c874-1b2f-4438...@googlegroups.com...
Good luck waiting for a solution. The compiler people like Fru left the Dev
team since CA abandoned the product.
Jamal
wrote in message
news:a168c874-1b2f-4438...@googlegroups.com...
Meinhard Schnoor-Matriciani
May 30, 2013, 4:05:58 PM5/30/13
to
Hi Kay-Victor,
just looked into the compiler to see what happens here. Roughly spoken
it happens the following (it is quite tricky to find these things out as
there is nearly no documentation) :
- the arithmetic expression is being evaluated on 32bit base even
shortints are only 16bits
- the comparison is done by using the right hand intermediate datatype
which is dword in this case ( which is wrong I admit, internally the
32bit registers are used and the values in there are just dwords ) I you
example a value of 0xfffff3 is used for the comparison and this is the
reason while the comparison fails
If you turn it the other way round ( nB - nC ) <= nA it works because
the result of (nB-nC) get converted to SHORTINT before the comparison
takes place because the right hand part results in the datatype SHORINT.
Another, abstruse way, to get around the problem is to use
IF nA > SHORTINT(DWORD((nB - nC ))) ....
I have checked the old versions as well and this problem is there at
least since 2.0 ....
Regards
Meinhard
Am 27.05.2013 17:53, schrieb kvste...@web.de:
> FUNCTION Start()
>
> LOCAL nA, nB, nC AS SHORTINT
>
> nA := 748
> nB := 22
> nC := 35
>
> IF nA > ( nB - nC )
> ? "748 is greater than (22 - 35)"
> ELSE
> ? "748 is not greater than (22 - 35). Oops!"
> ENDIF
>
> _Wait("")
just looked into the compiler to see what happens here. Roughly spoken
it happens the following (it is quite tricky to find these things out as
there is nearly no documentation) :
- the arithmetic expression is being evaluated on 32bit base even
shortints are only 16bits
- the comparison is done by using the right hand intermediate datatype
which is dword in this case ( which is wrong I admit, internally the
32bit registers are used and the values in there are just dwords ) I you
example a value of 0xfffff3 is used for the comparison and this is the
reason while the comparison fails
If you turn it the other way round ( nB - nC ) <= nA it works because
the result of (nB-nC) get converted to SHORTINT before the comparison
takes place because the right hand part results in the datatype SHORINT.
Another, abstruse way, to get around the problem is to use
IF nA > SHORTINT(DWORD((nB - nC ))) ....
I have checked the old versions as well and this problem is there at
least since 2.0 ....
Regards
Meinhard
Am 27.05.2013 17:53, schrieb kvste...@web.de:
> FUNCTION Start()
>
> LOCAL nA, nB, nC AS SHORTINT
>
> nA := 748
> nB := 22
> nC := 35
>
> IF nA > ( nB - nC )
> ? "748 is greater than (22 - 35)"
> ELSE
> ? "748 is not greater than (22 - 35). Oops!"
> ENDIF
>
> _Wait("")
kvste...@web.de
May 31, 2013, 6:43:30 AM5/31/13
to
Hi Meinhard,
thanks for checking that. So when I want to play it safe, I could say as a rule of thumb, "don't use expressions containing shortints on the right side of a comparison operator". Right?
In other words, I could prevent this sort of problem by either
- not using shortints at all, or
- evaluating shortint expressions into a separate variable before applying a comparison operator (The "nD" approach :)
Have a nice weekend
Kay-Viktor
thanks for checking that. So when I want to play it safe, I could say as a rule of thumb, "don't use expressions containing shortints on the right side of a comparison operator". Right?
In other words, I could prevent this sort of problem by either
- not using shortints at all, or
- evaluating shortint expressions into a separate variable before applying a comparison operator (The "nD" approach :)
Have a nice weekend
Kay-Viktor
0 new messages