Is there a bug in the way VB6 handles bitwise hex comparison?
Andy T
Private Sub Command1_Click()
Print Hex(&H123456 And &HFF)
Print Hex(&H123456 And &HFF00)
Print Hex(&H123456 And &HFF0000)
End Sub
Outputs:
56
123400
120000
Surely this should have given 56, 3400 and 120000. Is this a bug, or don't I
understand hex, bitwise comparison or both?
This arose because I was trying to extract red, green and blue values from
the POINT method. Anybody have a quick way of doing this? I don't think
there are any functions to accomplish this directly.
--
Andy T
MDW
The second line works properly if you write it as &HFF00&.
Try printing &HFF00& and then &HFF00.
The first gives you 65280, whereas the second gives you -256.
I'm not entirely sure why VB misinterprets &HFF00 - maybe someone
smarter than me can explain it.
Murray
Andy T wrote in message <7m5vht$fap$1...@news4.svr.pol.co.uk>...
John Keyston
The reason the VB interprets &FF00 as -256 is because it uses signed integers
and signed long integers rather than unsigned ones. The most significant bit is
used to indicate the sign. Murray pointed out that you will get what you
expected if you add an & to the end of the Hex representation. This apparently
tells VB to interpret the number as unsigned. However, I cannot find any
explanation of this in the VB6 MSDN Help, and I have seen no explanation in any
of the VB books I have bought over the last four years. I have seen quite a few
samples of code which use it, but it is always assumed that you somehow know
what that extra & means - a self-evident truth!
Of course it would be easier if the Hex function gave some help. However,
Hex(-256) and Hex(65280) both give FF00. This means that for VB, CInt("&H" &
Hex(65280)) = -256 is true!
Oh well.........
John......
Mike Williams
that data type), so it only performs the AND with the lower two bytes of the
four byte Long &H123456. This results in &H3400 being in the lo order two
bytes and the original &H0012 remaining unchanged in the hi order two bytes,
resulting in the answer &H123400. In order to make VB get it right then you
have to use:
Print Hex(&H123456 And &HFF00&)
For Visual Basic to get the answer wrong simply because you did not use the
& (Long type declaration character) is rather silly, but for it to also get
the following wrong (which it does) is completely inexcusable:
Dim h As Long, j As Long, k As Long
h = &H123456
j = &HFF00
k = h And j
Print Hex(k)
We have Dimensioned all the variables as Longs. You would therefore expect
that they would all be treated correctly as Longs and we would get the
correct result. But we don't. The result is wrong. VB is an idiot! When the
VB interpreter gets to the line j = &HFF00 then you would expect it to
realise that j is a Long (we have already told it so) and you would expect
it to deal with &HFF00 accordingly. After all, it is merely being asked to
put the thing into a pre defined variable - surely that can't be too
difficult! What actually happens is that VB sees that &HFF00 will fit into a
two byte Integer. It calculates that &HFF00 as a two byte Integer is -256
(decimal) and so it puts -256 into the Long. Now -256 as a four byte Long is
&HFFFFFF00, and so that it what it puts into the variable j. This is the
source of our error. I am almost about to swear here. VB is F*c*i*g stupid!
In order for it to get the bloody thing right then you have to use:
Dim h As Long, j As Long, k As Long
h = &H123456
j = &HFF00&
k = h And j
Print Hex(k)
What you learn from this is that when you use numbers of any sort in an
expression (even in a simple assignment as in the above code) *always* use
the appropriate Type Declaration Character. If you give Visual Basic even
the slightest opportunity to f*c* it up then it will do so!
Anyway, enough of my ranting. Here are a couple of different ways of getting
your RGB values:
' Method 1
Dim p As String
Dim red As Long, green As Long, blue As Long
'
p = Right("000000" & Hex(Point(100, 100)), 6)
red = "&H" & Right(p, 2)
green = "&H" & Mid(p, 3, 2)
blue = "&H" + Left(p, 2)
' Method 2
'In a module place:
Declare Sub MemCopy Lib "kernel32" Alias "RtlMoveMemory" (ByRef Destination
As Any, ByRef Source As Any, ByVal Bytes As Long)
'Use the following code:
Dim a As Long, b(1 To 4) As Byte
'
a = Point(100, 100)
MemCopy b(1), a, 4
Print "Red =", b(1)
Print "Green =", b(2)
Print "Blue =", b(3)
Mike
Andy T <An...@3-wind.freeserve.co.uk> wrote in message
news:7m5vht$fap$1...@news4.svr.pol.co.uk...
Jason
&HFF00 is seen as an integer (due to the lack of the Long type symbol &)
&HFF00 in binary is 1111111100000000
As you can see the left most bit is set, specifying a negative number.
When the Long Type symbol is appended (&HFF00&) it is really seen by VB as
&H0000FF00&, which in binary is:
00000000000000001111111100000000
Here, the leftmost bit is not set and therefore a positive number.
Hopefully that answers your question Murray.
Jason
Derek T. Asari
MDW <mur...@home.com> writes:
> Andy:
> The second line works properly if you write it as &HFF00&.
Try printing &HFF00& and then &HFF00. The first gives you
65280, whereas the second gives you -256. I'm not entirely
sure why VB misinterprets &HFF00 - maybe someone smarter
than me can explain it.
MS BASIC's use the "&H" prefix in numeric constants to indicate
an integer hexadecimal type. Long integer hexadecimal types are
denoted by using both the "&H" prefix and the "&" suffix.
-----------------------------------------------------------------
Derek Asari
eq...@cleveland.freenet.edu
Cheaporfre
Use a trailing ampersand & at the end
of your mask values and you will get the correct answers, ie $HFF&.
Here is how I tested this problem.
Debug.Print Hex(&H123456 And &HFF&)
Debug.Print Hex(&H123456 And &HFF00&)
Debug.Print Hex(&H123456 And &HFF0000)
Best Wishes.
Jeff
Fedor Solodovnik
It is bug in your code
Print Hex(&H123456 And &HFF&)
Print Hex(&H123456 And &HFF00&)
Print Hex(&H123456 And &HFF0000&)
Dont mix long and integer if you don't understand how conversion is done
&HFF is integer
&HFF00 is integer (like -256 or -257?)
if integer and long are included in statement:
1) everything is done long (including sign propagation)
2) STATEMENT IS EVALUATED
so &H123456 And &HFF00 = &H123456 And &HFFFFFF00 = &H123400
as &HFF00 is negative
Andy T wrote:
>
> Try running this code:
>
> Private Sub Command1_Click()
> Print Hex(&H123456 And &HFF)
> Print Hex(&H123456 And &HFF00)
> Print Hex(&H123456 And &HFF0000)
> End Sub
>
> Outputs:
> 56
> 123400
> 120000
>
> Surely this should have given 56, 3400 and 120000. Is this a bug, or don't I
> understand hex, bitwise comparison or both?
>
> This arose because I was trying to extract red, green and blue values from
> the POINT method. Anybody have a quick way of doing this? I don't think
> there are any functions to accomplish this directly.
>
> --
> Andy T
--
Mailto:fsol...@ntmail.desy.de