gawk integer conversions

[steve...@gmail.com]

Hi,

I am trying to pretty print some data presented as a bunch of hex
bytes:

0xf0 0xd8 - for a 16 bit example.

I use:
#little endian build a number
arg = 0
for (i = fld + len; i > fld ; i--) {
arg = (arg * 256) + strtonum($(i))
}

And this does collect the multi byte number 0xd8f0 and I can print it
as 55536 in decimal.

The problem is the data is signed. So -1 is represented as one byte of
0xff - which the above conversion will give me as 255.

Since gawk actually holds numbers as double floats, how can I sign
extend the integer sign?

Thanks, Steve

[Cesar Rabak]

steve...@gmail.com escreveu:

The best idea I can come is masking and testing the MSB multiplying the
result by -1 accordingly...

[John DuBois]

Negating won't produce the correct number; the description indicates that the
data is stored in 2's complement.

For 16-bit numbers:

if (arg > 32767)
arg = arg - 65536

Similar for 8-bit, etc.

John
--
John DuBois spc...@armory.com KC6QKZ/AE http://www.armory.com/~spcecdt/

[Grant]

grant@deltree:~$ awk -f xxx
0xf0 0xd8: -22768
0xf0 0x58: 22768
grant@deltree:~$ cat xxx
function convert_le(fld, len, arg, s)
{
# sign extend the first (ms) byte
if (and(strtonum($(fld -1 +len)), 0x80)) {
arg = 0xffffffff
s = 1
}
else
arg = s = 0
for (i = fld-1 + len; i >= fld ; i--) {

arg = (arg * 256) + strtonum($(i))
}

# assume 16 bit signed output
arg = and(arg, 0x7fff)
if (s)
return -arg
return arg
}
BEGIN {
$0 = "0xf0 0xd8"
printf "%s: %-d\n", $0, convert_le(1,2)
$0 = "0xf0 0x58"
printf "%s: %-d\n", $0, convert_le(1,2)
}

Hope this helps...

Grant.
--
http://bugsplatter.mine.nu/

[Cesar Rabak]

John DuBois escreveu:

Right. For 2's complement the math operation will have to be performed
afterward the conversion of more bit fiddling to arrive to the correct
result.

[spamtrap]

Grant <g_r_a...@dodo.com.au> wrote in
news:4hdd74lucnq1p8aus...@4ax.com:

> On Thu, 10 Jul 2008 09:23:32 -0700 (PDT), steve...@gmail.com wrote:
>
>>Hi,
>>
>>I am trying to pretty print some data presented as a bunch of hex
>>bytes:
>>
>>0xf0 0xd8 - for a 16 bit example.
>>
>>I use:
>>#little endian build a number
>> arg = 0
>> for (i = fld + len; i > fld ; i--) {

>> arg = (arg * 256) + strtonum($(i))Hi Grant,

You gave me the idea - Here is my final solution 2s complement, multi-
length byte strings, le conversion:

#little endian build a number

if (and(128,strtonum($(fld + len))) == 128 ) {
arg = -1;
} else {

arg = 0;
}
for (i = fld + len; i > fld ; i--) {
arg = (arg * 256) + strtonum($(i))
}

This way it doesn't even matter that it is an integer, everything happens
correctly as its internal double data type, and gawk knows it is negative,
because it built it that way. It is hard to quit thinking of everything as
integers and let double floats do their job.

Funny how this simple arithmetic stuff can confuse me!

I appreciate the help and ideas.

Regards, Steve

Hi Grant,

You gave me the idea - Here is my final solution 2s complement, multi-
length byte strings, le conversion:

#little endian build a number

if (and(128,strtonum($(fld + len))) == 128 ) {
arg = -1;
} else {

arg = 0;
}
for (i = fld + len; i > fld ; i--) {
arg = (arg * 256) + strtonum($(i))
}

This way it doesn't even matter that it is an integer, everything happens
correctly as its internal double data type, and gawk knows it is negative,
because it built it that way. It is hard to quit thinking of everything as
integers and let double floats do their job.

Funny how this simple arithmetic stuff can confuse me!

I appreciate the help and ideas.

Regards, Steve