Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Split a hex string into bytes and compute the xor of them.
134 views
Skip to first unread message
hongy...@gmail.com
Sep 3, 2023, 2:26:49 AM9/3/23
to
I want to split a hex string into bytes and compute their xor, as shown below:
werner@X10DAi:~$ data=$( sed 's/\(..\)/\1 /g' <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101 | sed -e 's/[ ]*$//' )
werner@X10DAi:~$ IFS=' ' read -ra ADDR <<< "$data"; xor=0; for i in "${ADDR[@]}"; do xor=$((xor ^ 0x$i)); done; printf '%x\n' $xor
ea
I wonder if there is a more concise and simple method for this purpose.
Regards,
Zhao
werner@X10DAi:~$ data=$( sed 's/\(..\)/\1 /g' <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101 | sed -e 's/[ ]*$//' )
werner@X10DAi:~$ IFS=' ' read -ra ADDR <<< "$data"; xor=0; for i in "${ADDR[@]}"; do xor=$((xor ^ 0x$i)); done; printf '%x\n' $xor
ea
I wonder if there is a more concise and simple method for this purpose.
Regards,
Zhao
marrgol
Sep 3, 2023, 7:48:27 AM9/3/23
to
DATA=23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101
XOR=0
while read -N2 BYTE; do ((XOR^=0x$BYTE)) ; done < <(echo $DATA)
printf '%02X\n' $XOR
Kenny McCormack
Sep 3, 2023, 9:26:23 AM9/3/23
to
In article <8404323f-40c3-473b...@googlegroups.com>,
$ gawk '{ x = 0; for (i=1; i<=NF; i++) x = xor(x,strtonum("0x" $i)); print x }' FPAT=.. <<< \
23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101
--
People sleep peaceably in their beds at night only because rough
men stand ready to do violence on their behalf.
George Orwell
23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101
--
People sleep peaceably in their beds at night only because rough
men stand ready to do violence on their behalf.
George Orwell
Ben Bacarisse
Sep 3, 2023, 10:09:03 AM9/3/23
to
clarity:
printf '%x\n' $(($(sed 's/\(..\)/0x\1^/g' <<<2324)0))
The 0 at the end is needed because the sed command adds a trailing ^
operator.
--
Ben.
hongy...@gmail.com
Sep 3, 2023, 11:27:06 AM9/3/23
to
werner@X10DAi:~$ gawk '{ x = 0; for (i=1; i<=NF; i++) x = xor(x,strtonum("0x" $i)); printf("%x\n", x) }' FPAT=.. <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101
ea
werner@X10DAi:~$ gawk '{ x = 0; for (i=1; i<=NF; i++) x = xor(x,strtonum("0x" $i)); printf "%x\n", x }' FPAT=.. <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101
ea
werner@X10DAi:~$ printf '%x\n' $(($(sed 's/\(..\)/0x\1^/g' <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101)0))
ea
> --
> Ben.
Regards,
Zhao
Janis Papanagnou
Sep 3, 2023, 6:58:04 PM9/3/23
to
a Kornshell solution without a shell loop or external commands
data=...
printf "%2x\n" $((${data//@(??)/0x\1^}0))
(functionally it's a variant of Ben's proposal). Unfortunately it seems
that bash (with extglob set) does not support the \1 in the expression?
Janis
> Regards,
> Zhao
>
hongy...@gmail.com
Sep 3, 2023, 9:00:17 PM9/3/23
to
werner@X10DAi:~$ ksh
Command 'ksh' not found, but can be installed with:
sudo apt install ksh93u+m # version 1.0.0~beta.2-1, or
sudo apt install mksh # version 59c-16
> Janis
>
> > Regards,
> > Zhao
> >
Janis Papanagnou
Sep 3, 2023, 10:00:55 PM9/3/23
to
On 04.09.2023 03:00, hongy...@gmail.com wrote:
> On Monday, September 4, 2023 at 6:58:04 AM UTC+8, Janis Papanagnou wrote:
>> On 03.09.2023 08:26, hongy...@gmail.com wrote:
>>> I want to split a hex string into bytes and compute their xor, as shown below:
>>>
>>> werner@X10DAi:~$ data=$( sed 's/\(..\)/\1 /g' <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101 | sed -e 's/[ ]*$//' )
>>> werner@X10DAi:~$ IFS=' ' read -ra ADDR <<< "$data"; xor=0; for i in "${ADDR[@]}"; do xor=$((xor ^ 0x$i)); done; printf '%x\n' $xor
>>> ea
>>>
>>> I wonder if there is a more concise and simple method for this purpose.
>> You've already got a couple solutions. So just for completeness here's
>> a Kornshell solution without a shell loop or external commands
>>
>> data=...
>> printf "%2x\n" $((${data//@(??)/0x\1^}0))
>>
>> (functionally it's a variant of Ben's proposal). Unfortunately it seems
>> that bash (with extglob set) does not support the \1 in the expression?
>
> What's your ksh version, and which one should I install for testing your code snippet, as listed below?
You should use any of the official or derived versions; this means
> On Monday, September 4, 2023 at 6:58:04 AM UTC+8, Janis Papanagnou wrote:
>> On 03.09.2023 08:26, hongy...@gmail.com wrote:
>>> I want to split a hex string into bytes and compute their xor, as shown below:
>>>
>>> werner@X10DAi:~$ data=$( sed 's/\(..\)/\1 /g' <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101 | sed -e 's/[ ]*$//' )
>>> werner@X10DAi:~$ IFS=' ' read -ra ADDR <<< "$data"; xor=0; for i in "${ADDR[@]}"; do xor=$((xor ^ 0x$i)); done; printf '%x\n' $xor
>>> ea
>>>
>>> I wonder if there is a more concise and simple method for this purpose.
>> You've already got a couple solutions. So just for completeness here's
>> a Kornshell solution without a shell loop or external commands
>>
>> data=...
>> printf "%2x\n" $((${data//@(??)/0x\1^}0))
>>
>> (functionally it's a variant of Ben's proposal). Unfortunately it seems
>> that bash (with extglob set) does not support the \1 in the expression?
>
> What's your ksh version, and which one should I install for testing your code snippet, as listed below?
ksh93u+ (or ksh93u+m) are fine. (I wouldn't use any clone that has
only incomplete support of origional AT&T Kornshell's features.)
>
> werner@X10DAi:~$ ksh
> Command 'ksh' not found, but can be installed with:
> sudo apt install ksh93u+m # version 1.0.0~beta.2-1, or
> sudo apt install mksh # version 59c-16
value that you _expect_ in case that the data string has an odd
number of digits; _if_ that can happen with your data you should
take appropriate precautions, e.g. by conditionally prepending a
leading zero in case of an odd number of digits in the data.
Janis
Ben Bacarisse
Sep 3, 2023, 10:39:52 PM9/3/23
to
Janis Papanagnou <janis_pap...@hotmail.com> writes:
> On 03.09.2023 08:26, hongy...@gmail.com wrote:
>> I want to split a hex string into bytes and compute their xor, as shown below:
>>
>> werner@X10DAi:~$ data=$( sed 's/\(..\)/\1 /g' <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101 | sed -e 's/[ ]*$//' )
>> werner@X10DAi:~$ IFS=' ' read -ra ADDR <<< "$data"; xor=0; for i in "${ADDR[@]}"; do xor=$((xor ^ 0x$i)); done; printf '%x\n' $xor
>> ea
>>
>> I wonder if there is a more concise and simple method for this purpose.
>
> You've already got a couple solutions. So just for completeness here's
> a Kornshell solution without a shell loop or external commands
>
> data=...
> printf "%2x\n" $((${data//@(??)/0x\1^}0))
Much neater.
> On 03.09.2023 08:26, hongy...@gmail.com wrote:
>> I want to split a hex string into bytes and compute their xor, as shown below:
>>
>> werner@X10DAi:~$ data=$( sed 's/\(..\)/\1 /g' <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101 | sed -e 's/[ ]*$//' )
>> werner@X10DAi:~$ IFS=' ' read -ra ADDR <<< "$data"; xor=0; for i in "${ADDR[@]}"; do xor=$((xor ^ 0x$i)); done; printf '%x\n' $xor
>> ea
>>
>> I wonder if there is a more concise and simple method for this purpose.
>
> You've already got a couple solutions. So just for completeness here's
> a Kornshell solution without a shell loop or external commands
>
> data=...
> printf "%2x\n" $((${data//@(??)/0x\1^}0))
> (functionally it's a variant of Ben's proposal). Unfortunately it seems
> that bash (with extglob set) does not support the \1 in the
> expression?
printf '0x%X\n' $((${data//??/0x&^}0))
and it can be turned on and off via a variable so not idea in something
portable.
--
Ben.
hongy...@gmail.com
Sep 4, 2023, 12:23:42 AM9/4/23
to
### begin quote ###
data=...
printf '0x%X\n' $((${data//??/0x&^}0))
explain the above ksh code.
The provided script line is an example of shell arithmetic expansion and parameter substitution in KornShell (ksh).
The code does the following:
1. `data=...`: This defines a variable `data`. It's supposed to have some hex value. The value isn't assigned in your example; it's merely given as ellipsis `...`.
2. `printf '0x%X\n' $((${data//??/0x&^}0))`: This is a more complex part.
- `printf '0x%X\n'`: This is a printf command that prints a number in hexadecimal format ('0x' at the beginning). The `%X` is a format specification for the hexadecimal output.
- `${data//??/0x&^}0`: This is a pattern substitution with arithmetic expansion.
- `${data//??/0x&}`: This will replace every two consecutive characters (`??`) in `data` with `0x&` (where `&` represents the matched part). So, if `data` was `1234`, after this operation it would become `0x120x34`.
- `${data//??/0x&^}0`: The `^` is bitwise XOR in ksh. It's used to XOR all the hexadecimal numbers together. The trailing `0` is necessary as the first XOR operation is between the `0` and the first replaced value.
The whole command is useful in converting multiple byte ASCII values into one big endian hexadecimal value. It opens up the byte string, calculates the XOR of all bytes, and prints out the final value in hexadecimal format.
### end quote ###
But I'm not sure if the above explanation is completely correct.
> and it can be turned on and off via a variable so not idea in something
> portable.
>
> --
> Ben.
Zhao
Ben Bacarisse
Sep 4, 2023, 2:47:11 PM9/4/23
to
"hongy...@gmail.com" <hongy...@gmail.com> writes:
> On Monday, September 4, 2023 at 10:39:52 AM UTC+8, Ben Bacarisse wrote:
>> Janis Papanagnou <janis_pap...@hotmail.com> writes:
>>
>> > On 03.09.2023 08:26, hongy...@gmail.com wrote:
>> >> I want to split a hex string into bytes and compute their xor, as shown below:
>> >>
>> >> werner@X10DAi:~$ data=$( sed 's/\(..\)/\1 /g' <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101 | sed -e 's/[ ]*$//' )
>> >> werner@X10DAi:~$ IFS=' ' read -ra ADDR <<< "$data"; xor=0; for i in "${ADDR[@]}"; do xor=$((xor ^ 0x$i)); done; printf '%x\n' $xor
>> >> ea
>> >>
>> >> I wonder if there is a more concise and simple method for this purpose.
>> >
>> > You've already got a couple solutions. So just for completeness here's
>> > a Kornshell solution without a shell loop or external commands
>> >
>> > data=...
>> > printf "%2x\n" $((${data//@(??)/0x\1^}0))
>> Much neater.
>> > (functionally it's a variant of Ben's proposal). Unfortunately it seems
>> > that bash (with extglob set) does not support the \1 in the
>> > expression?
Note that Janis is talking about bash now...
> On Monday, September 4, 2023 at 10:39:52 AM UTC+8, Ben Bacarisse wrote:
>> Janis Papanagnou <janis_pap...@hotmail.com> writes:
>>
>> > On 03.09.2023 08:26, hongy...@gmail.com wrote:
>> >> I want to split a hex string into bytes and compute their xor, as shown below:
>> >>
>> >> werner@X10DAi:~$ data=$( sed 's/\(..\)/\1 /g' <<< 23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101 | sed -e 's/[ ]*$//' )
>> >> werner@X10DAi:~$ IFS=' ' read -ra ADDR <<< "$data"; xor=0; for i in "${ADDR[@]}"; do xor=$((xor ^ 0x$i)); done; printf '%x\n' $xor
>> >> ea
>> >>
>> >> I wonder if there is a more concise and simple method for this purpose.
>> >
>> > You've already got a couple solutions. So just for completeness here's
>> > a Kornshell solution without a shell loop or external commands
>> >
>> > data=...
>> > printf "%2x\n" $((${data//@(??)/0x\1^}0))
>> Much neater.
>> > (functionally it's a variant of Ben's proposal). Unfortunately it seems
>> > that bash (with extglob set) does not support the \1 in the
>> > expression?
>> You can use & but I think it's new:
>>
>> printf '0x%X\n' $((${data//??/0x&^}0))
>
> I tried to understand the above code snippet with the help of
> chatgpt-4 as follows:
>
> ### begin quote ###
> data=...
> printf '0x%X\n' $((${data//??/0x&^}0))
>
> explain the above ksh code.
> The provided script line is an example of shell arithmetic expansion and parameter substitution in KornShell (ksh).
>
> The code does the following:
>
> 1. `data=...`: This defines a variable `data`. It's supposed to have some hex value. The value isn't assigned in your example; it's merely given as ellipsis `...`.
>
> 2. `printf '0x%X\n' $((${data//??/0x&^}0))`: This is a more complex part.
>
> - `printf '0x%X\n'`: This is a printf command that prints a number in hexadecimal format ('0x' at the beginning). The `%X` is a format specification for the hexadecimal output.
>
> - `${data//??/0x&^}0`: This is a pattern substitution with arithmetic expansion.
>
> - `${data//??/0x&}`: This will replace every two consecutive characters (`??`) in `data` with `0x&` (where `&` represents the matched part). So, if `data` was `1234`, after this operation it would become `0x120x34`.
>
> - `${data//??/0x&^}0`: The `^` is bitwise XOR in ksh. It's used to XOR all the hexadecimal numbers together. The trailing `0` is necessary as the first XOR operation is between the `0` and the first replaced value.
>
> The whole command is useful in converting multiple byte ASCII values into one big endian hexadecimal value. It opens up the byte string, calculates the XOR of all bytes, and prints out the final value in hexadecimal format.
> ### end quote ###
>
> But I'm not sure if the above explanation is completely correct.
check it yourself from the bash manual? Or you could ask the group to
explain it if there's some bit you don't follow.
--
Ben.
hongy...@gmail.com
Sep 5, 2023, 8:31:55 PM9/5/23
to
werner@X10DAi:~$ echo $SHELL
/bin/bash
werner@X10DAi:~$ data=23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101
werner@X10DAi:~$ printf '0x%X\n' $((${data//??/0x&^}0))
bash: 0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0: syntax error: operand expected (error token is "^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0")
werner@X10DAi:~$ printf '0x%X\n' $((${data//??/0x\1^}0))
0x0
> --
> Ben.
Zhao
Ben Bacarisse
Sep 5, 2023, 9:31:13 PM9/5/23
to
"hongy...@gmail.com" <hongy...@gmail.com> writes:
> I tried to test the above code snippet as follows:
>
> werner@X10DAi:~$ echo $SHELL
> /bin/bash
>
> werner@X10DAi:~$ data=23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101
> werner@X10DAi:~$ printf '0x%X\n' $((${data//??/0x&^}0))
> bash: 0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0: syntax error: operand expected (error token is "^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0")
> I tried to test the above code snippet as follows:
>
> werner@X10DAi:~$ echo $SHELL
> /bin/bash
>
> werner@X10DAi:~$ data=23230DFE4C5336413245313636504135303333333401000A130A1814300A01010101
> werner@X10DAi:~$ printf '0x%X\n' $((${data//??/0x&^}0))
> bash: 0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0: syntax error: operand expected (error token is "^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0x&^0")
I wrote:
You can use & but I think it's new:
printf '0x%X\n' $((${data//??/0x&^}0))
and it can be turned on and off via a variable so not ideal in something
You can use & but I think it's new:
printf '0x%X\n' $((${data//??/0x&^}0))
portable. (I should have said "via a shell option" and I mis-typed
"ideal".)
man bash says:
If the patsub_replacement shell option is enabled using shopt,
any unquoted instances of & in string are replaced with the
matching portion of pattern.
You may have an old version of bash or you may have patsub_replacement
turned off.
> werner@X10DAi:~$ printf '0x%X\n' $((${data//??/0x\1^}0))
meaning in a substitution. Every pair of digits simply gets replaced by
0x1^. The result evaluates to 0 or 1 depending on how many pairs there
are.
--
Ben.
0 new messages