Restore the destroyed disk image partition table by dd.

[hongy...@gmail.com]

I've two disk image files whose partuuid info are shown as below based on the method discussed here on <https://unix.stackexchange.com/questions/419104/what-is-partuuid-from-blkid-when-using-msdos-partition-table>:

$ dd if=openwrt-19.07.7-x86-64-combined-ext4.img bs=1 count=4 skip=440 2>/dev/null | od -t x4 -An
eb3ae1bb
$ dd if=disk.img bs=1 count=4 skip=440 2>/dev/null | od -t x4 -An
761b36b0

In order to change the partuuid of the 2nd disk image to the value of the 1st, I do the following operation:

$ dd if=openwrt-19.07.7-x86-64-combined-ext4.img of=disk.img bs=1 count=4 skip=440 seek=440
4+0 records in
4+0 records out
4 bytes copied, 0.000195385 s, 20.5 kB/s
$ dd if=disk.img bs=1 count=4 skip=440 2>/dev/null | od -t x4 -An
eb3ae1bb

But it turns out that the disk.img will be destroyed by the above operation. So, How can I restore it with the original data on it:

$ dd if=disk.img bs=1 count=4 skip=440 2>/dev/null | od -t x4 -An
761b36b0

Regards,
HY

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:

>In order to change the partuuid of the 2nd disk image to the value
>of the 1st, I do the following operation:

[deleted a shell command written in one over‐long line which
resembled the following shell command:]

dd if=openwrt-19.07.7-x86-64-combined-ext4.img \
of=disk.img \
bs=1 count=4 skip=440 seek=440

[…]

>But it turns out that the disk.img will be destroyed by the above
>operation.

Yes, that's true:  Unless the parameter 'conv=notrunc' is given to
'dd' when writing to a file, 'dd' will truncate the output file,
i. e. 'disk.img'. 

Do you have a backup of the 'disk.img' file?  If not, I'm sorry to
tell you, that your disk image file is lost.  You may issue the
command

ls -logd -- disk.img

in order to check if that's the case.

[hongy...@gmail.com]

On Tuesday, April 13, 2021 at 2:46:17 AM UTC+8, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongy...@gmail.com>:
> >In order to change the partuuid of the 2nd disk image to the value
> >of the 1st, I do the following operation:
> [deleted a shell command written in one over‐long line which
> resembled the following shell command:]
> dd if=openwrt-19.07.7-x86-64-combined-ext4.img \
> of=disk.img \
> bs=1 count=4 skip=440 seek=440
> […]
> >But it turns out that the disk.img will be destroyed by the above
> >operation.
> Yes, that's true: Unless the parameter 'conv=notrunc' is given to
> 'dd' when writing to a file, 'dd' will truncate the output file,
> i. e. 'disk.img'.

Do you mean if I have issued the 'dd' command with the 'conv = notrunc' option, then I can restore it?

> Do you have a backup of the 'disk.img' file?

No.

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:

>On Tuesday, April 13, 2021 at 2:46:17 AM UTC+8, Helmut Waitzmann wrote:
>> "hongy...@gmail.com" <hongy...@gmail.com>:

>> dd if=openwrt-19.07.7-x86-64-combined-ext4.img \
>> of=disk.img \
>> bs=1 count=4 skip=440 seek=440

>>>But it turns out that the disk.img will be destroyed by the above
>>>operation.
>> Yes, that's true: Unless the parameter 'conv=notrunc' is given to
>> 'dd' when writing to a file, 'dd' will truncate the output file,
>> i. e. 'disk.img'.
>
>Do you mean if I have issued the 'dd' command with the 'conv =
>notrunc' option, then I can restore it?

Yes.  You would have been able to restore it then. 


The 'conv=notrunc' parameter tells 'dd' not to truncate the output
file, that is, invoke the 'open()' system service without the
'O_TRUNC' flag (see the 'open(2)' manual page) on the output file: 
'dd' will overwrite (or create if they don't already exist) the file
contents at those positions where it writes to the file but let the
file contents at other file positions unmodified. 

On the other hand, if 'conv=notrunc' is not given to 'dd', it will
open the output file using the 'O_TRUNC' flag, and the consequence
of this will be (see the 'open(2)' manual page): 

If the file already exists and is a regular file and the
access mode allows writing (i.e., is O_RDWR or O_WRONLY) it
will be truncated to length 0. If the file is a FIFO or
terminal device file, the O_TRUNC flag is ignored.
Otherwise, the effect of O_TRUNC is unspecified.

[hongy...@gmail.com]

Thank you very much for your valuable notes.

Regards,
HY

[hongy...@gmail.com]

On Wednesday, April 14, 2021 at 4:54:02 AM UTC+8, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongy...@gmail.com>:
> >On Tuesday, April 13, 2021 at 2:46:17 AM UTC+8, Helmut Waitzmann wrote:
> >> "hongy...@gmail.com" <hongy...@gmail.com>:
> >> dd if=openwrt-19.07.7-x86-64-combined-ext4.img \
> >> of=disk.img \
> >> bs=1 count=4 skip=440 seek=440
>
> >>>But it turns out that the disk.img will be destroyed by the above
> >>>operation.
> >> Yes, that's true: Unless the parameter 'conv=notrunc' is given to
> >> 'dd' when writing to a file, 'dd' will truncate the output file,
> >> i. e. 'disk.img'.
> >
> >Do you mean if I have issued the 'dd' command with the 'conv =
> >notrunc' option, then I can restore it?

> Yes. You would have been able to restore it then.

Assuming I have used this option, what exact restore command should I use?

[William Unruh]

On 2021-04-14, hongy...@gmail.com <hongy...@gmail.com> wrote:
> On Wednesday, April 14, 2021 at 4:54:02 AM UTC+8, Helmut Waitzmann wrote:
>> "hongy...@gmail.com" <hongy...@gmail.com>:
>> >On Tuesday, April 13, 2021 at 2:46:17 AM UTC+8, Helmut Waitzmann wrote:
>> >> "hongy...@gmail.com" <hongy...@gmail.com>:
>> >> dd if=openwrt-19.07.7-x86-64-combined-ext4.img \
>> >> of=disk.img \
>> >> bs=1 count=4 skip=440 seek=440
>>
>> >>>But it turns out that the disk.img will be destroyed by the above
>> >>>operation.
>> >> Yes, that's true: Unless the parameter 'conv=notrunc' is given to
>> >> 'dd' when writing to a file, 'dd' will truncate the output file,
>> >> i. e. 'disk.img'.
>> >
>> >Do you mean if I have issued the 'dd' command with the 'conv =
>> >notrunc' option, then I can restore it?
>
>> Yes. You would have been able to restore it then.
>
> Assuming I have used this option, what exact restore command should I use?

If you had used that option, then the file would have been left along
except for the 4 bytes starting at location 440 that you changed. The
file would have been changed, but it would have all still been there.
To restore it, you would have had to have made a backup and then copied
the backup to the file location.

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:

[The 'conv=notrunc' option of 'dd']

>Assuming I have used this option, what exact restore command should
>I use?

First you should check, that the command


{
printf '%s\n' 'ibase=16' &&
printf '%s\n' b0 36 1b 76 |
LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
} | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
xargs -E '' -- printf '%b' |
od -v -A n -t x4

yields the output '761b36b0'.  If that's the case, you could have
done

{
printf '%s\n' 'ibase=16' &&
printf '%s\n' b0 36 1b 76 |
LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
} | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
xargs -E '' -- printf '%b' |
dd of=disk.img bs=1 seek=440 conv=notrunc

to restore the original partuuid. 


Read the documentation ('info', 'man') of the commands 'printf',
'tr', 'bc', 'xargs', 'od', and 'dd', to learn, what they are doing
in the commands above. 

[hongy...@gmail.com]

On Thursday, April 15, 2021 at 6:44:47 AM UTC+8, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongy...@gmail.com>:
>
> [The 'conv=notrunc' option of 'dd']
> >Assuming I have used this option, what exact restore command should
> >I use?
> First you should check, that the command
>
>
> {
> printf '%s\n' 'ibase=16' &&
> printf '%s\n' b0 36 1b 76 |
> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
> } | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
> xargs -E '' -- printf '%b' |
> od -v -A n -t x4
>
> yields the output '761b36b0'.

Yes, it does:

werner@M4600t:~$ {

> printf '%s\n' 'ibase=16' &&
> printf '%s\n' b0 36 1b 76 |
> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
> } | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
> xargs -E '' -- printf '%b' |
> od -v -A n -t x4

761b36b0

> If that's the case, you could have
> done

Why must the byte order be exchanged?

[hongy...@gmail.com]

On Thursday, April 15, 2021 at 6:44:47 AM UTC+8, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongy...@gmail.com>:
>
> [The 'conv=notrunc' option of 'dd']
> >Assuming I have used this option, what exact restore command should
> >I use?
> First you should check, that the command
>
>
> {
> printf '%s\n' 'ibase=16' &&
> printf '%s\n' b0 36 1b 76 |
> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
> } | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
> xargs -E '' -- printf '%b' |
> od -v -A n -t x4
>
> yields the output '761b36b0'. If that's the case, you could have
> done

Though I'm not sure if I completely understand your intention, I think you're judging the endianness (https://en.wikipedia.org/wiki/Endianness) used by the CPU architecture and the OS distribution in the question.

As noted by the above wiki website:

The Intel x86 and AMD64 / x86-64 series of processors use the little-endian format.

This is exactly my situation as shown below:

$ cat /proc/cpuinfo |grep -i 'model name'| sort -u
model name : Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:

>On Thursday, April 15, 2021 at 6:44:47 AM UTC+8, Helmut Waitzmann wrote:
>> "hongy...@gmail.com" <hongy...@gmail.com>:

>> [The 'conv=notrunc' option of 'dd']
>>
>>>Assuming I have used this option, what exact restore command should
>>>I use?
>> First you should check, that the command
>>
>>
>> {
>> printf '%s\n' 'ibase=16' &&
>> printf '%s\n' b0 36 1b 76 |
>> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
>> } | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
>> xargs -E '' -- printf '%b' |
>> od -v -A n -t x4
>>
>> yields the output '761b36b0'.
>>
>
>Yes, it does:
>
>
>werner@M4600t:~$ {
>> printf '%s\n' 'ibase=16' &&
>> printf '%s\n' b0 36 1b 76 |
>> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
>> } | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
>> xargs -E '' -- printf '%b' |
>> od -v -A n -t x4
> 761b36b0

[…]

>Why must the byte order be exchanged?
>

The byte order has to be re‐exchanged, because the command

od -v -A n -t x4

showed you the byte order exchanged, not the real byte order.  To
get the real byte order, you could have used the command

od -v -A n -t x1

which would have shown you the bytes in their real order:


b0 36 1b 76

If you want to extract a byte sequence using 'od' in order to
restore it later, you need the reverse function of 'od'.  The pipe
consisting of 'printf', 'tr', 'bc', and 'xargs' is that reverse of

od -v -A n -t x1

but not of

od -v -A n -t x4

which reverses the byte order on little‐endian architectures. 

[hongy...@gmail.com]

However, it seems that this simple task is made complicated by the above tricks.

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:

>On Friday, April 16, 2021 at 2:48:21 AM UTC+8, Helmut Waitzmann wrote:
>> "hongy...@gmail.com" <hongy...@gmail.com>:
>>>On Thursday, April 15, 2021 at 6:44:47 AM UTC+8, Helmut Waitzmann wrote:
>>>> "hongy...@gmail.com" <hongy...@gmail.com>:

>>>> {
>>>> printf '%s\n' 'ibase=16' &&
>>>> printf '%s\n' b0 36 1b 76 |
>>>> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
>>>> } | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
>>>> xargs -E '' -- printf '%b' |
>>>> od -v -A n -t x4

>> If you want to extract a byte sequence using 'od' in order to
>> restore it later, you need the reverse function of 'od'. The pipe
>> consisting of 'printf', 'tr', 'bc', and 'xargs' is that reverse
>

>However, it seems that this simple task is made complicated by the
>above tricks.

They are part of the unix tool chest.  I think, it's worth while
knowing them. 

However, if you just want to save the original partuuid, you don't
need to transform it using 'od'.  It suffices to save the raw 4
bytes and restore them both by just using 'dd'. 

[hongy...@gmail.com]

On Thursday, April 15, 2021 at 6:44:47 AM UTC+8, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongy...@gmail.com>:
>
> [The 'conv=notrunc' option of 'dd']
> >Assuming I have used this option, what exact restore command should
> >I use?
> First you should check, that the command

> {
> printf '%s\n' 'ibase=16' &&
> printf '%s\n' b0 36 1b 76 |
> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
> } | bc |
> xargs -E '' -- printf \''\\%.4o'\''\n' |

I find the following description from the pertinent man pages:

$ man xargs
-E eof-str
Set the end of file string to eof-str.

$ man 3 printf
o, u, x, X
The unsigned int argument is converted to unsigned octal (o),

But, I still can't figure out the purpose of the above command.

> xargs -E '' -- printf '%b' |

I find the following explanation:

$ man 1 printf
%b ARGUMENT as a string with '\' escapes interpreted, except that octal escapes are of the form \0 or \0NNN

But I still can't figure out how it works with the previous command.

> od -v -A n -t x4

$ man 1 od
-A, --address-radix=RADIX
output format for file offsets; RADIX is one of [doxn], for Decimal, Octal, Hex or None

-v, --output-duplicates
do not use * to mark line suppression

-t, --format=TYPE
select output format or formats

x[SIZE]
hexadecimal, SIZE bytes per integer

So, the above command means the following:

Use 4 bytes per integer format as the output, don't suppress duplicate lines, and without address-radix.

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:

>On Thursday, April 15, 2021 at 6:44:47 AM UTC+8, Helmut Waitzmann wrote:

>> {
>> printf '%s\n' 'ibase=16' &&
>> printf '%s\n' b0 36 1b 76 |
>> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
>> } | bc |
>> xargs -E '' -- printf \''\\%.4o'\''\n' |
>
>I find the following description from the pertinent man pages:
>
>$ man xargs
> -E eof-str
> Set the end of file string to eof-str.
>

I guess that your 'xargs' is GNU 'xargs'.  If that's the case,
note the following paragraph in its manual page:

SEE ALSO
find(1), locate(1), locatedb(5), updatedb(1),
fork(2), execvp(3), kill(1), signal(7),

The full documentation for xargs is maintained as a
Texinfo manual. If the info and xargs programs are
properly installed at your site, the command info
xargs should give you access to the complete manual.


Therefore the 'info' manual is the definitive reference for GNU
'xargs' (the 'man' documentation of 'xargs' is wrong here):

'-E EOF-STR'
'--eof[=EOF-STR]'
'-e[EOF-STR]'

Set the logical end of file marker string to EOF-STR.  If
the logical end of file marker string occurs as a line of
input, the rest of the input is ignored.  If EOF-STR is
omitted ('-e') or blank (either '-e' or '-E'), there is no
logical end of file marker string.  The '-e' form of this
option is deprecated in favour of the POSIX-compliant '-E'
option, which you should use instead.  As of GNU 'xargs'
version 4.2.9, the default behaviour of 'xargs' is not to
have a logical end of file marker string.  The POSIX
standard (IEEE Std 1003.1, 2004 Edition) allows this.

=>

xargs -E ''

won't ignore any input but read until the end of the data
has been reached.  GNU 'xargs' will behave the same without
that option, too.  But in general, POSIX 'xargs' might behave
different without it. 

Because that is my intention, that 'xargs' should not ignore any
input, I'll supply that option. 

>$ man 3 printf
> o, u, x, X
> The unsigned int argument is converted to unsigned octal (o),
>
>But, I still can't figure out the purpose of the above command.
>

For example:

printf \''\\%.4o'\''\n' 4 8 9 16 64 128 255

will output the following lines: 

'\0004'
'\0010'
'\0011'
'\0020'
'\0100'
'\0200'
'\0377'

>> xargs -E '' -- printf '%b' |

'xargs' will read the octal numbers, remove the apostrophes and
invoke printf, supplying the backslash‐prepended 4‐digit octal
numbers as addtional parameters. 

>
>I find the following explanation:
>
>$ man 1 printf
>%b ARGUMENT as a string with '\' escapes interpreted, except
>that octal escapes are of the form \0 or \0NNN
>
>But I still can't figure out how it works with the previous command.
>

Let me give an example:  The shell command line

printf '%b' '\0110' '\0145' '\0154' '\0154' '\0157' \
'\0054' '\0040' '\0167' '\0157' '\0162' '\0154' \
'\0144' '\0041' '\0012'

will output the line

Hello, world!

>> od -v -A n -t x4
>$ man 1 od
> -A, --address-radix=RADIX
> output format for file offsets; RADIX is one of
> [doxn], for Decimal, Octal, Hex or None
>
> -v, --output-duplicates
> do not use * to mark line suppression
>
> -t, --format=TYPE
> select output format or formats
>
> x[SIZE]
> hexadecimal, SIZE bytes per integer
>
>So, the above command means the following:
>
>Use 4 bytes per integer format as the output, don't suppress
>duplicate lines, and without address-radix.
>

You didn't construct the 'od' command you posted in the OP by
yourself but only crasped it out of the WWW?  So the WWW gave
bad advice to you, because it didn't supply the '-v' option and
supplied '-t' 'x4' rather than '-t' 'x1', causing the
little‐endian‐big‐endian trouble. 

>
>> yields the output '761b36b0'. If that's the case, you could have
>> done
>>
>> {
>> printf '%s\n' 'ibase=16' &&
>> printf '%s\n' b0 36 1b 76 |
>> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
>> } | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
>> xargs -E '' -- printf '%b' |
>> dd of=disk.img bs=1 seek=440 conv=notrunc
>>
>> to restore the original partuuid.
>>
>>
>> Read the documentation ('info', 'man') of the commands 'printf',
>> 'tr', 'bc', 'xargs', 'od', and 'dd', to learn, what they are doing
>> in the commands above.

Let me comment on the multi‐line shell command above:

# Print the sedecimal numbers, one per line, converted to
# decimal:
{
# Print 'ibase=16':
#

printf '%s\n' 'ibase=16' &&

# Print the sedecimal numbers, one per line:
#

printf '%s\n' b0 36 1b 76 |

# Feed the sedecimal numbers to 'tr' in order to convert
# all letters to upper case (because 'bc' doesn't accept
# lower case sedecimal numbers):
#

LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
} |

# Let bc convert the upper case sedecimal numbers to decimal
# numbers and print them one per line:
#
bc |

# Let 'xargs' read the decimal numbers and invoke 'printf',
# supplying the decimal numbers as additional parameters:
#
# 'printf' will convert each of the decimal numbers to a
# 4-digit octal number and prepend an apostrophe and a
# backslash before it and append an apostrophe after it,
# then output it, one line per number:
#
# printf \''\\%.4o'\''\n' the decimal numbers...
#

xargs -E '' -- printf \''\\%.4o'\''\n' |

# Let 'xargs' read each backslash-prepended
# apostrophe-surrounded 4-digit octal number and invoke
# 'printf', supplying the backslash-prepended 4-digit octal
# numbers as additional prameters:
#
# printf '%b' '\nnnn' '\nnnn' ...
#
# 'printf' will convert each octal value to a byte using
# that octal number as the code, i.e. convert the octal
# numbers to their raw form, which is essentially the
# inverse of
#
# od -A n -t o1
#

xargs -E '' -- printf '%b' |

#
# In summary: Output the raw values of the sedecimal coded
# bytes.  These can be fed to 'dd'.

[hongy...@gmail.com]

It seems to me that this form is more obscure than the following alternative one supplied by me.

>
> will output the following lines:
>
> '\0004'
> '\0010'
> '\0011'
> '\0020'
> '\0100'
> '\0200'
> '\0377'

I tried with the following form and obtain the same results as yours:

$ printf ''\''\%.4o'\''\n' 4 8 9 16 64 128 255

I've checked the manual page of 'od', but doesn't have so much experience in the selection of its various options.

> So the WWW gave
> bad advice to you, because it didn't supply the '-v' option and
> supplied '-t' 'x4' rather than '-t' 'x1', causing the
> little‐endian‐big‐endian trouble.

The endian conversion problem is architecture and OS platform dependent which only happens when the data is written into ram. I'm not sure if there is/are some convenient *nix tool(s) to check it.

> >
> >> yields the output '761b36b0'. If that's the case, you could have
> >> done
> >>
> >> {
> >> printf '%s\n' 'ibase=16' &&
> >> printf '%s\n' b0 36 1b 76 |
> >> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
> >> } | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
> >> xargs -E '' -- printf '%b' |
> >> dd of=disk.img bs=1 seek=440 conv=notrunc
> >>
> >> to restore the original partuuid.
> >>
> >>
> >> Read the documentation ('info', 'man') of the commands 'printf',
> >> 'tr', 'bc', 'xargs', 'od', and 'dd', to learn, what they are doing
> >> in the commands above.
> Let me comment on the multi‐line shell command above:
>
> # Print the sedecimal numbers, one per line, converted to

Should be wording as 'sedecimal' or 'hexadecimal'?

Thank you very much for your in-depth line-by-line explanations.

Regards,
HY

[Spiros Bousbouras]

On Sat, 17 Apr 2021 17:20:36 -0700 (PDT)
"hongy...@gmail.com" <hongy...@gmail.com> wrote:
> On Sunday, April 18, 2021 at 3:24:13 AM UTC+8, Helmut Waitzmann wrote:
> > For example:
> >
> > printf \''\\%.4o'\''\n' 4 8 9 16 64 128 255
>
> It seems to me that this form is more obscure than the following alternative
> one supplied by me.
>
> >
> > will output the following lines:
> >
> > '\0004'
> > '\0010'
> > '\0011'
> > '\0020'
> > '\0100'
> > '\0200'
> > '\0377'
>
> I tried with the following form and obtain the same results as yours:
>
> $ printf ''\''\%.4o'\''\n' 4 8 9 16 64 128 255

^^
This just prepends the empty string and can be omitted without
affecting the functionality. So the only difference between your construct
and Helmut's is that he uses \\ whereas you use \ . POSIX says

The interpretation of a backslash followed by any other sequence of
characters is unspecified.

so in particular the \% sequence in your construct is unspecified. It just
happened that it printed a backslash and then did the usual interpretation
corresponding to % .

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:

>On Sunday, April 18, 2021 at 3:24:13 AM UTC+8, Helmut Waitzmann wrote:

>> For example:
>>
>> printf \''\\%.4o'\''\n' 4 8 9 16 64 128 255
>
>It seems to me that this form is more obscure than the following
>alternative one supplied by me.
>
>>
>> will output the following lines:
>>
>> '\0004'
>> '\0010'
>> '\0011'
>> '\0020'
>> '\0100'
>> '\0200'
>> '\0377'
>
>I tried with the following form and obtain the same results as
>yours:
>
>$ printf ''\''\%.4o'\''\n' 4 8 9 16 64 128 255

At least according to the POSIX standard, '\%' in the formatting
parameter of 'printf' causes undefined behavior.  The posixly way of
letting the formatting parameter output one backslash is to provide
two:

printf ''\''\\%.4o'\''\n' 4 8 9 16 64 128 255

In the shell command line, the leading two apostrophes denote an
empty string, which, when prepended to an non‐empty string, just
disappears, so they may be omitted:

printf \''\\%.4o'\''\n' 4 8 9 16 64 128 255

If you prefer escaping (using a backslash) over quoting (using a
pair of apostrophes) in the shell command line you can do

printf \'\\\\%.4o\'\\n 4 8 9 16 64 128 255

instead. 

>I've checked the manual page of 'od', but doesn't have so much
>experience in the selection of its various options.
>
>> So the WWW gave bad advice to you, because it didn't supply the
>> '-v' option and supplied '-t' 'x4' rather than '-t' 'x1', causing
>> the little‐endian‐big‐endian trouble.
>
>The endian conversion problem is architecture and OS platform
>dependent which only happens when the data is written into ram. I'm
>not sure if there is/are some convenient *nix tool(s) to check it.

I'm not sure either. 


The conversion problem happens when bytes are combined into larger
units.  If 'od' is given the parameters '-t' 'x1' rather than '-t'
'x4' then there is no combining, hence no conversion problem:  'od'
just outputs the sedecimal representation of the raw bytes in the
same order as it reads them. 

And as there is no need to combine the 4 partuuid bytes into one
4‐byte number in order to copy them from one disk image to an other,
the endianess problem can easily be avoided. 


Just look at the 'od' options one after the other: 


I don't want line suppression to happen.  => I supply the '-v'
option. 

I don't want address information to be printed.  => I supply the
'-A' option together with its parameter 'n'. 

I don't want the endianness problem to happen.  => I supply the '-t'
option together with its parameter 'x1' rather than with 'x4'. 

That's all of it. 

>> # Print the sedecimal numbers, one per line, converted to
>>
>
>Should be wording as 'sedecimal' or 'hexadecimal'?
>

I prefer 'sedecimal', from Latin 'sedecim', sixteen.  YMMV. 

[Janis Papanagnou]

On 18.04.2021 06:32, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongy...@gmail.com>:
>>

>> Should be wording as 'sedecimal' or 'hexadecimal'?

As far as I can tell, "hexadecimal" is the predominant form.
Lexically it's a mixture of a Greek and a Latin part, while
"sedecimal" is consistent Latin, the pure Greek "hexadecadic"
(in German "hexadekadisch") seems rarely used (and where in
Greek they have the parts swapped, as in "dekaexadikó").

> I prefer 'sedecimal', from Latin 'sedecim', sixteen. YMMV.

There's a chance that it could get confused with the similar
word "sexagesimal" (which has another meaning). But generally
I suppose it's a matter of personal preference.

Janis

[hongy...@gmail.com]

On Sunday, April 18, 2021 at 12:32:35 PM UTC+8, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongy...@gmail.com>:
> >On Sunday, April 18, 2021 at 3:24:13 AM UTC+8, Helmut Waitzmann wrote:
>
> >> For example:
> >>
> >> printf \''\\%.4o'\''\n' 4 8 9 16 64 128 255
> >
> >It seems to me that this form is more obscure than the following
> >alternative one supplied by me.
> >
> >>
> >> will output the following lines:
> >>
> >> '\0004'
> >> '\0010'
> >> '\0011'
> >> '\0020'
> >> '\0100'
> >> '\0200'
> >> '\0377'
> >
> >I tried with the following form and obtain the same results as
> >yours:
> >
> >$ printf ''\''\%.4o'\''\n' 4 8 9 16 64 128 255
> At least according to the POSIX standard, '\%' in the formatting
> parameter of 'printf' causes undefined behavior. The posixly way of
> letting the formatting parameter output one backslash is to provide
> two:
> printf ''\''\\%.4o'\''\n' 4 8 9 16 64 128 255

If so, why in my case, it can still use one '\' to run smoothly?

[Keith Thompson]

Helmut Waitzmann <nn.th...@xoxy.net> writes:
> "hongy...@gmail.com" <hongy...@gmail.com>:
[...]

>> Should be wording as 'sedecimal' or 'hexadecimal'?
>
> I prefer 'sedecimal', from Latin 'sedecim', sixteen.  YMMV. 

I have never seen "sedecimal" before, and most people are unlikely to
understand it. dictionary.com doesn't recognize it. (A bit of Googling
indicates that "sedezimal" seems to be the German word for hexadecimal.)

--
Keith Thompson (The_Other_Keith) Keith.S.T...@gmail.com
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:
>On Sunday, April 18, 2021 at 12:32:35 PM UTC+8, Helmut Waitzmann wrote:

>> At least according to the POSIX standard, '\%' in the formatting
>> parameter of 'printf' causes undefined behavior. The posixly way of
>> letting the formatting parameter output one backslash is to provide
>> two:
>>
>> printf ''\''\\%.4o'\''\n' 4 8 9 16 64 128 255
>
>If so, why in my case, it can still use one '\' to run smoothly?
>

Probably, because the 'printf' on your system tolerates a single
backslash not part of an escape sequence to stand literally for
itself.  But I won't rely on it:  Different 'printf' implementations
might behave different.  Also, if one relies on that tolerance, one
might be surprised by the following effect: 

With the 'printf' on your system,


v=' ' && printf '\'"$v"'\'

will output '\ \'. 


v='' && printf '\'"$v"'\'

will output '\' rather than '\\'.  So, where is the second backslash
gone? 


On the other hand, if one doesn't rely on that tolerance, there are
no surprises:

v=' ' && printf '\\'"$v"'\\'

will output '\ \'. 


v='' && printf '\\'"$v"'\\'

will output '\\'. 

[Janis Papanagnou]

On 18.04.2021 12:11, Keith Thompson wrote:
> Helmut Waitzmann <nn.th...@xoxy.net> writes:
>> "hongy...@gmail.com" <hongy...@gmail.com>:
> [...]
>>> Should be wording as 'sedecimal' or 'hexadecimal'?
>>
>> I prefer 'sedecimal', from Latin 'sedecim', sixteen. YMMV.
>
> I have never seen "sedecimal" before, and most people are unlikely to
> understand it. dictionary.com doesn't recognize it.

The two online dictionaries I use both list it as English word.[*]

> (A bit of Googling
> indicates that "sedezimal" seems to be the German word for hexadecimal.)

As also Helmut mentioned it's a Latin form, thus incorporated as
loanword in many languages.

In Germany, at least in IT contexts, "hexadecimal" is predominant,
in informal speech commonly and inaccurately often abbreviated as
"hex".

Janis

[*] dict.cc and leo.org

[hongy...@gmail.com]

Indeed. Thank you for your wonderful examples.

HY

[William Unruh]

On 2021-04-18, Janis Papanagnou <janis_pa...@hotmail.com> wrote:
> On 18.04.2021 06:32, Helmut Waitzmann wrote:
>> "hongy...@gmail.com" <hongy...@gmail.com>:
>>>
>>> Should be wording as 'sedecimal' or 'hexadecimal'?
>
> As far as I can tell, "hexadecimal" is the predominant form.
> Lexically it's a mixture of a Greek and a Latin part, while
> "sedecimal" is consistent Latin, the pure Greek "hexadecadic"
> (in German "hexadekadisch") seems rarely used (and where in
> Greek they have the parts swapped, as in "dekaexadikó").
>
>> I prefer 'sedecimal', from Latin 'sedecim', sixteen. YMMV.

Language is a means of communication. You might prefer it, but since
everyone else uses hexadecimal in English, if you want to communicate it
might be better to use what others use. Note that this is an English
word, whereever it came from originally. It's history might be of
interest, but its current meaning is what is important for communcation.

Unless your purpose is to befuddle and impress rather than communicate.

[hongy...@gmail.com]

On Monday, April 19, 2021 at 1:56:28 AM UTC+8, William Unruh wrote:
> On 2021-04-18, Janis Papanagnou <janis_pa...@hotmail.com> wrote:
> > On 18.04.2021 06:32, Helmut Waitzmann wrote:
> >> "hongy...@gmail.com" <hongy...@gmail.com>:
> >>>
> >>> Should be wording as 'sedecimal' or 'hexadecimal'?
> >
> > As far as I can tell, "hexadecimal" is the predominant form.
> > Lexically it's a mixture of a Greek and a Latin part, while
> > "sedecimal" is consistent Latin, the pure Greek "hexadecadic"
> > (in German "hexadekadisch") seems rarely used (and where in
> > Greek they have the parts swapped, as in "dekaexadikó").
> >
> >> I prefer 'sedecimal', from Latin 'sedecim', sixteen. YMMV.
> Language is a means of communication. You might prefer it, but since
> everyone else uses hexadecimal in English, if you want to communicate it
> might be better to use what others use. Note that this is an English
> word, whereever it came from originally. It's history might be of
> interest, but its current meaning is what is important for communcation.
>
> Unless your purpose is to befuddle and impress rather than communicate.

I totally agree with this view

HY

[hongy...@gmail.com]

On Thursday, April 15, 2021 at 6:44:47 AM UTC+8, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongy...@gmail.com>:
>
> [The 'conv=notrunc' option of 'dd']
> >Assuming I have used this option, what exact restore command should
> >I use?
> First you should check, that the command
>
>
> {
> printf '%s\n' 'ibase=16' &&
> printf '%s\n' b0 36 1b 76 |
> LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
> } | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
> xargs -E '' -- printf '%b' |
> od -v -A n -t x4

I found the following interesting option of od, which may be used to simplify the above trick:

[...]
| od -v -A n -t x4 --endian=big
b0361b76

and

[...]
| od -v -A n -t x4 --endian=little
761b36b0

Regards,
HY

[hongy...@gmail.com]

On Sunday, April 18, 2021 at 12:32:35 PM UTC+8, Helmut Waitzmann wrote:

Another question I still not so clear is the following description from `man 1 printf':

%b ARGUMENT as a string with '\' escapes interpreted, except that
octal escapes are of the form \0 or \0NNN

As we can see, here we use the form '\0NNN' to denote the octal represented byte value. But as the man page excerpted above said: '... octal escapes are of the form \0 or \0NNN', so, how to use the form \0 as the octal escape?

Regards,
HY

[Ben Bacarisse]

"hongy...@gmail.com" <hongy...@gmail.com> writes:

> Another question I still not so clear is the following description
> from `man 1 printf':
>
> %b ARGUMENT as a string with '\' escapes interpreted, except that
> octal escapes are of the form \0 or \0NNN
>
>
> As we can see, here we use the form '\0NNN' to denote the octal
> represented byte value. But as the man page excerpted above said:
> '... octal escapes are of the form \0 or \0NNN', so, how to use the
> form \0 as the octal escape?

You just write \0:

printf "a\0b" | xargs -n1 -0 echo

The manual is being pedantic. Octal escapes start \0 and that applies
to a lone \0. The 0 is therefore an octal number. It make difference,
of course, because 0 has the same value no matter what base is assumed,
but computer people like this sort of thing. (Never ask "do you want
red or white?" at the computing lab's Christmas party, even if you are
holding a bottle of each. Everyone will just say "yes".)

--
Ben.

[jo...@schily.net]

In article <dc4a4cf6-c6b4-4409...@googlegroups.com>,

hongy...@gmail.com <hongy...@gmail.com> wrote:

>Another question I still not so clear is the following description from `man 1 printf':
>
> %b ARGUMENT as a string with '\' escapes interpreted, except that
> octal escapes are of the form \0 or \0NNN
>
>
>As we can see, here we use the form '\0NNN' to denote the octal represented byte value. But as the man page excerpted above said: '...
>octal escapes are of the form \0 or \0NNN', so, how to use the form \0 as the octal escape?

The standard says that \0 followed by a non-octal digit is a nul char.

\0 followed by up to 3 octal digits is also valid.

But be careful,

printf '%b\n' 'ab\0cd'

is not supported corectly by all printf(1) implementations.

In special dash(1) is affected.

--
EMail:jo...@schily.net Jörg Schilling D-13353 Berlin
Blog: http://schily.blogspot.com/
URL: http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/

[Keith Thompson]

It's not just being pedantic, it's incorrect. It implies that \0 has to
be followed by exactly 0 or 3 octal digits. In fact, it can be followed
by 0, 1, 2, or 3 octal digits. The info documentation ("info coreutils
printf") is correct, as is the POSIX specification. Quoting the info
manual:

• An additional directive ‘%b’, prints its argument string with ‘\’
escapes interpreted in the same way as in the FORMAT string,
except that octal escapes are of the form ‘\0OOO’ where OOO is 0
to 3 octal digits. If ‘\OOO’ is nine-bit value, ignore the
ninth bit. If a precision is also given, it limits the number of bytes
printed from the converted string.

So for example, the man page implies that
printf "%b" "\01x"
prints a null character, the digit '1', and the letter 'x'. In fact it
prints character 1 (Ctrl-A) and the letter 'x'.

[jo...@schily.net]

In article <87zgxrb...@nosuchdomain.example.com>,

Keith Thompson <Keith.S.T...@gmail.com> wrote:

>> You just write \0:
>>
>> printf "a\0b" | xargs -n1 -0 echo
>>
>> The manual is being pedantic. Octal escapes start \0 and that applies
>> to a lone \0. The 0 is therefore an octal number. It make difference,
>> of course, because 0 has the same value no matter what base is assumed,
>> but computer people like this sort of thing. (Never ask "do you want
>> red or white?" at the computing lab's Christmas party, even if you are
>> holding a bottle of each. Everyone will just say "yes".)
>
>It's not just being pedantic, it's incorrect. It implies that \0 has to
>be followed by exactly 0 or 3 octal digits. In fact, it can be followed
>by 0, 1, 2, or 3 octal digits. The info documentation ("info coreutils
>printf") is correct, as is the POSIX specification. Quoting the info

Be careful, the octal escape for printf arguments is \ followed by 1..3 octal
digits, while the %b format argument is \0 followed by 0..3 octal digits.

Some printf implementations do not handle that correctly.

[Helmut Waitzmann]

jo...@schily.net:

>But be careful,
>
> printf '%b\n' 'ab\0cd'
>
>is not supported corectly by all printf(1) implementations.
>
>In special dash(1) is affected.

… in some legacy version of Debian: 


Debian 6:


dash -c -- '"$@"' dash printf '%b\n' 'ab\0cd' |
od -A n -t c

yields

a b \n


Debian 10 (which is the current stable version):


dash -c -- '"$@"' dash printf '%b\n' 'ab\0cd' |
od -A n -t c

yields

a b \0 c d \n

[hongy...@gmail.com]

I can't find the null character:

$ printf "%b" "\01x" | od -v -A n -t o1
001 170

HY

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:

Yes.  In fact it does what Keith wrote.  If one uses the '\0NNN'
form (and single rather than double quotes)

printf '%b' '\00001x' | od -v -A n -t o1

then there is no ambiguity. 

[Keith Thompson]

"hongy...@gmail.com" <hongy...@gmail.com> writes:
> On Thursday, April 22, 2021 at 5:34:11 AM UTC+8, Keith Thompson wrote:

[...]

>> So for example, the man page implies that
>> printf "%b" "\01x"
>> prints a null character, the digit '1', and the letter 'x'. In fact it
>> prints character 1 (Ctrl-A) and the letter 'x'.
>
> I can't find the null character:
>
> $ printf "%b" "\01x" | od -v -A n -t o1
> 001 170

No, of course not. Why would you expect a null character?

The description in the man page implies that it would print a null
character. That description is incorrect; it doesn't match the info
manual, the POSIX specification, or the actual behavior.

[hongy...@gmail.com]

Yep. See below:

$ printf '%b' '\00001x' | od -v -A n -t o1
000 061 170

[jo...@schily.net]

In article <83sg3jf...@helmutwaitzmann.news.arcor.de>,

Helmut Waitzmann <oe.th...@xoxy.net> wrote:
>jo...@schily.net:
>
>>But be careful,
>>
>> printf '%b\n' 'ab\0cd'
>>
>>is not supported corectly by all printf(1) implementations.
>>
>>In special dash(1) is affected.
>

>… in some legacy version of Debian: 

There are some long term support Linux distros that did not update
the dash to a working one...