any way to get /var/log/messages on boot/install media?

[John Forkosh]

I'm using
slackware64-current/usb-and-pxe-installers/usbimg2disk.sh
to create a boot/install thumb drive. Is there any way to
modify either the script or the resulting thumb drive or
anything else so that when you boot that drive all the
boot messages get saved to a file similar to /var/log/messages.
Preferably all messages, as if *.* -/var/log/messages
were in /etc/syslog.conf .

Reason I'm asking is because the boot drive boots and installs
fine on two boxes, but hangs on a third, just at the last moment
when it prints that "Press 1 to select alternate keyboard" message.
And unforturnately, it blanks the screen just before printing
that. Retrying several thimes, I think I see a "kernel panic"
message, but can't make out any details before they all disappear.

And I subsequently tried booting an old slack 14.2 dvd, with
exactly the same outcome. However, I then downloaded an ubuntu
iso image, and that boots and installs just fine.
--
John Forkosh ( mailto: j...@f.com where j=john and f=forkosh )

[John Forkosh]

John Forkosh <for...@panix.com> wrote:
> Preferably all messages, as if *.* -/var/log/messages

That should be saying asterisk.asterisk -/var/log/messages

[John McCue]

John Forkosh <for...@panix.com> wrote:
> I'm using
> slackware64-current/usb-and-pxe-installers/usbimg2disk.sh
> to create a boot/install thumb drive. Is there any way to
> modify either the script or the resulting thumb drive or
> anything else so that when you boot that drive all the
> boot messages get saved to a file similar to /var/log/messages.
> Preferably all messages, as if *.* -/var/log/messages
> were in /etc/syslog.conf .

<snip>

I thought dmesg(1) had these messages, if they do
you can always add:
dmesg > filename
someplace once the system is up. Maybe /etc/rc.d/rc.local

[Henrik Carlqvist]

On Sun, 25 Apr 2021 09:16:00 +0000, John Forkosh wrote:

> I'm using
> slackware64-current/usb-and-pxe-installers/usbimg2disk.sh
> to create a boot/install thumb drive. Is there any way to modify either
> the script or the resulting thumb drive or anything else so that when
> you boot that drive all the boot messages get saved to a file similar to
> /var/log/messages. Preferably all messages, as if *.*
> -/var/log/messages were in /etc/syslog.conf .

Anything is possible depending upon how much you are willing to customize
your installation media. You could add syslogd with its configuration and
startup files to initrd.img

> Reason I'm asking is because the boot drive boots and installs fine on
> two boxes, but hangs on a third, just at the last moment when it prints
> that "Press 1 to select alternate keyboard" message.
> And unforturnately, it blanks the screen just before printing that.
> Retrying several thimes, I think I see a "kernel panic" message, but
> can't make out any details before they all disappear.

The sad thing with those log messages is that when you need them the most
you will probably not get them. The problem is that any data written to a
file will be cached, probably on several layers.

First you have your program which calls fwrite which only writes to a
buffer. If you are lucky that buffer might get flushed at every end of
line, but if you are unlycky your program will need to call fflush to
empty its buffer.

Then we have your file system which your kernel probably uses some part
of your RAM for caching data for a more responsive file system, to empty
that cache you will need to call fsync or unmount the file system. Not
until then will your OS think that data has been written to disk.

However, even after your OS thinks that data has been written to disk it
might still be in some cache on your raid controller or on your disk. If
your disk is removable that cache should be emtied if you tell your disk
that you are going to eject it.

If you get a kernel panic, your logged data migth still be in the fwrite
buffer, or in the OS cache. When you reboot you will most likely find
that the file system where you stored your log has been corrupted and
need to be fixed with some tool like fsck.

Sometimes you might be able to save logs which would be lost on a local
disk by logging to a log server on the network or by logging to a printer
or a serial port.

regards Henrik

[John Forkosh]

Thanks, but you snipped the "Reason I'm asking..." part.
The system never is up. Crashes before completely booting.

[John Forkosh]

Henrik Carlqvist <Henrik.C...@deadspam.com> wrote:
> On Sun, 25 Apr 2021 09:16:00 +0000, John Forkosh wrote:
>
>> I'm using
>> slackware64-current/usb-and-pxe-installers/usbimg2disk.sh
>> to create a boot/install thumb drive. Is there any way to modify either
>> the script or the resulting thumb drive or anything else so that when
>> you boot that drive all the boot messages get saved to a file similar to
>> /var/log/messages. Preferably all messages, as if *.*
>> -/var/log/messages were in /etc/syslog.conf .
>
> Anything is possible depending upon how much you are willing to customize
> your installation media. You could add syslogd with its configuration and
> startup files to initrd.img

Thanks for that suggestion, "You could add...". But that only works
for some values of "You", i.e., I've got no idea how to do that.
I'd assume everything needed (and then some) is in the /a/sysklogd-2...txz
package, particularly usr/sbin/syslogd and usr/lib64/libsyslog.so.0.0.0 .
But what do you do about etc/syslog.conf? And I've no idea how to
(re)build initrd.img with all that. But if I knew what I was doing,
then it doesn't sound too hard, at least not too hard to give it a try.
Is there a link to any slack howto, or similar, about all that?

>> Reason I'm asking is because the boot drive boots and installs fine on
>> two boxes, but hangs on a third, just at the last moment when it prints
>> that "Press 1 to select alternate keyboard" message.
>> And unforturnately, it blanks the screen just before printing that.
>> Retrying several thimes, I think I see a "kernel panic" message, but
>> can't make out any details before they all disappear.
>
> The sad thing with those log messages is that when you need them the most
> you will probably not get them. The problem is that any data written to a
> file will be cached, probably on several layers.

Yeah, I realize it's nowhere near guaranteed/foolproof to diagnose
the problem. But I wanted to give it a try and see what I get.
Won't be any worse off than I already am. So nothing to lose,
except the time and effort.

> First you have your program which calls fwrite which only writes to a
> buffer. If you are lucky that buffer might get flushed at every end of
> line, but if you are unlycky your program will need to call fflush to
> empty its buffer.
>
> Then we have your file system which your kernel probably uses some part
> of your RAM for caching data for a more responsive file system, to empty
> that cache you will need to call fsync or unmount the file system. Not
> until then will your OS think that data has been written to disk.
>
> However, even after your OS thinks that data has been written to disk it
> might still be in some cache on your raid controller or on your disk. If
> your disk is removable that cache should be emtied if you tell your disk
> that you are going to eject it.
>
> If you get a kernel panic, your logged data migth still be in the fwrite
> buffer, or in the OS cache. When you reboot you will most likely find
> that the file system where you stored your log has been corrupted and
> need to be fixed with some tool like fsck.
>
> Sometimes you might be able to save logs which would be lost on a local
> disk by logging to a log server on the network or by logging to a printer
> or a serial port.
>
> regards Henrik

Okay, I guess an easier solution is to modify that initrd.img in
some way so it doesn't blank the screen before printing that final
"Press 1 to select keyboard" message. If it does that by just
printing an ansi escape sequence, then maybe I could just vi the
file and change those hex instructions to no-op's. Seems like that
would be simple enough, if I just knew precisely where they were
and what to do about them. Then I could just read the messages
off the screen, without any logging/printing/etc.

[John McCue]

John Forkosh <for...@panix.com> wrote:
<snip>

> Thanks, but you snipped the "Reason I'm asking..." part.
> The system never is up. Crashes before completely booting.

Another option may be to set kernel DEBUG on boot.

I have never used that, but I believe lots of info
will be echoed out somewhere.

[Henrik Carlqvist]

On Mon, 26 Apr 2021 07:46:58 +0000, John Forkosh wrote:
> Thanks for that suggestion, "You could add...". But that only works for
> some values of "You", i.e., I've got no idea how to do that.

The initrd.img is really just a gzip compressed cpio archive, so to
unpack that you will need to do something like this as root:

mkdir /tmp/initrd_src
cd /tmp/initrd_src
gzip -d < /mnt/cdrom/isolinux/initrd.img | cpio -i

Once the initrd.img has been unpacked into initrd_src you can add
binaries and change configuration files or installation scripts.

> Okay, I guess an easier solution is to modify that initrd.img in some
> way so it doesn't blank the screen before printing that final "Press 1
> to select keyboard" message.

Sorry, I removed that question a long time ago on my custom Slackware
installation as I allways want the same keyboard layout. Places to start
looking for that might be bin/startup.sh or some of the files in usr/lib/
setup

Once you have made the changes you want to the directory structure you
can create a new initrd.img with the following command:

cd /tmp/initrd_src
cpio -o -H newc | gzip > /tmp/initrd.img

The last command does not have to be run as root, it will work also as an
ordinary user. However, unpacking initrd.img must be done as root to keep
ownership of files.

I assume that you know what to do with your new /tmp/initrd.img, maybe
you want to place it in the right directory structure and run mkisofs to
create your own custom iso. Maybe you can simply copy it to your USB
stick.

regards Henrik

[John Forkosh]

Thanks, again. You mean, as per
https://man7.org/linux/man-pages/man7/bootparam.7.html ?
I wasn't aware of that, and will give it a try. Can't hurt.
But as mentioned originally, the screen blanks just as the
"Press 1 to select keyboard" message prints and the machine
hangs, so any additional info will also likely just be lost.
There seems to be, at best, several tens of milliseconds
between the "kernel panic" message and screen blanking.
And my reactions haven't been quick enough to stop the boot
during that interval. And no info has been saved on the
usb boot medium. If kernel debug actually saves some there,
then that'll definitely help a lot. Otherwise, probably not.

[John Forkosh]

Henrik Carlqvist <Henrik.C...@deadspam.com> wrote:
> On Mon, 26 Apr 2021 07:46:58 +0000, John Forkosh wrote:
>> Thanks for that suggestion, "You could add...". But that only works
>> for some values of "You", i.e., I've got no idea how to do that.

Thanks again for the additional instructions below, Henrik.
That'll definitely help -- I think I can muddle my way through it
now, with your additional info (and maybe some additional googling).

> The initrd.img is really just a gzip compressed cpio archive, so to
> unpack that you will need to do something like this as root:
>
> mkdir /tmp/initrd_src
> cd /tmp/initrd_src
> gzip -d < /mnt/cdrom/isolinux/initrd.img | cpio -i
>
> Once the initrd.img has been unpacked into initrd_src you can add
> binaries and change configuration files or installation scripts.
>
>> Okay, I guess an easier solution is to modify that initrd.img in some
>> way so it doesn't blank the screen before printing that final "Press 1
>> to select keyboard" message.
>
> Sorry, I removed that question a long time ago on my custom Slackware
> installation as I allways want the same keyboard layout. Places to start
> looking for that might be bin/startup.sh or some of the files in usr/lib/
> setup

Great! This certainly seems like the easiest thing to try first,
especially since you've now confirmed it can be done, and be done
a whole lot more simply than I'd imagined.

> Once you have made the changes you want to the directory structure you
> can create a new initrd.img with the following command:
>
> cd /tmp/initrd_src
> cpio -o -H newc | gzip > /tmp/initrd.img
>
> The last command does not have to be run as root, it will work also as an
> ordinary user. However, unpacking initrd.img must be done as root to keep
> ownership of files.
>
> I assume that you know what to do with your new /tmp/initrd.img, maybe
> you want to place it in the right directory structure and run mkisofs to
> create your own custom iso. Maybe you can simply copy it to your USB
> stick.
>
> regards Henrik

Thanks again, John.

[Henrik Carlqvist]

On Mon, 26 Apr 2021 17:01:09 +0000, Henrik Carlqvist wrote:
> Once you have made the changes you want to the directory structure you
> can create a new initrd.img with the following command:
>
> cd /tmp/initrd_src
> cpio -o -H newc | gzip > /tmp/initrd.img

Whoops, my bad... The last commands should be:

cd /tmp/initrd_src
find . | cpio -o -H newc | gzip > /tmp/initrd.img


cpio is not like tar, it will read the files to put in the archive from
stdin.

regards Henrik