RISC-V Linux kernel image format standardization
347 views
Skip to first unread message
Karsten Merker
Apr 27, 2019, 6:44:56 PM4/27/19
to Anup Patel, Atish Patra, Bin Meng, Lukas Auer, Palmer Dabbelt, Rick Chen, RISC-V SW Dev
[CCing a number of people involved in kernel and bootloader
development to gather feedback]
Hello,
I'm trying to make booting a Linux distribution such as Debian
work out-of-the-box on riscv64, but have run into a problem
with the use of different kernel formats by different
bootloaders.
Currently the Debian kernel packages ship a vmlinux (i.e. an ELF
kernel image) because that is what BBL wants and BBL was the only
RISC-V bootloader available at the time when we added RISC-V
support to Debian's kernel packages:
$file /boot/vmlinux-4.19.0-4-riscv64
/boot/vmlinux-4.19.0-4-riscv64: ELF 64-bit LSB executable, UCB RISC-V, version 1 (SYSV), statically linked, BuildID[sha1]=b3b55da6473cddd2079e013062e0783997eb06b4, not stripped
Now I would like to move to OpenSBI and U-Boot, but there doesn't
seem to be a way to boot an ELF kernel plus an initramfs with
U-Boot on riscv64. If there is one, I'd appreciate very much if
somebody could point me into the right direction.
The U-Boot "bootm" command only seems to work with a flat kernel
image wrapped in a uImage container, but not with an ELF kernel
wrapped into a uImage container, and AIUI Atish's "booti" patches
for U-Boot and Linux remove the requirement for the uImage
container but also require a flat kernel image instead of an ELF
file.
So now I'm in the unfortunate situation that different
bootloaders require different and incompatible kernel image
formats but the infrastructure for handling kernels in Debian
assumes that there is _one_ canonical kernel image format per
architecture (which is true for all Debian architectures except
RISC-V), i.e. I cannot simply ship the kernel in two different
formats.
As we will see more different methods to boot a Linux kernel on
RISC-V in the future (e.g. BBL, "plain" U-Boot, EFI loaders) I
think it is important to come to a proper definition of _the_
canonical Linux kernel image format for RISC-V, and this has to
work for all methods of booting a kernel. Has there been any
attempt at defining an "official" standard kernel format?
I have looked through the various documents under
https://github.com/riscv and through the Linux kernel
"Documentation" directory but haven't found anything relevant.
Regards,
Karsten
--
Ich widerspreche hiermit ausdrücklich der Nutzung sowie der
Weitergabe meiner personenbezogenen Daten für Zwecke der Werbung
sowie der Markt- oder Meinungsforschung.
development to gather feedback]
Hello,
I'm trying to make booting a Linux distribution such as Debian
work out-of-the-box on riscv64, but have run into a problem
with the use of different kernel formats by different
bootloaders.
Currently the Debian kernel packages ship a vmlinux (i.e. an ELF
kernel image) because that is what BBL wants and BBL was the only
RISC-V bootloader available at the time when we added RISC-V
support to Debian's kernel packages:
$file /boot/vmlinux-4.19.0-4-riscv64
/boot/vmlinux-4.19.0-4-riscv64: ELF 64-bit LSB executable, UCB RISC-V, version 1 (SYSV), statically linked, BuildID[sha1]=b3b55da6473cddd2079e013062e0783997eb06b4, not stripped
Now I would like to move to OpenSBI and U-Boot, but there doesn't
seem to be a way to boot an ELF kernel plus an initramfs with
U-Boot on riscv64. If there is one, I'd appreciate very much if
somebody could point me into the right direction.
The U-Boot "bootm" command only seems to work with a flat kernel
image wrapped in a uImage container, but not with an ELF kernel
wrapped into a uImage container, and AIUI Atish's "booti" patches
for U-Boot and Linux remove the requirement for the uImage
container but also require a flat kernel image instead of an ELF
file.
So now I'm in the unfortunate situation that different
bootloaders require different and incompatible kernel image
formats but the infrastructure for handling kernels in Debian
assumes that there is _one_ canonical kernel image format per
architecture (which is true for all Debian architectures except
RISC-V), i.e. I cannot simply ship the kernel in two different
formats.
As we will see more different methods to boot a Linux kernel on
RISC-V in the future (e.g. BBL, "plain" U-Boot, EFI loaders) I
think it is important to come to a proper definition of _the_
canonical Linux kernel image format for RISC-V, and this has to
work for all methods of booting a kernel. Has there been any
attempt at defining an "official" standard kernel format?
I have looked through the various documents under
https://github.com/riscv and through the Linux kernel
"Documentation" directory but haven't found anything relevant.
Regards,
Karsten
--
Ich widerspreche hiermit ausdrücklich der Nutzung sowie der
Weitergabe meiner personenbezogenen Daten für Zwecke der Werbung
sowie der Markt- oder Meinungsforschung.
atish patra
Apr 27, 2019, 7:41:21 PM4/27/19
to Karsten Merker, Anup Patel, Bin Meng, Lukas Auer, Palmer Dabbelt, Rick Chen, RISC-V SW Dev
On Sat, Apr 27, 2019 at 3:44 PM Karsten Merker <mer...@debian.org> wrote:
[CCing a number of people involved in kernel and bootloader
development to gather feedback]
Hello,
I'm trying to make booting a Linux distribution such as Debian
work out-of-the-box on riscv64, but have run into a problem
with the use of different kernel formats by different
bootloaders.
Currently the Debian kernel packages ship a vmlinux (i.e. an ELF
kernel image) because that is what BBL wants and BBL was the only
RISC-V bootloader available at the time when we added RISC-V
support to Debian's kernel packages:
$file /boot/vmlinux-4.19.0-4-riscv64
/boot/vmlinux-4.19.0-4-riscv64: ELF 64-bit LSB executable, UCB RISC-V, version 1 (SYSV), statically linked, BuildID[sha1]=b3b55da6473cddd2079e013062e0783997eb06b4, not stripped
Now I would like to move to OpenSBI and U-Boot, but there doesn't
seem to be a way to boot an ELF kernel plus an initramfs with
U-Boot on riscv64. If there is one, I'd appreciate very much if
somebody could point me into the right direction.
The U-Boot "bootm" command only seems to work with a flat kernel
image wrapped in a uImage container, but not with an ELF kernel
wrapped into a uImage container, and AIUI Atish's "booti" patches
for U-Boot and Linux remove the requirement for the uImage
container but also require a flat kernel image instead of an ELF
file.
Yeah ELF support was not planned in booti patch. I am not even sure if U-Boot can load a ELF file
if it is not wrapped inside an uImage. Btw I recently updated the patch to accept compressed Image(Image.gz) as well.
So now I'm in the unfortunate situation that different
bootloaders require different and incompatible kernel image
formats but the infrastructure for handling kernels in Debian
assumes that there is _one_ canonical kernel image format per
architecture (which is true for all Debian architectures except
RISC-V), i.e. I cannot simply ship the kernel in two different
formats.
What is the standard kerne image format for arm64/x86 ?
As we will see more different methods to boot a Linux kernel on
RISC-V in the future (e.g. BBL, "plain" U-Boot, EFI loaders) I
think it is important to come to a proper definition of _the_
canonical Linux kernel image format for RISC-V, and this has to
work for all methods of booting a kernel. Has there been any
attempt at defining an "official" standard kernel format?
I have looked through the various documents under
https://github.com/riscv and through the Linux kernel
"Documentation" directory but haven't found anything relevant.
AFAIK, there is no standard related to Linux Image format in RISC-V land. It would be a good to have one and include
it in the Unix Platform specification. We can discuss pros/cons of different format and agree on something during the
first meeting which is scheduled next Wednesday (1st May 2019).
If you want you can join here.
https://lists.risc-v.org/g/tech-unixplatformspec/topics
https://lists.risc-v.org/g/tech-unixplatformspec/topics
Regards,
Atish
Regards,
Karsten
--
Ich widerspreche hiermit ausdrücklich der Nutzung sowie der
Weitergabe meiner personenbezogenen Daten für Zwecke der Werbung
sowie der Markt- oder Meinungsforschung.
--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/20190427224443.i2ea4q373jk4o4ec%40excalibur.cnev.de.
Regards,
Atish
David Abdurachmanov
Apr 27, 2019, 11:10:41 PM4/27/19
to atish patra, Karsten Merker, Anup Patel, Bin Meng, Lukas Auer, Palmer Dabbelt, Rick Chen, RISC-V SW Dev
objcopy to extract the binary.
See: https://github.com/riscv/riscv-pk/blob/537ae11ae506b6fa6c4dd9dc37c9cc818af84027/bbl/bbl.mk.in#L22
For the Fedora we are using Image.gz (same as on AArch64/arm64).
We haven't built ELF kernels for the last ~2 months.
For the next boot flow update we hope to use "booti" with Image.gz to
a void having wrapped kernel and ramdisk.
That will bring us closer to to how ARMv7-A boots right now in Fedora.
>
>>
>> So now I'm in the unfortunate situation that different
>> bootloaders require different and incompatible kernel image
>> formats but the infrastructure for handling kernels in Debian
>> assumes that there is _one_ canonical kernel image format per
>> architecture (which is true for all Debian architectures except
>> RISC-V), i.e. I cannot simply ship the kernel in two different
>> formats.
>>
>
> What is the standard kerne image format for arm64/x86 ?
>
>> As we will see more different methods to boot a Linux kernel on
>> RISC-V in the future (e.g. BBL, "plain" U-Boot, EFI loaders) I
>> think it is important to come to a proper definition of _the_
>> canonical Linux kernel image format for RISC-V, and this has to
>> work for all methods of booting a kernel. Has there been any
>> attempt at defining an "official" standard kernel format?
>> I have looked through the various documents under
>> https://github.com/riscv and through the Linux kernel
>> "Documentation" directory but haven't found anything relevant.
>>
>
> AFAIK, there is no standard related to Linux Image format in RISC-V land. It would be a good to have one and include
> it in the Unix Platform specification. We can discuss pros/cons of different format and agree on something during the
> first meeting which is scheduled next Wednesday (1st May 2019).
the specs. I would assume Image / Image.gz being the most popular
kernel targets.
david
>
> If you want you can join here.
> https://lists.risc-v.org/g/tech-unixplatformspec/topics
>
> Regards,
> Atish
>>
>> Regards,
>> Karsten
>> --
>> Ich widerspreche hiermit ausdrücklich der Nutzung sowie der
>> Weitergabe meiner personenbezogenen Daten für Zwecke der Werbung
>> sowie der Markt- oder Meinungsforschung.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
>> To post to this group, send email to sw-...@groups.riscv.org.
>> Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
>> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/20190427224443.i2ea4q373jk4o4ec%40excalibur.cnev.de.
>
>
>
> --
> Regards,
> Atish
>
> --
> You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
> To post to this group, send email to sw-...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
Karsten Merker
Apr 28, 2019, 4:38:10 AM4/28/19
to atish patra, Karsten Merker, Anup Patel, Bin Meng, Lukas Auer, Palmer Dabbelt, Rick Chen, RISC-V SW Dev
On Sat, Apr 27, 2019 at 04:4 1:05PM -0700, atish patra wrote:
> On Sat, Apr 27, 2019 at 3:44 PM Karsten Merker <mer...@debian.org> wrote:
[...]
> On Sat, Apr 27, 2019 at 3:44 PM Karsten Merker <mer...@debian.org> wrote:
> > So now I'm in the unfortunate situation that different
> > bootloaders require different and incompatible kernel image
> > formats but the infrastructure for handling kernels in Debian
> > assumes that there is _one_ canonical kernel image format per
> > architecture (which is true for all Debian architectures except
> > RISC-V), i.e. I cannot simply ship the kernel in two different
> > formats.
> >
> What is the standard kerne image format for arm64/x86 ?
For x86/amd64 bzImage, for armhf zImage and for arm64
> > bootloaders require different and incompatible kernel image
> > formats but the infrastructure for handling kernels in Debian
> > assumes that there is _one_ canonical kernel image format per
> > architecture (which is true for all Debian architectures except
> > RISC-V), i.e. I cannot simply ship the kernel in two different
> > formats.
> >
> What is the standard kerne image format for arm64/x86 ?
plain Image (similar to what your booti patches use).
> AFAIK, there is no standard related to Linux Image format in
> RISC-V land. It would be a good to have one and include it in
> the Unix Platform specification. We can discuss pros/cons of
> different format and agree on something during the first
> meeting which is scheduled next Wednesday (1st May 2019).
>
> If you want you can join here.
> https://lists.risc-v.org/g/tech-unixplatformspec/topics
very useful. Unfortunately I cannot join the Unix platform
working group discussion because it is only open to RISC-V
foundation members, but I am not a member and cannot become one
because doing so would require signing an NDA as part of the
membership agreement and that is unacceptable to me.
For arm64, the kernel format (flat kernel with a specific header)
and the boot interface are defined in the Linux kernel source
tree in Documentation/arm64/booting.txt, including the
conventions for non-EFI and EFI boot. Having a similar document
for RISC-V would IMHO solve the issue. Discussions about such a
document should be held in the open, though, in particular as
this should go into the Linux kernel and doing so requires public
discussion and review on the respective kernel mailinglists
anyway.
atish patra
Apr 29, 2019, 1:31:09 PM4/29/19
to Karsten Merker, Anup Patel, Bin Meng, Lukas Auer, Palmer Dabbelt, Rick Chen, RISC-V SW Dev
On Sun, Apr 28, 2019 at 1:38 AM Karsten Merker <mer...@debian.org> wrote:
On Sat, Apr 27, 2019 at 04:4 1:05PM -0700, atish patra wrote:
> On Sat, Apr 27, 2019 at 3:44 PM Karsten Merker <mer...@debian.org> wrote:
[...]
> > So now I'm in the unfortunate situation that different
> > bootloaders require different and incompatible kernel image
> > formats but the infrastructure for handling kernels in Debian
> > assumes that there is _one_ canonical kernel image format per
> > architecture (which is true for all Debian architectures except
> > RISC-V), i.e. I cannot simply ship the kernel in two different
> > formats.
> >
> What is the standard kerne image format for arm64/x86 ?
For x86/amd64 bzImage, for armhf zImage and for arm64
plain Image (similar to what your booti patches use).
Thanks for the information. So we can proceed with plain Image/Image.gz for now as that
suits both Fedora/Debian.
> AFAIK, there is no standard related to Linux Image format in
> RISC-V land. It would be a good to have one and include it in
> the Unix Platform specification. We can discuss pros/cons of
> different format and agree on something during the first
> meeting which is scheduled next Wednesday (1st May 2019).
>
> If you want you can join here.
> https://lists.risc-v.org/g/tech-unixplatformspec/topics
Getting to an agreement about a standard format would IMHO be
very useful. Unfortunately I cannot join the Unix platform
working group discussion because it is only open to RISC-V
foundation members, but I am not a member and cannot become one
because doing so would require signing an NDA as part of the
membership agreement and that is unacceptable to me.
For arm64, the kernel format (flat kernel with a specific header)
and the boot interface are defined in the Linux kernel source
tree in Documentation/arm64/booting.txt, including the
conventions for non-EFI and EFI boot. Having a similar document
for RISC-V would IMHO solve the issue. Discussions about such a
document should be held in the open, though, in particular as
this should go into the Linux kernel and doing so requires public
discussion and review on the respective kernel mailing lists
anyway.
Of course. Anything related to kernel documentation or source goes into kernel mailing list for discussion.
Me & David had a similar discussion about putting up a booting document for RISC-V similar to Documentation/arm64/booting.txt.
I will send it to the mailing list once the booti patches are accepted in kernel/U-Boot.
Regards,
Atish
Atish
Regards,
Karsten
--
Ich widerspreche hiermit ausdrücklich der Nutzung sowie der
Weitergabe meiner personenbezogenen Daten für Zwecke der Werbung
sowie der Markt- oder Meinungsforschung.
--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/20190428083757.nny4dsbzneyp32j4%40excalibur.cnev.de.
Regards,
Atish
Dennis Clarke
Apr 29, 2019, 4:22:33 PM4/29/19
to sw-...@groups.riscv.org
> > > So now I'm in the unfortunate situation that different
> > > bootloaders require different and incompatible kernel image
> > > formats but the infrastructure for handling kernels in Debian
> > > assumes that there is _one_ canonical kernel image format per
> > > architecture (which is true for all Debian architectures except
> > > RISC-V), i.e. I cannot simply ship the kernel in two different
> > > formats.
> > >
Very educational.
> > What is the standard kerne image format for arm64/x86 ?
>
> For x86/amd64 bzImage, for armhf zImage and for arm64
> plain Image (similar to what your booti patches use).
arm7$
arm7$ uname -a
Linux arm7 4.4.132+ #1 SMP Tue Oct 23 18:03:49 CST 2018 armv7l GNU/Linux
arm7$ ls -lapb /boot
total 53518
drwxr-xr-x 5 root root 1536 Jan 1 1970 ./
drwxr-xr-x 23 root root 4096 Mar 26 08:09 ../
-rwxr-xr-x 1 root root 3027355 Oct 27 2018 System.map-4.9.0-8-armmp-lpae
-rwxr-xr-x 1 root root 190506 Oct 27 2018 config-4.9.0-8-armmp-lpae
drwxr-xr-x 2 root root 512 Mar 24 16:47 extlinux/
drwxr-xr-x 2 root root 512 Mar 24 16:47 hdmi/
-rwxr-xr-x 1 root root 304 Oct 23 2018 hw_intf.conf
-rwxr-xr-x 1 root root 17349059 Mar 24 18:17 initrd.img
-rwxr-xr-x 1 root root 18576809 Mar 26 08:06 initrd.img-4.9.0-8-armmp-lpae
drwxr-xr-x 2 root root 2560 Mar 24 16:47 overlays/
-rwxr-xr-x 1 root root 72281 Oct 23 2018 rk3288-miniarm.dtb
-rwxr-xr-x 1 root root 3821600 Oct 27 2018 vmlinuz
-rwxr-xr-x 1 root root 3821600 Oct 27 2018 vmlinuz-4.9.0-8-armmp-lpae
-rwxr-xr-x 1 root root 7930936 Oct 23 2018 zImage
arm7$
Sadly getting that thing to boot anything else other than the 4.4.132+
version that gets shipped is some sort of a mystery to me. Also it is
stuck with using a FAT32 boot partition and thus the Debian update
scripts fails because the metadata for files simply doesn't exist.
Debian on ppc64 is very reasonable looking and behaves much like any
amd64/x86_64 boxen for kernel updates and such. With the obvious
exception being that there is openfirmware to deal with and that means
GRUB is still a bit of work.
ppc64$
ppc64$ uname -a
Linux hydra 4.19.0-2-powerpc64 #1 SMP Debian 4.19.16-1 (2019-01-17)
ppc64 GNU/Linux
ppc64$ cat /etc/debian_version
buster/sid
ppc64$
ppc64$ ls -lapb /boot/
total 56092
drwxr-xr-x 2 root root 4096 Apr 6 05:21 ./
drwxr-xr-x 22 root root 4096 Jan 14 00:27 ../
-rw-r--r-- 1 root root 170238 Jan 17 18:56 config-4.19.0-2-powerpc64
lrwxrwxrwx 1 root root 29 Jan 28 11:23 initrd.img ->
initrd.img-4.19.0-2-powerpc64
-rw-r--r-- 1 root root 26797511 Apr 6 05:21 initrd.img-4.19.0-2-powerpc64
-rw-r--r-- 1 root root 4300219 Jan 17 18:56 System.map-4.19.0-2-powerpc64
lrwxrwxrwx 1 root root 26 Jan 28 11:23 vmlinux ->
vmlinux-4.19.0-2-powerpc64
-rwxr-xr-x 1 root root 26072720 Jan 17 18:56 vmlinux-4.19.0-2-powerpc64
ppc64$
However I have grub booting and working fine on ppc64.
However arm and ppc64 are both slightly off topic here. Sorry.
Merely providing a reference point.
>
> Thanks for the information. So we can proceed with plain Image/Image.gz
> for now as that suits both Fedora/Debian.
>
> > AFAIK, there is no standard related to Linux Image format in
> > RISC-V land. It would be a good to have one and include it in
> > the Unix Platform specification. We can discuss pros/cons of
> > different format and agree on something during the first
> > meeting which is scheduled next Wednesday (1st May 2019).
is a wonderful thing but I doubt that Oracle folks will ever care to
port Solaris to RISC-V. With FreeBSD 13 I can boot with bbl now and
the FreeBSD kernel format is a bit of a mystery to me. Sorry. I may
look into that.
> >
> > If you want you can join here.
> > https://lists.risc-v.org/g/tech-unixplatformspec/topics
>
> Getting to an agreement about a standard format would IMHO be
> very useful. Unfortunately I cannot join the Unix platform
> working group discussion because it is only open to RISC-V
> foundation members, but I am not a member and cannot become one
> because doing so would require signing an NDA as part of the
> membership agreement and that is unacceptable to me.
doubt that folks from Oracle, IBM and SCO are in there together. If at
all.
>
> For arm64, the kernel format (flat kernel with a specific header)
> and the boot interface are defined in the Linux kernel source
> tree in Documentation/arm64/booting.txt, including the
> conventions for non-EFI and EFI boot. Having a similar document
> for RISC-V would IMHO solve the issue. Discussions about such a
> document should be held in the open, though, in particular as
> this should go into the Linux kernel and doing so requires public
> discussion and review on the respective kernel mailing lists
> anyway.
>
>
> Of course. Anything related to kernel documentation or source goes into
> kernel mailing list for discussion.
> Me & David had a similar discussion about putting up a booting document
> for RISC-V similar to Documentation/arm64/booting.txt.
> I will send it to the mailing list once the booti patches are accepted
> in kernel/U-Boot.
>
this discussion and they are from Western Digital and the other Sifive.
I guess that makes me the non-corporate old UNIX guy.
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken
GreyBeard and suspenders optional
Reply all
Reply to author
Forward
0 new messages