Debian on Apple M1 hardware

[Pip Cet]

Hello everyone,

I hope this is the right list. Sorry if it isn't.

This is mostly a success report: I'm running Debian sid on an M1
MacBook Pro and it's working very well. Many hardware features aren't
supported yet, but with one exception, it's kernel support that is
missing.

I'm using the kernel image provided at
https://downloads.corellium.info/linuxnvme.macho.

Unfortunately, so far, I haven't been able to recompile a working
kernel from the source tree at https://github.com/corellium/linux-m1
and the preloader at https://github.com/corellium/preloader-m1. I've
contacted the developer at Corellium about that and I think we're
working on a solution.

So far, I've only run into one Debian issue: Emacs doesn't work. It
works fine when compiled from the source tree (that's what I'm typing
this in), and the failure appears to be related to the page size in
use by the Corellium kernel (it's an unaligned mmap that fails).

So how do we proceed once a fully free kernel is available? As I said,
I think that's only a matter of time, apart from the non-free firmware
required for some of the hardware.

The one exception to hardware support being a kernel issue is that the
WiFi requires an extra dance to read ROM information before
generating, in userspace, the precise firmware that the module
requires.

Thanks for an OS that's already more usable than MacOS on this machine!

Pip

[Paul Wise]

On Sat, Mar 13, 2021 at 8:57 AM Pip Cet wrote:

> I'm using the kernel image provided at
> https://downloads.corellium.info/linuxnvme.macho.

I read that Corellium haven't participated in upstreaming Linux
support for the Apple M1 devices, so this is likely to be superseded
by the work Asahi Linux is doing within Linux mainline at some point.
More in the Hacker News discussion of the latest post from the Asahi
Linux folks about their progress bringing up Linux on M1 devices.

https://news.ycombinator.com/item?id=26423935
https://news.ycombinator.com/item?id=26421963
https://asahilinux.org/2021/03/progress-report-january-february-2021/
https://lwn.net/Articles/849108/

> So how do we proceed once a fully free kernel is available?

Once the Linux kernel changes are upstreamed, any needed config
options can be enabled in the Debian Linux kernel build. The same
process will be needed for u-boot, TianoCore or whichever bootloader
ends up being used. Then flash-kernel support can get added. Then the
d-i bits for M1 concatenated images can get added. There might need to
be some glue running on macOS in order to get d-i booted though given
the following item...

> apart from the non-free firmware required for some of the hardware.

As I understand it from the Asahi Linux post, a copy of the signed
iBoot2 blob appears to be needed on the boot partition in order to run
any alternatives to macOS.

--
bye,
pabs

https://wiki.debian.org/PaulWise

[Pip Cet]

On Sat, Mar 13, 2021 at 9:30 AM Paul Wise <pa...@debian.org> wrote:
> On Sat, Mar 13, 2021 at 8:57 AM Pip Cet wrote:
> > I'm using the kernel image provided at
> > https://downloads.corellium.info/linuxnvme.macho.
>
> I read that Corellium haven't participated in upstreaming Linux
> support for the Apple M1 devices,

Thanks for responding!

I think it's too early to conclude they're not interested in
upstreaming changes.

> so this is likely to be superseded
> by the work Asahi Linux is doing within Linux mainline at some point.

I'd like to disagree with the 'likely". It's possible, sure, but it's
also possible someone (possibly the Asahi Linux project) will polish
the Corellium work (which, well, works) and submit that to upstream.
It's Free Software, after all.

I don't think we have to, or want to, pick a side here.

> > So how do we proceed once a fully free kernel is available?
> Once the Linux kernel changes are upstreamed, any needed config
> options can be enabled in the Debian Linux kernel build. The same

You're describing this as what sounds like a lengthy process, and it
will be until everything is included in its "final" form, but I'd just
like to remark that I consider this architecture potentially important
and that taking a "there's no rush" attitude towards it might be the
wrong approach...

IOW, wouldn't it be nice to have something working soon? In
particular, independently of the question of who upstreams whose work?

> process will be needed for u-boot, TianoCore or whichever bootloader
> ends up being used. Then flash-kernel support can get added. Then the
> d-i bits for M1 concatenated images can get added. There might need to
> be some glue running on macOS in order to get d-i booted though given
> the following item...

> > apart from the non-free firmware required for some of the hardware.
>
> As I understand it from the Asahi Linux post, a copy of the signed
> iBoot2 blob appears to be needed on the boot partition in order to run
> any alternatives to macOS.

AIUI, the boot process does not involve macOS; installing a
kernel/bootloader image is currently only possible from the recovery
OS included with macOS, but that's not that unusual. It does mean an
inconvenient extra step installing a bootloader for users, which I
believe is precisely what Apple intended...

I'm admittedly unfamiliar with how Debian does these things. Any hints
or pointers to relevant documentation would be appreciated.

Pip

[Paul Wise]

On Sat, 2021-03-13 at 10:05 +0000, Pip Cet wrote:

> I think it's too early to conclude they're not interested in
> upstreaming changes.

Perhaps, but normally when companies are interested in upstreaming
changes, they engage with the Linux kernel community early and often;
posting an RFC patch sketch before it works, posting final patch series
when it works, posting new patch series in response to feedback etc.

> I'd like to disagree with the 'likely". It's possible, sure, but it's
> also possible someone (possibly the Asahi Linux project) will polish
> the Corellium work (which, well, works) and submit that to upstream.

The HN thread made it clear that Asahi Linux folks are not using a fair
bit of Corellium's work and that Corellium's work is often suboptimal
in terms of how the Linux kernel code normally works; reusing/tweaking
drivers vs duplicating them etc.

> I don't think we have to, or want to, pick a side here.

Ultimately what ends up in mainline is the side that will be picked.

> You're describing this as what sounds like a lengthy process, and it
> will be until everything is included in its "final" form, but I'd
> just like to remark that I consider this architecture potentially
> important and that taking a "there's no rush" attitude towards it
> might be the wrong approach...

For better or worse the Debian Linux kernel team does generally not
accept patchsets that aren't upstream or aren't near acceptance, the
same applies to all hardware including Apple M1 devices.

https://kernel-team.pages.debian.net/kernel-handbook/ch-source.html#s-acceptance

You can of course start working on supporting Apple M1 devices using
not yet upstreamed code, which is definitely a good idea, but of course
the final support will be solely based on upstreamed code.

> IOW, wouldn't it be nice to have something working soon?

I expect some folks who own or are able to own Apple M1 hardware would
like to have support for Linux, but I guess most people using Linux on
Apple M1 hardware will be doing so in virtual machines or Docker.
Personally I am stuck on decade old x86 hardware for now.

> independently of the question of who upstreams whose work?

Agreed that who does the work is not relevant.

> AIUI, the boot process does not involve macOS; installing a
> kernel/bootloader image is currently only possible from the recovery
> OS included with macOS, but that's not that unusual. It does mean an
> inconvenient extra step installing a bootloader for users, which I
> believe is precisely what Apple intended...

That is correct, but it does involve a proprietary iBoot2 Apple signed
binary blob, which according to Asahi folks has to be on the same
partition as the operating system kernel or the next part of the boot
chain, which for Asahi Linux will be:

SecureROM → iBoot1 (on NOR flash) → iBoot2 (on the OS partition) → m1n1 → U-Boot → GRUB → Linux
^ ^ ^ ^ ^ ^ ^
immutable \ proprietary & signed / \ open source and unsigned /

Presumably the iBoot2 blob isn't released under a license that allows
redistribution, so folks wanting to install Debian on Apple M1 devices
will need to grab a copy of that from their macOS partition, which
means that Debian cannot ship a d-i image that will just work, we have
to get the user to run something on macOS to grab iBoot2.

The Asahi Linux post I pointed at has relatively detailed descriptions
of how the Apple M1 devices boot and the requirements of each stage.

https://asahilinux.org/2021/03/progress-report-january-february-2021/
https://github.com/AsahiLinux/docs/wiki/Developer-Quickstart
https://github.com/AsahiLinux/docs/wiki/SW%3ABoot

The Raspberry Pi devices are in a similar situation, but at least there
is a chance the libre firmware reimplementation will be usable
eventually and for now the bootrom either doesn't require signatures or
parts of the boot chain are buggy enough that the boot restrictions are
easily bypassed.

https://github.com/librerpi/rpi-open-firmware
https://github.com/librerpi/rpi-open-firmware/blob/master/docs/cracking-rpi4-hmac.txt

[Wookey]

On 2021-03-13 10:05 +0000, Pip Cet wrote:
> On Sat, Mar 13, 2021 at 9:30 AM Paul Wise <pa...@debian.org> wrote:
> > On Sat, Mar 13, 2021 at 8:57 AM Pip Cet wrote:
> > > I'm using the kernel image provided at
> > > https://downloads.corellium.info/linuxnvme.macho.
> >

> > so this is likely to be superseded
> > by the work Asahi Linux is doing within Linux mainline at some point.

I hadn't realised there were two projects working on this. That is
good. But I agree with Paul that Corellium are goig to remain
irrelevant froma debian POV if they don't upstream their code (someone
else is welcome to try and do it for them, but if they don't engage
everyone will choose the Asahi stuff - I would too).

> AIUI, the boot process does not involve macOS; installing a
> kernel/bootloader image is currently only possible from the recovery
> OS included with macOS, but that's not that unusual. It does mean an
> inconvenient extra step installing a bootloader for users, which I
> believe is precisely what Apple intended...

I've not looked into this, and clearly it's progressed far enough that
I should. I have one of these machine and am very keen to see debian
on it so I can use the damn thing without having to learn macos.

I'm happy to do some work on making debian work on this platform,
although tuits are always in short supply...

> I'm admittedly unfamiliar with how Debian does these things. Any hints
> or pointers to relevant documentation would be appreciated.

What we want is to be provided with the UEFI environment then we don't
have to do anything special for the platform and debian will boot and
'just work' (insofar there is kernel support). The UEFI-a-like
environment that u-boot provides more-or-less does the job (there was
some issue with persistence of writeable variables IIRC) so if that's
what's provided then we are fairly happy too (and that is what the
Asahi linux people currently propose).

We do have the facility to do special things for a platform (via
flash-kernel) but it's much nicer not to have to: booting from a
standard platform interface is a good thing that separates firmware
development and distro development.

Wookey
--
Principal hats: Linaro, Debian, Wookware, ARM
http://wookware.org/

[Paul Wise]

On Sat, 2021-03-13 at 11:49 +0000, Wookey wrote:

> I've not looked into this, and clearly it's progressed far enough
> that I should.

Definitely, it is worth reading the Asahi post and their wiki.

> What we want is to be provided with the UEFI environment

Debian will have to ship our own UEFI environment if we want that,
since the Apple signed iBoot2 blobs only passes Apple Device Tree
(incompatible with Linux Device Tree) to the next boot layer.

> The UEFI-a-like environment that u-boot provides

The other libre implementation of UEFI is TianoCore/EDKII, which Debian
has a package of for virtual machines, but we do not have a package of
their hardware platform support codebase, which means everyone who uses
EDKII are stuck with vendor binaries of it.

[Pip Cet]

On Sat, Mar 13, 2021 at 11:49 AM Wookey <woo...@wookware.org> wrote:
> On 2021-03-13 10:05 +0000, Pip Cet wrote:
> > On Sat, Mar 13, 2021 at 9:30 AM Paul Wise <pa...@debian.org> wrote:
> > > so this is likely to be superseded
> > > by the work Asahi Linux is doing within Linux mainline at some point.
>
> I hadn't realised there were two projects working on this. That is
> good. But I agree with Paul that Corellium are goig to remain
> irrelevant froma debian POV if they don't upstream their code (someone

I agree with that, too, but it's a big if. I think it's a bad idea to
assume the worst.

> else is welcome to try and do it for them, but if they don't engage
> everyone will choose the Asahi stuff - I would too).

Well, it's not clear to me whether Asahi is supposed to be a Linux
distribution, in which case "choosing the Asahi stuff" means not
choosing Debian.

> > AIUI, the boot process does not involve macOS; installing a
> > kernel/bootloader image is currently only possible from the recovery
> > OS included with macOS, but that's not that unusual. It does mean an
> > inconvenient extra step installing a bootloader for users, which I
> > believe is precisely what Apple intended...
>
> I've not looked into this, and clearly it's progressed far enough that
> I should. I have one of these machine and am very keen to see debian
> on it so I can use the damn thing without having to learn macos.

I agree, you should. I think I've got a reasonable good understanding
of what's going on with the boot process, by now...

> I'm happy to do some work on making debian work on this platform,
> although tuits are always in short supply...

If there's anything I can do to help, I'd love to!

> > I'm admittedly unfamiliar with how Debian does these things. Any hints
> > or pointers to relevant documentation would be appreciated.
>
> What we want is to be provided with the UEFI environment then we don't
> have to do anything special for the platform and debian will boot and
> 'just work' (insofar there is kernel support). The UEFI-a-like
> environment that u-boot provides more-or-less does the job (there was
> some issue with persistence of writeable variables IIRC) so if that's
> what's provided then we are fairly happy too (and that is what the
> Asahi linux people currently propose).

That would be wonderful. I don't think it's going to happen unless
there's a change of heart at Apple.

> We do have the facility to do special things for a platform (via
> flash-kernel) but it's much nicer not to have to: booting from a
> standard platform interface is a good thing that separates firmware
> development and distro development.

I don't see any way to avoid telling people how to boot into 1TR, run
bputil (which disables the "Trusted Computing" mode), run kmutil
(which installs a kernel other than macOS), and then boot an
environment which ultimately loads the right image of Linux. I do
think we can make the rest of the process easier than it currently is.

So the process is:

- grab a USB image
- flash it to a USB drive
- boot your Mac and keep holding the power button for twenty seconds
or so until "boot options" are being loaded
- click on "options"
- click (this doesn't work with the keyboard) the menu bar and select
Utilities / Terminal
- run a shell script off the USB drive
- enter passwords and username at the prompt, repeatedly

The other problem is that I do not think that we can repartition Apple
file systems yet, which adds a "play around with the Apple
repartitioning software until you finally have a linuxable partition"
step (I did that, and it made me hate macOS)...

Pip

[Steve McIntyre]

On Sat, Mar 13, 2021 at 12:12:34PM +0000, Pip Cet wrote:
> On Sat, Mar 13, 2021 at 11:49 AM Wookey <woo...@wookware.org> wrote:
>> On 2021-03-13 10:05 +0000, Pip Cet wrote:
>> > On Sat, Mar 13, 2021 at 9:30 AM Paul Wise <pa...@debian.org> wrote:
>> > > so this is likely to be superseded
>> > > by the work Asahi Linux is doing within Linux mainline at some point.
>>
>> I hadn't realised there were two projects working on this. That is
>> good. But I agree with Paul that Corellium are goig to remain
>> irrelevant froma debian POV if they don't upstream their code (someone
>
>I agree with that, too, but it's a big if. I think it's a bad idea to
>assume the worst.
>
>> else is welcome to try and do it for them, but if they don't engage
>> everyone will choose the Asahi stuff - I would too).
>
>Well, it's not clear to me whether Asahi is supposed to be a Linux
>distribution, in which case "choosing the Asahi stuff" means not
>choosing Debian.

That's not really an issue. If they've done it right and upstreamed
it, then everybody wins. We all get to share what they've achieved,
just like they get to share what other people have. This is the key
benefit of Free Software, for me.

...

>I don't see any way to avoid telling people how to boot into 1TR, run
>bputil (which disables the "Trusted Computing" mode), run kmutil
>(which installs a kernel other than macOS), and then boot an
>environment which ultimately loads the right image of Linux. I do
>think we can make the rest of the process easier than it currently is.
>
>So the process is:
>
>- grab a USB image
>- flash it to a USB drive
>- boot your Mac and keep holding the power button for twenty seconds
>or so until "boot options" are being loaded
>- click on "options"
>- click (this doesn't work with the keyboard) the menu bar and select
>Utilities / Terminal
>- run a shell script off the USB drive
>- enter passwords and username at the prompt, repeatedly
>
>The other problem is that I do not think that we can repartition Apple
>file systems yet, which adds a "play around with the Apple
>repartitioning software until you finally have a linuxable partition"
>step (I did that, and it made me hate macOS)...

Ugh. That process looks hateful :-(

--
Steve McIntyre, Cambridge, UK. st...@einval.com
There's no sensation to compare with this
Suspended animation, A state of bliss

[Pip Cet]

On Sat, Mar 13, 2021 at 2:14 PM Steve McIntyre <st...@einval.com> wrote:
> On Sat, Mar 13, 2021 at 12:12:34PM +0000, Pip Cet wrote:
> >
> >> else is welcome to try and do it for them, but if they don't engage
> >> everyone will choose the Asahi stuff - I would too).
> >
> >Well, it's not clear to me whether Asahi is supposed to be a Linux
> >distribution, in which case "choosing the Asahi stuff" means not
> >choosing Debian.
>
> That's not really an issue. If they've done it right and upstreamed
> it, then everybody wins. We all get to share what they've achieved

> just like they get to share what other people have. This is the key
> benefit of Free Software, for me.

Absolutely! I think the same applies to the Corellium changes, though,
and we should not discount the possibility that they will do the right
thing and decide to go through the (expensive) process of getting
things upstreamed.

What I've seen of the code doesn't look so bad. Some of it needs to be
polished, and, yes, the keyboard driver should be standalone rather
than turning an existing file into a bowl of #ifdef spaghetti, but
compared to figuring out how undocumented hardware works, these
changes are minor.
(My main problem, as mentioned, is that the source on GitHub does not
match the binary they distribute; the binary works, the source code
recompiled even with the precise same utilities doesn't.)

> >So the process is:
> >
> >- grab a USB image
> >- flash it to a USB drive
> >- boot your Mac and keep holding the power button for twenty seconds
> >or so until "boot options" are being loaded
> >- click on "options"
> >- click (this doesn't work with the keyboard) the menu bar and select
> >Utilities / Terminal
> >- run a shell script off the USB drive
> >- enter passwords and username at the prompt, repeatedly
> >
> >The other problem is that I do not think that we can repartition Apple
> >file systems yet, which adds a "play around with the Apple
> >repartitioning software until you finally have a linuxable partition"
> >step (I did that, and it made me hate macOS)...
>
> Ugh. That process looks hateful :-(

Admittedly, we could skip the first two steps and change the sixth to
be "curl SOMEURLIDONTTRUST | bash". I don't think we should do,
though...

Pip

[Paul Wise]

On Sat, Mar 13, 2021 at 4:09 PM Pip Cet wrote:

> (My main problem, as mentioned, is that the source on GitHub does not
> match the binary they distribute; the binary works, the source code
> recompiled even with the precise same utilities doesn't.)

Hmm, that sounds like a GPL violation, not a good sign.