On Cap'nProto for generating Coyotos Kernel interfaces
41 views
Skip to first unread message
William ML Leslie
May 31, 2026, 1:32:41 AM (4 days ago) May 31
to cap-talk
I mentioned during the week that I have a work-in-progress project to generate the various Coyotos IDL stubs from Cap'nProto schemas, and that declaring exceptional return values was clunky. I have been playing with this and think I've settled on something I like enough to share. Note that the generated files don't /work/ yet, but if you want to generate some yourself, you can run this from the project root:
python3 -m capidl idl/coyotos/AddressSpace.capnp -I idl/
I've vendored the python dependencies (you'll just need the capnp compiler) because they are currently uninstallable.
The aim is that if your Cap'nProto methods meet some (hopefully reasonable) criteria, they can be turned into Coyotos kernel invocations in a way that is largely compatible with the methods generated with CapIDL. Most significantly, parameters aren't packed but live in registers easy to enumerate from the kernel and from assembler. In particular, the slightly funky choice of which parameters live in registers on x86 is influenced by the calls that the ProtoSpace needs to make: the opcode used is consistent, so those can live in a page mapped read-only, but the Program Counter for setSpaceAndPC needs to be drawn from elsewhere, and there is no writable memory mapped into a protospace, so it must come from a register.
The interface provided by Coyotos is semantically similar to rust's Result. There's a single bit in the status word that indicates whether the result is exceptional. We can treat this like a sum type - a union, in capnp parlance - to have different schemas for normal return values and exceptional results.
CapIDL goes beyond this. Exceptions in CapIDL return the exception's symbolic value, as a 64-bit unsigned integer, and no further data. This makes sense when working from C, as dealing with sum types as return values in C is quite unnatural. CapIDL generates functions that accept a reference to a variable for each output field and populate them on successful return. It's ugly, but unsurprising for C. I don't know if we need to preserve this /style/ of output function for anything but the existing APIs, but it is probably worth supporting.
Notice how in the following signature from CapIDL, l2slot and perms are returned by reference.
bool IDL_ENV_coyotos_AddressSpace_extendedFetch(
IDL_Environment *_env,
caploc_t _invCap,
coyotos_Cap_coyaddr_t offset,
coyotos_Memory_l2value_t l2arg,
coyotos_Memory_l2value_t *l2slot,
coyotos_Memory_restrictions *perms,
caploc_t _retVal
);
Here's how it currently looks from the new version. Note that I put the output cap (_retVal above) earlier myself in the IDL file, whoops.
static inline bool
coyotos_AddressSpace_extendedFetch(
IDL_Env_t *_env,
caploc_t _invCap,
uint64_t offset,
uint32_t l2arg,
caploc_t cap,
uint32_t *out_l2slot,
uint32_t *out_perms,
uint64_t *out_err,
)
What I'd love is, for languages that do have good support for sum types, for the method descriptions we support to feel natural. Here's the above method as described in the capnp schema.
struct ExtendedFetchResult {
union {
ok :group {
cap @0 :Cap;
l2slot @1 :Memory.L2Value;
perms @2 :Restrictions;
}
err @3 :Exception.T $exceptions([AddressSpace.opaqueSpace, Ex.requestError]);
}
}
extendedFetch @5 (offset :Cap.CoyAddrT, l2arg :Memory.L2Value)
-> ExtendedFetchResult;
Firstly, notice that the struct is declared explicitly, outside of the method declaration itself. As far as I can tell, there isn't a way to avoid this if you want the result struct to include a union like this. The first leg of the union is the success case. The remaining legs can be distinguished by the $exceptions() they are described by. Note that with this style, you can have more than one schema for your exceptional values.
using import "annotations.capnp".Exception;
using import "annotations.capnp".exceptions;
struct Result {
union {
ok :group {...}
err1 :group {
exception :Exception.T $exceptions([X.keyError]);
key :Data;
}
err2 :group {
exception :Exception.T $exceptions([X.indexError]);
index :UInt64;
}
}
}
Note that each leg of the union first declares a field with the $exceptions() annotation. The idea is that the bit in the coyotos status word that indicates exceptional results is used to select between the first leg and the remainder, and the value of the exception field is used to select between the remaining legs. Code generators need to handle the fact that a server could return a different exception (allowable exceptions are not enforced across a process boundary).
The downside to this approach is, of course, that it's nonstandard. Whether it will turn out to be reasonable for this specific case is unsure to me. I'll leave a note on the time machine to mention to Kenton that it'd be really helpful to support 64-bit descriminators and sum-type method returns around the time of the Cap'nProto release.
Oh! You want to know how to define exceptions. These are constants. I haven't implemented the algorithm to calculate the ordinal from the exception name yet, so you can just copy them from the generated IDL headers. The ordinal is the number actually sent via the kernel, and the name is used for code generation.
const opaqueSpace :Exception = (name = "opaqueSpace", ordinal = 1);
python3 -m capidl idl/coyotos/AddressSpace.capnp -I idl/
I've vendored the python dependencies (you'll just need the capnp compiler) because they are currently uninstallable.
The aim is that if your Cap'nProto methods meet some (hopefully reasonable) criteria, they can be turned into Coyotos kernel invocations in a way that is largely compatible with the methods generated with CapIDL. Most significantly, parameters aren't packed but live in registers easy to enumerate from the kernel and from assembler. In particular, the slightly funky choice of which parameters live in registers on x86 is influenced by the calls that the ProtoSpace needs to make: the opcode used is consistent, so those can live in a page mapped read-only, but the Program Counter for setSpaceAndPC needs to be drawn from elsewhere, and there is no writable memory mapped into a protospace, so it must come from a register.
The interface provided by Coyotos is semantically similar to rust's Result. There's a single bit in the status word that indicates whether the result is exceptional. We can treat this like a sum type - a union, in capnp parlance - to have different schemas for normal return values and exceptional results.
CapIDL goes beyond this. Exceptions in CapIDL return the exception's symbolic value, as a 64-bit unsigned integer, and no further data. This makes sense when working from C, as dealing with sum types as return values in C is quite unnatural. CapIDL generates functions that accept a reference to a variable for each output field and populate them on successful return. It's ugly, but unsurprising for C. I don't know if we need to preserve this /style/ of output function for anything but the existing APIs, but it is probably worth supporting.
Notice how in the following signature from CapIDL, l2slot and perms are returned by reference.
bool IDL_ENV_coyotos_AddressSpace_extendedFetch(
IDL_Environment *_env,
caploc_t _invCap,
coyotos_Cap_coyaddr_t offset,
coyotos_Memory_l2value_t l2arg,
coyotos_Memory_l2value_t *l2slot,
coyotos_Memory_restrictions *perms,
caploc_t _retVal
);
Here's how it currently looks from the new version. Note that I put the output cap (_retVal above) earlier myself in the IDL file, whoops.
static inline bool
coyotos_AddressSpace_extendedFetch(
IDL_Env_t *_env,
caploc_t _invCap,
uint64_t offset,
uint32_t l2arg,
caploc_t cap,
uint32_t *out_l2slot,
uint32_t *out_perms,
uint64_t *out_err,
)
What I'd love is, for languages that do have good support for sum types, for the method descriptions we support to feel natural. Here's the above method as described in the capnp schema.
struct ExtendedFetchResult {
union {
ok :group {
cap @0 :Cap;
l2slot @1 :Memory.L2Value;
perms @2 :Restrictions;
}
err @3 :Exception.T $exceptions([AddressSpace.opaqueSpace, Ex.requestError]);
}
}
extendedFetch @5 (offset :Cap.CoyAddrT, l2arg :Memory.L2Value)
-> ExtendedFetchResult;
Firstly, notice that the struct is declared explicitly, outside of the method declaration itself. As far as I can tell, there isn't a way to avoid this if you want the result struct to include a union like this. The first leg of the union is the success case. The remaining legs can be distinguished by the $exceptions() they are described by. Note that with this style, you can have more than one schema for your exceptional values.
using import "annotations.capnp".Exception;
using import "annotations.capnp".exceptions;
struct Result {
union {
ok :group {...}
err1 :group {
exception :Exception.T $exceptions([X.keyError]);
key :Data;
}
err2 :group {
exception :Exception.T $exceptions([X.indexError]);
index :UInt64;
}
}
}
Note that each leg of the union first declares a field with the $exceptions() annotation. The idea is that the bit in the coyotos status word that indicates exceptional results is used to select between the first leg and the remainder, and the value of the exception field is used to select between the remaining legs. Code generators need to handle the fact that a server could return a different exception (allowable exceptions are not enforced across a process boundary).
The downside to this approach is, of course, that it's nonstandard. Whether it will turn out to be reasonable for this specific case is unsure to me. I'll leave a note on the time machine to mention to Kenton that it'd be really helpful to support 64-bit descriminators and sum-type method returns around the time of the Cap'nProto release.
Oh! You want to know how to define exceptions. These are constants. I haven't implemented the algorithm to calculate the ordinal from the exception name yet, so you can just copy them from the generated IDL headers. The ordinal is the number actually sent via the kernel, and the name is used for code generation.
const opaqueSpace :Exception = (name = "opaqueSpace", ordinal = 1);
--
William ML Leslie
Taking the time to get it right, I guess?
Matt Rice
May 31, 2026, 4:36:21 AM (4 days ago) May 31
to cap-...@googlegroups.com
I guess that my thoughts primarily are about these constant values,
one of the nice things about rust enums is it will catch
reuse of discriminant
const oops :Exception = (name = "oops", ordinal = 1);
const ooops :Exception = (name = "ooops", ordinal = 1);
// error[E0081]: discriminant value `0` assigned more than once
// |
// 2 | enum Foo {
// | ^^^^^^^^
// 3 | Foo = 0,
// | - `0` assigned here
// 4 | Bar = 0,
// | - `0` assigned here
//
#[repr(u64)]
enum Foo {
Foo = 0,
Bar = 0,
}
As far as I know you can't include auxiliary constants in enums in rust.
I would tend to instead just throw those string values into a function/method
fn exception_name(self) -> &'static str {
match self {
Self::Foo => "foo",
Self::Bar => "bar",
}
}
There is one awkward bit of difference between rust style enums and c
style enums, which is that rust enums are closed
over the values, (so not all u64 values in the #[repr()] are valid bit
patterns for the type, only those given by the enums variants.
While for c++ and c style enums it is not undefined behavior to cast
any numeric value of the right type into an enum represented by it
whether or not the enum specifies a variant with that numeric code.
It seems like there is a distinction between c/c++ enums and cap'n
proto enums in that not all cap'n proto enums are documented as being
not necessarily numeric here
https://capnproto.org/language.html#enums. Alas it seems like when
serializing to c++ they emit c++ enums which are always numeric
https://capnproto.org/cxx.html#enums and unfortunately have the open
semantics described above (I find these two statements difficult to
reconcile).
It looks to me like the cap'n proto rust codegen for cap'n proto enums
correctly translates them into an open c++ like type, which can set
and receive arbitrary numeric values
rather than a direct translation into a rust enum with a numeric repr.
Doing otherwise it would be hard to use cap'n proto in a mixed
c++/rust fashion.
But it would be nice if cap'n proto actually had a closed enum for
these kinds of error codes.
Rusts numeric enums also can be extended with proc-macros to add
additional type checking in some crazy ways,
since you can access the exact constant value of a variant at compile
time, like you can ensure that all numeric values of an enum are
non-zero, or powers of two.
A check that might be impossible (uncertain) to add would be ensuring
that two enums have numeric reprs that are disjoint from one another.
That sort of check would rely on having the variants of both enums, or
some way of erroring late after both enums have been processed,
uncertain whether that is possible.
Anyhow just trying to think that adding disjointedness checking on
closed numeric enums would mirror much of the benefits of the whole c
style open enum numeric representation,
but with additional checking and principles... because then you could
tell them apart by their bit repr, certain that they do not overlap.
*shrug*
> https://gitlab.com/william-ml-leslie/wl-capidl-capnp/-/tree/main
>
> --
> William ML Leslie
> Taking the time to get it right, I guess?
>
> You received this message because you are subscribed to the Google Groups "cap-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/cap-talk/CAHgd1hEssPE5W7N6pioKUiXD85Tyd0tyRykpwz6tcLqAcCoonQ%40mail.gmail.com.
William ML Leslie
May 31, 2026, 7:02:32 AM (4 days ago) May 31
to cap-...@googlegroups.com
On Sun, 31 May 2026 at 18:36, Matt Rice <rat...@gmail.com> wrote:
Anyhow just trying to think that adding disjointedness checking on
closed numeric enums would mirror much of the benefits of the whole c
style open enum numeric representation,
but with additional checking and principles... because then you could
tell them apart by their bit repr, certain that they do not overlap.
*shrug*
It's only one line for me to add this, and it was always the plan.
CapnProto unions are disjoint, but I'm working around the native mechanism here so it's all on me. I'm just reflecting that CapIDL represented these as u64.
William ML Leslie
Oh dear, I have no idea what I'm doing.
Matt Rice
May 31, 2026, 4:27:39 PM (4 days ago) May 31
to cap-...@googlegroups.com
semantics of the cap'n proto c++ serialization,
mostly because of it differing from what the cap'n'proto official
docs/generators. I suppose as long as nobody tries to
combine the kernel interface generator and the cap'n proto c++ serializer?
I guess I should also say I'm kind of skeptical of external build time
dependencies including interpreters like python.
I can't imagine the python interpreter running natively on a
capability system, allowing the kernel interface generator to run on
the target system.
But I don't think there is anything inherent to cap'n proto that would
make a native port difficult though.
I guess the other concern with forking is probably not large enough
change to worry about keeping up with upstream cap'n proto (I don't
know as far as breaking changes how difficult it would be to keep up
with, I doubt it is a concern with this small of change, and cap'n
proto itself seems a pretty stable target)
William ML Leslie
May 31, 2026, 10:18:44 PM (3 days ago) May 31
to cap-...@googlegroups.com
There are a few details about my process that I should probably let you (and the rest of the capabilities mafia) in on, and I think this email is the motivation for me to talk about that publicly. I'll do that in my next email, here I'll just address points directly.
On Mon, 1 Jun 2026 at 06:27, Matt Rice <rat...@gmail.com> wrote:
On Sun, May 31, 2026 at 4:02 AM William ML Leslie
<william.l...@gmail.com> wrote:
>
> On Sun, 31 May 2026 at 18:36, Matt Rice <rat...@gmail.com> wrote:
>>
>> Anyhow just trying to think that adding disjointedness checking on
>> closed numeric enums would mirror much of the benefits of the whole c
>> style open enum numeric representation,
>> but with additional checking and principles... because then you could
>> tell them apart by their bit repr, certain that they do not overlap.
>> *shrug*
>
>
> It's only one line for me to add this, and it was always the plan.
>
> CapnProto unions are disjoint, but I'm working around the native mechanism here so it's all on me. I'm just reflecting that CapIDL represented these as u64.
>
Makes sense, but I suppose I am a little bit queasy about changing the
semantics of the cap'n proto c++ serialization,
mostly because of it differing from what the cap'n'proto official
docs/generators. I suppose as long as nobody tries to
combine the kernel interface generator and the cap'n proto c++ serializer?
I'm not changing the encoding of unions everywhere, I'm doing a different encoding of method return values that happen to have unions.
I would have liked to get this closer to rpc.capnp (which I guess is what you mean by "the official docs/generators") because I could then use any capnproto implementation in any language with rpc support to implement components. But since the aim is to make each method call literally a system call on an already well-defined interface, the specific encoding used for parameters in rpc.capnp had to change.
An adapter between an rpc.capnp message and the coytos kernel interface would be possible, too, but it seems a significant burden for small-space programs.
Note that the official documentation explicitly calls out that variation on this theme might make sense, almost as if Kenton had my exact use case in mind: https://capnproto.org/encoding.html#capabilities-interfaces
I guess I should also say I'm kind of skeptical of external build time
dependencies including interpreters like python.
According to my git history, the port of the Coyotos cross-environment from GCC 4.5 to GCC 10 took me from October 2020 to February 2022. If there's any dependency we should be skeptical of at this point, it's the C toolchain. Now, to be fair, I didn't have a lot of GNU toolchain knowledge when I started out (if you want to learn what I didn't know, the Linux From Scratch tutorial covers pretty much everything I was missing), but I'd just rather something that I have the depth of knowledge on that I could port it even if the entire internet went down, and given my history, that is Python.
This doesn't preclude anyone rewriting it in some other language at some point. Rewrites are trivial, getting the semantics nailed down is the tricky part.
Ideally, the only build dependency you'd need today is guix or nix and the software archive, which is totally achievable as far as I can tell.
I can't imagine the python interpreter running natively on a
capability system, allowing the kernel interface generator to run on
the target system.
Why? The first languages I'll be porting are the ones I want to use interactively. So while Monte, Idris, Pony and a scheme might beat out Python, I don't think it's far behind. I feel a lot more comfortable porting Python than I do GHC or GraalVM, valuable as they might be. I guess I'd better port a WASM runtime, too.
But I don't think there is anything inherent to cap'n proto that would
make a native port difficult though.
I guess the other concern with forking is probably not large enough
change to worry about keeping up with upstream cap'n proto (I don't
know as far as breaking changes how difficult it would be to keep up
with, I doubt it is a concern with this small of change, and cap'n
proto itself seems a pretty stable target)
So, this project does work with upstream capnpy, which is the capnproto implementation written by Antonio Cuni to have better performance than the wrapped C++ version. I once tried to add RPC support to it, but apparently never managed to get those changes integrated. So, I picked it because it's something I already know well. I don't imagine it has a lot of users, considering it was not installable when I tried. I vendored this project and its dependencies for your convenience, and deleted the stuff I wasn't using. It's really just used to interpret the output of the capnp compiler.
--
William ML Leslie
Omelette Factory Safety Advisory: It has been 0 days without a broken egg.
Matt Rice
Jun 1, 2026, 12:34:18 AM (3 days ago) Jun 1
to cap-...@googlegroups.com
On Sun, May 31, 2026 at 7:18 PM William ML Leslie
serialization' page, I was talking about earlier which had open enum
semantics.
> An adapter between an rpc.capnp message and the coytos kernel interface would be possible, too, but it seems a significant burden for small-space programs.
>
> Note that the official documentation explicitly calls out that variation on this theme might make sense, almost as if Kenton had my exact use case in mind: https://capnproto.org/encoding.html#capabilities-interfaces
>
>> I guess I should also say I'm kind of skeptical of external build time
>> dependencies including interpreters like python.
>
>
> According to my git history, the port of the Coyotos cross-environment from GCC 4.5 to GCC 10 took me from October 2020 to February 2022. If there's any dependency we should be skeptical of at this point, it's the C toolchain. Now, to be fair, I didn't have a lot of GNU toolchain knowledge when I started out (if you want to learn what I didn't know, the Linux From Scratch tutorial covers pretty much everything I was missing), but I'd just rather something that I have the depth of knowledge on that I could port it even if the entire internet went down, and given my history, that is Python.
>
> https://gitlab.com/william-ml-leslie/ccs-xenv/-/commits/master?ref_type=HEADS
> https://gitlab.com/william-ml-leslie/gnu-guix/-/commits/capos?ref_type=heads
>
> This doesn't preclude anyone rewriting it in some other language at some point. Rewrites are trivial, getting the semantics nailed down is the tricky part.
>
> Ideally, the only build dependency you'd need today is guix or nix and the software archive, which is totally achievable as far as I can tell.
>
Indeed the portability of c compilers to capability systems (without
posix emulation) is the major problem, I'm *well* aware of a good
number of pitfalls in *all* the gnu toolchain tools including make,
gas, gcc, and ld which make porting them to capability systems a
nightmare. The llvm tools are a bit better than the gnu ones, since at
least they were designed to be used as a library, and generally has
mechanisms in place where you can pass a filename as a string, along
with a file descriptor. The ability to compile c to webassembly is
another boon to freestanding bootstrap environments.
>>
>> I can't imagine the python interpreter running natively on a
>> capability system, allowing the kernel interface generator to run on
>> the target system.
>
>
> Why? The first languages I'll be porting are the ones I want to use interactively. So while Monte, Idris, Pony and a scheme might beat out Python, I don't think it's far behind. I feel a lot more comfortable porting Python than I do GHC or GraalVM, valuable as they might be. I guess I'd better port a WASM runtime, too.
>
As to why, there are a few reasons, i'll try and make a list
#1 I can't imagine python being successfully ported without posix emulation
Python's entire module system to me feels like it is reliant on
ambient authority and finding source files by filename "__init__.py" I
just can't imagine a port of it being successful without posix
emulation.
It just feels like it's accrued to much posix nonsense, since it's
predecessor/inspiration abc actually ran on the amoeba capability
system.
#2 If we're developing and cross-building on posix systems, tools we
write to make our lives easier as developers will more often target
posix systems.
My general feeling has been that systems like eros, coyotos, capros,
sel4 have not benefited from the tools that developers write to make
their own lives easier.
While the end product is a capability system, the majority of the
interaction is still with legacy tools.
#3 It complicates/bloats the bootstrap environment
Python is kind of a batteries included language, it was built under
the assumption it was running on an already bootstrapped system.
That makes a minimal bootstrap environment way bigger than it needs to be.
#4 Reliance on the host system tools and their release cadence.
The spread of python versions in use is very large, some distros
release stuff that is shockingly old with 10 year release cycles,
other systems are shockingly eager to drop support for non-latest
versions. There are really only a couple of sane ways to handle it,
(such as picking one host system you like for instance guix as you say).
I view another sane way is to make the bootstrap environment as
freestanding as possible so that it can run on basically any host
system or a host system with minimal requirements.
Inverting the build process, instead of using build tools that emulate
posix when porting to capability systems. Emulating capability apis
when running on posix within the bootstrap environment.
It probably doesn't matter until you start shipping, and people have
classrooms with students, and companies with teams running your
bootstrap environment.
With them running some combination of your bootstrap code along with
the output of their own teams/students the ability to fiddle with the
bootstrap environments seems to grind to a halt.
Because they'll only want to update ubuntu LTS once every 10 years so
everything ends up stuck in the mud.
But it always seems like linux distributions are one of the two
extremes of either hard to keep up with, or glacially slow.
Freestanding environments at least have the benefit of changing at
their own pace, so the host systems release cadence matters much less.
You can just change utilities within the build tree as they improve,
my hope is that some future capability systems will try to bootstrap
using this inverted capability APi method or make it a priority.
#5 It's entirely possible I'm both wrong and an idiot and python is
super easy to port, I don't know because I've avoided it primarily
because its module seems like a nightmare of ambient authority.
>> But I don't think there is anything inherent to cap'n proto that would
>> make a native port difficult though.
>>
>> I guess the other concern with forking is probably not large enough
>> change to worry about keeping up with upstream cap'n proto (I don't
>> know as far as breaking changes how difficult it would be to keep up
>> with, I doubt it is a concern with this small of change, and cap'n
>> proto itself seems a pretty stable target)
>
>
> So, this project does work with upstream capnpy, which is the capnproto implementation written by Antonio Cuni to have better performance than the wrapped C++ version. I once tried to add RPC support to it, but apparently never managed to get those changes integrated. So, I picked it because it's something I already know well. I don't imagine it has a lot of users, considering it was not installable when I tried. I vendored this project and its dependencies for your convenience, and deleted the stuff I wasn't using. It's really just used to interpret the output of the capnp compiler.
>
makes sense, especially for prototyping to pick something you're
familiar with. Sorry I don't mean to take like a decade of frustration
out upon you specifically,
But that has pretty much always been my focus, making these systems
bootstrap themselves. So those inclined can abandon posix bootstrap
environments.
> --
> William ML Leslie
> Omelette Factory Safety Advisory: It has been 0 days without a broken egg.
>
<william.l...@gmail.com> wrote:
>
> There are a few details about my process that I should probably let you (and the rest of the capabilities mafia) in on, and I think this email is the motivation for me to talk about that publicly. I'll do that in my next email, here I'll just address points directly.
>
> On Mon, 1 Jun 2026 at 06:27, Matt Rice <rat...@gmail.com> wrote:
>>
>> On Sun, May 31, 2026 at 4:02 AM William ML Leslie
>> <william.l...@gmail.com> wrote:
>> >
>> > On Sun, 31 May 2026 at 18:36, Matt Rice <rat...@gmail.com> wrote:
>> >>
>> >> Anyhow just trying to think that adding disjointedness checking on
>> >> closed numeric enums would mirror much of the benefits of the whole c
>> >> style open enum numeric representation,
>> >> but with additional checking and principles... because then you could
>> >> tell them apart by their bit repr, certain that they do not overlap.
>> >> *shrug*
>> >
>> >
>> > It's only one line for me to add this, and it was always the plan.
>> >
>> > CapnProto unions are disjoint, but I'm working around the native mechanism here so it's all on me. I'm just reflecting that CapIDL represented these as u64.
>> >
>>
>> Makes sense, but I suppose I am a little bit queasy about changing the
>> semantics of the cap'n proto c++ serialization,
>> mostly because of it differing from what the cap'n'proto official
>> docs/generators. I suppose as long as nobody tries to
>> combine the kernel interface generator and the cap'n proto c++ serializer?
>
>
> I'm not changing the encoding of unions everywhere, I'm doing a different encoding of method return values that happen to have unions.
>
> I would have liked to get this closer to rpc.capnp (which I guess is what you mean by "the official docs/generators") because I could then use any capnproto implementation in any language with rpc support to implement components. But since the aim is to make each method call literally a system call on an already well-defined interface, the specific encoding used for parameters in rpc.capnp had to change.
>
By "the official docs/generators I meant specifically the "C++
>
> There are a few details about my process that I should probably let you (and the rest of the capabilities mafia) in on, and I think this email is the motivation for me to talk about that publicly. I'll do that in my next email, here I'll just address points directly.
>
> On Mon, 1 Jun 2026 at 06:27, Matt Rice <rat...@gmail.com> wrote:
>>
>> On Sun, May 31, 2026 at 4:02 AM William ML Leslie
>> <william.l...@gmail.com> wrote:
>> >
>> > On Sun, 31 May 2026 at 18:36, Matt Rice <rat...@gmail.com> wrote:
>> >>
>> >> Anyhow just trying to think that adding disjointedness checking on
>> >> closed numeric enums would mirror much of the benefits of the whole c
>> >> style open enum numeric representation,
>> >> but with additional checking and principles... because then you could
>> >> tell them apart by their bit repr, certain that they do not overlap.
>> >> *shrug*
>> >
>> >
>> > It's only one line for me to add this, and it was always the plan.
>> >
>> > CapnProto unions are disjoint, but I'm working around the native mechanism here so it's all on me. I'm just reflecting that CapIDL represented these as u64.
>> >
>>
>> Makes sense, but I suppose I am a little bit queasy about changing the
>> semantics of the cap'n proto c++ serialization,
>> mostly because of it differing from what the cap'n'proto official
>> docs/generators. I suppose as long as nobody tries to
>> combine the kernel interface generator and the cap'n proto c++ serializer?
>
>
> I'm not changing the encoding of unions everywhere, I'm doing a different encoding of method return values that happen to have unions.
>
> I would have liked to get this closer to rpc.capnp (which I guess is what you mean by "the official docs/generators") because I could then use any capnproto implementation in any language with rpc support to implement components. But since the aim is to make each method call literally a system call on an already well-defined interface, the specific encoding used for parameters in rpc.capnp had to change.
>
serialization' page, I was talking about earlier which had open enum
semantics.
> An adapter between an rpc.capnp message and the coytos kernel interface would be possible, too, but it seems a significant burden for small-space programs.
>
> Note that the official documentation explicitly calls out that variation on this theme might make sense, almost as if Kenton had my exact use case in mind: https://capnproto.org/encoding.html#capabilities-interfaces
>
>> I guess I should also say I'm kind of skeptical of external build time
>> dependencies including interpreters like python.
>
>
> According to my git history, the port of the Coyotos cross-environment from GCC 4.5 to GCC 10 took me from October 2020 to February 2022. If there's any dependency we should be skeptical of at this point, it's the C toolchain. Now, to be fair, I didn't have a lot of GNU toolchain knowledge when I started out (if you want to learn what I didn't know, the Linux From Scratch tutorial covers pretty much everything I was missing), but I'd just rather something that I have the depth of knowledge on that I could port it even if the entire internet went down, and given my history, that is Python.
>
> https://gitlab.com/william-ml-leslie/ccs-xenv/-/commits/master?ref_type=HEADS
> https://gitlab.com/william-ml-leslie/gnu-guix/-/commits/capos?ref_type=heads
>
> This doesn't preclude anyone rewriting it in some other language at some point. Rewrites are trivial, getting the semantics nailed down is the tricky part.
>
> Ideally, the only build dependency you'd need today is guix or nix and the software archive, which is totally achievable as far as I can tell.
>
posix emulation) is the major problem, I'm *well* aware of a good
number of pitfalls in *all* the gnu toolchain tools including make,
gas, gcc, and ld which make porting them to capability systems a
nightmare. The llvm tools are a bit better than the gnu ones, since at
least they were designed to be used as a library, and generally has
mechanisms in place where you can pass a filename as a string, along
with a file descriptor. The ability to compile c to webassembly is
another boon to freestanding bootstrap environments.
>>
>> I can't imagine the python interpreter running natively on a
>> capability system, allowing the kernel interface generator to run on
>> the target system.
>
>
> Why? The first languages I'll be porting are the ones I want to use interactively. So while Monte, Idris, Pony and a scheme might beat out Python, I don't think it's far behind. I feel a lot more comfortable porting Python than I do GHC or GraalVM, valuable as they might be. I guess I'd better port a WASM runtime, too.
>
#1 I can't imagine python being successfully ported without posix emulation
Python's entire module system to me feels like it is reliant on
ambient authority and finding source files by filename "__init__.py" I
just can't imagine a port of it being successful without posix
emulation.
It just feels like it's accrued to much posix nonsense, since it's
predecessor/inspiration abc actually ran on the amoeba capability
system.
#2 If we're developing and cross-building on posix systems, tools we
write to make our lives easier as developers will more often target
posix systems.
My general feeling has been that systems like eros, coyotos, capros,
sel4 have not benefited from the tools that developers write to make
their own lives easier.
While the end product is a capability system, the majority of the
interaction is still with legacy tools.
#3 It complicates/bloats the bootstrap environment
Python is kind of a batteries included language, it was built under
the assumption it was running on an already bootstrapped system.
That makes a minimal bootstrap environment way bigger than it needs to be.
#4 Reliance on the host system tools and their release cadence.
The spread of python versions in use is very large, some distros
release stuff that is shockingly old with 10 year release cycles,
other systems are shockingly eager to drop support for non-latest
versions. There are really only a couple of sane ways to handle it,
(such as picking one host system you like for instance guix as you say).
I view another sane way is to make the bootstrap environment as
freestanding as possible so that it can run on basically any host
system or a host system with minimal requirements.
Inverting the build process, instead of using build tools that emulate
posix when porting to capability systems. Emulating capability apis
when running on posix within the bootstrap environment.
It probably doesn't matter until you start shipping, and people have
classrooms with students, and companies with teams running your
bootstrap environment.
With them running some combination of your bootstrap code along with
the output of their own teams/students the ability to fiddle with the
bootstrap environments seems to grind to a halt.
Because they'll only want to update ubuntu LTS once every 10 years so
everything ends up stuck in the mud.
But it always seems like linux distributions are one of the two
extremes of either hard to keep up with, or glacially slow.
Freestanding environments at least have the benefit of changing at
their own pace, so the host systems release cadence matters much less.
You can just change utilities within the build tree as they improve,
my hope is that some future capability systems will try to bootstrap
using this inverted capability APi method or make it a priority.
#5 It's entirely possible I'm both wrong and an idiot and python is
super easy to port, I don't know because I've avoided it primarily
because its module seems like a nightmare of ambient authority.
>> But I don't think there is anything inherent to cap'n proto that would
>> make a native port difficult though.
>>
>> I guess the other concern with forking is probably not large enough
>> change to worry about keeping up with upstream cap'n proto (I don't
>> know as far as breaking changes how difficult it would be to keep up
>> with, I doubt it is a concern with this small of change, and cap'n
>> proto itself seems a pretty stable target)
>
>
> So, this project does work with upstream capnpy, which is the capnproto implementation written by Antonio Cuni to have better performance than the wrapped C++ version. I once tried to add RPC support to it, but apparently never managed to get those changes integrated. So, I picked it because it's something I already know well. I don't imagine it has a lot of users, considering it was not installable when I tried. I vendored this project and its dependencies for your convenience, and deleted the stuff I wasn't using. It's really just used to interpret the output of the capnp compiler.
>
familiar with. Sorry I don't mean to take like a decade of frustration
out upon you specifically,
But that has pretty much always been my focus, making these systems
bootstrap themselves. So those inclined can abandon posix bootstrap
environments.
> --
> William ML Leslie
> Omelette Factory Safety Advisory: It has been 0 days without a broken egg.
>
> --
> You received this message because you are subscribed to the Google Groups "cap-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/cap-talk/CAHgd1hFV-e2xva081nW86NZ3a_RpnivEh3k1VQqtzHVw0j2xcQ%40mail.gmail.com.
> You received this message because you are subscribed to the Google Groups "cap-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
Matt Rice
Jun 1, 2026, 1:13:09 AM (3 days ago) Jun 1
to cap-...@googlegroups.com
machine based bootstrap environment,
which is host architecture neutrality, in at least capros/coyotos
we've had many architectures running the bootstrap environments
for many target architectures, which meant building the compiler
infrastructure for each
It was sort of a many:many combinatorial problem of host and target
architectures. In theory if we bootstrap with compilers running
on wasm we should be able to reduce that maintenance burden to
one:many bootstrap architectures to target architectures,
with the bootstrap architecture being some (any) kind of capability vm...
William ML Leslie
Jun 1, 2026, 3:34:08 AM (3 days ago) Jun 1
to cap-...@googlegroups.com
On Mon, 1 Jun 2026 at 12:18, William ML Leslie <william.l...@gmail.com> wrote:
There are a few details about my process that I should probably let you (and the rest of the capabilities mafia) in on, and I think this email is the motivation for me to talk about that publicly.
Ok.
The high level end goal is a capability system that:
- runs on my devices
- is free software
- has deep support for mutual suspicion (e.g. applications and filesystems don't have to trust each other)
- can run my existing applications - and not just in a VM as a second-class citizen, but with many of the benefits of running on a capability system, such as the ability to examine the filesystem transaction of the running program, control its network connections, and other hardware access
- with an interface that paid unwavering attention to Ka-Ping Yee's Secure Interface Design.
I'd be stoked if someone else did this, by the way. Norman and the GENODE team at ETH Zurich; Adam and the rest of the Fuchsia team at Google, and Now Jeremy and friends on the Redox team are all making good progress, and even if I'd like a little less legacy and more capabilities as both foundational elements and all the way to the UI layer I'd be more than happy to switch focus if I knew these were going to head in that direction.
I initially thought I could get there by iterating on things we'd already made plans about, that it would simply be a case of putting in the development work, but I hit some pretty core issues early on that sent me back to implementing async with KeyKOS-style resource accounting. This was pretty frustrating. I mentioned to Shap a couple of times that the ideal situation for me would be if he was basically my manager; I'd ask him what to build, follow his direction, and ask lots of questions, but I understand that's a lot to ask, especially between family commitments and running a company. I'd basically be outsourcing putting theory into practice. I have similar constraints myself, which is why I've been taking so long. I consider myself winning if I get one day a month to hack on this stuff at this point.
I have had to optimise for my own time, energy, and attention. I have very limited enthusiasm for C and C++, and I need to preserve it for the kernel (lest I rewrite it in Rocq or ATS) and for drivers, existing libraries, and libraries I need to port. So, unless someone else wants to maintain them, replacing the existing C++ tools streamlines maintenance for me in a way that compounds nicely, so not just because I'm almost 10-fold as productive in something like python as I am in C++. I'd be especially happy if I never had to update Boost again.
A fun side-effect of working solo so far is that I can pick tools that are unmaintained without question. Unmaintained tools are excellent, because there isn't going to be some change that breaks them for no reason. If a tool lets me deliver, it can become maintained again by virtue of my needing to keep it working. Nix and (to an even greater extent) Guix make this even easier: if you have a guix derivation that builds something, because builds are reproducible, you will always be able to build it. This has already saved me a heap of time, by not forcing me to update many autotools plugins.
I worry frequently that I let my perfectionism get the better of me, and without anyone else working on this, we'll get nowhere. So, progress at all costs. Figuring out which decisions I can't revisit later is important, though.
--
William ML Leslie
A tool for making incorrect guesses and generating large volumes of plausible-looking nonsense. Who is this very useful tool for?
William ML Leslie
Jun 1, 2026, 5:08:27 AM (3 days ago) Jun 1
to cap-...@googlegroups.com
On Mon, 1 Jun 2026 at 14:34, Matt Rice <rat...@gmail.com> wrote:
On Sun, May 31, 2026 at 7:18 PM William ML Leslie
<william.l...@gmail.com> wrote:
> I would have liked to get this closer to rpc.capnp (which I guess is what you mean by "the official docs/generators") because I could then use any capnproto implementation in any language with rpc support to implement components. But since the aim is to make each method call literally a system call on an already well-defined interface, the specific encoding used for parameters in rpc.capnp had to change.
>
By "the official docs/generators I meant specifically the "C++
serialization' page, I was talking about earlier which had open enum
semantics.
I think you are usually bound pretty tightly to your generator and wouldn't think of swapping it out. Would love to discover that I'm wrong, though.
#1 I can't imagine python being successfully ported without posix emulation
Python's entire module system to me feels like it is reliant on
ambient authority and finding source files by filename "__init__.py" I
just can't imagine a port of it being successful without posix
emulation.
It just feels like it's accrued to much posix nonsense, since it's
predecessor/inspiration abc actually ran on the amoeba capability
system.
As a last resort, I'm not too scared of doing posix emulation. I have a bunch of patches on glibc that got me quite far, and I have stubs for the handlers and services I'll need to get me off the ground.
The python import system is a known quantity, however, and exposing a read-only filesystem solely for its use isn't too complicated. I already started doing some of that work when I was porting Monte, which also loads source from the filesystem, despite being a capability system.
#2 If we're developing and cross-building on posix systems, tools we
write to make our lives easier as developers will more often target
posix systems.
My general feeling has been that systems like eros, coyotos, capros,
sel4 have not benefited from the tools that developers write to make
their own lives easier.
While the end product is a capability system, the majority of the
interaction is still with legacy tools.
Yes. We do have to start somewhere, though.
The approach of boiling the ocean is a lot more appealing to me than it used to be - rewrites don't scare me - but I think for the time being I need to have a way to use my posix tools on the new OS. At the very least, if I want to switch to it at work, then it needs to support whatever I'm using at work.
#3 It complicates/bloats the bootstrap environment
Python is kind of a batteries included language, it was built under
the assumption it was running on an already bootstrapped system.
That makes a minimal bootstrap environment way bigger than it needs to be.
If someone ends up with the time to do a https://bootstrappable.org/projects/mes.html style minimal bootstrap from Coyotos, they might want to revisit this, it's true. I don't imagine anyone is going to attempt that in the near future.
#4 Reliance on the host system tools and their release cadence.
The spread of python versions in use is very large, some distros
release stuff that is shockingly old with 10 year release cycles,
other systems are shockingly eager to drop support for non-latest
versions. There are really only a couple of sane ways to handle it,
(such as picking one host system you like for instance guix as you say).
Thing is, it used to be that python was backward compatible. If you knew which minimal feature set the program or library needed, you could use any version after that and you'd be fine. The same generally went for libraries, developers didn't break APIs once they were out in the wild. If you wanted to make a breaking change, you made a new class, or a new module. These days, cpython makes backwards incompatible changes on minor version updates. There is also the continuous damage that the python packaging authority are doing to the python ecosystem. This is why I vendored those dependencies - if we can avoid having to install any libraries, we might be able to avoid the absolute hellscape that comes from having to deal with python libraries post the broad adoption of setuptools.
But to the point - Guix is not a host system as much as it allows you to specify or pin a dependency. It's not about supporting a small set of versions, it's about being able to build and run the same software you did a decade ago with absolute fidelity.
I view another sane way is to make the bootstrap environment as
freestanding as possible so that it can run on basically any host
system or a host system with minimal requirements.
Inverting the build process, instead of using build tools that emulate
posix when porting to capability systems. Emulating capability apis
when running on posix within the bootstrap environment.
I left the compiler space because I didn't have the free time it required of me. I'd love to see solutions here, I just don't have that sort of time anymore.
It probably doesn't matter until you start shipping, and people have
classrooms with students, and companies with teams running your
bootstrap environment.
With them running some combination of your bootstrap code along with
the output of their own teams/students the ability to fiddle with the
bootstrap environments seems to grind to a halt.
Because they'll only want to update ubuntu LTS once every 10 years so
everything ends up stuck in the mud.
But it always seems like linux distributions are one of the two
extremes of either hard to keep up with, or glacially slow.
Freestanding environments at least have the benefit of changing at
their own pace, so the host systems release cadence matters much less.
You can just change utilities within the build tree as they improve,
my hope is that some future capability systems will try to bootstrap
using this inverted capability APi method or make it a priority.
So my earliest implementations of this were done in Monte. I'd love to describe in detail why I worried this might be more involved for newcomers specifically because it was foreign to the toolset they might be used to. That might be for another day, though. I'd love to revisit using Monte.
#5 It's entirely possible I'm both wrong and an idiot and python is
super easy to port, I don't know because I've avoided it primarily
because its module seems like a nightmare of ambient authority.
What doesn't have a concept mapping imports to filenames? SmallTalk? At least in python and java, the module system offers some degree of abstraction so you're not just importing a file.
> So, I picked it because it's something I already know well.
>
makes sense, especially for prototyping to pick something you're
familiar with. Sorry I don't mean to take like a decade of frustration
out upon you specifically,
But that has pretty much always been my focus, making these systems
bootstrap themselves. So those inclined can abandon posix bootstrap
environments.
I worry that it might be easiest to replace the wheels once the plane is in the air. I definitely welcome ideas, I am just limited in what I can implement on my own.
On virtual bootstrap environments: I don't have a maintenance burden around host architecture, at least not that I know of. I don't have any problem with the approach, it's just not solving a problem that's currently preventing my progress. But it would be smart to revisit it at some point, at least if we haven't replaced software kernels by better hardware.
--
William ML Leslie
A tool for making future me cry. Who is this very useful tool for?
Matt Rice
Jun 1, 2026, 7:42:02 AM (3 days ago) Jun 1
to cap-...@googlegroups.com
On Mon, Jun 1, 2026 at 2:08 AM William ML Leslie
It's a pretty delicate dance, which isn't exactly about eliminating
filenames. For instance, languages like rust you have an implicit
mod $(echo $(basename $filename) | sed s/-/_/g) {
file contents
}
So filenames may still exist within the data stream rather than
accessed by ambient authority.
Norm also had a page where he described a build system using directory
objects such that the 'make' program gave an interface like
a directory object on coyotos... Such that you receive a message with
the new data when file change, rather than invoking a compiler
which can tell that files changed because their timestamp is newer
than the prior output.
My point is largely that it isn't about ridding the thing of
filenames, but ambient authority.
If capabilities are about splitting filenames between user interface,
and computational core.
arguing they belong weaved throughout the entire toolchain is to argue
that the toolchain is entirely user interface
and contains no computational core.
The closest compilers to the system that Norm described I know of are
modern language server protocols, which evaluate
continuously as data changes though rather than based on ambient
authority. Which doesn't exist in the text buffers of an editor
thankfully.
Besides that it just seems like we value different approaches. Which
if anything is good to know...
>> > So, I picked it because it's something I already know well.
>> >
>>
>> makes sense, especially for prototyping to pick something you're
>> familiar with. Sorry I don't mean to take like a decade of frustration
>> out upon you specifically,
>> But that has pretty much always been my focus, making these systems
>> bootstrap themselves. So those inclined can abandon posix bootstrap
>> environments.
>
>
> I worry that it might be easiest to replace the wheels once the plane is in the air. I definitely welcome ideas, I am just limited in what I can implement on my own.
>
> On virtual bootstrap environments: I don't have a maintenance burden around host architecture, at least not that I know of. I don't have any problem with the approach, it's just not solving a problem that's currently preventing my progress. But it would be smart to revisit it at some point, at least if we haven't replaced software kernels by better hardware.
>
> --
> William ML Leslie
> A tool for making future me cry. Who is this very useful tool for?
>
filenames. For instance, languages like rust you have an implicit
mod $(echo $(basename $filename) | sed s/-/_/g) {
file contents
}
So filenames may still exist within the data stream rather than
accessed by ambient authority.
Norm also had a page where he described a build system using directory
objects such that the 'make' program gave an interface like
a directory object on coyotos... Such that you receive a message with
the new data when file change, rather than invoking a compiler
which can tell that files changed because their timestamp is newer
than the prior output.
My point is largely that it isn't about ridding the thing of
filenames, but ambient authority.
If capabilities are about splitting filenames between user interface,
and computational core.
arguing they belong weaved throughout the entire toolchain is to argue
that the toolchain is entirely user interface
and contains no computational core.
The closest compilers to the system that Norm described I know of are
modern language server protocols, which evaluate
continuously as data changes though rather than based on ambient
authority. Which doesn't exist in the text buffers of an editor
thankfully.
Besides that it just seems like we value different approaches. Which
if anything is good to know...
>> > So, I picked it because it's something I already know well.
>> >
>>
>> makes sense, especially for prototyping to pick something you're
>> familiar with. Sorry I don't mean to take like a decade of frustration
>> out upon you specifically,
>> But that has pretty much always been my focus, making these systems
>> bootstrap themselves. So those inclined can abandon posix bootstrap
>> environments.
>
>
> I worry that it might be easiest to replace the wheels once the plane is in the air. I definitely welcome ideas, I am just limited in what I can implement on my own.
>
> On virtual bootstrap environments: I don't have a maintenance burden around host architecture, at least not that I know of. I don't have any problem with the approach, it's just not solving a problem that's currently preventing my progress. But it would be smart to revisit it at some point, at least if we haven't replaced software kernels by better hardware.
>
> --
> William ML Leslie
> A tool for making future me cry. Who is this very useful tool for?
>
> --
> You received this message because you are subscribed to the Google Groups "cap-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/cap-talk/CAHgd1hGBBaqzk900AvJWb4jefuG7-%3DbC8Jy8-ceJQS-xvjVRNg%40mail.gmail.com.
> You received this message because you are subscribed to the Google Groups "cap-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
Matt Rice
Jun 1, 2026, 7:49:34 AM (3 days ago) Jun 1
to cap-...@googlegroups.com
names as part of the data stream,
there are vast differences in how amiliable it is to port any actual
compilers to using this data stream approach.
If the compiler internally builds this data stream already it's pretty simple.
For rust on gcc it would still be a complete rewrite from the bottom
up, because bfd library wants to open/close files
randomly to stay under the file descriptor limits. There is just no
data layer which doesn't have ambient authority woven throughout the
computational core.
Matt Rice
Jun 1, 2026, 4:38:30 PM (3 days ago) Jun 1
to cap-...@googlegroups.com
On Mon, Jun 1, 2026 at 12:34 AM William ML Leslie
FWIW I agree with all these except I don't give much of a hoot about
compatibility with existing applications.
Where I sit is I'd rather be working on toolchains, and bootstrap
environments and improving the systems we've built.
I tried doing some of these improvements on the seL4 bootstrap
process, but was stifled by ubuntu LTS, alas the version of some
utility (cmake)
with a change in my freestanding build was unfortunately two ubuntu
LTS releases behind, and twenty years from now maybe I'll begin
working on it again.
(Even though seL4 isn't really my cup of tea due to its lack of persistence).
I'm not really someone who has any interest in building OS kernels,
I'm much more interested in developer tools, so for me it's finding
the right project to contribute to.
Until then I'm just going to be outside enjoying the sunshine. At one
point I was actively working on latch (least authority toolchain) by
myself with a hope If I just built it
some operating system would pick it up and use it as their bootstrap
environment, but at some point I had a back injury and spent a million
years in recovery, and ...
One thing is there *are* language level security problems, which
capabilities do have answers for, but not all the problems are with
the toolchains themselves, they inherit a lot of problems from the
garbage legacy operating systems their running on too. So I guess at
this point I'm just jaded with no toolchain to run and no os to run it
on.
>
> I'd be stoked if someone else did this, by the way. Norman and the GENODE team at ETH Zurich; Adam and the rest of the Fuchsia team at Google, and Now Jeremy and friends on the Redox team are all making good progress, and even if I'd like a little less legacy and more capabilities as both foundational elements and all the way to the UI layer I'd be more than happy to switch focus if I knew these were going to head in that direction.
>
> I initially thought I could get there by iterating on things we'd already made plans about, that it would simply be a case of putting in the development work, but I hit some pretty core issues early on that sent me back to implementing async with KeyKOS-style resource accounting. This was pretty frustrating. I mentioned to Shap a couple of times that the ideal situation for me would be if he was basically my manager; I'd ask him what to build, follow his direction, and ask lots of questions, but I understand that's a lot to ask, especially between family commitments and running a company. I'd basically be outsourcing putting theory into practice. I have similar constraints myself, which is why I've been taking so long. I consider myself winning if I get one day a month to hack on this stuff at this point.
>
> I have had to optimise for my own time, energy, and attention. I have very limited enthusiasm for C and C++, and I need to preserve it for the kernel (lest I rewrite it in Rocq or ATS) and for drivers, existing libraries, and libraries I need to port. So, unless someone else wants to maintain them, replacing the existing C++ tools streamlines maintenance for me in a way that compounds nicely, so not just because I'm almost 10-fold as productive in something like python as I am in C++. I'd be especially happy if I never had to update Boost again.
>
> A fun side-effect of working solo so far is that I can pick tools that are unmaintained without question. Unmaintained tools are excellent, because there isn't going to be some change that breaks them for no reason. If a tool lets me deliver, it can become maintained again by virtue of my needing to keep it working.
I guess I have a different approach, if I'm going to assume
maintainership it better be *exactly* what I need, otherwise I'm more
likely to just do a stupid rewrite which provides the 50% of the
functionality I actually use,
and the changes I want. Because even with well designed things like
cap'n'proto I see c-isms I'd rather not.
> Nix and (to an even greater extent) Guix make this even easier: if you have a guix derivation that builds something, because builds are reproducible, you will always be able to build it. This has already saved me a heap of time, by not forcing me to update many autotools plugins.
> I worry frequently that I let my perfectionism get the better of me, and without anyone else working on this, we'll get nowhere. So, progress at all costs. Figuring out which decisions I can't revisit later is important, though.
>
> --
> William ML Leslie
> A tool for making incorrect guesses and generating large volumes of plausible-looking nonsense. Who is this very useful tool for?
>
<william.l...@gmail.com> wrote:
>
> On Mon, 1 Jun 2026 at 12:18, William ML Leslie <william.l...@gmail.com> wrote:
>>
>> There are a few details about my process that I should probably let you (and the rest of the capabilities mafia) in on, and I think this email is the motivation for me to talk about that publicly.
>
>
> Ok.
>
> The high level end goal is a capability system that:
> - runs on my devices
> - is free software
> - has deep support for mutual suspicion (e.g. applications and filesystems don't have to trust each other)
> - can run my existing applications - and not just in a VM as a second-class citizen, but with many of the benefits of running on a capability system, such as the ability to examine the filesystem transaction of the running program, control its network connections, and other hardware access
> - with an interface that paid unwavering attention to Ka-Ping Yee's Secure Interface Design.
I guess here is where I'm coming from,
>
> On Mon, 1 Jun 2026 at 12:18, William ML Leslie <william.l...@gmail.com> wrote:
>>
>> There are a few details about my process that I should probably let you (and the rest of the capabilities mafia) in on, and I think this email is the motivation for me to talk about that publicly.
>
>
> Ok.
>
> The high level end goal is a capability system that:
> - runs on my devices
> - is free software
> - has deep support for mutual suspicion (e.g. applications and filesystems don't have to trust each other)
> - can run my existing applications - and not just in a VM as a second-class citizen, but with many of the benefits of running on a capability system, such as the ability to examine the filesystem transaction of the running program, control its network connections, and other hardware access
> - with an interface that paid unwavering attention to Ka-Ping Yee's Secure Interface Design.
FWIW I agree with all these except I don't give much of a hoot about
compatibility with existing applications.
Where I sit is I'd rather be working on toolchains, and bootstrap
environments and improving the systems we've built.
I tried doing some of these improvements on the seL4 bootstrap
process, but was stifled by ubuntu LTS, alas the version of some
utility (cmake)
with a change in my freestanding build was unfortunately two ubuntu
LTS releases behind, and twenty years from now maybe I'll begin
working on it again.
(Even though seL4 isn't really my cup of tea due to its lack of persistence).
I'm not really someone who has any interest in building OS kernels,
I'm much more interested in developer tools, so for me it's finding
the right project to contribute to.
Until then I'm just going to be outside enjoying the sunshine. At one
point I was actively working on latch (least authority toolchain) by
myself with a hope If I just built it
some operating system would pick it up and use it as their bootstrap
environment, but at some point I had a back injury and spent a million
years in recovery, and ...
One thing is there *are* language level security problems, which
capabilities do have answers for, but not all the problems are with
the toolchains themselves, they inherit a lot of problems from the
garbage legacy operating systems their running on too. So I guess at
this point I'm just jaded with no toolchain to run and no os to run it
on.
>
> I'd be stoked if someone else did this, by the way. Norman and the GENODE team at ETH Zurich; Adam and the rest of the Fuchsia team at Google, and Now Jeremy and friends on the Redox team are all making good progress, and even if I'd like a little less legacy and more capabilities as both foundational elements and all the way to the UI layer I'd be more than happy to switch focus if I knew these were going to head in that direction.
>
> I initially thought I could get there by iterating on things we'd already made plans about, that it would simply be a case of putting in the development work, but I hit some pretty core issues early on that sent me back to implementing async with KeyKOS-style resource accounting. This was pretty frustrating. I mentioned to Shap a couple of times that the ideal situation for me would be if he was basically my manager; I'd ask him what to build, follow his direction, and ask lots of questions, but I understand that's a lot to ask, especially between family commitments and running a company. I'd basically be outsourcing putting theory into practice. I have similar constraints myself, which is why I've been taking so long. I consider myself winning if I get one day a month to hack on this stuff at this point.
>
> I have had to optimise for my own time, energy, and attention. I have very limited enthusiasm for C and C++, and I need to preserve it for the kernel (lest I rewrite it in Rocq or ATS) and for drivers, existing libraries, and libraries I need to port. So, unless someone else wants to maintain them, replacing the existing C++ tools streamlines maintenance for me in a way that compounds nicely, so not just because I'm almost 10-fold as productive in something like python as I am in C++. I'd be especially happy if I never had to update Boost again.
>
> A fun side-effect of working solo so far is that I can pick tools that are unmaintained without question. Unmaintained tools are excellent, because there isn't going to be some change that breaks them for no reason. If a tool lets me deliver, it can become maintained again by virtue of my needing to keep it working.
maintainership it better be *exactly* what I need, otherwise I'm more
likely to just do a stupid rewrite which provides the 50% of the
functionality I actually use,
and the changes I want. Because even with well designed things like
cap'n'proto I see c-isms I'd rather not.
> Nix and (to an even greater extent) Guix make this even easier: if you have a guix derivation that builds something, because builds are reproducible, you will always be able to build it. This has already saved me a heap of time, by not forcing me to update many autotools plugins.
> I worry frequently that I let my perfectionism get the better of me, and without anyone else working on this, we'll get nowhere. So, progress at all costs. Figuring out which decisions I can't revisit later is important, though.
>
> --
> William ML Leslie
> A tool for making incorrect guesses and generating large volumes of plausible-looking nonsense. Who is this very useful tool for?
>
> --
> You received this message because you are subscribed to the Google Groups "cap-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/cap-talk/CAHgd1hExffeYcYn0JiwMMRDfCcb9sse_M%2BKBK7tsb59HVphj8g%40mail.gmail.com.
> You received this message because you are subscribed to the Google Groups "cap-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
Matt Rice
Jun 2, 2026, 3:39:18 PM (2 days ago) Jun 2
to cap-...@googlegroups.com
point I was trying to make across here.
Basically computing feels like one of the most taxing things for my
back health. So if I'm going to contribute
in any significant fashion, it has to be to something I'm fairly
excited about, like non-emulator driven self hosted capability
systems...
William ML Leslie
Jun 3, 2026, 4:01:34 AM (yesterday) Jun 3
to cap-...@googlegroups.com
On Wed, 3 Jun 2026 at 05:39, Matt Rice <rat...@gmail.com> wrote:
Sorry for yet another response, but I really don't think I got the
point I was trying to make across here.
Basically computing feels like one of the most taxing things for my
back health. So if I'm going to contribute
in any significant fashion, it has to be to something I'm fairly
excited about, like non-emulator driven self hosted capability
systems...
No, that's understandable.
I don't think of what I'm doing as an end state - my hope is that once I have the system and people can use it, the path is clear for any other implementations of general-purpose capability systems on commodity hardware. It's answering the questions that only come from implementation, in much the same way that I hope some of my async work ends up ported to seL4.
I'd love to bootstrap up from scheme, honestly. I just know attempting that now means I accomplish nothing.
--
William ML Leslie
A dork biting off far more than he can chew? I mean, a second time?
Reply all
Reply to author
Forward
0 new messages