wleslie update thread
19 views
Skip to first unread message
William ML Leslie
Oct 1, 2026, 1:59:28 AMOct 1
to cap-talk
Making this a thread for now.
Most of September has been aimed at one test on my CapnProto/CapIDL implementation, exercising the synchronous client and server stubs for a single message (or, a single method call). The arguments are 15 arrays of different types, plus one small struct. It returns the same type. This is a good example of a "slow path" message, as in, it can't go into registers on Coyotos so it's one indirect string both ways.
Most of September has been aimed at one test on my CapnProto/CapIDL implementation, exercising the synchronous client and server stubs for a single message (or, a single method call). The arguments are 15 arrays of different types, plus one small struct. It returns the same type. This is a good example of a "slow path" message, as in, it can't go into registers on Coyotos so it's one indirect string both ways.
https://gitlab.com/william-ml-leslie/wl-capidl-capnp/-/blob/main/tests/idl/tests/Silly.capnp?ref_type=heads#L16
The aim was to verify that you could populate the arrays, use the client stub, and have the server stub decode the message and be able to read back the same values. In particular, array handling in CapnProto seems especially finicky, so I wanted some tests that at least this implementation could read the values it wrote. We now have a test that does exactly that. Well, if you exclude the fact that this is running on the host, so the actual invoke_capability call is stubbed by the test framework.
The aim was to verify that you could populate the arrays, use the client stub, and have the server stub decode the message and be able to read back the same values. In particular, array handling in CapnProto seems especially finicky, so I wanted some tests that at least this implementation could read the values it wrote. We now have a test that does exactly that. Well, if you exclude the fact that this is running on the host, so the actual invoke_capability call is stubbed by the test framework.
https://gitlab.com/william-ml-leslie/wl-capidl-capnp/-/blob/main/tests/silly.c?ref_type=heads#L191
There are a total of 36 commits this month, which is a good measure of what I lost by not adding tests first. I think this month will be quieter, if only a little.
I would like to:
- have a test that reads some messages generated by the capnp compiler, e.g. using schema.capnp.
- support big endian.
- test arrays of floats.
- test/implement arrays of bits.
- support the Coyotos 0.6.2 ABI in indirect string support, which I removed at some point.
- test the upgrade paths, e.g. upgrading a list of ints into a list of structs with one int field.
I don't think I'll get to async tests this month, but that's where I'm headed.
I'm undecided as to whether I should let register allocation be determined by 32-bit. As in, which argument goes into which register is determined regardless of platform, and specifically, we always allocate two registers for a 64-bit value, even if it's unnecessary. I suspect that running 32-bit applications on a 64-bit system is rare enough that we can pay the cost of repacking these only when needed. I mean, we don't have a large installed base of 32-bit Coyotos Native applications we need to support.
In fact, for parameters passed in softregs, the 32-bit syscall API already wouldn't be compatible with 64-bit, since it relies on the layout of a struct where each of the registers are naturally aligned. So, maybe it's time to not worry about compatibility there.
There are a total of 36 commits this month, which is a good measure of what I lost by not adding tests first. I think this month will be quieter, if only a little.
I would like to:
- have a test that reads some messages generated by the capnp compiler, e.g. using schema.capnp.
- support big endian.
- test arrays of floats.
- test/implement arrays of bits.
- support the Coyotos 0.6.2 ABI in indirect string support, which I removed at some point.
- test the upgrade paths, e.g. upgrading a list of ints into a list of structs with one int field.
I don't think I'll get to async tests this month, but that's where I'm headed.
I'm undecided as to whether I should let register allocation be determined by 32-bit. As in, which argument goes into which register is determined regardless of platform, and specifically, we always allocate two registers for a 64-bit value, even if it's unnecessary. I suspect that running 32-bit applications on a 64-bit system is rare enough that we can pay the cost of repacking these only when needed. I mean, we don't have a large installed base of 32-bit Coyotos Native applications we need to support.
In fact, for parameters passed in softregs, the 32-bit syscall API already wouldn't be compatible with 64-bit, since it relies on the layout of a struct where each of the registers are naturally aligned. So, maybe it's time to not worry about compatibility there.
--
William ML Leslie
William ML Leslie
Nov 4, 2026, 3:57:04 AM (11 days ago) Nov 4
to cap-talk
I ended up spending a week in hospital with pneumonia and not flying anywhere, which was terrible for my social life but pretty good for progress on mundane tasks.
Our CapIDL replacement now has tests for lists of void, bits, floats and doubles.
I have a really simple async test working; specifically, it tests using a continuation directly. It tests this with messages that fit into the remaining registers, and one that uses all of the registers available to a synchronous call. An async system that supports using all of the registers available to a regular coyotos IPC can support backwards-compatible calls.
I have another weird use of IPC that I think requires me to add another flag to the control word. The continuation wants to be interrupted when attempting to send in some cases. For example, if I have a continuation that I've configured to perform some task, and now I want to abort that task, it's pretty easy to do so - unless it's trying to send synchronously; which it might be doing if it needs to talk to an existing synchronous object.
The existing non-blocking send doesn't quite work for this case. Normally, with a non-blocking send, you don't care if the send was successful. But we'd still like for the send to happen - we might just need to process a message first. In order to know whether to retry the send, we need to know if it happened. Another problem with non-blocking send in this case is that it may still block. Now your cancel needs to wait for the service to have some of its address space paged in.
What I need is something that attempts to send (synchronously) unless there's a message waiting to be received. It always receives and processes existing messages before sending, and waiting to send needs to be interrupted by an incoming message.
Next steps, probably:
- re-enable indirect strings
- generate interfaces for true asynchronous send and receive (using send and receive buffers)
- port the existing kernel codebase to the new tools
The first step is a prereq to the other two, but I don't know if I'll do them in that order.
Our CapIDL replacement now has tests for lists of void, bits, floats and doubles.
I have a really simple async test working; specifically, it tests using a continuation directly. It tests this with messages that fit into the remaining registers, and one that uses all of the registers available to a synchronous call. An async system that supports using all of the registers available to a regular coyotos IPC can support backwards-compatible calls.
I have another weird use of IPC that I think requires me to add another flag to the control word. The continuation wants to be interrupted when attempting to send in some cases. For example, if I have a continuation that I've configured to perform some task, and now I want to abort that task, it's pretty easy to do so - unless it's trying to send synchronously; which it might be doing if it needs to talk to an existing synchronous object.
The existing non-blocking send doesn't quite work for this case. Normally, with a non-blocking send, you don't care if the send was successful. But we'd still like for the send to happen - we might just need to process a message first. In order to know whether to retry the send, we need to know if it happened. Another problem with non-blocking send in this case is that it may still block. Now your cancel needs to wait for the service to have some of its address space paged in.
What I need is something that attempts to send (synchronously) unless there's a message waiting to be received. It always receives and processes existing messages before sending, and waiting to send needs to be interrupted by an incoming message.
Next steps, probably:
- re-enable indirect strings
- generate interfaces for true asynchronous send and receive (using send and receive buffers)
- port the existing kernel codebase to the new tools
The first step is a prereq to the other two, but I don't know if I'll do them in that order.
--
William ML Leslie
Reply all
Reply to author
Forward
0 new messages