capnweb

[๏̯͡๏ Jasvir Nagra]

https://blog.cloudflare.com/capnweb-javascript-rpc-library/

...from Kenton.

-- 

Jasvir Nagra

[William ML Leslie]

A supported ocapn in javascript? That's going to be a great time saver.  Love the trick with array encoding; Cowbert will be proud.

--
You received this message because you are subscribed to the Google Groups "friam" group.
To unsubscribe from this group and stop receiving emails from it, send an email to friam+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/friam/CAK1HgMiHFczR_4_SfwWkOvuQJ3fpiQCSERCYezC3FVvHVK3WCg%40mail.gmail.com.

--

William ML Leslie

[Kenton Varda]

So, I hope this doesn't make everyone mad at me but... it's not actually an implementation of OCapN.

But it's definitely an ocap protocol so should be reasonably isomorphic!

Cap'n Web is a new protocol I designed, loosely based on Cap'n Proto, which is in turn based on CapTP.

Though I think Cap'n Web is actually closer to CapTP than Cap'n Proto is. Cap'n Proto treated an outstanding call differently from a promise -- calls would eventually get a "return" message, whereas promises would get a "resolve". At the time I designed it (2013) I thought I had good reasons for this, but I no longer remember exactly what they were. In Cap'n Web a call just creates a promise, like in CapTP. I admit, it turned out way simpler.

Cap'n Web also merges the imports and questions tables into a single table: negative numbers are imports, positive numbers are questions. (Zero is the "bootstrap" or "main" capability (if any).)

I now actually want to revise Cap'n Proto's RPC layer to use this design as well... though I worry about breaking compatibility. Need to think it through.

I didn't use OCapN for a few reasons:

* It's really important for this project that we can exactly match the semantics of Cloudflare Workers's built-in worker-to-worker RPC (which we launched a year and a half ago). That includes things like the exact set of supported types.

* I'm optimizing for browsers specifically, where code footprint matters and using the built-in JSON parser is really the ideal trade-off between speed and code footprint.

* I personally have a different opinion about the practicality of distributed GC, which I think has some fairly deep implications for protocol design.

* I honestly didn't realize that there were working implementations of OcapN, because the web site and github don't seem to mention them! I thought it was still in the design phase. I didn't want to be the first one implementing a protocol I didn't play a part in designing. I should probably have asked.

BTW, Cloudflare Workers is extensively capability-based, mostly under the hood so far, but it's going to be getting a lot more explicit, as we're going to be pitching it as the correct way to handle sandboxing AI. More on that coming Friday...

-Kenton

To view this discussion visit https://groups.google.com/d/msgid/friam/CAHgd1hG7jnH-paruZR0pVFiMkbpuwimUtc94Rn%3DJWoUgj1R3qQ%40mail.gmail.com.

[๏̯͡๏ Jasvir Nagra]

Coming Friday as in you'll be at friam? Or that you have more rolling thunder to launch on Friday!

Either way color me excited!

-- 

Jasvir Nagra

To view this discussion visit https://groups.google.com/d/msgid/friam/CAJaLmO43k_ccDC5QYC0cNNQ-gvuj9nqjB65J8cYxeztJN8NCGQ%40mail.gmail.com.

[Kenton Varda]

Oh sorry... I mean that I have another announcement blog post coming up on Friday.

Would love to see you all sometime but this week is a bit busy!

-Kenton 

To view this discussion visit https://groups.google.com/d/msgid/friam/CAK1HgMi4NKtXXqdm9ZqLeZe5QHnM-mrVtzeo-ayAXSs0hJLS6w%40mail.gmail.com.

[William ML Leslie]

On Tue, 23 Sept 2025 at 11:47, Kenton Varda <temp...@gmail.com> wrote:

So, I hope this doesn't make everyone mad at me but... it's not actually an implementation of OCapN.

But it's definitely an ocap protocol so should be reasonably isomorphic!

Cap'n Web is a new protocol I designed, loosely based on Cap'n Proto, which is in turn based on CapTP.

I tend to re-implement just enough of CapTP to get by on a lot of projects.  Having something I can just pull in as a library will be superb.  It also doesn't help that every time I want to use CapTP, I'm working in a different language, but using JSON over websockets as the transport makes it not too difficult to do the server side.  I have often tended to do the same using a WAMP-inspired protocol (turning data-e's M expressions into JSON arrays).

This really just means I don't need to worry about the client side, it's just a bit of javascript ffi.

It's probably a few weeks before I can try it seriously, but I am looking forward to it.

--

William ML Leslie

[Mark S. Miller]

Kenton, your s-expression-like JSON embedding looks obviously extensible by adding new names in the first position. For example, to convey NaN, +- Infinity, -0, bigints. Before I think about actually proposing any (e.g., as would be needed to cover ocapn use cases), are you open to considering such extensions for future versions of Cap'n Web? Perhaps with qualifications on interop, such as not sending those your receiver does not recognize? Or treating those you don't recognize as opaque but retransmissible?

--

You received this message because you are subscribed to the Google Groups "friam" group.
To unsubscribe from this group and stop receiving emails from it, send an email to friam+un...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/friam/CAHgd1hFN0%3DZeJ9FLEAdiMhUVj7S7AkG%2BE7uWJtoRQ0K0r5WZAA%40mail.gmail.com.

--

  Cheers,
  --MarkM

[Kenton Varda]

(Sorry for slow replies, getting lots of email this week.)

Yes, I'm open to adding more supported types to the protocol.

Bigint is actually already supported but I forgot to document it. As is Uint8Array.

NaN and +-Infinity definitely make sense to support. (-0 I'm not so sure about.)

I'm hesitant to allow apps to register their own pass-by-value types, though, as it seems like a pretty big security footgun (as evidenced by all the vulnerabilities coming out of Java serialization). So my rule has been that applications can only define pass-by-reference types.

-Kenton

To view this discussion visit https://groups.google.com/d/msgid/friam/CAK5yZYi6zoR2qtRqPW45S%2BzADBiF7m28K5qCN3%3DXV99B9zT7Yw%40mail.gmail.com.

[Kris Kowal]

My intuition is the same regarding user-defined types. Over at OCapN, we’ve got most of our data model and inter-language mapping ironed out except particulars of error. The most no

https://github.com/ocapn/ocapn/blob/main/draft-specifications/Model.md

The data model differs in a few small ways:

OCapN does have “tagged values”.

OCapN has no Date. We would conventionally use a UTC ISO string with a tag. In EndoJS, that looks like {[@passStyle]: 'tagged', tag: 'data', payload: '2025-09-25etc'}.

OCapN’s byte arrays get reïfied as ArrayBuffer and not Uint8Array (a point we might adjust given movement at TC39).

OCapN has “symbols”.

Symbols have been a sore for the JavaScript side because registered symbols would introduce a registry stuffing attack and anonymous symbols with equal descriptions are not “equal” in the conventional sense, so we’re moving in the direction of using ordinary objects with a symbol-key to represent them. That’s our escape hatch in general: an OCapN object on the wire will never have a symbol-named key, so we can reliably partition data objects from various other kinds. The Scheme folks will not suffer an omission of symbols.

I’m pretty well against adding any of the types you aspire to add like regular expressions. We use “tagged” values to ship immutable representations of maps, bags, and sets, which just obligates the recipient to map them with their light client or data structure of choice. This is probably fine. We’re overdue to have a conversation about “perimeter types” which will probably be a tug-of-war on the perimeter of OCapN’s data model. I figure, if you want a richer data model, you put a membrane around a larger extent and leave the protocol alone.

That is to say, that CapnWeb’s data model is pretty close to compatible. Not close enough to call it an OCapN flavor, but maybe they can converge. I’m working on a Noise Protocol that we can use to straddle multiple transport protocols and stand on the same assurances about confidentiality and authenticity of peers and, for example, do three-party-hand-off with transport, encoding, and captp negotiation. Would be neat to support Capnweb as a variant. Locator URLs might look like ocap://ed25519.np/swissnum@ws+capnweb:example.com@tcp+ns+cbor:example.com@tor+ws+cbor:onion.

Kris Kowal

[Matt Rice]

Perhaps more controversial and less trivial to get right than the
suggestions of Mark, but (one of)
my serialization pet peeves has been the total lack of support for
partial synchronization in serialization formats.

I think that a first class range type, e.g. [0,5) to use interval
notation (unlikely to be a good choice for the format, but using it
regardless)
including support for serializing partial arrays [0,0,0,0]@[0,4)

I've at least found in my work that easily being able to do a query,
and respond that something is within some address range
of my address space, which can be followed up by a query for/transfer
of that address range, or updates to it or some such.

Anyhow this is a pretty far leap from json, python does allow
serializing ranges but in possibly the most unhelpful way by sending
arrays of int.
Definitely not as obvious that this is a reasonable ask as the types
Mark mentioned though.

> To view this discussion visit https://groups.google.com/d/msgid/friam/CAJaLmO4JQ1z_oboOcicMBGMHyjbmmAz4z_C6dLtuJvJn41hAuQ%40mail.gmail.com.

[Pierre Thierry]

Le 26/09/2025 à 05:19, Matt Rice a écrit :

(one of) my serialization pet peeves has been the total lack of support for
partial synchronization in serialization formats.

What do you mean by that?

Curiously,
Pierre Thierry
--

pie...@nothos.net
0xD9D50D8A

[Matt Rice]

On Fri, Sep 26, 2025 at 5:36 PM Pierre Thierry <pie...@nothos.net> wrote:
>
> Le 26/09/2025 à 05:19, Matt Rice a écrit :
>
> (one of) my serialization pet peeves has been the total lack of support for
> partial synchronization in serialization formats.
>
> What do you mean by that?

What I meant specifically in this case was sending a sub slice of an
array using ranges.
So, replicating an address space by sending/receiving ranges of raw memory,
without sending the whole array by value.

In for example, something like a client/server debugger like gdb
connected to gdbserver or some such,
you rarely need to the entire address space in the debugger client,
but you want to transfer portions of the
address space to the client as needed, and respond to queries e.g.
about symbol locations.
Similarly there are fairly well defined invalidation points during
symbol loading for much static data.

For these kinds of things I find it easier to deal with a
serialization format that can fairly accurately serialize/invalidate
pieces of an address space lazily, as opposed to the typical
serialization formats which are limited to sending complete arrays.
Does that clarify things?