capt'n proto schema for webassembly

[mjb...@gmail.com]

Hello, 

I'm currently investigating whether capt'n proto schema would be appropriate for describing Webassebly interfaces. I'm attempting to translate this interface to capn proto schema. I several questions and I'm sure I'll have more as the work progresses.

1. Webassebly will soon have opaque references. How would you represent reference types in capt'n proto?

2. Is annotation the correct way to represent pointers? for example

  using Pointer = UInt32;

  using TimestampT = Uint64

 # Return the resolution of a clock.

 # Note: This is similar to clock_getres in POSIX.
  clockResGet @2 (
    clock_id :ClockId, # The clock for which to return the resolution.
    resolution :Pointer $type("TimestampT") # The resolution of the clock.
  ) -> (
    error :ErrnoT
  );

 Here I need to generate the corrisponding wasm signature

 

(func $clockResGet(param $clock_id i32)(param $resolution_pntr  i32) (return i32))

But what I really want to write in captn proto is

  using Pointer = UInt32;
  using TimestampT = Uint64
  # Return the resolution of a clock.
  # Note: This is similar to clock_getres in POSIX.
  clockResGet @2 (
     clock_id :ClockId, # The clock for which to return the resolution.
  ) -> ( 

    error :ErrnoT,

        resolution :TimestampT

  );

But I can't do this because because wasm doesn't have mutiple return values and I can't automatically generate the correct wasm signature in all cases since I'm porting a old interface and some times the output pointers are not always in the same order.

Another Option I thought about was this.    

  using Pointer = UInt32;
  using TimestampT = Uint64
  # Return the resolution of a clock.
  # Note: This is similar to clock_getres in POSIX.
  clockResGet @2 (
     clock_id :ClockId, # The clock for which to return the resolution.
  ) -> ( 
    error :ErrnoT, 
    resolution :TimestampT  $arg_index(1)
  );

Here the annotation "$arg_index" say what param index to put the output pointer in wasm function signature.  I think this solution might be the best for now. But I would love to hear some feedback. 

[Kenton Varda]

Hi mjbecze,

This is interesting! Is this related to:

https://github.com/WebAssembly/design/issues/1274

?

On Sat, Jun 15, 2019 at 9:08 AM <mjb...@gmail.com> wrote:

Hello, 

I'm currently investigating whether capt'n proto schema would be appropriate for describing Webassebly interfaces. I'm attempting to translate this interface to capn proto schema. I several questions and I'm sure I'll have more as the work progresses.

1. Webassebly will soon have opaque references. How would you represent reference types in capt'n proto?

Cap'n Proto's "Capability" type is exactly what you want here: it represents an external reference to "something". Note that interface types in Cap'n Proto are all subclasses of "Capability" -- i.e. "Capability" is kind of like Java's "Object". But it's totally valid to use just the type "Capability" to represent a handle to some opaque thing that has no particular methods you can call.

Also worth noting that in Cap'n Proto's real RPC system, capabilities are referenced through table indices, just like Wasm opaque references apparently will be. Check out:

https://github.com/capnproto/capnproto/blob/77f20b4652e51b5a7ebda414e979e059a6c7c27c/c++/src/capnp/rpc.capnp#L116

Also, Cap'n Proto's C++ implementation has the ability to attach file descriptors to capabilities and pass them between processes (via SCM_RIGHTS on unix sockets):

https://github.com/capnproto/capnproto/pull/821

So there's lots of precedent here.

2. Is annotation the correct way to represent pointers? for example

Oof, this is pretty awkward, especially in that it presumes the callee has the ability to reach into the caller's address space. Obviously, Cap'n Proto hasn't been designed with that sort of thing in mind.

With that said, I like your second solution, transforming return values into things that are stored to the pointer. This allows the schema to express the semantics in a way that makes sense for classic RPC, and then return-by-write-to-pointer becomes part of the "calling convention" that maps Cap'n Proto to Wasm.

-Kenton

[Mark S. Miller]

[+Michael, +Dan] 

I like this direction!

Where can I find a bnf for the Cap'n Proto schema/idl language? I want to start from there to create a concrete syntax for the Chainmail (formal specification language for ocap patterns) abstract syntax. I had previously started with a Jessie-like syntax, but I think Cap'n Proto would be a better place to start. As part of this, I want to write specifications for the ERTP and SwingSet APIs as Cap'n Proto files that I can imagine would grow into Chainmail specs. Together with WASM/WASI bindings for Cap'N Proto, this would be a start for enabling non-JS WASM instances to plug into our SwingSet kernel.

To close this loop, we should define the subset of Cap'n Proto that we expect to compile to WASM/WASI bindings. SwingSet should then restrict itself to that subset.

Is there a standard representation of the AST that a Cap'n Proto schema/idl file parses into? With such an AST translated to JSON, I would also like to explore changing our marshaling system so that it only allows through calls described in the IDL and made available to the comm system at runtime via a derived JSON representation suitable for that purpose. That would put us in a good position to explore eventually cerealizing to Cap'n Proto itself.

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQ%3DYWW-L4Rp5JZHD5CGTbhiKypWuoP5HJ10CxKqL8A6Z%3Dg%40mail.gmail.com.

--

  Cheers,
  --MarkM

[Mark S. Miller]

(From Martin BZ and MarkM jointly)

Hi Kenton, could you look at 

https://github.com/wanderer/WASI/tree/master/schema

We want to suggest this to the WASI folks, to suggest that this be the way WASI IDLs should be specified. Their current plans are to start with WebIDL, which is rather horrible. So we'd like to send this to them as soon as reasonable, but not sooner. So, at this stage, we're not looking for detailed feedback on particulars, but rather, whether this is a reasonable style for using the Cap'n Proto schema language.

Thanks!

--

  Cheers,
  --MarkM

[Kenton Varda]

Hi Mark,

Yes, this seems like a valid use of the capnp schema language. Certainly people have commonly used capnp (and protobuf) schemas to describe things other than capnp (or protobuf) serialization, and I think it's generally gone well.

It does strike me as a tough sell to use something other than WebIDL, considering that I assume there's already a lot of momentum around building Wasm bindings for WebIDL-specified interfaces. But I think it'd be pretty cool if you can convince people.

capnp schemas have a compiled format which is itself specified in capnp:

https://github.com/capnproto/capnproto/blob/master/c++/src/capnp/schema.capnp

It's not quite an AST -- a few additional compilation steps have been performed at this point. This format is used as the input to code generators, and can also be loaded dynamically by many Cap'n Proto runtime implementations to operate on types that weren't known at compile time.

There is also an internal intermediate format that is more of an AST, specified here:

https://github.com/capnproto/capnproto/blob/master/c++/src/capnp/compiler/grammar.capnp

I don't have any formal BNF specification of the Cap'n Proto grammar. The parser is built using a parser-combinators library, so the grammar is specified in C++ code. You can find it here, FWIW:

https://github.com/capnproto/capnproto/blob/master/c++/src/capnp/compiler/parser.c++

-Kenton

[M Bz]

> This is interesting! Is this related to:

> https://github.com/WebAssembly/design/issues/1274

not exactly I don't want to propose or be dependent on full bindings yet. I mainly just want to describe the WASI interface in term of capn proto schema. Full bindings might be nice in the future though! Thanks for the input.

[indol...@gmail.com]

Did anything every come of this?

To unsubscribe from this group and stop receiving emails from it, send an email to capn...@googlegroups.com.

Visit this group at https://groups.google.com/group/capnproto.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQ%3DYWW-L4Rp5JZHD5CGTbhiKypWuoP5HJ10CxKqL8A6Z%3Dg%40mail.gmail.com.

--

  Cheers,
  --MarkM

--

  Cheers,
  --MarkM

[M Bz]

Witx replaced the cap'n proto proposal https://github.com/WebAssembly/WASI/blob/master/docs/witx.md

You received this message because you are subscribed to a topic in the Google Groups "Cap'n Proto" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/capnproto/faUdrdqBVtI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/e9ee73e2-9950-45f6-ba83-c8864f217a32%40googlegroups.com.

[Zach Lym]

Is there an opportunity to layer witx into Cap'n Proto's design at all?