getTimer in RPC & transparent proxy?

[pepij...@gmail.com]

Hey,

Another one of these things that's probably good design but somewhat puzzling.

How do I obtain a timer in my RPC server?

I found you need to get them from the async IO provider.

The easyrpc server has a method to get it, but then I'm stuck.

I need to somehow pass the timer to my server implementation, but before I create the RPC server I can't access the IO provider, and after I created the RPC server my implementation got moved into a private property.

Honestly maybe the answer is: if you need a timer in your RPC server you're doing it wrong. It's basically doing a waitloop on my work thread because I got stuck with my queue implementation. It seemed like it'd require a lot of overhead in copying and locking, while the waitloop just has to check one uncontested lock while the worker thread fills a vector. Basically what I need is an async Locked::wait(cond)

The second question is: I saw some release notes talking about a proxy not having to know the exact message format it's proxying, but is there actually a way to make a generic capnproto proxy? Similarly I saw a few mentions of HTTP and websocket stuff, but is there actually a way to use Capnproto over a websocket? I need none of that right now, but it's interesting to know what the options are.

Here is my project so far btw https://github.com/NyanCAD/SimServer

Thanks for all the help so far :)

Cheers,

Pepijn

[Ian Denhardt]

Quoting pepij...@gmail.com (2021-03-25 12:38:43)

> Similarly I saw a few mentions of HTTP and websocket stuff, but is
> there actually a way to use Capnproto over a websocket? I need none of
> that right now, but it's interesting to know what the options are.

Not out of the box in the C++ implementation; I did some preliminary
refactoring to enable this a while back but have yet to actually add the
relevant support. Someone would need to add an implementation of
MessageStream backed by a websocket:

https://github.com/capnproto/capnproto/blob/b5ab41ea033874cb3c144b9ad87e3442a297ed55/c%2B%2B/src/capnp/serialize-async.h#L36

(I just noticed that the comment on that class is bitrotten, since
at some point the functionality "above" got moved below...)

The relevant issue is here:

https://github.com/capnproto/capnproto/issues/1063

It wouldn't be a lot of work; I did something similar for the Go
implementation as a one-off for an app I was writing here:

https://github.com/zenhack/sandstorm-rendezvous/blob/master/websocket-capnp.go

...which took me < 10 min.

> Here is my project so far btw https://github.com/NyanCAD/SimServer

Looks like you accidentally committed some merge conflicts at the bottom
of the .capnp.

-Ian

[Ryan Patterson]

On Friday, March 26, 2021 at 12:38:43 AM UTC+8 pepij...@gmail.com wrote:

Similarly I saw a few mentions of HTTP and websocket stuff, but is there actually a way to use Capnproto over a websocket?

This might not be what you're looking for, but I took the fasterthanlime fork of capnp-ts and upgraded all of the dependencies, and in my test application I was able to use a go capnp server with a Typescript capnp client over web sockets. Relevant links:

• my updated fork of capnp-ts
  
• glue between capnp-ts and web sockets
  
• the original work on capnp-ts RPC (to whom the credit is owed)
  

[Kenton Varda]

On Thu, Mar 25, 2021 at 11:38 AM pepij...@gmail.com <pepij...@gmail.com> wrote:

Hey,

Another one of these things that's probably good design but somewhat puzzling.

How do I obtain a timer in my RPC server?

I found you need to get them from the async IO provider.

The easyrpc server has a method to get it, but then I'm stuck.

I need to somehow pass the timer to my server implementation, but before I create the RPC server I can't access the IO provider, and after I created the RPC server my implementation got moved into a private property.

The EZ interfaces turn out to be problematic in a lot of ways.

I recommend skipping them and instead using kj::setupAsyncIo() and capnp::TwoPartyClient/TwoPartyServer to set up your async and RPC environments more explicitly. Then you'll be able to get the timer before you construct your server object.

Also note that you can take a (non-owning) reference to your server before you pass it off to the RPC system. You can assume the server won't be moved or destroyed while the RPC system is active, so the reference remains valid.

 

The second question is: I saw some release notes talking about a proxy not having to know the exact message format it's proxying, but is there actually a way to make a generic capnproto proxy?

You can take a capability received from one RPC connection and pass it over any other RPC connection. Cap'n Proto will automatically arrange to proxy requests in this case. In this case, only the caller and callee actually need to know the schema for the methods being called; the proxy doesn't need to know. So, for example, you could cast a capability to the type `capnp::Capability::Client` (which is the base type of all `Client` objects), then pass it around from machine to machine, and then eventually use `client.castAs<MyInterface>()` to cast back to the object's real type (or any superclass), and it'll work. You can make calls and they'll pass through all the proxies to the final object, even if the proxies don't have the schema for `MyInterface` or only have an older version.

 

Similarly I saw a few mentions of HTTP and websocket stuff, but is there actually a way to use Capnproto over a websocket? I need none of that right now, but it's interesting to know what the options are.

Ian mentioned MessageStream. In theory it should be easy to combine that with KJ HTTP's WebSocket implementation -- at least to the extent that doing anything with KJ HTTP is "easy". It's like all other KJ interfaces, weird at first but nice when you get to know it.

-Kenton

 

Here is my project so far btw https://github.com/NyanCAD/SimServer

Thanks for all the help so far :)

Cheers,

Pepijn

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/d9072ac4-2751-4e9e-91f3-a27be38da0dbn%40googlegroups.com.

[Ian Denhardt]

Quoting Ryan Patterson (2021-03-25 22:04:01)

> This might not be what you're looking for, but I took the
> fasterthanlime fork of capnp-ts and upgraded all of the dependencies,
> and in my test application I was able to use a go capnp server with a
> Typescript capnp client over web sockets. Relevant links:

> * [1]my updated fork of capnp-ts
> * [2]glue between capnp-ts and web sockets
> * [3]the original work on capnp-ts RPC (to whom the credit is owed)

How did you build this, and how well does it actually work? I tried forking the
fasterthanlime fork it myself, but it looked to me like it was left in a broken
state; I fixed a few things on my own fork[1], but when running tsc
there are still a couple errors:

$ tsc
packages/capnpc-ts/lib/generators.ts:931:9 - error TS2345: Argument of type 'undefined' is not assignable to parameter of type 'readonly Expression[]'.

931 __ // arguments
~~

packages/capnpc-ts/lib/generators.ts:985:17 - error TS2345: Argument of type 'undefined' is not assignable to parameter of type 'readonly Expression[]'.

985 __ // parameters
~~
I assume those had been placeholders. Did you just plow ahead using the
generated code anyway (in which case I assume your schema didn't trigger
that case), or are you doing something that I'm missing when building?

I had a heck of a time trying to get the gulpfile to work, and ended up
just giving up, since running tsc from the top level seemed to work with
some tweaking, modulo the obviously legitimate errors above.

-Ian

[1]: https://github.com/zenhack/capnp-ts/tree/rpc

[Ryan Patterson]

OK, maybe not all the credit is owed there. Check my fork again, notice that I upgraded both the built-in capnp schema files as well as the gulpfile. You should be able to do a clean "yarn build" after that point.

Node v15.6.0 | Yarn 1.19.1

$ gulp --version

CLI version: 2.3.0

Local version: 4.0.2

[Julián Díaz]

Original author of capnp-ts here: I've been watching this thread from a distance and am glad people are finding my old work in progress here useful enough to try forking it – the situation with me personally has been ~4 years of being locked up with an employer that did not allow me opportunity to contribute back to this project, even on my free time. Hence the complete radio silence on the repo and in these discussion groups.

Fortunately that job is going away in a few weeks and I look forward to finally getting a chance to upgrade the dependencies/build tooling, and get everything into good working order before taking a look at the state of RPC and what's been happening with the forks (especially Ryan's fork). Please reach out if you want to work together with me on this!

I regret switching the build process from a Makefile to gulp; that was at the behest of a very loud contributor who hated it simply because he didn't understand it. I'll probably bring that back because the gulpfile never worked quite as well as the Makefile did. I'm also open to other suggestions on a good build tool for this because it's been some time since I've been embedded in the JS ecosystem.

--

You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.

To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/83caf638-062f-416b-b726-3c83742ace2fn%40googlegroups.com.

[Ian Denhardt]

Quoting Julián Díaz (2021-03-29 15:58:17)

> I'll probably bring that back because the gulpfile never worked quite
> as well as the Makefile did.

+1.