How to get server-side implementation from client capability ?

[Théophile B.]

I'm not sure I'm usign the right terminology in my question, but I hope you will understand my problem with the following example.

I'm developing a media library where the user can play/pause/stop different kinds of "pipelines" (source ~ camera, filter ~ processing, sink ~ display). Depending on their type, these pipeline can listen to each other, or not (e.g: a source pipeline cannot listen). I'm now working on a "capnp wrapper" of the library, so that it can be controlled via RPC.

I have replicated the class inheritance with success:

interface Publisher {}

interface Subscriber {
listenTo @0 (publisher :Publisher);
}

interface Pipeline {
play @0 ();
pause @1 ();
stop @2 ();
}

interface SourcePipeline extends(Pipeline, Publisher) {}

interface FilterPipeline extends(Pipeline, Publisher, Subscriber) {}

interface SinkPipeline extends(Pipeline, Subscriber) {}

Server-side I have those impl classes, which holds the references to the underlying objects of the media library (pub_ & sub_).

class PublisherImpl : public virtual GstDaemon::Publisher::Server {

protected:

PublisherImpl(my_underlying_library::Publisher* pub) : pub_(pub) {}

private:

my_underlying_library::Publisher* pub_;

};

class SubscriberImpl : public virtual GstDaemon::Subscriber::Server {

protected:

SubscriberImpl(my_underlying_library::Subscriber* sub) : sub_(sub) {}

kj::Promise<void> listenTo(ListenToContext context) override {

auto pub = context.getParams().getPublisher(); // GstDaemon::Publisher::Client

// ??? = PublisherImpl::pub_

sub_->listenTo(???)

}

private:

my_underlying_library::Subscriber* sub_;

};

Now, from the client, I wanted to do something like this:

//GstDaemon::Subscriber::Client subscriber; // already returned by the server

//GstDaemon::Publisher::Client publisher; // already returned by the server

auto request = subscriber.listenToRequest();

request.setPublisher(publisher);

auto promise = request.send();

However, as you can see, on the server it's not possible to get the pub_ pointer when the listenTo method is called. Is there any way I can get a reference to the PublisherImpl here ?

[Kenton Varda]

Hi Théophile,

Take a look at CapabilityServerSet:

https://github.com/capnproto/capnproto/blob/6b5bcc2c6e954bc6e167ac581eb628e5a462a469/c++/src/capnp/capability.h#L607-L633

When you convert your `Server` object to a `Client`, you need to do it using a CapabilityServerSet. Later, you can use the same set to unwrap the `Client` and get the underlying `Server` back -- even if the client has been passed over the network and back in the meantime.

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/488a63c6-92bc-4bc8-9ab8-e648d3a577bfn%40googlegroups.com.

[Théophile B.]

Thank you for your quick answer!

The `Server` to `Client` "conversion" is done via the bootstrap capability, but I need the underlying `Server` back when calling capabilities returned by the bootstrap capability.

Here is a simplified example where I (tried) to implement your advice:

interface MediaLibrary {

   addPublisher @0 (name :Text) -> (pub :Publisher);

   addSubscriber @1 (name :Text) -> (sub :Subscriber);

  interface Publisher {}

  interface Subscriber {
     listenTo @0 (publisher :Publisher);
   }

}

class MediaLibraryImpl final : public MediaLibrary::Server {

public:

kj::Promise<void> addPublisher(AddPublisherContext context) override {

     my_underlying_library::Publisher* output = ...; // call the underlying lib

    MediaLibrary::Publisher::Client pub = publishers_.add(kj::heap<PublisherImpl>(output));

    context.getResults().setPub(pub);

   return kj::READY_NOW;

}

kj::Promise<void> addSubscriber(AddSubscriberContext context) override {

     my_underlying_library::Subscriber* output = ...; // call the underlying lib

    MediaLibrary::Subscriber::Client sub = subscribers_.add(kj::heap<SubscriberImpl>(output));

    context.getResults().setSub(sub);

    return kj::READY_NOW;

}

private:

capnp::CapabilityServerSet<MediaLibrary::Subscriber> subscribers_;

capnp::CapabilityServerSet<MediaLibrary::Publisher> publishers_;

};

class PublisherImpl : public virtual GstDaemon::Publisher::Server {

protected:

PublisherImpl(my_underlying_library::Publisher* pub) : pub_(pub) {}

private:

my_underlying_library::Publisher* pub_;

}

class SubscriberImpl : public virtual GstDaemon::Subscriber::Server {

protected:

SubscriberImpl(my_underlying_library::Subscriber* sub) : sub_(sub) {}

kj::Promise<void> listenTo(ListenToContext context) override {

auto pub = context.getParams().getPublisher(); // GstDaemon::Publisher::Client

// No access to CapabilityServerSet !

// We we would like to do: sub_->listenTo("pub_")

}

private:

my_underlying_library::Subscriber* sub_;

}

If I understand well,  I can now do subscribers_.getLocalServer("GstDaemon::Publisher::Client") from `MediaLibraryImpl`.

Bu what about SubscriberImpl, where the `listenTo` needs `PublisherImpl` ?

Is it possible to have access to `MediaLibraryImpl` from `SubscriberImpl` ?

Instead, should I pass references to the `CapabilityServerSet` to `PublisherImpl` and `SubscriberImpl` ?

[Kenton Varda]

You need your `SubscriberImpl` to hold a regular C++ pointer back to the `MediaLibraryImpl` that created it, so that you can access its `publishers_` set through that pointer. You can pass the pointer to `SubscriberImpl`'s constructor.

You'll also want to make sure the pointer doesn't become a dangling pointer, so `SubscriberImpl` should also hold a `MediaLibrary::Client` which points to the same `MediaLibraryImpl` -- holding this counts as a strong reference so Cap'n Proto won't destroy the `MediaLibraryImpl`. Inside `addSubscriber()` you can use `thisCap()` to get a `MediaLibrary::Client` pointing to `this`, then you'd pass that to `SubscriberImpl`'s constructor, along with the plain C++ pointer to `this`.

-Kenton

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/45982dae-5fb0-4adb-8b15-9fea70506d25n%40googlegroups.com.

[Théophile B.]

It works ! Thank you for the expanded explanation, it helped a lot. I initially though there would be a "prettier" out-of-the-box solution, but as long as it works...

While I have you, I have a more general question. Is it common case to distribute Cap'n'Proto genererated headers as part of a library, or everything must stay private ?

Let's say I have defined some enum in the .capnp and I want the users of my library to use them.

For the moment I have a duplicated the definition of this enum in a public header, and I do static casts before serialization.

[Kenton Varda]

I would say that when you are distributing a precompiled binary copy of your library, you should distribute the `.capnp.h` header along with it, if you want programs using your library to be able to use that type. Those programs will need to link against the exact version of the schema that you compiled into your library, so you might as well provide them with the matching header. Otherwise, if they try to generate their own copy, they may run into linker errors due to duplicate symbols -- or worse, subtle runtime ODR errors if their version didn't exactly match what's in your library.

With all that said, there's a second layer of this problem, which is that your library and applications using it need to link against exactly the same version of Cap'n Proto. Unfortunately, different versions of the Cap'n Proto library are not ABI-compatible, and headers generated for one version won't necessarily work with a different version of the library. If you don't want your library to impose a specific version of Cap'n Proto on the applications linking with it, then either you need to let each app compile your library themselves (using the version of capnp they like), or you need to do symbol-renaming stuff in order to embed a private copy of capnp into your binary. In the latter case, you can't really distribute the .capnp.h header since it would not work with any version of Cap'n Proto other than your private one.

-Kenton

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/8c279b5e-0fe0-4dac-af58-22eeda468c92n%40googlegroups.com.

[Théophile B.]

Ok I see, generated headers indeed include capnproto/kj headers, so Cap'n Proto would be a public dependency of my library.

The version mismatch issue could be solved by using a CMake package, with the right version of Cap'n Proto required in the CMake configuration file.

Based on all this, I decided that Cap'nProto should stay a private dependency (I'm building it inside my project with CPM.cmake), even if I need to static_cast my enums on both sides.

Thank you again for your time !