Define pubsub like rpc in protobuf file

[Alex Barlow]

Hi all,

I was hoping for some advice for designing my publish subscribe system on top of protobuf and rpc.

The idea is that a developer may have defined set of messages, a normal service and some rpc methods which they then augment with some publishers and subscribers.

For example, given this ordinary service..

syntax = "proto3";

package helloworld;

service Greeter {

  rpc SayHello (HelloRequest) returns (HelloReply) {}

}

message HelloRequest {

  string name = 1;

}

message HelloReply {

  string message = 1;

}

I'm thinking of creating my own extensions and options so a developer can describe which messages it publishes and what messages it subscribes to but I'm struggling with how to implement it, here's a proposal of a annotated proto

syntax = "proto3";

package helloworld;

enum Broadcasts {

  HELLO = 0;

}

service Greeter {

  rpc SayHello (HelloRequest) returns (HelloReply) {}

  option (subscribers) = {

    publisher {

      name: HELLO
      returns: HelloReply

    }

    subscriber {

      name: HELLO

    }

    subscriber {

      name: some_service.CREATED

    }

  };

}

message HelloRequest {

  string name = 1;

}

message HelloReply {

  string message = 1;

}

The problem is in defining the .proto for the subscribers option, how do I create a message/extention which accepts an ENUM or Message type as a value to be set.

I'm also open to completely different ways of designing this, but I was hoping to have a way to define subscribers and publishers in a typed way that shows me the dependencies so I can then generate the code.

Any help would be appreciated! Thanks

[Feng Xiao]

Not sure if this is what you are asking for, but the following proto compiles for me:
syntax = "proto3";
import "google/protobuf/descriptor.proto";

enum Broadcasts {
  HELLO = 0;

  CREATED = 1;
}
enum ReturnTypes {
  HELLO_REPLY = 0;
}
message Publisher {
  Broadcasts name = 1;
  ReturnTypes returns = 2;
}
message Subscriber {
  Broadcasts name = 1;
}
message Subscribers {
  repeated Publisher publisher = 1;
  repeated Subscriber subscriber = 2;
}

extend google.protobuf.ServiceOptions {
  Subscribers subscribers = 4444;
}

service Greeter {

  option (subscribers) = {
    publisher {
      name: HELLO

      returns: HELLO_REPLY
    }  
    subscriber {
      name: HELLO
    }  
    subscriber {
      name: CREATED
    }  
  };  
}

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Alex Barlow]

Ah yes,

So I've done something similar, however the problem is that the pub sub extension would be a library. So using the 'Broadcasts' enum wouldn't scale as I couldn't foresee all the events that a particular service might implement and I'd want the developer to define their own enum. So I guess my question is, how can I take an enum or string constant (which an enum is generated to in some libraries).

Essentially, since I can't define RPC methods as they're async and don't reference other services, I'm looking for a way to reference other messages and constants in my options. Perhaps there's another way of going about it? I could just define strings, but then you would have to make sure they're the same between protobufs and using a code generator, I would have to find that other protobuf and link them together

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.

[Feng Xiao]

On Tue, Apr 11, 2017 at 3:59 PM, 'Alex Barlow' via Protocol Buffers <prot...@googlegroups.com> wrote:

Ah yes,

So I've done something similar, however the problem is that the pub sub extension would be a library. So using the 'Broadcasts' enum wouldn't scale as I couldn't foresee all the events that a particular service might implement and I'd want the developer to define their own enum. So I guess my question is, how can I take an enum or string constant (which an enum is generated to in some libraries).

Essentially, since I can't define RPC methods as they're async and don't reference other services, I'm looking for a way to reference other messages and constants in my options. Perhaps there's another way of going about it?

Maybe not a good idea, but you can get something like the below with extensions:

syntax = "proto2";

import "google/protobuf/descriptor.proto";

package xiaofeng;

message MyOptions {

  extensions 1000 to max;

}

extend google.protobuf.ServiceOptions {

  optional MyOptions any_option = 4444;

}

message MyMessage {

  extend MyOptions {

    optional MyMessage extended_option = 1000;

  }

  optional int32 value = 1;

}

service MyService {

  option (any_option) = {

    [xiaofeng.MyMessage.extended_option] {

      value: 10

    }

  };

}

Basically you define your custom option with extensions, and let users extend your option to add custom data into your option. This only works with proto2 though as it uses extensions.

 

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.