Binlog as a gRPC service?

[to...@spotify.com]

Is there a plan to support on-demand streaming of binary logs by providing it as a gRPC service or maybe a reason not to support it?

With this example service definition:   

service BinaryLogz {

  rpc GetBinaryLogs(BinaryLogRequest) returns (stream BinaryLogResponse);

}

we wouldn't require any additional logging infrastructure for the simple case of taping into a single instance request/response flow.  

As simple CLI could be use to stream the response to stdout or dump it to a file to be replayed/inspected later:   

gdump -d '{"log_filter": "Foo/*,Foo/Bar{m:256}"}' grpc.binarylog.v1alpha.BinaryLogz/GetBinaryLogs -o binlog.rio

I understand that this might be complicated to implement for all languages and might not make sense to have in core, but I wanted to to float the idea.  

[Carl Mastrangelo]

I'm not sure that there is a service in mind, but it would need some sort of special treatment if it did exist. (i.e. do binlog gRPCs themselves get logged?)

Also, what happens if the log service hangs, gets backed up, or is unreachable?  

[Tommy Ulfsparre]

> Also, what happens if the log service hangs, gets backed up, or is unreachable?

Unreachable how? The producer would be implemented in-process and the consumer would be the stream connected to the service. 

If you have slow consumers or if the producer can't publish/write entries fast enough I would assume the only reasonable thing to do is drop entries to not effect RPCs. 

This is basically just to improve the ergonomics of using binlog, which Im not sure is worth the added complexity? 

A smaller change and something we can implement ourself with the current interface is toggle the binlog with a settable/dynamic flags. 

We would have to have an extra step for getting the locally written binlog out of the container but I guess that is not terrible.

--
You received this message because you are subscribed to a topic in the Google Groups "grpc.io" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/grpc-io/_-t-QWNVwyY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d5c7b043-1669-4506-8407-1c1fe3e0d561%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.