GRFC L20: Core API to estimate the memory usage of channels and servers

[apo...@google.com]

Discussion of https://github.com/grpc/proposal/pull/55

this thread supersedes https://groups.google.com/forum/#!topic/grpc-io/FPoXprcT0d4

[Craig Tiller]

Let's enumerate the things that need sizes in this grfc... At first cut I'd suggest:

- channels
- servers
- calls
- completion queues

Do we also need slices, slice buffers?
What about metadata?

Server memory can change drastically over time since it also maintains a list of channels and incoming but not yet delivered calls.

Should we additionally account for this? If yes, how? Is it polled? Is there a callback model?

On Thu, Jan 18, 2018, 8:56 PM apolcyn via grpc.io <grp...@googlegroups.com> wrote:

Discussion of https://github.com/grpc/proposal/pull/55

this thread supersedes https://groups.google.com/forum/#!topic/grpc-io/FPoXprcT0d4

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/ee85c861-e4a7-4cbb-a0eb-444b3ea05ce8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[apo...@google.com]

Good point, I think that really anything consuming large/unbounded memory should potentially be tracked.

As for the API, I am thinking:

- If the API is synchronous, we could provide a rough estimate of an object upon it's creation that doesn't change throughout it's lifetime. Perhaps the estimate could change change throughout the lifetime of the process though. For example, after a channel allocates a large buffer once, future channels can use that to increase their estimate. 

- An callback or queue based API might be more accurate though.

One example of something to account for is the cert file (what originally motivated this). In ruby, if the user uses the default channel SSL creds, then we end up loading a ~300KB cert file (https://github.com/grpc/grpc/blob/master/etc/roots.pem), which is apparently copied (at least partially and at least once) roughly per channel.

[apo...@google.com]

Personally, for the API I am thinking it would be flexible if core exposed a single global callback, e.g., "notify_of_change_in_memory(amount)", where amount is positive or negative for alloc/free.

[Craig Tiller]

Alternative (sketch) proposal:

Track this via resource quota's, since we have an API that's supposed to deal with this already.

I'd imagine one new API:

void grpc_resource_quota_notify_on_memory_change(

    grpc_resource_quota* resource_quota, 

    size_t min_memory_usage, 

    size_t max_memory_usage, 

    size_t* current_memory_usage,

    void* tag,

    grpc_completion_queue* cq);

With the semantics that 'tag' would be emitted on 'cq' whenever memory usage falls outside of the min/max. On entry current_memory_usage would be what the application last observed, prior to queuing the tag it would be updated to be the instantaneous current value, and so have the thrash-prevention properties of channel state.

This will (of course) need the resource quota code to be informed of (most) of the allocations within core, and that hookup will be time consuming.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/1c77c9c9-e310-4e61-a520-b14a9c94099a%40googlegroups.com.