Distributed Component invocation on all instances

89 views
Skip to first unread message

Cody Craven

unread,
Mar 25, 2023, 4:17:48 PM3/25/23
to Service Weaver
Are there any plans to support function invocations that run on all instances of a distributed component? This may be something that's already supported, I couldn't find anything in the docs that led me to believe it was though.

Background:
I'm currently learning Service Weaver as a lower-effort replacement for K8s for projects that it is suitable for, and I keep thinking of cases where I'd want functionality such as cache invalidation. I could certainly do this by incorporating something like pubsub separately deployed from the service weaver application, however it seems like it would fit nicely within Service Weaver natively since it is already aware of all instances of the distributed application.

Srdjan Petrovic

unread,
Mar 26, 2023, 2:31:12 PM3/26/23
to Cody Craven, Service Weaver
Hi Cody,

Thanks for your question. The underlying Service Weaver runtime has all of the information needed for us to implement this type of "invoke-all" functionality. Note however that the semantics of this solution would be best-effort. Because instance creation is dynamic (e.g., increase/decrease # of instances based on load), there would be no way to guarantee that *all* of the instances have been reached. It is also possible that some instances that are currently running have been marked for scale-down, so some of the instance calls may return errors. How to handle such errors, and what it means to you as a developer, is a tricky business.

For example, in your case, suppose that you issue a "cache-invalidate" to all of your running instances. And furthermore, suppose you get an error. Unless we give you some more information, you have no idea how many instances were reached. And suppose we pass that information to you, say, 80% instances were reached. Should you be able to do a retry? How do you know if the retry has reached the remaining 20% of instances? And how to ensure that the instances whose cache has already been invalidated haven't been invalidated again.

It almost feels like RPC may not be a right solution for this functionality. But all other solutions as well (e.g., pub-sub) may cause over-invalidation or under-validation for your caches. This generally feels like a hard problem. :)

Thanks,
-Srdjan

--
You received this message because you are subscribed to the Google Groups "Service Weaver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to serviceweave...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/serviceweaver/082d0be8-5c23-4fc7-a1d2-e1708db4e9e0n%40googlegroups.com.

Cody Craven

unread,
Mar 28, 2023, 11:42:07 AM3/28/23
to Service Weaver
Hi Srdjan,

I definitely understand the nuances you're describing. Could Service Weaver defer on the hard parts by returning a slice of currently known components that the application developer could then iterate and handle each error as they see fit?

Something like:
```
// Retrieves defined component instances that exist at that moment. A more
// flexible future solution could use a channel to be notified of status changes
// (instances spinning up, down, or status becomes unknown) so that
// applications that need precise control could have it.
myComps, err := weaver.CurrentInstances[MyComponent](root)
if err != nil {
t.Fatal(err)
}
// Application could perform this behavior async with a wait group or channel
// sized for len(myComps), simple example shown just for discussion.
for _, myComp := range myComps {
err = myComp.MyFunc(ctx, "invalidate-cache")
if errors.Is(err, weaver.ErrRetriable) {
// retry, omitted for example
continue
}
// Could have an error type for when component is still instantiating
if errors.Is(err, weaver.ErrInitializing) {
// apps would likely just skip the instance, but could do something
// like place it in a queue to retry
continue
}
// Could have an error type for when component is spinning down
if errors.Is(err, weaver.ErrStopping) {
// again, apps would most likely skip the instance, but could
// choose to do something
continue
}
}
```
Reply all
Reply to author
Forward
0 new messages