Ensure context timeout behaviour from caller, or callee ?

[Jerry Londergaard]

When using a context.WithTimeout, I always felt that it should be the function where the context was created should be the one that ensures that it (the current function) does not block longer than intended. That is to say, it should call context.WithTimeout(), and then run the subsequent code in a goroutine (passing in the context most likely), whilst using a select multiplexer to listen to when the context is cancelled (by the timeout), and then resume. Here's an example of this https://go.dev/play/p/EGNlJqo3hY5

However, when I was looking at this on the go database docs, it seems to say to push that logic further down the stack to the 'child' (or callee) https://go.dev/doc/database/cancel-operations#:~:text=Canceling%20database%20operations%20after%20a%20timeout

I think this is an alternative implementation of my first example, that is analogous to what the database package is suggesting https://go.dev/play/p/DABt-Z36T-F

Whilst I understand that the child should listen for the signal and clean-up what it's doing in the event of the cancellation, that's doesn't feel like the same thing as the caller *relying* on that behaviour for the caller function to resume running.

How should I think about this? Am I missing something here?

[burak serdar]

Both patterns have their uses. If the new goroutine can periodically
check context cancellation and needs to do some cleanup, then it is
best to pass the context down, clean up, and return on cancellation.
If the new goroutine cannot check context cancellation, then the
goroutine starting that new goroutine can deal with cancellation like
you did, hoping that the new goroutine eventually ends. I usually use
the former, but the latter has its uses if you are calling a
third-party library or some operation you cannot cancel. If that
operation never ends, the goroutine leaks.

>
> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f45936f2-36b3-46cb-aa93-9bbbf9438843n%40googlegroups.com.

[Jerry Londergaard]

If the new goroutine can periodically
check context cancellation and needs to do some cleanup, then it is
best to pass the context down, clean up, and return on cancellation.

Understood. I would always pass the context down anyway,

letting that callee decide whether or not it wants to listen for a context cancellation (regardless

of what the caller is doing with the context).

Are you comfortable though, *relying* on this behaviour down the stack to enforce the timeout that was declared

further up the stack (leaving potentially leaked goroutines out of it...)?

If the new goroutine cannot check context cancellation, then the
goroutine starting that new goroutine can deal with cancellation like
you did, hoping that the new goroutine eventually ends.

Ok, I think you're referring to this example of mine https://go.dev/play/p/EGNlJqo3hY5

I usually use
the former, but the latter has its uses if you are calling a
third-party library or some operation you cannot cancel.

This is close to the heart of the question. My example of that third-party library is the database one,

and it seems to say (based on its suggested usage) "You shouldn't need to handle your context timeout, we'll make sure we do".

Which, maybe works fine in that specific case, but in general feels a bit awkward. It seems you would agree?

 

If that operation never ends, the goroutine leaks.

That's a good point. If you don't run code in a goroutine you can never leak goroutines :-p

It seems to me though, that if we are running things in goroutines, and passing down the context,

and the code being run in those goroutines isn't listening for cancellation signals, 

which results in the running goroutine becoming leaked, then that feels like a bug in that code (i.e not the callers responsibility). 

 

 

[burak serdar]

On Wed, Sep 20, 2023 at 5:04 PM Jerry Londergaard
<jlonde...@gmail.com> wrote:
>
> Are you comfortable though, *relying* on this behaviour down the stack to enforce the timeout that was declared
> further up the stack (leaving potentially leaked goroutines out of it...)?

That's the nature of handling requests using this layered approach. As
you move down layers in a concurrent program, those that are at a
higher layer set the context for the lower layers. Those that are
lower the stack are not enforcing timeouts or cancellations, they are
merely getting a notification of it.

>
>> If the new goroutine cannot check context cancellation, then the
>> goroutine starting that new goroutine can deal with cancellation like
>> you did, hoping that the new goroutine eventually ends.
>
>
> Ok, I think you're referring to this example of mine https://go.dev/play/p/EGNlJqo3hY5
>
>> I usually use
>> the former, but the latter has its uses if you are calling a
>> third-party library or some operation you cannot cancel.
>
>
> This is close to the heart of the question. My example of that third-party library is the database one,
> and it seems to say (based on its suggested usage) "You shouldn't need to handle your context timeout, we'll make sure we do".
> Which, maybe works fine in that specific case, but in general feels a bit awkward. It seems you would agree?

I am a bit confused about what you are asking here. I find it awkward
and risky if I cannot pass down a cancelable context down the stack
where things may not return. But then, that's the only way to deal
with such third-party code. If goroutines leak, they leak, and
eventually you run out of memory, crash, and restart. Sometimes
failing and recovering is cheaper than fixing.

[Jerry Londergaard]

I find it awkward
and risky if I cannot pass down a cancelable context down the stack
where things may not return.

Absolutely, we always want to be able to pass that down regardless.

Those that are
lower the stack are not enforcing timeouts or cancellations, they are
merely getting a notification of it.

Right, this is how I understand it as well. In this case the database lib methods takes the context and act on the 

cancellation (rightly so), but also seemed to suggest to not bother enforcing that timeout at the higher level. Perhaps I'm

reading too much into their example though.

I am a bit confused about what you are asking here.

Apologies. I think you've already answered my question with your previous statement ('Those that are lower down the stack are not enforcing timeouts...').

Thanks for your patience, this has been illuminating.