Adding a timeout to a script interpreter (without leaking a goroutine)

[Ben Hoyt]

I'm looking at adding a timeout option to my GoAWK script interpreter using either a timeout or a context value. This is mainly for safety when using it in an embedded context, for example to avoid someone doing a denial-of-service attack by submitting a `BEGIN { while (1); }` script.

This seems simple to do using a timer or context.WithTimeout, and it will time out and return to the caller ... but will continue to run the while loop in the background, eating CPU cycles and leaking a goroutine.

So I'm trying to figure out the best way for the interpreter to check for the timeout at various points and terminate if the timeout has been reached. github.com/yuin/gopher-lua does this by checking ctx.Done() every time around its bytecode loop, but it adds a lot of overhead. I think in GoAWK I'd at least need to check the timeout when handling the "for", "while", and "do-while" statements, as well as the constructs that call os/exec (presumably by passing the ctx down). And I think it'd have to check every statement execution as well. And I'm sure it'll cause a fair bit of performance degradation.

Are there better / more performant ways to handle this?

GitHub issue reference: https://github.com/benhoyt/goawk/pull/26

Thanks,

Ben

[Max]

Using `context` is the recommended mechanism to pass timeouts and other information to functions,

so it's probably already the "best" way, at least from the design point of view.

I recommend benchmarking it before searching for other solutions:

if it turns out to cause excessive slowdown, there are some tricks you can use.

A quite hackish solution that just came to my mind is this:

in the GoAWK interpreter you will surely have some "interpreter context" struct, probably full of often-used pointers.

To set a timeout, you could spawn a goroutine that, after a specified time, sets one or more of these pointers to nil.

The interpreter would then panic due to a nil pointer dereference, and you'd need to recover() from that panic in the top-most interpreter function/method.

And of course you also need to save those pointers somewhere, in order to restore their values at the beginning of the next interpreter call.

As I said, quite hackish.

Regards,

Cosmos72

[Ben Hoyt]

Using `context` is the recommended mechanism to pass timeouts and other information to functions,

so it's probably already the "best" way, at least from the design point of view.

I recommend benchmarking it before searching for other solutions:

if it turns out to cause excessive slowdown, there are some tricks you can use.

Yeah, I think I'm going to try using context and switching on Done() in execute() or eval() and at least see what the performance degradation is. I have some pretty good benchmarks so will easily be able to see the change.

 

A quite hackish solution that just came to my mind is this:

in the GoAWK interpreter you will surely have some "interpreter context" struct, probably full of often-used pointers.

To set a timeout, you could spawn a goroutine that, after a specified time, sets one or more of these pointers to nil.

The interpreter would then panic due to a nil pointer dereference, and you'd need to recover() from that panic in the top-most interpreter function/method.

And of course you also need to save those pointers somewhere, in order to restore their values at the beginning of the next interpreter call.

Heh, this is a funky idea. I don't love it and probably won't use it for real :-), but it's a neat idea.

Thanks for the response.

-Ben

[Max]

A bit less "funky" idea - I currently use it in my https://github.com/cosmos72/gomacro
to interrupt the interpreter if the user hits Ctrl+C.
Caveat: I don't know how difficult is to adapt it to GoAWK.

The idea is conceptually simple: loop unrolling. The details are slightly tricky:

unroll the interpreter main loop that executes statements by something like 20 times,
and after the 20 iterations check *once* for externally-set flags: timeout, Ctrl+C, etc.
The tricky part is: you also need to handle the case where you are somewhere in the middle
of the 20 unrolled iterations but suddenly there are no more statements to execute.

The skeleton code might look similar to
https://github.com/cosmos72/gomacro/blob/master/fast/code.go#L169
or https://github.com/cosmos72/gomacro/blob/master/fast/code.go#L204

Regards,

Cosmos72

[roger peppe]

Another idea might be to avoid making the costly Done check every time. Something like this perhaps (untested)?

    // NewDoneChecker returns a DoneChecker value

    // that amortizes the cost of calling ctx.Done by only

    // calling ctx.Done every interval times that

    // DoneChecker.Done is called.

    func NewDoneChecker(ctx context.Context, interval int) *DoneChecker {

        return &DoneChecker{

            ctx:      ctx,

            interval: interval,

        }

    }

    

    type DoneChecker struct {

        ctx      context.Context

        interval int

        n        int

        done     bool

    }

    

    // Done reports whether the underlying context has reported

    // that it is done. It only does the check every so often.

    func (c *DoneChecker) Done() bool {

        c.n--

        if c.n > 0 || c.done {

            return c.done

        }

        select {

        case <-c.ctx.Done():

            c.done = true

            return true

        default:

            c.n = c.interval

            return false

        }

    }

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/d7484265-0a61-4c48-9e4e-0d6aaa3d447b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[adon...@google.com]

On Tuesday, 21 May 2019 01:18:34 UTC-4, Ben Hoyt wrote:

I'm looking at adding a timeout option to my GoAWK script interpreter...

Are there better / more performant ways to handle this?

 

Hi Ben, imposing resource bounds is a tricky problem. It's possible to do it in an interpreter implemented in C++, but it requires careful discipline throughout the implementation. It is essentially impossible to do in a target language whose variables are recycled by the garbage collector of the host language. Turing incompleteness of the target language (bounded recursion only) seems like it ought to help but in fact does not; a bounded program can still use all your memory and take ~forever.

The only reliable way to impose bounds is to use the operating system. Put the untrusted code in a different process, impose a limit on its maximum memory size, and kill it if it hasn't finished by your deadline.

 

[Thomas Bushnell, BSG]

plus you'd lose the ability to compute ackerman's function, which i'm doing all the time

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/b4182f6d-8a3c-41a2-9cb0-e9d3c7fb1780%40googlegroups.com.