How can I kill a JS function at runtime for a V8 unit test?

[Isiah Meadows]

What I need to do for a specific unit test is to run a specific method + arguments, and if it takes too long, stop the call mid-cycle and fail the test. This is for a performance-related unit test for my patch (which is a perf patch itself, anyways). Is this possible, and if so, how would I do it? I'm not nearly as well versed in C++ as I am in JavaScript, but I could write it in C++ if I had to.

[Ben Noordhuis]

It sounds like you would need to call V8::TerminateExecution() from a
watchdog thread. That said, tests that rely on wall clock time are
unreliable and will almost certainly get rejected during code review.

[Isiah Meadows]

How's the best way to test for a performance regression (as in "this test should not hang")?

--
--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to a topic in the Google Groups "v8-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/v8-users/AMVUNpV-xLw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to v8-users+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jakob Kummerow]

That depends entirely on the kind of performance delta we're talking about.

If a particular testcase went from actually hanging (endless loop, or running for hours) to finishing quickly (say, in 1 millisecond), then by all means land that testcase. (You don't have to detect the hang yourself, the test driver already has a timeout for each test.)

Ideally you're spending your time on changes that visibly improve one of the major benchmarks we're tracking anyway. In that case, there's no need to add a separate performance test; but please do add correctness tests for the feature you're adding/modifying.

Other than that we usually don't have fine-grained performance tests. Optimizing for micro-benchmarks is easy but often pointless. Also, it's important to realize that the test suite is supposed to assert correctness while finishing as quickly as possible, whereas measuring performance typically requires running the tests in question for a considerable amount of time to get reliable results.

[Isiah Meadows]

On Thursday, September 25, 2014 2:01:03 PM UTC-4, Jakob Kummerow wrote:

That depends entirely on the kind of performance delta we're talking about.

If a particular testcase went from actually hanging (endless loop, or running for hours) to finishing quickly (say, in 1 millisecond), then by all means land that testcase. (You don't have to detect the hang yourself, the test driver already has a timeout for each test.)

Okay. That's pretty much what I needed. My test case would be for the former case there (actually hanging). I was more looking for a specific case that shouldn't hang. How long is the timeout?
 

[Jakob Kummerow]

On Fri, Sep 26, 2014 at 3:18 PM, Isiah Meadows <impi...@gmail.com> wrote:

On Thursday, September 25, 2014 2:01:03 PM UTC-4, Jakob Kummerow wrote:

That depends entirely on the kind of performance delta we're talking about.

If a particular testcase went from actually hanging (endless loop, or running for hours) to finishing quickly (say, in 1 millisecond), then by all means land that testcase. (You don't have to detect the hang yourself, the test driver already has a timeout for each test.)

Okay. That's pretty much what I needed. My test case would be for the former case there (actually hanging). I was more looking for a specific case that shouldn't hang. How long is the timeout?

It's chosen dynamically based on a variety of factors, IIRC between a couple seconds and a few minutes. Do not depend on it. Make sure your testcase finishes fast (1 millisecond is a good target to aim for).

[Isiah Meadows]

Problem then, it should take no more than a second or two. Basically it is a "this should not hang the engine" test.

--

[Jakob Kummerow]

I don't understand what you're saying. Do you mean your testcase should, when all is well, finish in a second or two? That's much too slow. Or do you mean that after 1-2 seconds it would be OK to cancel it and treat it as a failure, because in the good case it finishes much faster? As I said before, there's no need to test this specifically, just make sure all affected code paths are tested for correctness.

Can you provide more detail about what precisely it is that you're trying to do? I'm getting a bit tired of the guesswork.

[Isiah Meadows]

It is primarily a perf patch, where str.repeat(2^20), etc. no longer hangs the engine's execution thread.

I'm wanting to assert the following two things:
1. Large numbers as arguments must not hang the engine in the normal case. The patch provides a reduction in the worst case of algorithmic runtime complexity.
2. Repeating an empty string should be near instantaneous. The patch also includes a special case where repeating an empty string should be near instantaneous, regardless of the number of repeats. This case should take no longer than a few milliseconds on an old, excessively bogged down machine (even to the point the desktop is hanging), almost ready to fry itself.

--

[Jakob Kummerow]

And why would anyone care how long it takes to repeat a string a million times? Especially if that string is empty?

Don't make the test suite slower for this. Ensure correctness, and leave it at that.

[Isiah Meadows]

I'll just leave it out, then.