info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Recent spate of jit-test failures
17 views
Skip to first unread message
Bill McCloskey
Sep 13, 2013, 9:55:06 PM9/13/13
to JS Internals list, Ed Morley, philri...@gmail.com
Does anyone have any idea what's going on here? It seems like there are maybe 5 or 10 a day getting filed. Examples:
https://bugzilla.mozilla.org/show_bug.cgi?id=916388
https://bugzilla.mozilla.org/show_bug.cgi?id=916386
They all have the form "X is undefined" or "X is not a function" or some other JS error. Is this a new occurrence or were they just never filed before? Is anyone looking into the problem? Would it make sense to group them all into one big intermittent jit-tests bug? It's probably annoying for the sheriffs to have to file a new bug for every jit-test.
As far as I can tell, they all happen on Windows. Not that that helps much.
-Bill
https://bugzilla.mozilla.org/show_bug.cgi?id=916388
https://bugzilla.mozilla.org/show_bug.cgi?id=916386
They all have the form "X is undefined" or "X is not a function" or some other JS error. Is this a new occurrence or were they just never filed before? Is anyone looking into the problem? Would it make sense to group them all into one big intermittent jit-tests bug? It's probably annoying for the sheriffs to have to file a new bug for every jit-test.
As far as I can tell, they all happen on Windows. Not that that helps much.
-Bill
Jan de Mooij
Sep 14, 2013, 5:03:32 AM9/14/13
to Bill McCloskey, Ed Morley, philri...@gmail.com, JS Internals list
I looked into this a while ago, bug 899697 [0] has some info.
Summary: It's definitely Windows only. The only tests that fail are the
ones that expect a return code != 0. Most of these tests expect return code
3 (because they have a "|jit-test| error" comment), but I think I've also
seen tests fail that expect 6 (timeout). This happens also with very simple
tests and with shell flags that disable the JITs.
When the tests fail, they do throw the error we expect. If you look at bug
916836 for instance:
TEST-UNEXPECTED-FAIL |
e:\builds\moz2_slave\m-in-w32-000000000000000000000\build\js\src\jit-test\tests\parallel\bug858582.js
| --baseline-eager --no-ti --no-fpu: @evaluate:1:679 TypeError: cmpFunction
is not a function
The test that fails has "// |jit-test| error: TypeError", this TypeError is
the one we expect and see on other platforms. So the test itself happily
succeeds but the shell somehow returns 0 instead of 3 and the test fails.
If we wait a few more months, the sheriffs will have filed bugs for all
tests that have "|jit-test| error" or "|jit-test| exitstatus". That's only
~330 bugs...
Even if I make the shell's main() always "return 3", we reach that return
but the process exit code is still 0 every X thousand runs. Apparently this
is possible on Windows in some cases [1]. I think it requires a thread-safe
shell, it's probably something NSPR related. I also thought it didn't
happen if I make main() return before JS_DestroyRuntime, but I may be wrong
about that.
The problem started after bug 776043 [2] landed, it made the jit-test
harness check the return code. My suggestion in bug 899697 was to always
allow return code == 0 on Windows to stop these intermittent failures. It's
hiding the problem somehow, but unfortunately I don't have the time to get
to the bottom of this.. It doesn't help that it can take > 30 minutes
before a test fails, so it's hard to say whether a change "fixes" it..
Jan
[0] https://bugzilla.mozilla.org/show_bug.cgi?id=899697
[1] http://blogs.msdn.com/b/oldnewthing/archive/2008/05/06/8461730.aspx
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=776043
> _______________________________________________
> dev-tech-js-engine-internals mailing list
> dev-tech-js-en...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals
>
Summary: It's definitely Windows only. The only tests that fail are the
ones that expect a return code != 0. Most of these tests expect return code
3 (because they have a "|jit-test| error" comment), but I think I've also
seen tests fail that expect 6 (timeout). This happens also with very simple
tests and with shell flags that disable the JITs.
When the tests fail, they do throw the error we expect. If you look at bug
916836 for instance:
TEST-UNEXPECTED-FAIL |
e:\builds\moz2_slave\m-in-w32-000000000000000000000\build\js\src\jit-test\tests\parallel\bug858582.js
| --baseline-eager --no-ti --no-fpu: @evaluate:1:679 TypeError: cmpFunction
is not a function
The test that fails has "// |jit-test| error: TypeError", this TypeError is
the one we expect and see on other platforms. So the test itself happily
succeeds but the shell somehow returns 0 instead of 3 and the test fails.
If we wait a few more months, the sheriffs will have filed bugs for all
tests that have "|jit-test| error" or "|jit-test| exitstatus". That's only
~330 bugs...
Even if I make the shell's main() always "return 3", we reach that return
but the process exit code is still 0 every X thousand runs. Apparently this
is possible on Windows in some cases [1]. I think it requires a thread-safe
shell, it's probably something NSPR related. I also thought it didn't
happen if I make main() return before JS_DestroyRuntime, but I may be wrong
about that.
The problem started after bug 776043 [2] landed, it made the jit-test
harness check the return code. My suggestion in bug 899697 was to always
allow return code == 0 on Windows to stop these intermittent failures. It's
hiding the problem somehow, but unfortunately I don't have the time to get
to the bottom of this.. It doesn't help that it can take > 30 minutes
before a test fails, so it's hard to say whether a change "fixes" it..
Jan
[0] https://bugzilla.mozilla.org/show_bug.cgi?id=899697
[1] http://blogs.msdn.com/b/oldnewthing/archive/2008/05/06/8461730.aspx
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=776043
> dev-tech-js-engine-internals mailing list
> dev-tech-js-en...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals
>
Ryan VanderMeulen
Sep 16, 2013, 9:37:28 AM9/16/13
to
On 9/14/2013 5:03 AM, Jan de Mooij wrote:
> I looked into this a while ago, bug 899697 [0] has some info.
>
> Summary: It's definitely Windows only. The only tests that fail are the
> ones that expect a return code != 0. Most of these tests expect return code
> 3 (because they have a "|jit-test| error" comment), but I think I've also
> seen tests fail that expect 6 (timeout). This happens also with very simple
> tests and with shell flags that disable the JITs.
>
While it's true that Windows presents the biggest problem, I will point
> I looked into this a while ago, bug 899697 [0] has some info.
>
> Summary: It's definitely Windows only. The only tests that fail are the
> ones that expect a return code != 0. Most of these tests expect return code
> 3 (because they have a "|jit-test| error" comment), but I think I've also
> seen tests fail that expect 6 (timeout). This happens also with very simple
> tests and with shell flags that disable the JITs.
>
out that there have been more occurrences on OSX recently as well. The
below link shows the examples filed in the last few weeks.
https://bugzilla.mozilla.org/buglist.cgi?keywords=intermittent-failure%2C%20&keywords_type=allwords&list_id=7921457&short_desc=jit-test&resolution=---&op_sys=Mac%20OS%20X&chfieldto=Now&query_format=advanced&chfield=[Bug%20creation]&chfieldfrom=2013-08-26&short_desc_type=allwordssubstr
Thank you very much for looking into this! Here's to hoping you can find
a workable solution soon! :-)
-Ryan
0 new messages