info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
mochitest logging changes
46 views
Skip to first unread message
Nathan Froyd
Feb 27, 2014, 10:00:10 AM2/27/14
to dev-pl...@lists.mozilla.org
Hi,
Mochitest (M1, M2, M3, M4, M5, and M-oth on TBPL) logging has seen several changes over the past couple of weeks. The intent of these changes is to reduce the runtime of the tests, particularly for tests which produce many messages, and reduce the log file sizes. The current status on inbound, post-bug 977198 landing, is:
- Tests with no failures (all TEST-PASS or TEST-KNOWN-FAIL) produce only TEST-START and TEST-END messages. This is similar to what xpcshell has been doing for some time now.
- If there is a TEST-FAIL or a TEST-UNEXPECTED-PASS, then the failure itself is logged, along with some number of messages (currently capped at 100) since the start of the test or the last failure. The intent here was to provide most of the benefits of suppressing messages while still showing some record of what the test was doing prior to the failure.
- If you run a single test locally, then all the messages produced by the test will be shown.
(Please note that these rules do not apply to browser-chrome and metro tests, due to the different logging framework they use. Hooray for consistency.)
If you require more context for your TEST-FAILs, then you may call SimpleTest.requestCompleteLog() in your test and all messages after the call will be displayed. Please treat this as a debugging aid for Try or for intermittent oranges; it shouldn't be necessary to have SimpleTest.requestCompleteLog() calls lying around in our test suite.
For those interested in the nuts and bolts of the changes, you can refer to https://bugzilla.mozilla.org/show_bug.cgi?id=937181 and its dependent/blocking bugs.
Thanks,
-Nathan
Mochitest (M1, M2, M3, M4, M5, and M-oth on TBPL) logging has seen several changes over the past couple of weeks. The intent of these changes is to reduce the runtime of the tests, particularly for tests which produce many messages, and reduce the log file sizes. The current status on inbound, post-bug 977198 landing, is:
- Tests with no failures (all TEST-PASS or TEST-KNOWN-FAIL) produce only TEST-START and TEST-END messages. This is similar to what xpcshell has been doing for some time now.
- If there is a TEST-FAIL or a TEST-UNEXPECTED-PASS, then the failure itself is logged, along with some number of messages (currently capped at 100) since the start of the test or the last failure. The intent here was to provide most of the benefits of suppressing messages while still showing some record of what the test was doing prior to the failure.
- If you run a single test locally, then all the messages produced by the test will be shown.
(Please note that these rules do not apply to browser-chrome and metro tests, due to the different logging framework they use. Hooray for consistency.)
If you require more context for your TEST-FAILs, then you may call SimpleTest.requestCompleteLog() in your test and all messages after the call will be displayed. Please treat this as a debugging aid for Try or for intermittent oranges; it shouldn't be necessary to have SimpleTest.requestCompleteLog() calls lying around in our test suite.
For those interested in the nuts and bolts of the changes, you can refer to https://bugzilla.mozilla.org/show_bug.cgi?id=937181 and its dependent/blocking bugs.
Thanks,
-Nathan
Boris Zbarsky
Feb 27, 2014, 11:30:14 AM2/27/14
to
On 2/27/14 10:00 AM, Nathan Froyd wrote:
> - If there is a TEST-FAIL or a TEST-UNEXPECTED-PASS, then the failure itself is logged, along with some number of messages (currently capped at 100) since the start of the test or the last failure.
Does that include test timeouts?
> - If there is a TEST-FAIL or a TEST-UNEXPECTED-PASS, then the failure itself is logged, along with some number of messages (currently capped at 100) since the start of the test or the last failure.
Do I read this right that this will show the first 100 messages from the
test, not the last 100 messages before the failure? Or am I
misunderstanding?
-Boris
Benjamin Smedberg
Feb 27, 2014, 4:00:06 PM2/27/14
to Nathan Froyd, dev-pl...@lists.mozilla.org
On 2/27/2014 10:00 AM, Nathan Froyd wrote:
> - If there is a TEST-FAIL or a TEST-UNEXPECTED-PASS, then the failure itself is logged, along with some number of messages (currently capped at 100) since the start of the test or the last failure. The intent here was to provide most of the benefits of suppressing messages while still showing some record of what the test was doing prior to the failure.
Is there a reason why it's not every message since the start of the
> - If there is a TEST-FAIL or a TEST-UNEXPECTED-PASS, then the failure itself is logged, along with some number of messages (currently capped at 100) since the start of the test or the last failure. The intent here was to provide most of the benefits of suppressing messages while still showing some record of what the test was doing prior to the failure.
current test or the 100 most recent, whichever is greater? I imagine
that seeing the entire output of the single test that failed might be
useful in many circumstances.
>
> - If you run a single test locally, then all the messages produced by the test will be shown.
whether it's a single test or an entire directory/manifest?
--BDS
Nathan Froyd
Feb 27, 2014, 5:04:07 PM2/27/14
to Benjamin Smedberg, dev-pl...@lists.mozilla.org
----- Original Message -----
> On 2/27/2014 10:00 AM, Nathan Froyd wrote:
> > - If there is a TEST-FAIL or a TEST-UNEXPECTED-PASS, then the failure
> > itself is logged, along with some number of messages (currently capped at
> > 100) since the start of the test or the last failure. The intent here was
> > to provide most of the benefits of suppressing messages while still
> > showing some record of what the test was doing prior to the failure.
> Is there a reason why it's not every message since the start of the
> current test or the 100 most recent, whichever is greater? I imagine
> that seeing the entire output of the single test that failed might be
> useful in many circumstances.
Sorry, my explanation was bogus. The intent and implementation is that you see the last 100 messages produced by the test. Of course it is possible that you would see fewer if your test hasn't produced that many messages yet.
> On 2/27/2014 10:00 AM, Nathan Froyd wrote:
> > - If there is a TEST-FAIL or a TEST-UNEXPECTED-PASS, then the failure
> > itself is logged, along with some number of messages (currently capped at
> > 100) since the start of the test or the last failure. The intent here was
> > to provide most of the benefits of suppressing messages while still
> > showing some record of what the test was doing prior to the failure.
> Is there a reason why it's not every message since the start of the
> current test or the 100 most recent, whichever is greater? I imagine
> that seeing the entire output of the single test that failed might be
> useful in many circumstances.
> > - If you run a single test locally, then all the messages produced by the
> > test will be shown.
> Is there a mach flag which can force full logging of an entire run
> whether it's a single test or an entire directory/manifest?
-Nathan
0 new messages