Errors related to calendar code in C-C TB xpcshell tests. (Pass --verbose flag. to see them.)

[ISHIKAWA,chiaki]

Folks,

I create patches for TB under Debian GNU/Linux on my local PC and run
mochitest and xpcshell tests locally.

Recently, due to the time it took me to modify my patches to cater to
the directory layout changes in C-C and an issue (OS version string not
handled correctly) that prevented me from running xpchsell test locally,
I could not run xpcshell-test for a while.
Today I have finally been able to run xpcshell test for the first time
in 3 months or so.

Then I am surprised. There are simply so many errors/warnings that I did
not see before.

For those developers who use local linux PC for development, can you
check yourself that xpcshell test actually trigger so many errors and
warnings that are simply NOT reported to you UNLESS you pass --verbose
flag? You may want to pass --sequential flag also.

Since there are now so many of such errors and warnings when I pass
--verbose flag to |mach xpcshell-tests|, I have no idea where to begin.
I have filed a few bugs/warnings related to calendar codes so far, but
there are simply too many to handle by me alone.
https://bugzilla.mozilla.org/show_bug.cgi?id=1667966
https://bugzilla.mozilla.org/show_bug.cgi?id=1667962

Among the errors/warnings, I have found out that there so many
lightening-related errors/warnings or configuration issue (maybe) in
xpcshell test log.
I don't think I saw many of them in June.
I urge whoever is developing calendar code under linux to check the log
by passing --verbose and --sequential to |mach xpcshell-test| locally.
Then you will see that there are many errors/warnings issued for
lightening code (and others as well, of course) and not reported by
default xpcshell log on try-comm-central server.

Too bad xpcshell-test framework does not simply fail such tests that
produce errors in the log.

In any case, these errors and warnings are hidden in xpcshell test log
on tryserver unless you figure out how to pass --verbose flag to it.
And unless --sequential flag is also supplied, the resulting log is hard
to decipher.

TIA

Chiaki

[ISHIKAWA,chiaki]

I have uploaded the crude SUMMARY of xpcshell test log of C-C TB when I
ran the test with --verbose and --sequential.

It is at the following URL.:

https://paste.mozilla.org/7coyptJM

Obviously, there is a configuration issue regarding timezone (I live in
Japan and so the OS uses Japanese timezone.) But there is an issue
regarding a timezone for Berline, also.I wonder why.

There are verbose lines that are dumped by local patch for debugging
purposes. You won't see them when you try the test with --verbose and
--sequential.

But you get the idea that there *ARE* many warning/error lines that are
not printed by xpcshell tests usually (without --verbose) on, say,
try-comm-central.

I ran the tests sequentially by modules.
"By modules", I mean I invoked tests by specifying unit test directory
one at a time for debugging purposes. And it was also important to
run xpcshell test under valgrind for checking memory errors.
(Running test by modules worked rather well until June.
However, recently, xpcshell.ini seems to have a feature to include other
files as part of its specification, and
the xpcshell.ini files got split up in some logical manner. Good for
maintenance purposes maybe.
However, my local script to run the tests sequentially by modules
now fails or rather tries to invoke a few non-meaningful test
directories because the shell script tries to locate the test
directories by simply looking for xpcshell*.ini files under C-C ./comm
subdirectory.Oh well.)


The script to summarize the log was originally used for tidying up |make
mozmill| log for quite someitime (maybe 5 years or so). It worked very
well. However, it has been modified (still incompletely and not so
satisfactorily) to work with the log of |mochitest| since January, 2020.

I only began checking the "--verbose" and "--sequential" output of
xpcshell test since December 2020 and so the xpcshell summary is not as
clean or meaningful as I (and presumably other developers) want it to be.

Enjoy.

Chiaki

[ISHIKAWA,chiaki]

On 2020/09/30 7:26, ISHIKAWA,chiaki wrote:
> On 2020/09/29 20:28, ISHIKAWA,chiaki wrote:

I have no idea why the text lines are wrapped in the paste bin page.

BTW, there *ARE* some frivolous and not so frivolous errors/warnings not
shown in the uploaded log.
I applied local patches to reduce such noise. However, we have so many
errors/warnings still. That is why I called for help to clean up the
current state of the affairs.

TIA

Chiaki