TB Tryserver WIN32 Build question: merge failure of the change to M-C portion of the source.

[ishikawa]

I have a question regarding submitting win32 build for Thunderbird TryServer.

I have created and tested patches for COMM-CENTRAL source tree
locally on my own PC (linux 32bits: Debian GNU/Linux)

Once they compile clean and pass "make mozmill" test locally more or less,
I submit the patches to Thunderbird Tryserver and have them compiled and
executed for linux (32bits and 64bits).

Linux builds are OK.

However, I have been unable to do so for changes made to m-c
(mozilla-central portion) of the files for WIN32 build.

The changes made to mozilla-central portion of the files fail to be
merged for WIN32 build. So the compilation never started for
WIN32 build.

Today, I bit the bullet and took out the patches created for M-C
portion of the files, and then FINALLY(!) the compilation started.
(Well, it failed because I used bzero() without declaring it. VC
complained. But that is a great progress for me although not being
able to use the patch created for m-c portion of the files is
discouraging.
To be honest, I think the code is logically OK since linux32, and
linux64 work but we may hit these type of compiler compatibility
issues. So I wonder if I can make the win32 build work for changes in
m-c part.)

Has anyone seen the problem of the patches to M-C portion of the
COMM-CENTRAL not accepted by Thunderbird TryServer because the patches
fail although linux32 and linux64 build in the same submission work as expected?

TIA


BACKGROUND and my guess where something is incorrect:

For those who are unfamiliar with comm-central source tree,
the following may have a bearing on this issue. So I will explain.

Comm-central (c-c) contains the source for thunderbird (and
spidermonkey or whatever), I will simply call it MAILER in this post.
But it requires the CORE components available in Mozilla-Central (m-c)
source tree.
So c-c ADDS m-c under its top-level directory as "./mozilla"
subdirectory.
Everything in m-c is under this "./mozilla" directory.
(So I think we can build Firefox under this mozilla directory. I have
not tried, but mozilla web page suggests so.)

We must pay attention to the following issue carefully: Mercurial
repositories for c-c's MAILER portion and for this "m-c as
part of c-c" are independent and separate.

There are ".hg" subdirectory "./.hg" immediately in the top-level
directory of c-c source tree to manage c-c (MAILER) source tree, AND
there is another "./mozilla/.hg" subdirectory that manages the "m-c as
part of c-c" source tree.

This means that "hg status", if you issue it in the c-c's MAILER
portion of the source tree, say at the top-level and its
subdirectories such as ./mail, ./mailnews, etc., it will print ONLY
the status of c-c's MAILER portion of the source tree. It will not
print out the modifications made to files in "m-c as part of c-c"
source tree.

The reverse is also true.

If I am under "./mozilla" subdirectory and its subdirectories ("m-c as
part of c-c"), "hg status" will print out only the modification under
the ./mozilla subdirectory and no changes made to the MAILER portion (say,
./mail, ./mailnews, etc.) of "c-c" are printed.

We have to be careful when creating patches so that we will not leave
out the changes made in the other repository.

C-C's client.py takes care of this issue when updating the source
files from c-c repository, by calling hg commands for these
independent mercurial repositories multiple times as necessary.

python client.py update

First it updates c-c's MAILER portion of the local repository, then
it descends into "./mozilla" subdirectory and then reissues
hg commands to update the "m-c as part of c-c" repository.

Enough for a local management (on one's local PC).
[Well, after writing up to this point, I realize that there ought to be
a writeup about these pitfalls somewhere in mozilla's web documentation for
thunderbird source tree, etc. I have found out about the above only after I
fetched the source from comm-central and began fixing bugs.]

This setup makes it a little tricky to submit a job to Thunderbird
Tryerver.

After all, at the superficial level during Thunderbird TryServer job
submission, we only say we have made changes to the top-level "c-c" (MAILER)
source tree.

How do we tell the Thunderbird TryServer we have made changes to the
files in "m-c as part of c-c" source tree?

The trick used by Thunderbird Tryserver currently is as follows.: I am
quoting this from
https://wiki.mozilla.org/Thunderbird/Infrastructure/TryServer

--- quote begin
Pushing mozilla-central patches

There's two steps to this process.

Edit client.py-args: remove the hgtool option, add --apply-patches
Example patch here
Do this as a separate item in your mq and you can use whenever you want
Note: hgtool option removal is because I'm not convinced if applying
a patch affects the hg share or not
Take your mozilla-central patch and name it something like:
mozilla-<anything>.patch
hg add your patch
Then hg commit your changes, or use hg qnew for a new item on your patch
queue
Push your patches to try-comm-central

The client.py code will automatically apply your patch when the code is
checked out. Any apply failures will cause the builds to be aborted.

--- end quote

According to the above, we create the patch from "m-c as part of c-c"
source tree, created in the "./mozilla" subdirectory and then add the
patch with a file name that looks like "mozilla-some-string.patch" in
the top-level directory of c-c (I assume so and tested), and record the
addition of this file as the hg repository of "c-c" (MAILER) itself,
and tell the Thunderbird tryserver about this addition.
Then it will cleverly figure out this mozilla-some-string.patch is a
patch to be applied to "m-c as part of c-c" repository.
A clever hack, indeed.

Unfortunately, for me, this clever hack did not seem to work for
WIN32 build. It seems to work for linux build (32bits, 64bits). The
changes to file(s) "m-c as part of c-c" file caused merge error and
the job is aborted.

Any tips will be appreciated.

(If I had screwed up the patch creation. I don't think
linux32 and linux64 build in the same job submission would have worked. So I
think the problem is WIN32 build specific.)

From the recent posts, I believe Joshua Cranmer is working on to merge
the source trees (c-c and m-c) into one. That will be great. We can avoid
these problems I am facing.


PS: I like thunderbird. This is one of the few portable mail clients that
run on different OSs today. I wish it can be made more robust.
My previous e-mail client crashed maybe less than 10 times in 15 years.
(To be honest, it never crashed on its own. I used RMAIL mode of Emacs and
emacs crashed due to compiler code generation error and such about a dozen
times during that time. RMAIL never crashed on its own and never lost a
single e-mail even under dire conditions such as file system overflow, etc.
It was quite a feat.)
I have seen TB crash more than half a dozen times since last October. This
is way too often for a mail-client, IMHO.

[ishikawa]

(Sorry for posting this twice in mozilla.dev.builds. I forgot to add
mozilla.dev.platform. Iput mozilla.dev.platform as followup-to newsgroup. TIA)

I have a question regarding
submitting win32 build for Thunderbird TryServer.

I have created and tested patches for COMM-CENTRAL source tree

locally on my own PC.

Once they compile clean and test passes, I submit the patches to

Thunderbird Tryserver and have them compiled and executed for linux
(32bits and 64bits).

Linux versions are OK.

However, I have been unable to do so for changes made to m-c
(mozilla-central portion) of the files for WIN32 build.

The changes made to mozilla-central portion of the files fail to be

merged even for WIN32 build. So the compilation never started for

WIN32 build.

Today, I bit the bullet and took out the patches created for M-C
portion of the files, and then FINALLY(!) the compilation started.
(Well, it failed because I used bzero() without declaring it. VC
complained. But that is a great progress for me although not being
able to use the patch created for m-c portion of the files is
discouraging.
To be honest, I think the code is logically OK since linux32, and
linux64 work but we may hit these type of compiler compatibility
issues. So I wonder if I can make the win32 build work for changes in
m-c part.)

Has anyone seen the problem of the patches to M-C portion of the
COMM-CENTRAL not accepted by Thunderbird TryServer because the patches

fail although linux32 and linux64 work as expected?

TIA


BACKGROUND and my guess where something is incorrect:

For those who are unfamiliar with comm-central source tree,
the following may have a bearing on this issue. So I will explain.

Comm-central (c-c) contains the source for thunderbird (and
spidermonkey or whatever), I will simply call it MAILER in this post.
But it requires the CORE components available in Mozilla-Central (m-c)
source tree.
So c-c ADDS m-c under its top-level directory as "./mozilla"
subdirectory.
Everything in m-c is under this "./mozilla" directory.
(So I think we can build Firefox under this mozilla directory. I have
not tried, but mozilla web page suggests so.)

We must pay attention to the following issue carefully: Mercurial

repositories for c-c's MAILER portion and mercurial for this "m-c as

part of c-c" are independent and separate.

There are ".hg" subdirectory "./.hg" immediately in the top-level
directory of c-c source tree to manage c-c (MAILER) source tree, AND
there is another "./mozilla/.hg" subdirectory that manages the "m-c as
part of c-c" source tree.

This means that "hg status", if you issue it in the c-c's MAILER
portion of the source tree, say at the top-level and its
subdirectories such as ./mail, ./mailnews, etc., it will print ONLY
the status of c-c's MAILER portion of the source tree. It will not
print out the modifications made to files in "m-c as part of c-c"
source tree.

The reverse is also true.

If I am under "./mozilla" subdirectory and its subdirectories ("m-c as
part of c-c"), "hg status" will print out only the modification under
the ./mozilla subdirectory and no changes made to the MAILER portion

of "c-c" are printed.

We have to be careful when creating patches so that we will not leave
out the changes made in the other repository.

C-C's client.py takes care of this issue when updating the source
files from c-c repository, by calling hg commands for these
independent mercurial repositories multiple times as necessary.

python client.py update

First it updates c-c's MAILER portion of the local repository, then
it descends into "./mozilla" subdirectory and then reissues
hg commands to update the "m-c as part of c-c" repository.

Enough for a local management (on one's local PC).

This setup makes it a little tricky to submit a job to Thunderbird

Tryserver.

After all, at the superficial level during Thunderbird TryServer job

submission, we only tell the thunderbird TryServer we have made

patch with a file name that looks like "mozilla-some-string.patch" (in
the top-level directory of c-c, I assume and tested), and record the

addition of this file as the hg repository of "c-c" (MAILER) itself,
and tell the Thunderbird tryserver about this addition.
Then it will cleverly figure out this mozilla-some-string.patch is a
patch to be applied to "m-c as part of c-c" repository.

A clever hack.

Unfortunately, for me, this clever hack does not seem to work for

WIN32 build. It seems to work for linux build (32bits, 64bits). The
changes to file(s) "m-c as part of c-c" file caused merge error and
the job is aborted.

Any tips will be appreciated.

From the recent posts, I believe Joshua Cranmer is working on to merge

the source trees into one. That will be great. We can avoid these
problems I am facing.