In addition, I have made a production release of MozillaBuild, version 1.0.
The MozillaBuild package [1] is now the recommended build environment for
all Mozilla developers on Windows. There were a few bugs with the 1.0
releases that have now been corrected in a MozillaBuild 1.1 release
candidate [2].
After the Tinderboxes have been upgraded to use the MozillaBuild package on
trunk, I am going to be discontinuing support for the cygwin build environment.
--BDS
followup-to mozilla.dev.builds please
[1]
http://ftp.mozilla.org/pub/mozilla.org/mozilla/libraries/win32/MozillaBuildSetup-1.0.exe
[2] http://benjamin.smedbergs.us/tests/MozillaBuildSetup-1.1.exe
Does msys still require Unix line breaks? This is a deal-breaker for me.
--
James Ross <sil...@warwickcompsoc.co.uk>
ChatZilla and Venkman Developer
Can we access the build scripts and deploy it for the contributed builds?
Thanks,
-Alfred
Does that mean one won't be able to build branches and trunk on the same
machine as one requires cygwin and the other requires MozillaBuild/msys?
Or can they be installed next to each other without problems?
Robert Kaiser
As documented on MDC, mozillabuild works with all of our active branches.
--BDS
> Does msys still require Unix line breaks? This is a deal-breaker for me.
Yes, msys uses unix-style line breaks, and I'm sorry that's a problem for you.
You might be able to work around that using wincvs instead of the
mozillabuild builtin CVS, but I'm not sure what else that might break.
--BDS
It is for Windows only, and I do not have any plans to make a unified
package for other OSes. Most unix/linux systems come with all of our build
prerequisites by default, or they can be easily obtained from a package
manager or built from source; there is little value in a prepackaged build
solution.
> Can we access the build scripts and deploy it for the contributed builds?
The tinderbox and patcher build scripts are all available in public CVS
(mozilla/tools/tinderbox and mozilla/tools/release). There is admittedly not
a lot of documentation for how they all fit together.
--BDS
Is it advisable or necessary to uninstall CygWin as part of this?
It is unnecessary. You are of course welcome to uninstall all traces of
cygwin after you make sure the new build system works.
--BDS
I have tried your MozillaBuildSetup-1.1 together with VC8 Express and
its belonging SDK successfully, but I have made the following observations:
1. Now my build platform target identifies itself as i686-pc-mingw32
Is this correct? I thought Mingw is not really supported (?)
2. I have been unable to export the environmental variables
MOZILLA_OFFICIAL=1 and BUILD_OFFICIAL=1 unless using mk_add_options in
the .mozconfig.
Using export inside the unix shell does not work, nor using SET in the
start-msvc8.bat.
3. Finally I noticed that writing $PATH in the unix shell reports
:/d/mozilla-build/moztools/bin: No such file or directory as the last
entry in the path. I think this is a kind of a bug in MSYS as the
moztools/bin really is found during the build.
Good move. Windows is slow enough already, Cygwin just made a bad
situation worse. I've also made some fixes to MSYS recently that make it
more usable as a development environment (e.g., better propagation of
signals to native Windows apps). That made life a lot easier for us with
our testing environment too.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
Chief Architect, OpenLDAP http://www.openldap.org/project/
> 1. Now my build platform target identifies itself as i686-pc-mingw32
> Is this correct? I thought Mingw is not really supported (?)
You are perhaps mistaking the mingw/msys *environment*, which is what the
build package contains, with the mingw gcc *compiler*, which is not provided
and is only marginally supported.
> 2. I have been unable to export the environmental variables
> MOZILLA_OFFICIAL=1 and BUILD_OFFICIAL=1 unless using mk_add_options in
> the .mozconfig.
> Using export inside the unix shell does not work, nor using SET in the
> start-msvc8.bat.
When did you set the variables. Please note that you have to set them before
running configure. Changes to the environment after you run configure do not
take effect inside our build system, because we cache the configure
environment settings of those variables in autoconf.mk:
http://lxr.mozilla.org/mozilla/source/config/autoconf.mk.in#555
--BDS
http://ximbiot.com/cvs/manual/cvs-1.11.22/cvs_2.html#SEC26
I realize the Mozilla build process doesn't need or use this, but I have
come to use it in my Verbosio project for getting snippets of code from
other places. Can you help me figure out why this doesn't work in the
MozillaBuild environment?
Alex
I believe this is because the msys environment comes with an older version
of cvs than 1.11.22. I'm not sure there's much you can do about it.
--BDS
I just found a changelog for the 1.11 series, and this issue was fixed
in 1.11.1:
http://ftp.gnu.org/non-gnu/cvs/source/stable/1.11.22/NEWS.1.11.22
At present, I'm trying to figure out who could get that upgraded in the
msys side of things...
Alex
Earnie Boyd, the MSYS project lead, has gone on sabbatical, so that may
take some time. It shouldn't be too hard to spin an updated cvs binary
though. You might try asking on the msys-devel mailing list. I have
commit rights there but it would be best to coordinate thru their
mailing list.
Sorry for my screw-ups earlier. I've got it working now, but I have one
more observation:
In the beginning when configure is executing, it is issuing the command
grep -P. This is not understood by the grep (version 2.4.2) included
in the package. Later versions of grep (e.g cygwin grep 2.5.1) can
handle -P.
I do not know if this at all is important - as I said, it is an
observation of mine...
I'm going to send that note fairly soon. I keep getting the following
message:
cvs [checkout aborted]: received broken pipe signal
This issue appears to have been fixed in cvs 1.11.13.
What are the advantages of replacing autoconf?
-David
--
L. David Baron <URL: http://dbaron.org/ >
Technical Lead, Layout & CSS, Mozilla Corporation
> What are the advantages of replacing autoconf?
I discussed a little bit here:
http://benjamin.smedbergs.us/blog/2006-11-30/plans-for-the-mozilla-build-system/
The modularity and maintainability of autoconf scripts is extremely poor,
and the learning curve is very high. I believe this can be made better, and
I'd like to standardize on as few languages as possible in our build system:
python is the natural fit for many reasons including its standard library
and readability.
--BDS
I agree that the learning curve is high, but it also seems like a lot of
work to do the replacement.
I'm also a little skeptical about modularity gains -- it seems like the
modularity of the build system is pretty closely tied to the modularity
of the code, and until we're building apps on top of a distinct core,
we're going to have a lot of mess in the build system reflecting the way
each app builds the core differently, etc.
> I'm also a little skeptical about modularity gains -- it seems like the
> modularity of the build system is pretty closely tied to the modularity
> of the code, and until we're building apps on top of a distinct core,
> we're going to have a lot of mess in the build system reflecting the way
> each app builds the core differently, etc.
We are moving towards the core -> apps pretty quickly. You can already build
Firefox-on-XR and the nascent Mozilla Composer-on-XULRunner. There are plans
in the works to do the same for Camino and Sunbird, I believe. The
mailnews-carrying apps (Tbird and SeaMonkey) are harder, because the
mailnews code uses strings so badly that porting it to frozen linkage will
be massively hard. But I am definitely moving in that direction. Except for
the configure issues, we already have the ability to plug arbitrary
directories into the build system and configure with
--with-libxul-sdk=path/to/XR/dist --enable-application=myapp which will read
build.mk.
--BDS
It sure seems to me like this is going to force a lot of work on certain
projects that just don't have the manpower or inclination to rework their
build systems right now.
If a project is building with autoconf, and it "ain't broke", then why fix
it?
I guess projects that have full time people to maintain and rewrite their
makefiles will be happy about this. Full employment for build engineers.
But for projects where makefile work takes man hours directly away from
the work on the project's main deliverable, this seems like a nuisance.
--
Nelson B
> It sure seems to me like this is going to force a lot of work on certain
> projects that just don't have the manpower or inclination to rework their
> build systems right now.
>
> If a project is building with autoconf, and it "ain't broke", then why fix
> it?
>
> I guess projects that have full time people to maintain and rewrite their
> makefiles will be happy about this. Full employment for build engineers.
> But for projects where makefile work takes man hours directly away from
> the work on the project's main deliverable, this seems like a nuisance.
Nelson, which projects are you talking about? I'm not planning on touching
NSS/coreconf build any time soon, though I find its lack of proper
dependency calculation and hardcoded platform build rules somewhat
depressing. I may eventually port the NSPR build system over, but that would
be a far-future task. Because they are each independent, we can upgrade one
without mucking with the other.
--BDS
Running start-msvc8.bat also loses things that I had in my PATH. Just
another oddness that either should be fixed or documented so that people
don't spend too much time on it.
Thanks,
--Aaron
Benjamin Smedberg wrote:
> Starting next week the Mozilla build system is going to require python on
> all platforms. This will allow us to gradually convert various build scripts
> which are currently written in perl, and start hacking on an autoconf
> replacement written in python. Any python 2.3 or higher will be acceptable.
>
> In addition, I have made a production release of MozillaBuild, version 1.0.
> The MozillaBuild package [1] is now the recommended build environment for
> all Mozilla developers on Windows. There were a few bugs with the 1.0
> releases that have now been corrected in a MozillaBuild 1.1 release
> candidate [2].
>
> After the Tinderboxes have been upgraded to use the MozillaBuild package on
> trunk, I am going to be discontinuing support for the cygwin build environment.
>
Since I set HOME=USERPROFILE in cygwin as well as msys, I don't really know
that much about the default cygwin configuration. If you could write up what
msys does differently, I'd appreciate that.
> Running start-msvc8.bat also loses things that I had in my PATH. Just
> another oddness that either should be fixed or documented so that people
> don't spend too much time on it.
This is by design, because many machines have some compiler or another
configured in the default environment. I don't really want to document this
in the main Windows Build Prerequisites page, but I'd be interested in
creating a "peculiarities of MozillaBuild" page.
--BDS
It would probably be useful to have some info on working with
MozillaBuild. For example, you can put settings in ~/.profile, which
winds up in $APPDATA.
-Ted
On Windows in the previous build world, SSH keys by default were kept in
a directory called '.ssh' under your cygwin home directory. The default
name for the keys were 'id_dsa' and 'id_dsa.pub'. Msys by default
expects the SSH keys to be kept in a directory named '.ssh' under the
directory specified in the USERPROFILE environment variable. It also
expects the SSH keys to be named 'identity' and 'identity.pub'. This
difference will affect you, for example, if you are used to running
'ssh-add' with no parameters.
I don't really know that much about SSH, keys, cygwin, etc. to write
anything more detailed. I assume that the SSH mozilla page will have to
be re-written, too? -> http://www.mozilla.org/cvs-ssh-faq.html For the
new people getting write access?
--Aaron
Looking at the man page of ssh, the .ssh/identity file is used for
protocol version 1, .ssh/id_dsa for protocol version 2.
Is this a setup problem with ssh in msys?
Axel
on XP pro I installed
http://benjamin.smedbergs.us/tests/MozillaBuildSetup-1.1.exe
ran start-msvc8.bat
it apparently starts in c:\documents and settings\....
then...
cvs -d :pserver:anon...@cvs-mirror.mozilla.org:/cvsroot co
mozilla/client.mk
cvs -d :pserver:anon...@cvs-mirror.mozilla.org:/cvsroot co -r
MOZILLA_1_8_BRANCH mozilla/mail/config/mozconfig
cd mozilla
make -f client.mk checkout MOZ_CO_PROJECT=mail
yields
client.mk:421: *** The mozilla directory cannot be located in a path
with spaces.. Stop.
I don't see this problem listed at
http://developer.mozilla.org/en/docs/Mozilla_Build_FAQ#Win32-specific_questions
> client.mk:421: *** The mozilla directory cannot be located in a path
> with spaces.. Stop.
When I wrote this error message I thought it was pretty self-explanatory.
What don't you understand about it? You need to use a path without any
spaces in it. I typically use c:\builds
--BDS
the message may be clear to someone who has built before.
problem is this is a first timer following the web page's
windows-specific instructions to the letter. other than that problem
it's going OK.
I added the following changes to C:\mozilla-build\start-msvc8.bat and
called it mystart.bat
rem gary custom settings -- add my mozconfig
set MOZCONFIG=c:\moz\mozconfig.txt
rem --set some cvs stuff
SET CVSROOT=:pserver:anon...@cvs-mirror.mozilla.org:/cvsroot
SET CVS_RSH=ssh
I also changed C:\mozilla-build\msys\etc\profile.d\profile-homedir.sh
to CD to C:\moz instead of USERPROFILE
Niccolo Machiavelli concluded: "...there is nothing more difficult to
carry out, nor more doubtful of success, nor more dangerous to handle,
than to initiate a new order of things. For the reformer has enemies
in all those who profit by the old order, and only lukewarm defenders
in all those who would profit by the new order..."
Build systems - a royal pain in the ass and a constant reminder that
there is even more that we have to know.
Why can't these things just build themselves?
TIA
"Benjamin Smedberg" <benj...@smedbergs.us> wrote in message
news:1fKdne4Ne8DszXXY...@mozilla.org...
There's several reports of improved build times just from using MSYS
instead of Cygwin. See https://bugzilla.mozilla.org/show_bug.cgi?id=294122
for some history.
-d
I just did a comparison with GnuPG (clean checkout, auto*, ./configure and make)
between MSYS and Cygwin. I used GnuPG because the test wouldn't take nearly as
long as Seamonkey would.
MSYS:
real 36m3.411s
user 4m9.277s
sys 6m1.302s
Cygwin:
real 60m35.342s
user 15m29.710s
sys 12m9.897s
Around 40% faster with MSYS.
--
John P. Clizbe Inet: JPClizbe(a) tx DAWT rr DAHT con
"Be who you are and say what you feel because those who mind don't matter
and those who matter don't mind." - Dr Seuss, "Oh the Places You'll Go"
checking for make... (cached) /c/mozilla-build//make-3.81/bin/make
*** /c/mozilla-build//make-3.81/bin/make is not GNU Make. You will
not be able to build Mozilla without GNU Make.
*** Fix above errors and then restart with "make -f
client.mk build"
make[1]: *** [configure] Error 1
make[1]: Leaving directory `/d/mozillaXulRunnerSource/mozilla'
make: *** [/d/mozillaXulRunnerSource/mozilla/obj-i686-pc-mingw32/
Makefile] Error 2
Oh, what to do now. I'm so very frustrated already with all the *nix
type rigamarole....
I have no clue what this is... it's certainly not a common problem. Can you
check config.log and see what the log says about make?
--BDS
s%@MAKE@%/c/mozilla-build//make-3.81/bin/make%g
There is no such folder, speaking in Windows terms anyhow.
As for the config.log, the last four lines in it are:
configure:3571: checking for zip
configure:3612: checking for makedepend
configure:3647: checking for xargs
configure:3898: checking for make
Hmmmmm..... After typing/pasting the above, I started snooping around
a bit more. I discovered in the config.cache the same apparently-
invalid file path to make... so I just deleted it. Got a lot
farther... but NOW I get an alert <<xpidl.exe - Unable To Locate
Component... This application has failed to start because
libIDL-0.6.dll was not found. Re-installing the application may fix
this problem. <OK> >>. I reinstalled already which is when I started
getting the "make is not GNU" problem and wound up with the new v1.1
mozilla-build.
Signed,
Frustrated TunerGeek (Paul)
(and THANKS!)
I did not yet have the time to read the whole thread, neither
the links, but I'm totally against writing an completely own
build system, just for mozilla, and not in an still quite
some exotic language like python (perl wouldn't be such bad).
Yes, autoconf+friends have their problems, and I you google around
you'll learn, I'll be the last one praising autotools.
The big question for me is: are that really autoconf problems
or just dirty hacks in the .in files ?
Well, I wasn't in favour of autoconf in the xorg-modularize
project (I'm working on my own buildsystem with an completely
different approach, but it's not yet ready for production:
http://treebuild.metux.de/), but now it works quite good.
Far from being perfect, but quite good. Autoconf requires
great care, because it's just an bunch of macros and scripts,
without an strictly defined model (which ie. treebuild is
focusing on).
An really bad point of autoconf is that it allows the dev
to write shit code very easily. Yes, and it's done too much.
But will an self-written, mozilla-only buildsystem do better ?!
The really good point of autoconf is that it's an standard.
Using standard buildsystems makes packagers happy.
Why not an own buildsystem ?
Simply because it will be only for mozilla. Each new buildsystem
requires people to learn a new one. And if I look through the
current code structure, I really doubt it will be easy to learn.
Let's tell me some little episode from my daily work:
I'm supplying images for embedded systems, and therefore
crosscompiling in an sysroot'ed environment is an MUST. Builds
have to be really clean and deterministic. Manual work is not
just a waste of time, but an big risk of errors.
Now I had to do some fixes in OpenSSL's buildsystem, just add
a few options to leave out some stuff. They wrote their own
configure script in perl. It's far from being easy to understand
if you're not an openssl-dev. And it's totally specific to
OpenSSL. With some stardard buildsystem, ie. autoconf, the
changes would have been done in a fews mins, but understanding
this perl script took several hours.
If we really want to do better than autoconf, we should *first*
develop some new buildsystem, get it running quite stable,
and *then* migrate mozilla to it. This takes time, and it
requires *good* concepts. Evryting else is useless redundance,
wasted time.
For example we could take treebuild :)
*BUT*! Treebuild is an completely different paradigm. It is
no makefile generator, no macro system or something like that.
Treebuild is purely descriptive: it models the *structure*
of the software (ie. module hierachy, etc) and completely
takes the build process out of the dev's hands. It solves a
lot of problems, ie. crosscompiling, specific build layouts,
etc are not in the scope of the devs anymore.
Moving to this paradigm requires an careful refactoring of the
whole code structure. Evrything has to be strictly modularized,
with clear (one-way!) dependencies.
And now we're at the real point. No matter if treebuild will ever
be an option, if we stay at autoconf or use something else, the
real problem is monolithic structure. The tree grows and grows,
and each day it gets harder and harder to maintain. And some day
we're at the point where meta works eat up much more time than
actual development.
Xorg learned this lesson and modularized. It went really good
with that.
So my vote is: stay at autoconf for now, do heavy modularization
instead. Once we got some modules moved out cleanly, we can think
about using some different buildsystem for them, separately.
The first steps could be, completely kicking off evrything what's
not mozilla specific. For example third party packages like libjpeg.
Then fork off the extension API stuff and import this to the
monolithic mozilla. So we can easily move out extensions, one
by one.
This way we can do an modularization quite easily, step by step,
wihtout breaking too much.
Just my $0.02
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
I was able to use xcopy in my scripts in cygwin.
so I am changing
xcopy "C:\CookBook\MySidebar.htm" "C:\CBSOURCE
\gj53run\jar\content\gj53run\CookBook\*.*" /y
to
cp "C:/CookBook/MySidebar.htm" "C:/CBSOURCE/
gj53run/jar/content/gj53run/CookBook"
But
xcopy "C:\CookBook\XXSoftwareTools\GraphicWidgets\*.*" "C:\CBSOURCE
\gj53run\jar\content\gj53run\CookBook\XXSoftwareTools\GraphicWidgets
\*.*" /y
to this
cp "C:/CookBook/XXSoftwareTools/GraphicWidgets/*" "C:/CBSOURCE/
gj53run/jar/content/gj53run/CookBook/XXSoftwareTools/GraphicWidgets"
does not work
and
xcopy "C:\CookBook\XXSoftwareTools\js\*.js" "C:\CBSOURCE
\gj53run\jar\content\gj53run\CookBook\XXSoftwareTools\js\*.*" /y
to
cp "C:/CookBook/XXSoftwareTools/js/*.js" "C:/CBSOURCE/
gj53run/jar/content/gj53run/CookBook/XXSoftwareTools/js"
does not work.
I know this is trivial, but it some times takes me a long time to
figure out very simple unix stuff.
I also got used to running dos2unix on everything as a precaution;
however, it appears to be missing in msys and I saw something about
the 2s being broken. Does anybody know of anything in the mozilla-
build that will do the same thing?
The quotes around your filenames are giving you your trouble;
they prevent shell expansion of the filenames; eg
$ ls "*"
ls: *: No such file or directory
Remove the quotes and things should work with cp. You don't need them.
If you're using double quotes to get around spaces in directory or file names,
try using 8.3 style names or escaping the spaces with \.
cp --help will give you the options to use with cp
> I also got used to running dos2unix on everything as a precaution;
> however, it appears to be missing in msys and I saw something about
> the 2s being broken. Does anybody know of anything in the mozilla-
> build that will do the same thing?
I can't recall ever needing to run dos2unix when building in the last four
years, but... "Google Is Your Friend™". Here they are in perl:
dos2unix
#!/usr/bin/perl -pi
s/\r\n/\n/;
unix2dos
#!/usr/bin/perl -pi
s/\n/\r\n/;
You may have to adjust your path to perl
--
John P. Clizbe Inet: JPClizbe (a) tx DAWT rr DAHT con
GingerBear Networks PGP/GPG KeyID: 0x608D2A10