Forcing alphabetical order in moz.build files

[Robert O'Callahan]

I have a request ... can we require lists in moz.build files to be in
alphabetical order, and actually enforce with some build-system check? I'm
always annoyed by Makefiles where lists are sometimes unordered and it's
hard to find items and know where to add items.

Rob
--
q“qIqfq qyqoquq qlqoqvqeq qtqhqoqsqeq qwqhqoq qlqoqvqeq qyqoquq,q qwqhqaqtq
qcqrqeqdqiqtq qiqsq qtqhqaqtq qtqoq qyqoquq?q qEqvqeqnq qsqiqnqnqeqrqsq
qlqoqvqeq qtqhqoqsqeq qwqhqoq qlqoqvqeq qtqhqeqmq.q qAqnqdq qiqfq qyqoquq
qdqoq qgqoqoqdq qtqoq qtqhqoqsqeq qwqhqoq qaqrqeq qgqoqoqdq qtqoq qyqoquq,q
qwqhqaqtq qcqrqeqdqiqtq qiqsq qtqhqaqtq qtqoq qyqoquq?q qEqvqeqnq
qsqiqnqnqeqrqsq qdqoq qtqhqaqtq.q"

[Gregory Szorc]

On 4/16/2013 10:15 PM, Robert O'Callahan wrote:
> I have a request ... can we require lists in moz.build files to be in
> alphabetical order, and actually enforce with some build-system check? I'm
> always annoyed by Makefiles where lists are sometimes unordered and it's
> hard to find items and know where to add items.

Theoretically, yes. And I do empathize with the desire to have them
alphabetical.

However, it's not the easiest to implement. And it's all because of
assignments behind condition blocks.

A few months ago I experimented with Python's built-in AST module [1] to
statically analyze moz.build files. I didn't make much progress after a
few hours and decided my time was better spent elsewhere.

We have bug 774381 [2] open to implement a style checking tool for
moz.build files. However, because it's non-trivial and because I think
time is better spent on making the build faster (making the export tier
fully non-recursive is within grasp and experimenting with different C++
compilation techniques like PCH should be attainable once CPPSRCS and
friends are in moz.build), I doubt you'll see any of the usual suspects
work on it any time soon. But, if someone authors a patch [hopefully
with decent test coverage], I'll gladly review it.

If it's any consolation, the tool we're using to automatically convert
Makefile.in variables to moz.build files sorts lists. I hope enough
people realize the lists are sorted and follow the convention until a
tool can enforce it.

[1] http://docs.python.org/2/library/ast.html
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=774381

[Ms2ger]

On 04/17/2013 07:15 AM, Robert O'Callahan wrote:
> I have a request ... can we require lists in moz.build files to be in
> alphabetical order, and actually enforce with some build-system check? I'm
> always annoyed by Makefiles where lists are sometimes unordered and it's
> hard to find items and know where to add items.

Fully agree, with the exception that I'm not sure this is a safe
transformation for *DIRS.

--
Ms2ger

[Robert Kaiser]

Gregory Szorc schrieb:

> Theoretically, yes. And I do empathize with the desire to have them
> alphabetical.

As Ms2ger notes, we probably cannot have that for *DIRS, though. I know
at least one case where this is crucial, and that's Mac packaging. I
added a comment for that a long time ago in SeaMonkey that is now in
http://mxr.mozilla.org/comm-central/source/suite/moz.build#32 but the
same is true for Firefox and others.

Robert Kaiser

[Ralph Giles]

On 13-04-17 8:40 AM, Ms2ger wrote:

> Fully agree, with the exception that I'm not sure this is a safe
> transformation for *DIRS.

It certainly isn't for add_tier_dir.

-r

[Mike Hommey]

Also, any place where PARALLEL_DIRS can't be used probably has some
assumptions on the order things are being traversed.

Typical example:
http://mxr.mozilla.org/mozilla-central/source/mozglue/moz.build

"build" needs to be last.

I'm not saying this can't be fixed, it will just require a lot more
work on the build system.


Mike

[Robert O'Callahan]

Good point. That's fine though, we can just not care about that.

Likewise we can not care about conditional blocks. I would simply say that
if we have a literal array being appended to one of a known set of
order-independent variables (including EXPORTS, CPPSRCS, etc), its elements
should be in sorted order.

[Gregory Szorc]

On 4/17/13 2:37 PM, Robert O'Callahan wrote:
> On Thu, Apr 18, 2013 at 3:40 AM, Ms2ger <ms2...@gmail.com> wrote:
>

> Good point. That's fine though, we can just not care about that.
>
> Likewise we can not care about conditional blocks. I would simply say that
> if we have a literal array being appended to one of a known set of
> order-independent variables (including EXPORTS, CPPSRCS, etc), its elements
> should be in sorted order.

That's a fair compromise and should be relatively easy to implement.
I've filed bug 863069.

[Gregory Szorc]

On 4/16/13 10:15 PM, Robert O'Callahan wrote:
> I have a request ... can we require lists in moz.build files to be in
> alphabetical order, and actually enforce with some build-system check? I'm
> always annoyed by Makefiles where lists are sometimes unordered and it's
> hard to find items and know where to add items.

This just landed in inbound. For variables where order isn't important
(pretty much everything except *DIRS), every incoming list append or
replacement must be sorted by byte order of the UTF-8 encoding of the
string (typically strings are ASCII). If it isn't, you'll see an error
message like:

ERROR PROCESSING MOZBUILD FILE
==============================

The error occurred while processing the following file:

/Users/gps/src/firefox/dom/devicestorage/moz.build

The error was triggered on line 15 of this file:

'DeviceStorage.h',

An error was encountered as part of executing the file itself. The error
appears to be the fault of the script.

The error as reported by Python is:

['UnsortedError: An attempt was made to add an unsorted sequence to
a list. The incoming list is unsorted starting at element 0. We expected
"DeviceStorage.h" but got "nsDeviceStorage.h"\n']


As you can see, it tells you exactly what it expects to aid you in
fixing the problem.