I'm trying to build the NSS libraries on Mac OS 10.6.5, on a 64-bit machine, using the instructions here:
https://developer.mozilla.org/en/NSS_reference/Building_and_installing_NSS/Build_instructions
but using the source code from this archive rather than the cvs checkout:
http://ftp.mozilla.org/pub/mozilla.org/security/nss/releases/NSS_3_12_9_RTM/src/nss-3.12.9-with-nspr-4.8.7.tar.gz
I can extract the archive fine, but when I try the "make nss_build_all" step, the build exits with an error:
> ncraike@ncraikework 15:27:40 ~/Installs/nss-3.12.9/mozilla/security/nss
> $ make nss_build_all
> cd ../coreconf ; make
> ...
> drbg.c: In function ‘RNG_RandomUpdate’:
> drbg.c:516: error: size of array ‘arg’ is negative
> make[3]: *** [Darwin10.5.0_DBG.OBJ/Darwin_SINGLE_SHLIB/drbg.o] Error 1
> make[2]: *** [libs] Error 2
> make[1]: *** [libs] Error 2
> make: *** [libs] Error 2
The line in question (line 516 of mozilla/security/nss/lib/freebl/drbg.c) seems to be an assertion that a particular type is of an expected size:
> PR_STATIC_ASSERT(sizeof(size_t) <= 4);
If I write a quick C program to print sizeof(size_t), it prints "8". Should this assertion be failing? Is there some problem with building on a 64-bit machine? From the build instructions:
> By default, NSS builds for the 32-bit environment on all platforms except Alpha/OSF1.
Is this the case for Mac OS X? Is it possible to build the 32-bit version on a 64-bit Mac? The Mac OS X man page for gcc describes an "Apple only" option -arch:
> -arch arch
> Compile for the specified target architecture arch. The allowable values are
> i386, x86_64, ppc and ppc64.
...but same error occurs (the failing assertion) if I run the make script with this option, as below:
> ncraike@ncraikework 15:54:40 ~/Installs/nss-3.12.9/mozilla/security/nss
> $ CC="gcc -arch i386" CXX="g++ -arch i386" make nss_build_all
If I set the USE_64 environment variable:
> ncraike@ncraikework 16:00:09 ~/Installs/nss-3.12.9/mozilla/security/nss
> $ USE_64=1 make nss_build_all
...the build does succeed without errors (creating a mozilla/dist/Darwin10.5.0_64_DBG.OBJ directory), but I assume this produces the 64-bit version. I'd eventually like to include the built libraries in software deployed an any machine running Mac OS X 10.6, which may be Intel 32-bit or 64-bit machines, so a build which works on both architectures would be preferred (either 32-bit or some kind of universal binary, I assume).
Should I be able to build the 32-bit version of NSS? Should this assertion be failing?
I'm also trying to build python-nss on Mac OS X, with some issues, and this may be something I ask about later. I also have a Stack Overflow question about the two issues as a whole, if this interests anyone (or is easier to read):
http://stackoverflow.com/questions/4733108/how-do-i-build-python-nss-and-nss-for-mac-os-x
I'm more experienced with Python than C, so if my problem is a simple build config error, sorry for the trouble.
Any help is greatly appreciated.
Regards,
Nathan Craike
> I'd eventually like to include the built libraries in software
> deployed an any machine running Mac OS X 10.6, which may be Intel
> 32-bit or 64-bit machines, so a build which works on both
> architectures would be preferred (either 32-bit or some kind of
> universal binary, I assume).
That's certainly doable, but I don't think the NSS build system has
support for building universal binaries (you'd have to fiddle with lipo
yourself).
> Should I be able to build the 32-bit version of NSS? Should this
> assertion be failing?
I also ran into this previously, and added "-DNS_PTR_GT_32" to the
"OS_REL_CFLAGS = -Di386" line in Darwin.mk. Not sure if that's the
proper way of fixing things, though.
Kaspar
My guess is that no one has set the compiler flags correct for building
32-bit binaries on Mac 64.
>> Should I be able to build the 32-bit version of NSS? Should this
>> assertion be failing?
> I also ran into this previously, and added "-DNS_PTR_GT_32" to the
> "OS_REL_CFLAGS = -Di386" line in Darwin.mk. Not sure if that's the
> proper way of fixing things, though.
My guess is, no. Intel platforms do not have greater then 32 bit
pointers when running in 32 bit mode, so I suspect you are building 64
bit binaries. NS_PTR_GT_32 or NS_PTR_LE_32 are variables which you set
if your native build is a 32 build but your native pointer is actually
later or your native build is 64 bit, but the pointer is less. I think
you just wound up producing 64 bit binaries with the 32 bit directory names.
If you have the proper changes to coreconf to produce 32 bits on a Mac
64 bit system, we would certainly be interested.
bob
> Kaspar
I know about this problem. Apple's compiler generates 64-bit binaries
by default now. So passing USE_64=1 to make is one solution to the
build problem, but it produces a 64-bit build.
To generate a 32-bit build, your workaround of passing CC="gcc -arch
i386" CXX="g++ -arch i386" to make should work. I don't know why it
doesn't.
Wan-Teh
The correct command is to pass CC and CXX (or rather, CCC, which is
the variable name used by NSS) to make as make variables as opposed to
environment variables:
$ make nss_build_all CC="gcc -arch i386" CCC="g++ -arch i386"
> If I set the USE_64 environment variable:
>
>> ncraike@ncraikework 16:00:09 ~/Installs/nss-3.12.9/mozilla/security/nss
>> $ USE_64=1 make nss_build_all
Similarly, this should be
$ make nss_build_all USE_64=1
although passing USE_64=1 as an environment variable also works.
Wan-Teh
bob
This is correct.
It is also possible to build for two arches in one pass, for example,
gcc -arch i386 -arch x86_64. GCC still compiles each file twice in
this method. This method does have one limitation -- you can't define
an arch-specific macro, such as -Di386 or -DNSS_X64 on the compiler
command line.
Wan-Teh
I just realized that my message from 22 January never made it to Google
Groups (most likely due to the presence of attachments), so I'm
repeating some information here. [1]
The patches previously attached have now been filed as
https://bugzilla.mozilla.org/show_bug.cgi?id=645460 (NSS) and
https://bugzilla.mozilla.org/show_bug.cgi?id=645459 (NSPR).
> Building universal binaries for NSS in a single step, on the other hand,
> is something which doesn't work properly yet. In theory, this could be
> achieved by setting CC="gcc -arch x86_64 -arch i386" with the make
> command (which instructs GCC to compile separate versions and assemble
> them with lipo afterwards), but this will currently fail at drbg.c:
>
>> gcc -arch x86_64 -arch i386 -o Darwin10.6.0_OPT.OBJ/Darwin_SINGLE_SHLIB/drbg.o -c -O2 -fPIC -Di386 -Wall -fno-common -pipe -DDARWIN -DHAVE_STRERROR -DHAVE_BSD_FLOCK -DXP_UNIX -DSHLIB_SUFFIX=\"dylib\" -DSHLIB_PREFIX=\"lib\" -DSHLIB_VERSION=\"3\" -DSOFTOKEN_SHLIB_VERSION=\"3\" -DRIJNDAEL_INCLUDE_TABLES -UDEBUG -DNDEBUG -DNSS_ENABLE_ECC -DUSE_UTIL_DIRECTLY -DMP_API_COMPATIBLE -I../../../../dist/Darwin10.6.0_OPT.OBJ/include -I../../../../dist/public/nss -I../../../../dist/private/nss -Impi -Iecl drbg.c
>> drbg.c: In function 'RNG_RandomUpdate':
>> drbg.c:516: error: size of array 'arg' is negative
>
> Looking at the current logic in drgb.c
>
>> #if defined(NS_PTR_GT_32) || (defined(NSS_USE_64) && !defined(NS_PTR_LE_32))
>> PR_STATIC_ASSERT(sizeof(size_t) > 4);
>> #else
>> PR_STATIC_ASSERT(sizeof(size_t) <= 4);
>> #endif
>
> this is due to GCC trying to compile for x86_64, but without NSS_USE_64
> being set. Maybe this could be addressed by modifications to drbg.c?
> (Adding USE_64=1 to the make command won't help, because that makes the
> i386 part fail.)
On 24.1.11 22:08, Wan-Teh Chang wrote:
> It is also possible to build for two arches in one pass, for example,
> gcc -arch i386 -arch x86_64. GCC still compiles each file twice in
> this method. This method does have one limitation -- you can't define
> an arch-specific macro, such as -Di386 or -DNSS_X64 on the compiler
> command line.
That's not completely true, as I found out in the meantime. There is a
solution to this problem - the "Xarch" option (Apple-specific gcc
extension):
> -Xarch_arch option
> Apply option to the command line for architecture arch. This is
> useful for specifying an option that should only apply to one
> architecture when building a "universal" binary. (APPLE ONLY)
With the following make command, it's actually possible to compile
3-architecture universal binaries in a single step:
> make nss_build_all CC="gcc -arch x86_64 -Xarch_x86_64 -DNSS_USE_64\ -DNSS_X64\ -DNSS_X86_OR_X64 -arch i386 -Xarch_i386 -DNSS_X86\ -DNSS_X86_OR_X64 -arch ppc" BUILD_OPT=1 NSS_ENABLE_ECC=1 CPU_ARCH=dummy
("CPU_ARCH=dummy" is kind of a hack, currently - it's only purpose is to
prevent coreconf from applying the x86 related settings in the ifeq...
CPU_ARCH block. It will add -Dppc to the CC command, but this doesn't do
any harm, I think. Note that the backslashes are required to protect the
following blanks.)
This will give you fat binaries in the respective dist/ subtree:
> $ file dist/Darwin10.7.0_OPT.OBJ/bin/certutil
> dist/Darwin10.7.0_OPT.OBJ/bin/certutil: Mach-O universal binary with 3 architectures
> dist/Darwin10.7.0_OPT.OBJ/bin/certutil (for architecture x86_64): Mach-O 64-bit executable x86_64
> dist/Darwin10.7.0_OPT.OBJ/bin/certutil (for architecture i386): Mach-O executable i386
> dist/Darwin10.7.0_OPT.OBJ/bin/certutil (for architecture ppc7400): Mach-O executable ppc
Depending on how important support for universal build is considered,
this could be added as a further option to Darwin.mk?
Kaspar
[1] See
http://thread.gmane.org/gmane.comp.mozilla.crypto/15825/focus=15831 for
an archived copy
As for how to roll the below into Darwin.mk, I haven't tested it, but I'd suggest adding pseudo-platform support to Darwin.mk before or after the block that current starts at line 52:
ifneq (,$(findstring darwin-universal,${CPU_ARCH}))
$(warning "${CPU_ARCH} is a universal target with experimental support. Cross-check settings against new Makefiles, and do not use with a non-Apple gcc. Some architectures may no longer be supported by later Apple releases of gcc.")
UNIVERSAL_ARCHES = x64 x86 ppc64
ARCHES = $(subst +, ,$(patsubst darwin-universal-%,%,${CPU_ARCH}))
UNKNOWN_ARCHES = $(filter-out ${UNIV_ARCHES},${ARCHES})
ifneq (,${UNKNOWN_ARCHES})
$(error "Do not know how to make universal arches ${UNKNOWN_ARCHES}")
ifneq (,$(filter x86,${ARCHES}))
CC += -arch i386
OS_REL_CFLAGS += -Xarch_i386 -DNSS_X86 \
-Xarch_i386 -DNSS_X86_OR_X64
endif
ifneq (,$(filter x64,${ARCHES}))
CC += -arch x86_64
OS_REL_CFLAGS += -Xarch_x86_64 -DNSS_USE_64 \
-Xarch_x86_64 -DNSS_X64 \
-Xarch_X86_OR_X64 -DNSS_X86_OR_X64
endif
ifneq (,$(filter ppc64,${ARCHES}))
CC += -arch ppc64
OS_REL_CFLAGS += -Xarch_ppc64 -Dppc
endif
endif
> --
> dev-tech-crypto mailing list
> dev-tec...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-crypto
I would avoid this. -Xarch_arch is implemented as an Apple GCC
driverdriver option and isn’t available in mainline GCC or even the
Apple GCC’s CPU-specific frontends (such as i686-apple-darwin10-
gcc-4.2.1). -Xarch_arch would allow things to work with Apple GCC and
would even enable single-pass universal builds, but would ruin things
for users of non-Apple GCC. NSPR and NSS have striven to be flexible
with respect to build configurations in the past. I’d be surprised if
either project were now willing to mandate that Apple GCC (or some
other Apple-supplied compiler) would be the only compiler that could
be used to target Mac OS X or Darwin.
Wan-Teh and I discussed adding a new header to handle this problem,
security/nss/lib/freebl/blconfig.h, that could be #included as needed
where definitions of macros like NSS_X86 and NSS_USE_64 are
significant. This could be structured like
#ifdef DARWIN
#ifdef __i386__
#define NSS_X86
#define NSS_X86_OR_X64
#endif /* __i386__ */
#ifdef __x86_64__
#define NSS_X64
#define NSS_X86_OR_X64
#endif /* __x86_64__ */
#ifdef __LP64__
#define NSS_USE_64
#endif /* __LP64__ */
#endif /* DARWIN */
and definitions of these macros could be removed from security/nss/lib/
freebl/Makefile when building for Darwin, along the lines of
ifneq ($(OS_TARGET),Darwin)
...
endif
> ("CPU_ARCH=dummy" is kind of a hack, currently - it's only purpose is to
> prevent coreconf from applying the x86 related settings in the ifeq...
> CPU_ARCH block. It will add -Dppc to the CC command, but this doesn't do
> any harm, I think. Note that the backslashes are required to protect the
> following blanks.)
For my purposes, when I’m working with single-pass universal builds
and am expected to provide a CPU specification at some level that’s
too general to actually know the target CPU, I usually use “mac.”
Mandating Apple compilers is fine. Until one or two years ago,
NSPR and NSS were using the -Wmost flag, which is an Apple
extension. I was the one who noticed the problem. That's
strong evidence that no one else had tried to compile NSPR
or NSS on the Mac with non-Apple GCC.
But we should use -Xarch_arch only if it offers a simpler solution
than defining arch-specific macros in lib/freebl/blconfig.h.
Wan-Teh
Sounds good. security/nss/lib/jar is currently the other place which
also depends on the NSS_X* macros, i.e. it should be a header file which
can be used by files outside freebl, too.
>> ("CPU_ARCH=dummy" is kind of a hack, currently - it's only purpose is to
>> prevent coreconf from applying the x86 related settings in the ifeq...
>> CPU_ARCH block. It will add -Dppc to the CC command, but this doesn't do
>> any harm, I think. Note that the backslashes are required to protect the
>> following blanks.)
>
> For my purposes, when I’m working with single-pass universal builds
> and am expected to provide a CPU specification at some level that’s
> too general to actually know the target CPU, I usually use “mac.”
The command line I provided in my earlier message was just meant as a
proof of concept - under the assumption that you use NSS trunk as-is
(i.e., you can use it right now w/o touching any files in the tree).
When adding support for universal binaries to Darwin.mk, we don't
necessarily have to use CPU_ARCH to specify the options a universal
build, I think. A separate option, let's say something like
"NSS_OSX_MULTIARCH") could be used to specify the list of architectures
to compile (and would be handled in Darwin.mk in the way proposed by
Bayard). If specified, it would unset CPU_ARCH at the same time.
Kaspar
Note that this is also true for the "-arch" option (and "-arch x86_64"
was added to Darwin.mk two years ago with
https://bugzilla.mozilla.org/show_bug.cgi?id=469738, and I guess nobody
complained about build failures with USE_64=1).
> But we should use -Xarch_arch only if it offers a simpler solution
> than defining arch-specific macros in lib/freebl/blconfig.h.
Agreed.
Kaspar
I see. security/nss/lib/util/secport.h is a header where we can define
these NSS_* macros. It already defines a few such macros:
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/util/secport.h&rev=1.23&mark=53-74#49
The only problem is that secport.h is a public header, so those
NSS_* macro definitions will also be seen by NSS customers.
Another option is to fix security/nss/lib/jar so that it doesn't
use the NSS_X86_OR_X64 macro. lib/jar is only used by
the NSS command-line tools modutil and signutil, so it isn't
important to rely on x86/x86_64 processors' ability to do
unaligned memory access.
Wan-Teh
Right, I had another look at libjar, and there's only one place in the
code which relies on NSS_X86_OR_X64, in jarfile.c:
> #ifdef NSS_X86_OR_X64
> #define x86ShortToUint32(ii) ((const PRUint32)*((const PRUint16 *)(ii)))
> #define x86LongToUint32(ii) (*(const PRUint32 *)(ii))
> #else
> static PRUint32
> x86ShortToUint32(const void *ii);
>
> static PRUint32
> x86LongToUint32(const void *ll);
> #endif
(later on, there are implementations of x86ShortToUint32 and
x86LongToUint32 which simulate "an x86 (little endian, unaligned) uint
fetch from any address")
If we agree that we can sacrifice this optimization for x86 in
jarfile.c, then it seems fairly straightforward to nuke NSS_X86_OR_X64
from libjar, so that there would be no further obstacle to using
freebl/blconfig.h (which, IMO, seems preferrable to putting things into
the public secport.h header).
Kaspar