Reg 64-bit architecture support for Spidermonkey 1.5 and NSPR 4.4.1 components
Rajesh N
I would like to get some information regarding 64-bit platform architecture
support for Spidermonkey and NSPR components.
For a server application, we are using Spidermonkey JS Engine 1.5 version
and NSPR 4.4.1 version components.
Till now we have been using 32-bit libraries of these components in
different UNIX-based platforms.
We are evaluating technical aspects of migrating the application to 64-bit
platforms (including x86_64 and IA64 models). Towards this, I have a few
queries which I am listing down below:
a) Are Spidermonkey 1.5 and NSPR 4.4.1 version codebases from the Mozilla
(http://ftp.mozilla.org/pub/mozilla.org/js/older-packages/) 64-bit
architecture compliant?
b) If yes, what points do I need to take care of while building using 64-bit
compiler? I am not very proficient in the code-base, but I see certain flags
like JS_BYTES_PER_LONG, JS_BYTES_PER_WORD etc. Do I need to explicitly set
any of these flags?
c) If no, then should we take care of 64-bit porting for Spidermonkey and
NSPR codebases with respect to issues like data structure alignment,
padding, and structure size etc?
If anyone has worked on using Spidermonkey/NSPR components on 64-bit
platforms, please share some guiding points.
Thanks
Rajesh
Benjamin Smedberg
> a) Are Spidermonkey 1.5 and NSPR 4.4.1 version codebases from the Mozilla
> (http://ftp.mozilla.org/pub/mozilla.org/js/older-packages/) 64-bit
> architecture compliant?
Yes. 64-bit builds of Firefox and other applications are available for
x86-64 at least. I would think that IA64 might also work, but I don't have
any personal experience with it.
Note, however, that the tracing JIT that comes with SpiderMonkey has not
been implemented for either of the 64-bit architectures, at least not in any
release builds. The x86 version may be faster, perhaps much faster than the
64-bit version for this reason. I believe people are currently working on
x86-64 support for tracing, but I don't know the current status. I don't
know of anyone working on IA64 support.
> b) If yes, what points do I need to take care of while building using 64-bit
> compiler? I am not very proficient in the code-base, but I see certain flags
> like JS_BYTES_PER_LONG, JS_BYTES_PER_WORD etc. Do I need to explicitly set
> any of these flags?
The configure script should take care of this for you without any manual
intervention.
--BDS
Boris Zbarsky
> I believe people are currently working on
> x86-64 support for tracing, but I don't know the current status.
Fixed on mozilla-central since Sept 16. See
https://bugzilla.mozilla.org/show_bug.cgi?id=489146
-Boris
Mike Shaver
> On 11/2/09 7:56 AM, Rajesh N wrote:
>
>> a) Are Spidermonkey 1.5 and NSPR 4.4.1 version codebases from the Mozilla
>> (http://ftp.mozilla.org/pub/mozilla.org/js/older-packages/) 64-bit
>> architecture compliant?
>
> Yes. 64-bit builds of Firefox and other applications are available for
> x86-64 at least. I would think that IA64 might also work, but I don't have
> any personal experience with it.
Spidermonkey 1.5 is pretty old; I would be hesitant to make any claims
about its 64-bit readiness without testing on the toolchain in
question at the least.
NSPR should be fine.
Mike
Rajesh N
Thanks for the reply.
I took JS 1.8.0 (js-1.8.0-rc1.tar.gz dated 09-Mar-2009) from
http://ftp.mozilla.org/pub/mozilla.org/js/
From an initial source browsing, I understand that JS 1.8 codebase handles
64-bit issues. I find certain new macros like JS_PTR_TO_INT32,
JS_UINT32_TO_PTR etc which have been added to handle data type conversions
(which is an important 64-bit compatibility related issue).
I have some more doubts which are listed below:
a) Is JS 1.8.0-rc1 an official formal release? Is this the final version
before the new JS 1.8.1 (with Tracing JIT) gets released?
b) We are currently using JS 1.5 embedded with our application. Is JS1.8
backward compatible with respect to JS 1.5? I know there have been
intermediate 1.6 and 1.7 versions. However we did not migrate to any of them
as we found JS 1.5 suiting our application requirements well enough.
As per our analysis, we think Spidermonkey 1.5 is not 64-bit compatible. As
we are not conversant with the source well, we tried building JS 1.5 on Suse
Linux 10 (X86-64) and observed that many warnings have been reported.
While trying to compile JS 1.8 on x86-64, we faced couple of warnings
related to the va_list usage while invoking TryArgumentFormatter(). These
warnings for JS 1.8 in 64-bit environment were fixed with the following
build update:
a) config/Linux_All.mk ----> Added -DHAVE_VA_LIST_AS_ARRAY (I assume gcc in
64-bit environment handles va_list as array)
It would be great if someone could share us more pointers regarding 64-bit
support evaluation for Spidermonkey.
Thanks
Rajesh
--BDS
_______________________________________________
dev-tech-js-engine mailing list
dev-tech-...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Jason Orendorff
> a) Is JS 1.8.0-rc1 an official formal release? Is this the final version
> before the new JS 1.8.1 (with Tracing JIT) gets released?
JS 1.8.0 RC1 is an official release candidate. The core Mozilla JS
developers never released anything called "JS 1.8.0 Final", but a pair
of active community contributors, Wes Garland and Ash Berlin, are
working on that.
You can follow that work here:
http://code.google.com/p/js18/wiki/BugzillaStatus
> b) We are currently using JS 1.5 embedded with our application. Is JS1.8
> backward compatible with respect to JS 1.5? I know there have been
> intermediate 1.6 and 1.7 versions. However we did not migrate to any of them
> as we found JS 1.5 suiting our application requirements well enough.
We have tried to keep API-breaking changes to a minimum, but there have
been a few over the years.
JS1.5 is before my time, but I documented all known issues migrating
from 1.7 to 1.8 in the release notes:
https://developer.mozilla.org/en/SpiderMonkey/1.8
See the two sections "Known issues" and "Migrating to JavaScript 1.8".
> While trying to compile JS 1.8 on x86-64, we faced couple of warnings
> related to the va_list usage while invoking TryArgumentFormatter(). These
> warnings for JS 1.8 in 64-bit environment were fixed with the following
> build update:
> a) config/Linux_All.mk ----> Added -DHAVE_VA_LIST_AS_ARRAY (I assume gcc in
> 64-bit environment handles va_list as array)
Yes, this is a known issue. I think your fix is correct. (The new build
system in SpiderMonkey 1.8.1 doesn't have this bug.)
-j
Rajesh N
Thanks for the clarifications. Couple of more queries:
a) Is JS 1.8.0 RC1 64-bit compatible? Does the sourcebase take care of
issues like address alignment, padding, typecasts (pointer<->integer) and
conversions?
b) Yes, I observe that JS 1.6 added E4X, JS 1.7 added new concepts like
iterators,generators etc. My understanding is JS 1.8.0 builds over the same
code base. However, has these enhancements affected any basic features that
a JS engine embedded application would need - like create runtime, create
context, compile and execute scripts, compile and call functions, define/set
properties etc?
Thanks
Rajesh
-----Original Message-----
From: dev-tech-js-engine-bounces+rajeshn=huawe...@lists.mozilla.org
[mailto:dev-tech-js-engine-bounces+rajeshn=huawe...@lists.mozilla.org] On
Behalf Of Jason Orendorff
Sent: Monday, November 16, 2009 11:29 PM
To: dev-tech-...@lists.mozilla.org
Subject: Re: Reg 64-bit architecture support for Spidermonkey 1.5 and
NSPR4.4.1components
-j
Jason Orendorff
> a) Is JS 1.8.0 RC1 64-bit compatible? Does the sourcebase take care of
> issues like address alignment, padding, typecasts (pointer<->integer) and
> conversions?
Yes.
> b) Yes, I observe that JS 1.6 added E4X, JS 1.7 added new concepts like
> iterators,generators etc. My understanding is JS 1.8.0 builds over the same
> code base. However, has these enhancements affected any basic features that
> a JS engine embedded application would need - like create runtime, create
> context, compile and execute scripts, compile and call functions, define/set
> properties etc?
I don't know. Try it!
-j
Rajesh N
I noticed a few points in JS 1.8 code where I had some queries on the 64-bit
compatibility aspects.
Could you please help me get clear on these issues?
a) File jsarena.c
Function: JS_INIT_NAMED_ARENA_POOL
Here argument "align" is of type size_t (8 bytes on 64-bit system). This
argument "align" is passed as argument to JS_CeilingLog2 function.
JS_CeilingLog2 prototype specifies that its argument is of type JSUint32
(unsigned int, even on LP64 systems, as per the definition in jstypes.h).
Won't such an invocation lead to data loss or truncation from unsigned long
to unsigned int??
b) Data struture definitions do not take care of padding explicitly.
Example: JSConstDoubleSpec, JSPropertySpec etc in jspai.h
c) Defnition of struct BinaryDigitReader (jsnum.c)
struct BinaryDigitReader
{
uintN base; /* Base of number; must be a power of 2 */
uintN digit; /* Current digit value in radix given by base */
uintN digitMask; /* Mask to extract the next bit from digit */
const jschar *digits; /* Pointer to the remaining digits */
const jschar *end; /* Pointer to first non-digit */
};
Even on a 64-bit LP64 system, uintN is unsigned int (4 bytes). So,the
pointer memebr digits is not aligned to be on 8-byte aligned address.
d) JS 1.8 code still has many uses of JS_PTR_TO_UINT32, JS_PTR_TO_UINT32 and
reverse conversion macros. On 64-bit systems, how can we guarantee there is
no data loss when a pointer value of 8 bytes is typecated/assigned to a 4
byte integer?
Thanks
Rajesh
-----Original Message-----
From: dev-tech-js-engine-bounces+rajeshn=huawe...@lists.mozilla.org
[mailto:dev-tech-js-engine-bounces+rajeshn=huawe...@lists.mozilla.org] On
Behalf Of Jason Orendorff
Sent: Wednesday, November 18, 2009 2:20 AM
To: dev-tech-...@lists.mozilla.org
Subject: Re: Reg 64-bit architecture support for Spidermonkey 1.5
andNSPR4.4.1components
Yes.
-j
Mike Shaver
> a) File jsarena.c
> Function: JS_INIT_NAMED_ARENA_POOL
> Here argument "align" is of type size_t (8 bytes on 64-bit system). This
> argument "align" is passed as argument to JS_CeilingLog2 function.
>
> JS_CeilingLog2 prototype specifies that its argument is of type JSUint32
> (unsigned int, even on LP64 systems, as per the definition in jstypes.h).
>
> Won't such an invocation lead to data loss or truncation from unsigned long
> to unsigned int??
Only if you pass an extremely large value to it -- I don't think we'll
be aligning to anything that huge.
> b) Data struture definitions do not take care of padding explicitly.
> Example: JSConstDoubleSpec, JSPropertySpec etc in jspai.h
>
> c) Defnition of struct BinaryDigitReader (jsnum.c)
> struct BinaryDigitReader
> {
> uintN base; /* Base of number; must be a power of 2 */
> uintN digit; /* Current digit value in radix given by base */
> uintN digitMask; /* Mask to extract the next bit from digit */
> const jschar *digits; /* Pointer to the remaining digits */
> const jschar *end; /* Pointer to first non-digit */
> };
>
> Even on a 64-bit LP64 system, uintN is unsigned int (4 bytes). So,the
> pointer memebr digits is not aligned to be on 8-byte aligned address.
That's the compiler's problem: it needs to produce appropriate
alignment of members for its ABI; this is not unique to 64-bit, and
just basic C-compiler requirements.
> d) JS 1.8 code still has many uses of JS_PTR_TO_UINT32, JS_PTR_TO_UINT32 and
> reverse conversion macros. On 64-bit systems, how can we guarantee there is
> no data loss when a pointer value of 8 bytes is typecated/assigned to a 4
> byte integer?
You'd have to give the examples -- in most (all?) the operands of
these operations were produced by converting an int to a pointer,
meaning that it's known to be safe. You could add assertions in the
macros I guess, if you thought you'd found a case that was wrong, or
just post here with details.
Mike
Jason Orendorff
> d) JS 1.8 code still has many uses of JS_PTR_TO_UINT32, JS_PTR_TO_UINT32 and
> reverse conversion macros. On 64-bit systems, how can we guarantee there is
> no data loss when a pointer value of 8 bytes is typecated/assigned to a 4
> byte integer?
The name of that macro is a lie: the type of the result is actually
jsuword, which is pointer-sized.
I have filed a bug to get rid of the last few remaining uses of these
macros, bug 530583. But the code is 64-bit safe already, despite the
misleading INT32 in the macro names.
-j
Puneet
> On 11/23/2009 10:31 AM, Rajesh N wrote:
>
> > d) JS 1.8 code still has many uses of JS_PTR_TO_UINT32, JS_PTR_TO_UINT32 and
> > reverse conversion macros. On 64-bit systems, how can we guarantee there is
> > no data loss when a pointer value of 8 bytes is typecated/assigned to a 4
> > byte integer?
>
> The name of that macro is a lie: the type of the result is actually
> jsuword, which is pointer-sized.
jsuword is pointer sized but, JS_PTR_TO_UINT32 is actually truncation
of pointer information, as it typecast to jsint, which is a 4 byte
integer on 64 bit platform.
Macro : jstypes.h
#define JS_PTR_TO_INT32(x) ((jsint)((char *)(x) - (char *)0))
File : jsapi.c
3101: static JSBool
3102 DefineProperty(JSContext *cx, JSObject *obj, const char *name,
jsval value,
3103: JSPropertyOp getter, JSPropertyOp setter, uintN
attrs,
3104: uintN flags, intN tinyid)
3105: {
3106: jsid id;
3107: JSAtom *atom;
3108:
3109: if (attrs & JSPROP_INDEX) {
3110: id = INT_TO_JSID(JS_PTR_TO_INT32(name));
For eg, in func DefineProperty, it first typecast to jsid from jsint
and same jsid is accessed as an atom pointer after converting it back
to jsint in func DefineNativeProperty.
Filet : jsapi.c
3119: if (flags != 0 && OBJ_IS_NATIVE(obj)) {
3120: return js_DefineNativeProperty(cx, obj, id, value,
getter, setter,
3121 attrs, flags, tinyid,
NULL);
It does a series of operations, which does not seems to guarantee
restoration of truncated data.
> I have filed a bug to get rid of the last few remaining uses of these
> macros, bug 530583. But the code is 64-bit safe already, despite the
> misleading INT32 in the macro names.
> -j
-Puneet
Puneet Sharma
> > d) JS 1.8 code still has many uses of JS_PTR_TO_UINT32, JS_PTR_TO_UINT32
and
> > reverse conversion macros. On 64-bit systems, how can we guarantee there
is
> > no data loss when a pointer value of 8 bytes is typecated/assigned to a
4
> > byte integer?
> The name of that macro is a lie: the type of the result is actually
> jsuword, which is pointer-sized.
> I have filed a bug to get rid of the last few remaining uses of these
> macros, bug 530583. But the code is 64-bit safe already, despite the
> misleading INT32 in the macro names.
> -j
-Puneet
Jason Orendorff
> On Nov 23, 11:43 pm, Jason Orendorff<jorendo...@mozilla.com> wrote:
>> On 11/23/2009 10:31 AM, Rajesh N wrote:
>>
>>> d) JS 1.8 code still has many uses of JS_PTR_TO_UINT32, JS_PTR_TO_UINT32 and
>>> reverse conversion macros. On 64-bit systems, how can we guarantee there is
>>> no data loss when a pointer value of 8 bytes is typecated/assigned to a 4
>>> byte integer?
>>
>> The name of that macro is a lie: the type of the result is actually
>> jsuword, which is pointer-sized.
>
> jsuword is pointer sized but, JS_PTR_TO_UINT32 is actually truncation
> of pointer information, as it typecast to jsint, which is a 4 byte
> integer on 64 bit platform.
>
> Macro : jstypes.h
> #define JS_PTR_TO_INT32(x) ((jsint)((char *)(x) - (char *)0))
Oops! Yes, you are completely right about that. The macro truncates.
> For eg, in func DefineProperty, it first typecast to jsid from jsint
> and same jsid is accessed as an atom pointer after converting it back
> to jsint in func DefineNativeProperty.
if (attrs & JSPROP_INDEX) {
id = INT_TO_JSID(JS_PTR_TO_INT32(name));
This is OK because JSPROP_INDEX means that `name` is actually an integer
that fits in 31 bits. The data is not pointer data at all.
https://developer.mozilla.org/En/JS_GetPropertyAttributes#Property_attributes
>> I have filed a bug to get rid of the last few remaining uses of these
>> macros, bug 530583. But the code is 64-bit safe already, despite the
>> misleading INT32 in the macro names.
I still believe this last sentence is true.
-j