Error: gpsee.module.ca.page.gffi.Memory.constructor.size

[Christoph Dorn]

Getting the following error 100% now running of tip:

[pinf-loader] Error:
gpsee.module.ca.page.gffi.Memory.constructor.size: nan is not a valid
memory size

Memory((void 0))@:0
workingDirectory()@/opt/local/gpsee/libexec/fs-base.js:359
([object gpsee.module.Exports])@/usr/local/lib/node_modules/pinf-
loader-js/lib/pinf-loader-js/./adapter/gpsee.js:48
()@/usr/local/lib/node_modules/pinf-loader-js/./lib/pinf-loader-js/
loader.js:145
@/usr/local/lib/node_modules/pinf-loader-js/pinf-loader.js:2

I presume you are getting closer to the mysterious segfault?

Christoph

[Wes Garland]

> I presume you are getting closer to the mysterious segfault?

Yes.

I have identified what I believe to be the root cause of the crashes I am seeing on your loader (dated from last week - I don't want to lose my test case).

It's a quite interesting bug.  In GPSEE, you can attach finalizers to FFI'd C functions, with a sort of C Function closure. (BTW - this is a feature completely unique to GPSEE AFAIK.) This closure contains the arguments to pass to the C function, and a reference to a piece of memory which is tied to the JS CFunction instance representing the function which gets run upon finalization.  You can think of this as ... a sort of JS weak reference which is implemented in C.

And this is the crux of the issue - it's a weak reference which we are treating a strong reference.  This is normally not a problem, as CFunction instances are normally declared as globals, and finalization of active FFI return value objects happens long before the program terminates.

The PINF loader *really* churns the garbage collector around FFI'd functions, in particular, the ones related to opening and closing fs-base Streams.  When GSR shuts down the JS engine, we release the global object and trigger a GC.  This in turn triggers Stream finalization, which uses the C function closures, which use the weak references to the CFunction instances.

But JS GC is unordered, like Java and unlike Cedar.  This means that even if there is a defined relationship in the object graph between two objects, they can be collected in arbitrary order when neither is reachable from any GC root.

So, in some cases during GSR shutdown, we collect the CFunction instance, free() the memory it was occupying, and then try to use that same memory when finalizing an fs-base Stream instance (or other FFI return object with a finalizer).  This memory may now have been overwritten by other operations, resulting in garbage. When we use pointers stored here, they then point to random addresses; hence the segmentation violation.

The fix is to let the garbage collector know about this C-side relationship, making it a strong reference that just happens to be invisible from JS-land, and then to finalize any pending-finalization FFI objects before dropping the global object's GC root and shutting down the JS runtime.

That fix is underway now.

Incidentally  - thank you VERY MUCH for both the test case and the bug report.  This has been a long, arduous hunt but it helps everybody in the long run.

Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102

[Christoph Dorn]

On Oct 3, 5:48 pm, Wes Garland <w...@page.ca> wrote:
> Incidentally  - thank you VERY MUCH for both the test case and the bug
> report.  This has been a long, arduous hunt but it helps everybody in the
> long run.

I am glad I could help and thank you for jumping on this right away.
It allows me to trust in GPSEE as a platform that I can invest in and
build on.

Once the loader adapter for GPSEE is fully implemented I have various
user-level applications built against node (on PINF) that I will be
porting which will make further test cases for GPSEE and CommonJS
portability. I am pretty confident that once this stuff is ported,
GPSEE will be stable for typical uses relating to the PINF loader and
we will have the ability to write portable applications that run on
two platforms that solve very different needs but are compatible (if
not using node's event loop features).

v8cgi and ringo are also turning out to be quite hassle-free although
ringo/rhino support will need some more work at the compilation level
(need to cache and ship compiled source as it takes way too long to
compile).

Christoph

[Christoph Dorn]

We discussed that this error was related to an empty file. I think
there is something else going on.

Here is an updated trace:

JS strict warning #162 in fs-base.js at line 359 - reference to
undefined property dh.FILENAME_MAX

[pinf-loader] Error:
gpsee.module.ca.page.gffi.Memory.constructor.size: nan is not a valid
memory size

Memory((void 0))@:0
workingDirectory()@/opt/local/gpsee/libexec/fs-base.js:359
([object gpsee.module.Exports])@/usr/local/lib/node_modules/pinf-

loader-js/lib/pinf-loader-js/./adapter/gpsee.js:61

()@/usr/local/lib/node_modules/pinf-loader-js/./lib/pinf-loader-js/
loader.js:145
@/usr/local/lib/node_modules/pinf-loader-js/pinf-loader.js:2

Where ./adapter/gpsee.js:61 corresponds to:

require("fs-base").workingDirectory();

You may be able to reproduce with the following (I can on one OSX Lion
system, not on another):

cd ~
# gsr -h
# node -h
npm install -g pinf-loader-js
commonjs -v --platform node --script test
https://github.com/pinf/test-programs-js/zipball/master

and the error should show up in the log. It will indicate the command
that can be run for the single GPSEE test (you are running the cross-
platform test suite via node with the last command above).

If you don't want to install the loader via `npm install` you can just
clone the source and do `node .../loader.js --script ...`.

Let me know if you cannot reproduce this.

Christoph


On Oct 3, 12:28 pm, Christoph Dorn <christ...@christophdorn.com>
wrote:

[Christoph Dorn]

It is breaking on a new OSX Lion system that I just got. I am
wondering if I have the wrong regular expression library again. What
did we do to fix that again?

Christoph

On Oct 4, 8:25 pm, Christoph Dorn <christ...@christophdorn.com> wrote:
> We discussed that this error was related to an empty file. I think
> there is something else going on.
>
> Here is an updated trace:
>
> JS strict warning #162 in fs-base.js at line 359 - reference to
> undefined property dh.FILENAME_MAX
> [pinf-loader] Error:
> gpsee.module.ca.page.gffi.Memory.constructor.size: nan is not a valid
> memory size
>
>   Memory((void 0))@:0
>   workingDirectory()@/opt/local/gpsee/libexec/fs-base.js:359
>   ([object gpsee.module.Exports])@/usr/local/lib/node_modules/pinf-
> loader-js/lib/pinf-loader-js/./adapter/gpsee.js:61
>   ()@/usr/local/lib/node_modules/pinf-loader-js/./lib/pinf-loader-js/
> loader.js:145
>   @/usr/local/lib/node_modules/pinf-loader-js/pinf-loader.js:2
>
> Where ./adapter/gpsee.js:61 corresponds to:
>
>   require("fs-base").workingDirectory();
>
> You may be able to reproduce with the following (I can on one OSX Lion
> system, not on another):
>
>   cd ~
>   # gsr -h
>   # node -h
>   npm install -g pinf-loader-js

>   commonjs -v --platform node --script testhttps://github.com/pinf/test-programs-js/zipball/master

[Wes Garland]

On 4 October 2011 23:29, Christoph Dorn <chri...@christophdorn.com> wrote:

It is breaking on a new OSX Lion system that I just got. I am
wondering if I have the wrong regular expression library again. What
did we do to fix that again?

Install the mac ports grep package, and make sure mac ports is at the start of your path.

I bumped into this on one Lion system at work, but another was fine.

Safari and Firefox had a similar regression in their RegExp class due to a bug in Apple's YARR which replaced PCRE, I wonder if the same thing happened to Apple's grep?

Wes

[Christoph Dorn]

That did it. Thanks! It's now part of the install instructions:

https://github.com/pinf/platform-js/blob/master/docs/osx-lion.md

Christoph


On Oct 5, 5:12 am, Wes Garland <w...@page.ca> wrote: