No worries: I needed to see what was needed first
before asking.
I actually managed to get as far as this (valgrind on the segfault below):
[jbj@wp3 wes-js185]$ gsr -c 'print("w00t!")'
w00t!
Segmentation fault
==26340== 1 errors in context 1 of 87:
==26340== Invalid read of size 8
==26340== at 0x4CC5140: js::MaybeGC(JSContext*) (in /X/src/wes-js185/spidermonkey/builds/release/jsapi/lib/libmozjs185.so.1.0.0)
==26340== by 0x518F96B: gpsee_maybeGC (gpsee.c:639)
==26340== by 0x518FDF4: gpsee_operationCallback (gpsee.c:470)
==26340== by 0x4C7B167: js_InvokeOperationCallback(JSContext*) (in /X/src/wes-js185/spidermonkey/builds/release/jsapi/lib/libmozjs185.so.1.0.0)
==26340== by 0x4C7B1BB: js_HandleExecutionInterrupt(JSContext*) (in /X/src/wes-js185/spidermonkey/builds/release/jsapi/lib/libmozjs185.so.1.0.0)
==26340== by 0x4CD0234: js::Interpret(JSContext*, JSStackFrame*, unsigned int, JSInterpMode) (in /X/src/wes-js185/spidermonkey/builds/release/jsapi/lib/libmozjs185.so.1.0.0)
==26340== by 0x4CDEA6C: js::RunScript(JSContext*, JSScript*, JSStackFrame*) (in /X/src/wes-js185/spidermonkey/builds/release/jsapi/lib/libmozjs185.so.1.0.0)
==26340== by 0x4CE0C35: js::Execute(JSContext*, JSObject*, JSScript*, JSStackFrame*, unsigned int, js::Value*) (in /X/src/wes-js185/spidermonkey/builds/release/jsapi/lib/libmozjs185.so.1.0.0)
==26340== by 0x4C5990F: EvaluateUCScriptForPrincipalsCommon(JSContext*, JSObject*, JSPrincipals*, unsigned short const*, unsigned int, char const*, unsigned int, unsigned long*, JSVersion) (in /X/src/wes-js185/spidermonkey/builds/release/jsapi/lib/libmozjs185.so.1.0.0)
==26340== by 0x4C59A04: JS_EvaluateUCScriptForPrincipals (in /X/src/wes-js185/spidermonkey/builds/release/jsapi/lib/libmozjs185.so.1.0.0)
==26340== by 0x4C5B291: JS_EvaluateScriptForPrincipals (in /X/src/wes-js185/spidermonkey/builds/release/jsapi/lib/libmozjs185.so.1.0.0)
==26340== by 0x4C5B331: JS_EvaluateScript (in /X/src/wes-js185/spidermonkey/builds/release/jsapi/lib/libmozjs185.so.1.0.0)
==26340== Address 0x120 is not stack'd, malloc'd or (recently) free'd
which isn't too bad.
Now I'm asking:
What needs doing to get GPSEE linked to JS 1.8.5?
Wes: If you can prioritize the issues, I'll try to make that happen.
73 de Jeff
> Hey, Jeff!
>
> BTW - when did you pull? I merged in 0.2 changes and pushed last night. Have you read the GPSEE blog? It covers some of what I've been doing (and why).
>
My checkout was likely a ship passing at midnight, I'll pull soon.
I've read the GPSEE blog several times now: as always, you words
are quite useful.
It is JS (and commonJS) context I need in JS land because my focus is largely on custom
C modules which is rather different than most.
> The issues remaining are pretty straightforward if non-deterministic:
> 1 - Fix the constructor bug
> 2 - Get the modules compiling
> 3 - Test and Debug until stable
> 4 - Rinse, lather, repeat
> 5 - Now do RPM5's modules :)
>
One of the first issues that I will attempt is AutoFu for GPSEE in order to
use "system" -lffi and -lmozjs and curl and more.
Dunno whether you want to change how GPSEE is built to use AutoFu or not.
I need something I can hack efficiently: your Makefiles are quite carefully
done and likely useful for your "production" builds but …
<snipped for later contemplation>
Thanks!
Time for round #2 … ding!
73 de Jeff
> Dunno whether you want to change how GPSEE is built to use AutoFu or not.
I don't have an ideological exception here; in fact, I have re-invented parts of autoconf as it stands. When I started GPSEE, I had never done any autoconf hacking, although I haven't been able to completely avoid it since. (I've had to fix Moz's stuff a few times). If I had a complete, working, autoconf build for GPSEE, I would probably even switch internally. Two build systems is almost always worse than one.
The main question to answer, if converting to autoconf, is how to communicate appropriate options to the makefiles. You can either generate the Makefile from Makefile.in and then build, or you can you can create local_config.mk, which is what our current ./configure script does.
Switching to a normal autoconf Makefile.in will also have some sanity advantages, particularly for the gffi build, where it's a bit precarious the way it compiles code which generates CPPFLAGS which are then included in the Makefile, etc.
One thing to remember -- GPSEE doesn't do a lot of "how the hell do I compile" tests -- it relies on SpiderMonkey's js-config output for this. But js-config is buggy, although I hope at least the *.pc file I built with js-1.8.5 is not. (See the ridiculous work around in spidermonkey/Makefile where I edit ldflags etc).
Also, it's important to support the probe-jsapi functionaly, that detects variances in the JSAPI. As well as alternate C compiler path, path to curl, ah, well, you can do a configure --help too. :)
Your Solaris/BSD style is going to meet stiff resistance and many complaintsif/when released.
We both agree that external -lmozjs and -lffi needs to be attempted.
So with 30 minutes of hackery I've achieved:
You da GPSEE maintainer, and a consenting adult:Choose whatever poison works for you.
> Hey, Jeff!
>
> BTW - when did you pull? I merged in 0.2 changes and pushed last night. Have you read the GPSEE blog? It covers some of what I've been doing (and why).
>
Does ".. merged … pushed …" mean I should stick with the wes-js185 checkout or
change pack to the gpsee checkout?
Presumably I stick with wes-js185 … I tried AutoFu into gpsee checkout Just In Case too.
After some dreadful hackery I'm back (on a wes-js185 tree insturmented with AutoFu)
where I was yesterday:
[jbj@wp3 gpsee]$ ./gsr -c 'print("w00t!")'
w00t!
Segmentation fault
Mebbe 10% of the way along, but layering in the boilerplate is the first step,
now mostly in place.
The important part in the round #2 build is now using external -lmozjs185 -lffi.
On round #3 (later tonight or tomorrow), I will likely achieve
make install
which is sufficient to start resurrecting RPM+JS module bit-rot.
Note that the install will be "portable" in the sense that /usr/lib or /iusr/lib64
will be used as needed (or I screwed something, equally likely atm).
73 de Jeff
>
> On round #3 (later tonight or tomorrow), I will likely achieve
> make install
> which is sufficient to start resurrecting RPM+JS module bit-rot.
>
OK, "make install" is sufficiently functional in gpsee-0.3 now for me to
start scrubbing RPM+GPSEE bit rot, and likely attempt a *.rpm package soonishly.
I've also added enough AutoFu so that "make distcheck" succeeds
doing a recursive build from a tar ball. The "make distcheck" is useful to
ensure that manifests are correct.
There's a start at gettext i18n stu8bbed in, and loader maps, and versioned
symbols, and more present. I haven't a clue what works or not yet: most has
been swiped from RPM where its worked fine for years.
The tar ball can be found at
http://rpm5.org/files/3rd/gpsee-0.3.DEVEL.tar.gz
The gpsee-0.3.DEVEL.tar.gz is merely a starting reference point: there's loads of trash that
I need to haul out, and quite a bit more to polish up with the AutoFu name space, and
at least half the modules aren't yet enabled, nor are the tests using AutoFu etc etc etc …
hth
73 de Jeff
There's a start at gettext i18n stu8bbed in, and loader maps, and versioned
symbols, and more present.
Wes Garland
23 September, 2011 7:47 AM
Wow, somebody's been plowing through the coffee and pizza!
Is there any way I could convince you to push to a mercurial repository on either Google Code or Bitbucket? That's the easiest way for me to track development and take patches into the mainline.
There's a start at gettext i18n stu8bbed in, and loader maps, and versioned
symbols, and more present.
Hey, if you get us to the point where we can have localizations, I'll do the first pass at the French one; that will give us a good test case. (Anybody else here bilingual?)
Wow, somebody's been plowing through the coffee and pizza!
Is there any way I could convince you to push to a mercurial repository on either Google Code or Bitbucket? That's the easiest way for me to track development and take patches into the mainline.
There's a start at gettext i18n stu8bbed in, and loader maps, and versioned
symbols, and more present.
Hey, if you get us to the point where we can have localizations, I'll do the first pass at the French one; that will give us a good test case. (Anybody else here bilingual?)
Christoph
>
> Google PEPIPOPUM to see what I'm contemplating. That is one page
> of PHP to handle PO files: what I'm suggesting is the same approach
> in C with a look-aside cache locally accessed through the usual
> gettext/dgettext C API used widely for i18n.
Since there's expertise and skill with CommonJS here, I really should
suggest "Internationalization" for server side JS as a more serious undertaking.
Here is PEPIPOPUM
http://pepipopum.dixo.net/
that fits into all the usual i18n processes.
I think COmmonJS would benefit from think through how
module/package i18n SHOULD be done including a "best effort"
automatic translation component.
There also needs to be some local cache that can bre redacted
when one encounters "Ruglish" or "Franglais" returned from
GOOG Translate, and also to implemement caching.
What is typically done with i18n on web sites is rather deficient
from an engineering POV (but whatever works).
hth
73 de Jeff
> Here is PEPIPOPUM
> http://pepipopum.dixo.net/
> that fits into all the usual i18n processes.
>
> I think COmmonJS would benefit from think through how
> module/package i18n SHOULD be done including a "best effort"
> automatic translation component.
>
> There also needs to be some local cache that can bre redacted
> when one encounters "Ruglish" or "Franglais" returned from
> GOOG Translate, and also to implemement caching.
>
> What is typically done with i18n on web sites is rather deficient
> from an engineering POV (but whatever works).
I ran into an aspect of this yesterday:
https://github.com/pinf/test-packages-js/blob/master/Common/lib/greetings.js#L4
I'll do my part to implement i18ninto the package and module system of
PINF. Not sure what we need there yet, but I can see a bunch of use-cases.
Christoph
>
> I ran into an aspect of this yesterday:
>
> https://github.com/pinf/test-packages-js/blob/master/Common/lib/greetings.js#L4
>
> I'll do my part to implement i18ninto the package and module system of PINF. Not sure what we need there yet, but I can see a bunch of use-cases.
>
Following the usual C conventions => JS should get you
most of the way to a sane proposal for a translation process.
But the critical design element (and the deficiency in all current
translation processes) is that PO are file containers.
What SHOULD be done is string lookups using UUIDv3/UUIDv5
onto the "msgid" text (adorned however you wish to avoid
the vanishing probabilty of collisions but to ensure
strong administrative controls) to retrieve the "msgstr"
in whatever locale is desired.
MongoDB (also in use by dupal under transifex afaik) would
rock! as a distributed store for msgstr UUID's.
Note that I have several MongoDB's available for use any
time you want:
http://harwich.jbj.org:28010
and I pay for commercial hosting at MongoHQ as well.
Access available to either for the asking.
Inserting a local look-aside cache is dirt simple (until you
have to discuss where the cache belongs in FHS: that discussion
will take years and likely will never reach a consensus).
Make sense?
73 de Jeff
I don't know about the C conventions. Do you have a link or quick summary?
> But the critical design element (and the deficiency in all current
> translation processes) is that PO are file containers.
Right. So they need to be shipped with the code thus coupling
translations to releases?
> What SHOULD be done is string lookups using UUIDv3/UUIDv5
> onto the "msgid" text (adorned however you wish to avoid
> the vanishing probabilty of collisions but to ensure
> strong administrative controls) to retrieve the "msgstr"
> in whatever locale is desired.
Why not URIs? Too long? URIs would provide implicit namespaces allowing
for mappings and defaults etc...
Christoph
> msgstr ""
>
> #: gsr.c:122
> #, c-format
> msgid "\aFatal Error in "
"\aSchwerer Fehler in "
> msgstr ""
>
> #: gsr.c:125
> #, c-format
> msgid "Fatal Error: %s"
"Schwerer Fehler: %s "
Christoph
Hey, if you get us to the point where we can have localizations, I'll do the first pass at the French one; that will give us a good test case. (Anybody else here bilingual?)
I'll add fr.po next go around.
> Jeff Johnson wrote:
>> Following the usual C conventions => JS should get you
>> most of the way to a sane proposal for a translation process.
>
> I don't know about the C conventions. Do you have a link or quick summary?
>
The C convention is basically this and nothing more:
--- a/gsr.c Tue Sep 20 22:58:19 2011 -0400
+++ b/gsr.c Fri Sep 23 12:12:14 2011 -0400
@@ -56,6 +56,8 @@
static __attribute__((unused)) const char rcsid[]="$Id: gsr.c,v 1.28 2010/12/02 21:57:03 wes Exp $";
+#include "system.h"
+
#define PRODUCT_VERSION "1.0-rc2"
#if !defined(GPSEE_DEBUGGER)
@@ -92,6 +94,8 @@
extern rc_list rc;
+#include "debug.h"
+
/** Handler for fatal errors. Generate a fatal error
* message to surelog, stdout, or stderr depending on
* whether our controlling terminal is a tty or not.
@@ -115,14 +119,14 @@
haveTTY = isatty(currentStderrFileno);
if (!message)
- message = "UNDEFINED MESSAGE - OUT OF MEMORY?";
+ message = N_("UNDEFINED MESSAGE - OUT OF MEMORY?");
if (haveTTY)
{
- fprintf(stderr, "\007Fatal Error in " PRODUCT_SHORTNAME ": %s\n", message);
+ fprintf(stderr, _("\007Fatal Error in " PRODUCT_SHORTNAME ": %s\n"), message);
}
else
- gpsee_log(NULL, SLOG_EMERG, "Fatal Error: %s", message);
+ gpsee_log(NULL, SLOG_EMERG, _("Fatal Error: %s"), message);
exit(1);
}
The #defines are in system.h which does essentially this:
#if defined(HAVE_LOCALE_H)
# include <locale.h>
#endif
#if !defined(HAVE_SETLOCALE)
# define setlocale(Category, Locale) /* empty */
#endif
#if defined(ENABLE_NLS) && !defined(__LCLINT__)
# include <libintl.h>
# define _(Text) dgettext (PACKAGE, Text)
# define D_(Text) Text
#else
# undef bindtextdomain
# define bindtextdomain(Domain, Directory) /* empty */
# undef textdomain
# define textdomain(Domain) /* empty */
# define _(Text) Text
# define D_(Text) Text
# undef dgettext
# define dgettext(DomainName, Text) Text
#endif
#define N_(Text) Text
Since JS is so close to C, one would need the analogue (either global
methods or loadable module or some JS functions prefixed) of the C #defines.
There's also some setup/initialization like
#if defined(ENABLE_NLS) && !defined(__LCLINT__)
(void) setlocale(LC_ALL, "" );
(void) bindtextdomain(PACKAGE, __localedir);
(void) textdomain(PACKAGE);
#endif
which is essentially mapping the usual envvars into paths where the compiled
PO files can be found.
>> But the critical design element (and the deficiency in all current
>> translation processes) is that PO are file containers.
> Right. So they need to be shipped with the code thus coupling translations to releases?
>
>> What SHOULD be done is string lookups using UUIDv3/UUIDv5
>> onto the "msgid" text (adorned however you wish to avoid
>> the vanishing probabilty of collisions but to ensure
>> strong administrative controls) to retrieve the "msgstr"
>> in whatever locale is desired.
>
> Why not URIs? Too long? URIs would provide implicit namespaces allowing for mappings and defaults etc…
>
Well some of uglix and i18n doesn't use CSS or HTML5 ;-)
No reason why URI's can't be substituted for paths. But there's *lots*
of lookups for strings, and fetching a URI and relying on a URI content
cache aren't the only problems that need solving. There will be the inevitable
discussion about bandwidth usage if i18n is attempted directly trhough a URI.
> Why not URIs? Too long? URIs would provide implicit namespaces allowing for mappings and defaults etc� >
Well some of uglix and i18n doesn't use CSS or HTML5 ;-) No reason why URI's can't be substituted for paths. But there's *lots* of lookups for strings, and fetching a URI and relying on a URI content cache aren't the only problems that need solving. There will be the inevitable discussion about bandwidth usage if i18n is attempted directly trhough a URI.
en-us"
http://meta.org/messages/errors/fatalen-us"
http://meta.org/messages/errorsen-us"
http://meta.org/messagesJeff Johnson wrote:> Why not URIs? Too long? URIs would provide implicit namespaces allowing for mappings and defaults etc… >
Well some of uglix and i18n doesn't use CSS or HTML5 ;-) No reason why URI's can't be substituted for paths. But there's *lots* of lookups for strings, and fetching a URI and relying on a URI content cache aren't the only problems that need solving. There will be the inevitable discussion about bandwidth usage if i18n is attempted directly trhough a URI.
There is lots of options here:
$curl --header "Accept-Language:en-us" http://meta.org/messages/errors/fatal
Fatal Error
$curl --header "Accept-Language:en-us" http://meta.org/messages/errors
fatal=Fatal Error
outOfMemory=Out Of Memory
$curl --header "Accept-Language:en-us" http://meta.org/messages
errors/fatal=Fatal Error
errors/outOfMemory=Out Of Memory
You setup a remote repository of strings that can be mirrored incrementally based on the scope of namespace you need.
Local storage is by ID:
.../meta.org/messages/errors.list
or without hostname if string repository is mirrored:
.../messages/errors/fatal
.../messages/errors.list
Jeff -
HG repo is ready with your devel tarball at http://bitbucket.org/wesgarland/gpsee-rpm; you should have an invite waiting. I've also invited Christoph Dorn for write access in case he decides he wants to contribute to de.po, even with his kleine Wortshatz :) .
While importing your tar ball, I discovered the following issue, which I throw out here mostly because I haven't thought about them:
version.c: how do we generate this with auto-fu?
gpsee_config.h: ditto - this is generated by spidermonkey/probe-jsapi.sh
lightning code review --
vm/vm.c -- JSPropertyStub -- is NULL really an acceptable value there?
modules/binary/bytehings.c -- good patch, how did you find that?
gpsee.h: I put the constructor warning back in, it's a REAL BIG BUG that we don't do that. Discussed as item #1 in preview email. But it some place smarter. ;)
gpsee.h: observation: you moved non-threadsafe from error to warning. non-threadsafe builds are not supported and almost certainly buggy. (Moz is no longer supporting)
modules/gpsee/gpsee.c - moving JS_NewDoubleValue() here won't work, that's the require("gpsee") module and not part of gpsee-core. Suggest making non-static and putting in /gpsee.c or nearby. Symbol needs to be accessible both to gpsee and users of libgpsee.so.
gpsee-realms.c -- this will break on JS < 1.8.2. Not losing sleep over this, since I'm not testing older than that.
Observation: gpsee.h and gpsee-realms.c changes make me wonder if you have JS 1.8.5 header mismatch problems, or maybe it really is not thread-safe
The real flaw is the JS package I'm using that passes JS_THREADSAFEon the compile line (so its enabled) but doesn't copy the valueinto whatever include file GPSEE is expecting and testing.
The proper fix will be to change the Fedora packaging: but as soon as I do that,then I'm running a custom JS package or have to teach some package monkeyho0w to peel a banana correctly: not gonna happen in finite time.
Jeff -
HG repo is ready with your devel tarball at http://bitbucket.org/wesgarland/gpsee-rpm; you should have an invite waiting. I've also invited Christoph Dorn for write access in case he decides he wants to contribute to de.po, even with his kleine Wortshatz :) .
While importing your tar ball, I discovered the following issue, which I throw out here mostly because I haven't thought about them:
version.c: how do we generate this with auto-fu?
gpsee_config.h: ditto - this is generated by spidermonkey/probe-jsapi.sh
lightning code review --
vm/vm.c -- JSPropertyStub -- is NULL really an acceptable value there?
modules/binary/bytehings.c -- good patch, how did you find that?
gpsee.h: I put the constructor warning back in, it's a REAL BIG BUG that we don't do that. Discussed as item #1 in preview email. But it some place smarter. ;)
On 23 September 2011 13:04, Jeff Johnson <n3...@mac.com> wrote:The real flaw is the JS package I'm using that passes JS_THREADSAFEon the compile line (so its enabled) but doesn't copy the valueinto whatever include file GPSEE is expecting and testing.
GRRR
That somebody trying to make JS 1.8.5 build "the same" as JS 1.4-1.8.0rc1, even though they have completely separate build systems. (Jim Blandy retrofitted the Moz autoconf-2.13 build system onto JS 1.8.1 ~ Jan 2008)
The proper fix will be to change the Fedora packaging: but as soon as I do that,then I'm running a custom JS package or have to teach some package monkeyho0w to peel a banana correctly: not gonna happen in finite time.
Yeah. I'll try and grab Colin Walters the next time I see him online.
FWIW, Ubuntu should have a good package, I worked with Chris Coulson on that, and Debian has "none". (Mike Hommey is shipping -lmozjs from a random recent Firefox, but I'm not sure why. GJS, maybe)
Well JS_THREADSAFE is only recently MANDATORY and thereis some hysteresis involved rebuilding packages.
-I/usr/include/js185/
Hang back a bit and I'll load your gun with a patch to make Colin'slife as easy as possible.
On 23 September 2011 14:17, Jeff Johnson <n3...@mac.com> wrote:Well JS_THREADSAFE is only recently MANDATORY and thereis some hysteresis involved rebuilding packages.
True - but JS_THREADSAFE should no longer be specified on the make command line (like when we had Makefile.ref), and instead should be the result of --enable-threadsafe.
(data point: after JS 1.8.5, --enable-threadsafe will be probably be the default)
I have been suggesting that distros build -lmozjs185 with --enable-threadsafe --with-system-nspr --enable-ctypes. What happens is that one of the header files gets a "#define JS_THREADSAFE" when emitted by configure. I think "js-config.h" but not 100% sure.