Google Groups

Re: The future of binary embedding

lkcl luke Feb 27, 2012 6:42 AM
Posted in group:
allo ben, thanks for taking this up.

On Mon, Feb 27, 2012 at 1:53 PM, Benjamin Smedberg
<> wrote:
> On 2/25/2012 3:53 AM, wrote:
>>> We never solved it fully, and each embedder has had to
>>>       spend lots of time tweaking things.
>>  perhaps it might be worthwhile for the mozilla foundation to find - and
>> fund - the people who _do_ understand these things, rather than totally
>> abandon existing efforts and leave entire communities completely without
>> support:
> There is no free lunch. As I noted, finding and funding embedding support
> must necessarily come at the expense of something we could be doing instead.
> Overall I'm not saying that we don't want Mozilla to be embeddable, but that
> there are more important things to work on, both strategically and
> technically. This is not a permanent calculus, and it can change again in
> the future.
> Part of this decision is merely making clear that there hasn't been a
> community maintaining this code effectively. If you'd like to build that
> community, I encourage you to do so!

 well...  there's the problem.

 there's a community *depending* on this stuff *working*, but they are
all python programmers.  they shouldn't _have_ to learn c or c++ -
massive amounts of it.

 and, because this code doesn't actually need "development" - just
maintenance - there's no "community" to actually build!

 there's a critical CRITICAL difference between middleware and the
code that *uses* that middleware.  the maintenance and development
cycles are _completely_ different.

 * code that uses middleware requires continuous upkeep, development
etc. in order to stay relevant.  it's also orders of magnitude larger
than the middleware it uses

 * middleware does NOT require constant development.  it works or it
doesn't.  if it doesn't work, it was designed WRONG in the first
place.  middleware DOES however require maintenance to keep up-to-date
with its dependencies (mozilla-core in this case).

 ... how can a community be built around something that requires only
a tiny amount of occasional maintenance?

 ... it can't, can it?  once every year to two years, a couple of
hard-core developers get together, bash out a few updates on the
dependencies, and then go away again?  that's *not* a "community", is
it, ben.

 you see what i'm getting at?

 just because xpcom (and hulahop) appear not to have an "active
community" does *not* mean that they are "shit" or "unmaintained" or
"should be relegated to third party status" or "removed entirely from
the mozilla codebase".

 that's the mistake that's been made.

>>  no it's not.  if you have things like XPCOM which is a dynamic "object
>> model" interface, you don't *need* "binary compatibility" - you just need to
>> make sure that XPCOM works.
>>  XPCOM, which was based on (inspired by) microsoft's COM, is just as
>> misunderstood, it seems, as COM is.
>>  the benefits of Common Object Model technology are just... i cannot
>> emphasise enough how powerful they are.  this is something that the Free
>> Software Community as a whole simply does NOT get, to its detriment.
>>  microsoft dominated computing for nearly two decades thanks to COM/DCOM.
>>  apple dominates its sector through Objective-C, at the heart of which is,
>> yes, you guessed it: a type of Dynamic Object Model system.
> objective-c and COM are fundamentally different in how they achieve
> backwards compatibility. XPCOM "working" correctly will prevent you from
> crashing, but it only achieves the goal of being *compatible* if you don't
> change the interfaces.

 ah.... in static programming languages, yes.  in dynamic programming
languages such as python where you use the trick of lazy / dynamic
binding (based on the name) - absolutely NOT.  the changing of the
interfaces is COMPLETELY irrelevant.

> We have tried this, and it's clear that our current
> interfaces are not suited to freezing.

 ben - python doesn't *care* about the interface GUIDs, and as long as
the interfaces are designed by adding extra default arguments (rather
than changing the number or order of the arguments), python doesn't
even care if you _add_ to the interfaces.

 even if you change the type of one of the arguments from say a
nsHTMLImage to a nsHTMLElement python *still* doesn't care!

 this is really really frustrating for me to have to be explaining
this stuff because you should _know_ how it works.

 but there's more.  do you remember this bugreport?

 do you remember that i said that adding that "optional_arg" concept
was a really bad idea [instead of doing proper IDispatch (coclass)
stuff where you can merge / look up multiple COM interfaces]?

 the reason for that is clear when you look at microsoft's COM IDL
files for MSHTML.  you don't need to look at _microsoft's_ COM IDL
file(s), you can look on the Wine web site, here:

 take a look for HTMLElement1, 2, 3 and 4, then look at line 7414.  do
you notice the use of that coclass thing?  what that does is it
*merges* all of the functions - each of which have the same name but
have *different* parameters (usually different numbers of parameters)
- so that you achieve the same thing that c++ has through function

 what's significant about this is that the interface UUIDs for
HTMLElement1, 2, 3 and 4 have *NEVER CHANGED*.  EVER.

 if a new function was ever needed, or a change to a parameter was
ever needed, they just added an extra definition (HTMLElement5) and
the advantage of that is that older (static languages, c, c++, java)
*still work*, and dynamic languages get to take care of accessing the
right thing via "lazy dynamic binding".

 do you see why this is so important?

> The XPCOM object model in gecko does
> not suit the needs of binary compatibility.

 again: not in a dynamic programming language is this even remotely a problem.

> We are focused primarily on the *script* compatibility, and to the extent
> that we want a good embedding API in the future, it will be designed around
> embedding via script, and primarily JavaScript since that is the language of
> the web. We do make strong compatibility guarantees for JS, and we should be
> able to do that for embedders as well.

 the static-language (c,c++,java) embedders are on their own, thanks
to XPCOM's design *not* having taken on the full features of COM which
was "inspired" by.

 the dynamic language embedders, when you have good "lazy / dynamic"
bindings such as that which todd (or... wait.. it was mark hammond
wasn't it?) created, simply *don't* care.

>> so do you think it would be a good start for the mozilla foundation to
>> adopt python-hulahop as the beginnings of reversing this trend towards
>> isolation of the mozilla foundation from absolutely everything but
>> javascript, javascript, javascript and firefox, firefox, firefox?
> What kind of "adoption"? I don't think that python-hulahop is strategically
> important to our core goals of innovation on the internet.

 ok - i'll pass that on to the pyjamas community.  i'll let them know
that not only has the mozilla foundation badly let down the OLPC
community but is quite prepared to abandon the pyjamas community as

 but before i do that, i think it might be a good idea if you ran
pyjamas-desktop and actually saw and understood what it can do.
you'll need to use debian 6.0 and do "apt-get install pyjamas" - it
specifically has to be debian 6.0 because that was the last time it
was possible to get a stable build of xulrunner 1.9, xpcom and
python-hulahop all together in the same distribution.

> So I don't think
> that in terms of the Mozilla mission our core community should make
> maintenance and development of that code a priority. I'd be happy to provide
> a place for you or somebody else to build a community around python-hulahop,
> though.

 the summary is: there *is* no expertise within the community that
*is* built around python-hulahop, and you've never run pyjamas-desktop
(not that i'm aware of) so it is very hard for you to comprehend (and
even harder for me to explain) what it is that i'm on about.

 so that just leaves... me.  the burden of responsibility for getting
this code working is entirely on me - and i'm not very happy about it.

 i've managed to hack python-xpcom into submission: i had to remove
the pyloader module because it relies on "MOZILLA_INTERNAL_API" stuff,
uses internal strings when PyXPCOM.h declares external ones; i tried
getting the into line with one of the js loaders
(mozJSComponentLoader) that i found: adding LIBXUL_LIBRARY=1 and
STATIC_LIBRARY=1 into pyxpcom's pyloader had *zero*
effect, i ended up with missing functions at linker time (ones that
were in -lxul *despite* linking with -lxul) - in the end i went "fuck
it" and commented the code out.

then i moved on to hulahop, and immediately ran into missing header
files that "make -f install" hadn't bothered to install
(large numbers of js* files).  i manually copied those over, and then
immediately ran into the build error below, at which point i decided
i'd had enough.

now, these are the kinds of things that will be blindingly-obvious to
you, because you're very very familiar with the ongoing day-to-day

... but we "outsiders" are *not*, and it's fucking annoying to have to
get up-to-speed _just_ the once, knowing full well that by the time
it's done, there's a good chance that there will have been *another*
release - 14, 15, 16, 17, 18 - of xulrunner - leaving absolutely NO
chance of getting this into the mainline linux distributions.

do you see what i'm getting at?

even if i did this work, getting hulahop up-and-running with each
release, it would be pointless, because the mozilla foundation "rushes
ahead" with release, release, release, release.

if you want to help, you need to include hulahop in the release cycle.
 bugs in hulahop because of changes made to xulrunner should be a
*critical* block on a xulrunner release.


In file included from /usr/local/include/xulrunner-13.0a1/jscntxt.h:56:0,
/usr/local/include/xulrunner-13.0a1/jsgc.h: In static member function
'static js::gc::Chunk*
/usr/local/include/xulrunner-13.0a1/jsgc.h:787:76: warning: invalid
access to non-static data member 'js::gc::Chunk::info'  of NULL object
/usr/local/include/xulrunner-13.0a1/jsgc.h:787:76: warning: (perhaps
the 'offsetof' macro was used incorrectly)
In file included from
                 from /usr/local/include/xulrunner-13.0a1/jscntxt.h:57,
/usr/local/include/xulrunner-13.0a1/vm/String.h: At global scope:
/usr/local/include/xulrunner-13.0a1/vm/String.h:67:6: error: multiple
definition of 'enum js::InternBehavior'
/usr/local/include/xulrunner-13.0a1/jsatom.h:555:6: error: previous
definition here
/usr/local/include/xulrunner-13.0a1/vm/String.h:69:23: error:
conflicting declaration 'DoNotInternAtom'
/usr/local/include/xulrunner-13.0a1/jsatom.h:557:23: error:
'js::DoNotInternAtom' has a previous declaration as
'js::InternBehavior js::DoNotInternAtom'
/usr/local/include/xulrunner-13.0a1/vm/String.h:70:18: error:
conflicting declaration 'InternAtom'
/usr/local/include/xulrunner-13.0a1/jsatom.h:558:18: error:
'js::InternAtom' has a previous declaration as 'js::InternBehavior
/usr/local/include/xulrunner-13.0a1/vm/String.h:80:91: error: default
argument given for parameter 3 of 'JSAtom*
js_AtomizeString(JSContext*, JSString*, js::InternBehavior)'
/usr/local/include/xulrunner-13.0a1/jsatom.h:573:1: error: after
previous specification in 'JSAtom* js_AtomizeString(JSContext*,
JSString*, js::InternBehavior)'
In file included from