The future of binary embedding

[Benjamin Smedberg]

Last summer, I led a session at the Mozilla summit to discuss whether
and how we ought to continue supporting our various embedding efforts
(gtkmozembed, javaxpcom, the ActiveX control, the NSView embedding
widget, etc) given the effort involved in preserving their various
degrees of code and binary compatibility with Mozilla core. We came the
following conclusions:

* Embedding Mozilla rendering into other processes is a tough
problem. We never solved it fully, and each embedder has had to
spend lots of time tweaking things.
* Firefox is the key product of the Mozilla project; to the extent
that supporting embedding takes away from Firefox, we should
strongly prioritize Firefox.
* Binary compatibility of our embedding bindings is a high cost
which is not worth the benefits.
* As we move Firefox into a multiple-process model, the embedding
solution we really want is very different from the one we
currently have: we really want embedders to be simple containers
for a separate Firefox process which would do the actual web
rendering.

Because of this, I'm planning on making the following changes in our code:

* Remove gtkmozembed and its supporting code. The promise of
gtkmozembed was a binary-compatible wrapper that GTK applications
could use to easily get web rendering into their application.
Without significant supporting code to deal with profiles,
certificates, prompts, and other details, this code doesn't work
correctly, and is unmaintained.
* Remove javaxpcom and its supporting code.
* Remove the ActiveX control and plugin, and the IDispatch code
which was created to support interconnecting the ActiveX code with
our DOM.

Various people have expressed interest in taking of maintenance of these
embedding solutions, but I lost their email addresses in a recent
computer crash. Anyone with interest should please email me, and I will
happily work with you to set up a Mozilla repository for the continued
maintenance of these projects.

As a project, we aren't going to spend effort trying to solve the
problems associated with in-process embedding. Once separate-process
rendering is implemented in Firefox, we may consider ways to make really
simple multi-process embedding a possibility. If you are interested in
helping to begin implementation of this multiprocess embedding solution,
please let me know, and I will help guide you in the right direction.

--BDS
Module Owner, Embedding

[Grant Gayed]

Hi Benjamin,

> As a project, we aren't going to spend effort trying to solve the problems
> associated with in-process embedding.

Can you clarify what this means? Is it just referring to the dropping of
gtkmozembed/javaxpcom/ActiveX control? Or is it saying that once mozilla
has entered the multi-process world that in-process embedding like can
currently be done via typical XPCOM will no longer be possible?

Thanks,
Grant

"Benjamin Smedberg" <benj...@smedbergs.us> wrote in message
news:mailman.771.130134620...@lists.mozilla.org...

[Martin Lutken]

I would like to know that too.
I don't use gtkmozembed or other stuff. I also don't need binary
compatibility since I compile Mozilla myself.
But does this mean that in the future I will not be able to embed
Mozilla in my own application using the functions 'XRE_InitEmbedding2'
and similar ?

If not will there be other ways to achive the same? Apart from
switching to Webkit that is.

-Martin

[Benjmain Smedberg]

On 4/1/11 1:21 PM, Grant Gayed wrote:
> Hi Benjamin,
>
>> As a project, we aren't going to spend effort trying to solve the problems
>> associated with in-process embedding.
> Can you clarify what this means? Is it just referring to the dropping of
> gtkmozembed/javaxpcom/ActiveX control? Or is it saying that once mozilla
> has entered the multi-process world that in-process embedding like can
> currently be done via typical XPCOM will no longer be possible?

I am not going to peer that far into the future and make grand
pronouncements. Today, the decision is very focused on removing these
embedding bindings which we don't have the community or capacity to
support. We are not removing the core nsIWebBrowser and such, so
in-process embedding is still possible.

--BDS

[sden...@rocketmail.com]

Benjamin,

1) I'm unclear of the impacts of removing gtkmozembed. Can you please clarify which APIs and functionality will be impacted? For example, will XRE_InitEmbedding go away with these near-term changes?

2) I too rebuild mozilla for a special xulrunner application that plays nicely with legacy gtk+/wx applications to provide an embedded browser. However, I utilize a wrapper for my access - wxWebConnect which is supported by Kirix:

http://www.kirix.com/labs/wxwebconnect.html

Will the changes you identify here impact that project and the apis/utilities they use?

3) What is the timeframe for these changes?

Thanks,
Susan

[CMB]

Would this terminate embedding gecko in IDE's (ex eclipse), TopStyle ecc.?

[Benjamin Smedberg]

On 4/7/11 9:41 AM, CMB wrote:
> Would this terminate embedding gecko in IDE's (ex eclipse), TopStyle ecc.?

Eclipse uses javaxpcom to do embedding. If somebody wishes to maintain
that code, they may continue to do so, but it will not be built into
XULRunner or available in the main mozilla codebase.

--BDS

[Benjamin Smedberg]

On 4/7/11 9:29 AM, sden...@rocketmail.com wrote:
> Benjamin,
>
> 1) I'm unclear of the impacts of removing gtkmozembed. Can you please clarify which APIs and functionality will be impacted? For example, will XRE_InitEmbedding go away with these near-term changes?

gtkmozembed is the code in embedding/browser/gtk and primarily the
functions in gtkmozembed.h. XRE_InitEmbedding is not affected.

> 2) I too rebuild mozilla for a special xulrunner application that plays nicely with legacy gtk+/wx applications to provide an embedded browser. However, I utilize a wrapper for my access - wxWebConnect which is supported by Kirix:
>
> http://www.kirix.com/labs/wxwebconnect.html
>
> Will the changes you identify here impact that project and the apis/utilities they use?

I don't know, that depends on how they do their embedding. But of
course, if they use gtkmozembed, they can import that code and continue
to use and maintain it.

> 3) What is the timeframe for these changes?

Immediately for Firefox 5.

--BDS

[sdenning]

Thank you so much for the response!

4) Even if we do not use gtkmozembed directly, are there other components that will be affected? Specifically, will there be an impact to programs like the wxEmbed test driver: mozilla/embedding/tests/wxEmbed/

5) Without javaxpcom, would there still be access to XPCOM objects via javascript?

Thanks!
Susan

[Benjamin Smedberg]

On 4/7/11 3:42 PM, sdenning wrote:
> 4) Even if we do not use gtkmozembed directly, are there other components that will be affected? Specifically, will there be an impact to programs like the wxEmbed test driver: mozilla/embedding/tests/wxEmbed/

I don't think that code is built, and will likely be removed also.

> 5) Without javaxpcom, would there still be access to XPCOM objects via javascript?

Yes, javaxpcom is XPCOM access from java, not javascript.

--BDS

[sdenning]

On Thursday, April 7, 2011 10:54:13 PM UTC, Benjamin Smedberg wrote:
> On 4/7/11 3:42 PM, sdenning wrote:
> > 4) Even if we do not use gtkmozembed directly, are there other components that will be affected? Specifically, will there be an impact to programs like the wxEmbed test driver: mozilla/embedding/tests/wxEmbed/
> I don't think that code is built, and will likely be removed also.

It doesn't get built automatically, but it does serve as a good sample program and in a xulrunner-1.9.0.6 distribution I was able to build it. I guess my concern is whether the APIs a wx-based C++ embedded app would be affected with the removal of gtkmozembed. It seems like gtkmozembed was just the embedding of low-level gtk, where embedded applications that use wx (wrapper for gtk) would be unaffected?

[Grant Gayed]

Actually Eclipse uses XPCOM to embed xulrunner, and exposes JavaXPCOM to
clients. So the second part is of course dependent on JavaXPCOM's fate, but
the embedding part could continue on without JavaXPCOM.

Grant

"Benjamin Smedberg" <benj...@smedbergs.us> wrote in message

news:mailman.1997.13022046...@lists.mozilla.org...

[luke.l...@gmail.com]

On Monday, March 28, 2011 10:03:11 PM UTC+1, Benjamin Smedberg wrote:
> Last summer, I led a session at the Mozilla summit to discuss whether
> and how we ought to continue supporting our various embedding efforts
> (gtkmozembed, javaxpcom, the ActiveX control, the NSView embedding
> widget, etc) given the effort involved in preserving their various
> degrees of code and binary compatibility with Mozilla core. We came the
> following conclusions:
>
> * Embedding Mozilla rendering into other processes is a tough
> problem.

no it's not - the python-hulahop widget uses GTK just like firefox does: it works perfectly, and it even provides access to XPCOM so that python-xpcom can get hold of the NPAPI and use DOM functionality in exactly the same way that javascript does [when xulrunner is embedded in firefox]

> We never solved it fully, and each embedder has had to
> spend lots of time tweaking things.

perhaps it might be worthwhile for the mozilla foundation to find - and fund - the people who _do_ understand these things, rather than totally abandon existing efforts and leave entire communities completely without support:

http://lists.sugarlabs.org/archive/sugar-devel/2011-June/031865.html

> * Firefox is the key product of the Mozilla project; to the extent
> that supporting embedding takes away from Firefox, we should
> strongly prioritize Firefox.
> * Binary compatibility of our embedding bindings is a high cost

no it's not. if you have things like XPCOM which is a dynamic "object model" interface, you don't *need* "binary compatibility" - you just need to make sure that XPCOM works.

XPCOM, which was based on (inspired by) microsoft's COM, is just as misunderstood, it seems, as COM is.

the benefits of Common Object Model technology are just... i cannot emphasise enough how powerful they are. this is something that the Free Software Community as a whole simply does NOT get, to its detriment. microsoft dominated computing for nearly two decades thanks to COM/DCOM. apple dominates its sector through Objective-C, at the heart of which is, yes, you guessed it: a type of Dynamic Object Model system.

> which is not worth the benefits.

what??? completely abandoning and orphaning entire communities is seen as "beneficial"??

> * As we move Firefox into a multiple-process model, the embedding
> solution we really want is very different from the one we
> currently have: we really want embedders to be simple containers
> for a separate Firefox process which would do the actual web
> rendering.
>
> Because of this, I'm planning on making the following changes in our code:
>
> * Remove gtkmozembed and its supporting code.

that's actually very sensible. gtkmozembed is pretty lame. it doesn't go nearly far enough. without dynamic language bindings - i.e. without access to what's actually *in* the web page, who the hell just wants to drop a web rendering engine into an application?

you didn't go far *enough*, ben.

people want to be able to drop in widgets into the middle of the HTML page: they want to be able to respond interactively to events (DOM events) - but not just in c, it has to happen in *all* programming languages, be it c, c++, java, python, ruby, perl - anything.

now.

that *is* possible with python-hulahop.

so do you think it would be a good start for the mozilla foundation to adopt python-hulahop as the beginnings of reversing this trend towards isolation of the mozilla foundation from absolutely everything but javascript, javascript, javascript and firefox, firefox, firefox?

> The promise of
> gtkmozembed was a binary-compatible wrapper that GTK applications
> could use to easily get web rendering into their application.
> Without significant supporting code to deal with profiles,
> certificates, prompts, and other details, this code doesn't work
> correctly, and is unmaintained.

hulahop manages absolutely fine. all that's written in python. it just works.

> As a project, we aren't going to spend effort trying to solve the
> problems associated with in-process embedding. Once separate-process
> rendering is implemented in Firefox, we may consider ways to make really
> simple multi-process embedding a possibility.

it's not enough.

you need to have the "dual worlds" functionality. a foot in the "embedded widget" door *and* a foot in the DOM world, via XPCOM.

hulahop provides exactly that.

the problem is that because it's "third party", it's absolute hell for non-funded part-time free software community members to keep it up-to-date, and the mozilla-foundation-driven release cycle does *not* include hulahop, meaning that releases like xulrunner 10.0.2 go out the door with serious memory corruption problems that, more by luck than judgement, happen *not* to be hit by firefox... but that hulahop runs smack into with a segfault.

what is the mozilla foundation intending to do about this, especially given that entire free software communities have been completely cut off by these decisions (the OLPC team and the pyjamas-desktop team), and that the mozilla foundation is at risk of not fulfilling its manifesto as a result.

l.

[luke.l...@gmail.com]

On Monday, March 28, 2011 10:03:11 PM UTC+1, Benjamin Smedberg wrote:

> Last summer, I led a session at the Mozilla summit to discuss whether
> and how we ought to continue supporting our various embedding efforts
> (gtkmozembed, javaxpcom, the ActiveX control, the NSView embedding
> widget, etc) given the effort involved in preserving their various
> degrees of code and binary compatibility with Mozilla core. We came the
> following conclusions:
>
> * Embedding Mozilla rendering into other processes is a tough
> problem.

no it's not - the python-hulahop widget uses GTK just like firefox does: it works perfectly, and it even provides access to XPCOM so that python-xpcom can get hold of the NPAPI and use DOM functionality in exactly the same way that javascript does [when xulrunner is embedded in firefox]

> We never solved it fully, and each embedder has had to
> spend lots of time tweaking things.

perhaps it might be worthwhile for the mozilla foundation to find - and fund - the people who _do_ understand these things, rather than totally abandon existing efforts and leave entire communities completely without support:

http://lists.sugarlabs.org/archive/sugar-devel/2011-June/031865.html

> * Firefox is the key product of the Mozilla project; to the extent
> that supporting embedding takes away from Firefox, we should
> strongly prioritize Firefox.
> * Binary compatibility of our embedding bindings is a high cost

no it's not. if you have things like XPCOM which is a dynamic "object model" interface, you don't *need* "binary compatibility" - you just need to make sure that XPCOM works.

XPCOM, which was based on (inspired by) microsoft's COM, is just as misunderstood, it seems, as COM is.

the benefits of Common Object Model technology are just... i cannot emphasise enough how powerful they are. this is something that the Free Software Community as a whole simply does NOT get, to its detriment. microsoft dominated computing for nearly two decades thanks to COM/DCOM. apple dominates its sector through Objective-C, at the heart of which is, yes, you guessed it: a type of Dynamic Object Model system.

> which is not worth the benefits.

what??? completely abandoning and orphaning entire communities is seen as "beneficial"??

> * As we move Firefox into a multiple-process model, the embedding
> solution we really want is very different from the one we
> currently have: we really want embedders to be simple containers
> for a separate Firefox process which would do the actual web
> rendering.
>
> Because of this, I'm planning on making the following changes in our code:
>
> * Remove gtkmozembed and its supporting code.

that's actually very sensible. gtkmozembed is pretty lame. it doesn't go nearly far enough. without dynamic language bindings - i.e. without access to what's actually *in* the web page, who the hell just wants to drop a web rendering engine into an application?

you didn't go far *enough*, ben.

people want to be able to drop in widgets into the middle of the HTML page: they want to be able to respond interactively to events (DOM events) - but not just in c, it has to happen in *all* programming languages, be it c, c++, java, python, ruby, perl - anything.

now.

that *is* possible with python-hulahop.

so do you think it would be a good start for the mozilla foundation to adopt python-hulahop as the beginnings of reversing this trend towards isolation of the mozilla foundation from absolutely everything but javascript, javascript, javascript and firefox, firefox, firefox?

> The promise of
> gtkmozembed was a binary-compatible wrapper that GTK applications
> could use to easily get web rendering into their application.
> Without significant supporting code to deal with profiles,
> certificates, prompts, and other details, this code doesn't work
> correctly, and is unmaintained.

hulahop manages absolutely fine. all that's written in python. it just works.

> As a project, we aren't going to spend effort trying to solve the
> problems associated with in-process embedding. Once separate-process
> rendering is implemented in Firefox, we may consider ways to make really
> simple multi-process embedding a possibility.

[luke.l...@gmail.com]

ben, there's a line of questioning that i'd like to engage you on.

please could you review the pyjd/mshtml.py code, which you can find here:
http://pyjs.org/pygit/

the first question is this: how the heck is it possible that i could create, in under 3 weeks, an entire port of pyjamas-desktop to use the MSHTML IE engine, *without* requiring access to the proprietary MSHTML Trident Engine?

and, how come that pyjd MSHTML port works, without fail, utterly flawlessly, with the exact same MSHTML engine from IE6, IE7, IE8 _and_ IE9, and i have reasonable confidence that it will continue to work without fail, without modifications required, on *all* future versions of the MSHTML Trident Engine.

the key question is this: what is it about the microsoft technology that makes this possible? this is not a rhetorical question. this is not a sarcastic question. i genuinely require that you actually think, ben, and find out the answer (hint: you already implement one. or used to. until it was pulled).

and if microsoft can do it, why can't the mozilla foundation?

more questions once you've answered the key question.

l.

[luke.l...@gmail.com]

[L. David Baron]

On Saturday 2012-02-25 00:53 -0800, luke.l...@gmail.com wrote:
> On Monday, March 28, 2011 10:03:11 PM UTC+1, Benjamin Smedberg wrote:
> > * Embedding Mozilla rendering into other processes is a tough
> > problem.
>
> no it's not - the python-hulahop widget uses GTK just like
> firefox does: it works perfectly, and it even provides access to
> XPCOM so that python-xpcom can get hold of the NPAPI and use DOM
> functionality in exactly the same way that javascript does [when
> xulrunner is embedded in firefox]

Has PyXPCOM been hooked up to the cycle collector yet, or do uses of
APIs that depend on cycle collection just leak memory?

-David

--
𝄞 L. David Baron http://dbaron.org/ 𝄂
𝄢 Mozilla http://www.mozilla.org/ 𝄂

[lkcl luke]

On Sat, Feb 25, 2012 at 4:59 PM, L. David Baron <dba...@dbaron.org> wrote:
> On Saturday 2012-02-25 00:53 -0800, luke.l...@gmail.com wrote:

>> On Monday, March 28, 2011 10:03:11 PM UTC+1, Benjamin Smedberg wrote:
>> >     * Embedding Mozilla rendering into other processes is a tough
>> >       problem.
>>
>>  no it's not - the python-hulahop widget uses GTK just like
>>  firefox does: it works perfectly, and it even provides access to
>>  XPCOM so that python-xpcom can get hold of the NPAPI and use DOM
>>  functionality in exactly the same way that javascript does [when
>>  xulrunner is embedded in firefox]
>

> Has PyXPCOM been hooked up to the cycle collector yet, or do uses of
> APIs that depend on cycle collection just leak memory?

david, thank you for responding on this.

... now, you see... this kind of information is entirely missing -
and it's the relegation of pyxpcom to "third party status" within
mozilla's project management - i.e. its removal from mozilla-central -
that results in things like that massive PRBool->bool substitution,
and other straightforward maintenance, being entirely missed.

so the code is just left to rot, or is left to someone who really
doesn't have the time or resources to focus 100% on keeping absolutely
without fail without exception absolutely 100% up-to-date with every
single one of the advances and modifications made to the mozilla core
codebase such that they even *know* about things like "the cycle
collector".

now, as it's a small codebase, i'm certain that someone who *does*
know about this "cycle collector", or any other code modifications,
could probably bring pyxpcom up-to-date prior to each xulrunner stable
release in very short order, especially when working alongside someone
like todd who is familiar with the code.

... but the fact that nobody has even bothered, and has pretty much
abandoned the code, leaves it as worse than useless, and a massive
drain on the time and resources of the free software community, who
have to learn or "re-learn" about each and every single API change,
each and every single mozilla core codebase change each and every
monotonous time *right* before a release.

that's massively unfair to expect members of the free software community to do.

hence why i have requested - formally on mozilla.governance - for
advice on how to go about making a formal request that the mozilla
foundation a) adopt python-hulahop as a 1st level priority project b)
reinstate pyxpcom as a 1st level priority project - preferably *back*
into mozilla-central. c) introduce unit tests that, if not passed,
will block the release of *any* xulrunner release at the time until
such time as the bugs are fixed and the unit tests pass.

the end result will actually be an *improvement* in the quality of
the xulrunner and the firefox codebase because it will result in a
different kind of testing - from a different angle with different
code-paths of the exact same core components that are utilised in
firefox.

do you happen to know to whom such formal requests should be directed?

many thanks,

l.

[L. David Baron]

On Saturday 2012-02-25 19:38 +0000, lkcl luke wrote:
> now, as it's a small codebase, i'm certain that someone who *does*
> know about this "cycle collector", or any other code modifications,
> could probably bring pyxpcom up-to-date prior to each xulrunner stable
> release in very short order, especially when working alongside someone
> like todd who is familiar with the code.

It's not. It's a massive task. (I'd guess it's roughly equivalent
in complexity to everything else in PyXPCOM. That is, I'd guess
that this would be about half the work of writing PyXPCOM from
scratch with it included. That's a very rough guess, though.)

> ... but the fact that nobody has even bothered, and has pretty much
> abandoned the code, leaves it as worse than useless, and a massive
> drain on the time and resources of the free software community, who
> have to learn or "re-learn" about each and every single API change,
> each and every single mozilla core codebase change each and every
> monotonous time *right* before a release.
>
> that's massively unfair to expect members of the free software
> community to do.

Why do you consider Mozilla employees different from other members
of the software community? Mozilla is paying its employees to be
members of the community -- and to perform the tasks where they'd
most effectively advance Mozilla's mission
( http://www.mozilla.org/about/manifesto.en.html ). There are other
companies who pay people to work on Mozilla code (e.g., in the area
of embedding, a number of Linux distributors have at various times).

> hence why i have requested - formally on mozilla.governance - for
> advice on how to go about making a formal request that the mozilla
> foundation a) adopt python-hulahop as a 1st level priority project b)
> reinstate pyxpcom as a 1st level priority project - preferably *back*
> into mozilla-central. c) introduce unit tests that, if not passed,
> will block the release of *any* xulrunner release at the time until
> such time as the bugs are fixed and the unit tests pass.

Demanding that somebody else do work for you doesn't seem like an
approach likely to succeed.

> the end result will actually be an *improvement* in the quality of
> the xulrunner and the firefox codebase because it will result in a
> different kind of testing - from a different angle with different
> code-paths of the exact same core components that are utilised in
> firefox.

That's at least a more reasonable argument. I don't believe it,
though.

I think any future in our embedding API should be based around APIs
designed to be an embedding API, not around a collection of internal
functions gathered up and labeled as an API. The problem with the
latter is that it's hard to avoid changing them, because they
weren't designed to be permanent. You're not going to turn the
latter into the former by shaking bugs out through additional
testing.

[lkcl luke]

allo david, thanks for responding here.

On Sun, Feb 26, 2012 at 4:20 AM, L. David Baron <dba...@dbaron.org> wrote:
> On Saturday 2012-02-25 19:38 +0000, lkcl luke wrote:
>>  now, as it's a small codebase, i'm certain that someone who *does*
>> know about this "cycle collector", or any other code modifications,
>> could probably bring pyxpcom up-to-date prior to each xulrunner stable
>> release in very short order, especially when working alongside someone
>> like todd who is familiar with the code.
>
> It's not.  It's a massive task.  (I'd guess it's roughly equivalent
> in complexity to everything else in PyXPCOM.  That is, I'd guess
> that this would be about half the work of writing PyXPCOM from
> scratch with it included.  That's a very rough guess, though.)

*whistles*. if you have time, could you help me to assess what's
actually involved? what code uses the "cycle collector", what patch
implemented a change-over to the functions behind "cycle collector"
etc. etc.

i note for example in xpcom/src/PyISupports.cpp that there are
functions utilised called nsMemory::Alloc and nsMemory::Free, and a
quick grep | wc shows 45 such functions in the 7,000 lines comprising
xpcom/src/*.cpp but i have no actual idea which c/c++ functions you're
referring to, yet.

>>  ... but the fact that nobody has even bothered, and has pretty much
>> abandoned the code, leaves it as worse than useless, and a massive
>> drain on the time and resources of the free software community, who
>> have to learn or "re-learn" about each and every single API change,
>> each and every single mozilla core codebase change each and every
>> monotonous time *right* before a release.
>>
>>  that's massively unfair to expect members of the free software
>>  community to do.
>
> Why do you consider Mozilla employees different from other members
> of the software community?  Mozilla is paying its employees to be
> members of the community -- and to perform the tasks where they'd
> most effectively advance Mozilla's mission
> ( http://www.mozilla.org/about/manifesto.en.html ).  There are other
> companies who pay people to work on Mozilla code (e.g., in the area
> of embedding, a number of Linux distributors have at various times).

this is a good point - i can't explain it though. i know that
something's "not right", but i don't have the... how-to-say... i
haven't been able to accurately articulate what i feel is the matter.

does that make any sense? i'm sitting here waving my hands, trying
to think how to express the frustration that i feel but the words just
aren't coming to me :)

>>  hence why i have requested - formally on mozilla.governance - for
>> advice on how to go about making a formal request that the mozilla
>> foundation a) adopt python-hulahop as a 1st level priority project b)
>> reinstate pyxpcom as a 1st level priority project - preferably *back*
>> into mozilla-central. c) introduce unit tests that, if not passed,
>> will block the release of *any* xulrunner release at the time until
>> such time as the bugs are fixed and the unit tests pass.
>
> Demanding that somebody else do work for you doesn't seem like an
> approach likely to succeed.

for "me"? no. you misunderstand.

if this was for *me* i would go and - personally - use pythonwebkit.
which i - me - *i* wrote that.
http://www.gnu.org/software/pythonwebkit. pythonwebkit does the
*exact* same job as xpcom+hulahop, from the high-level (python-based)
API perspective.

when i say the exact same job i mean that by the time you get to the
DOM/HTML5 level, all of the thousands of properties and functions, and
the hundreds of objects that are accessible to python, are EXACTLY the
same [near as damnit, and the pyjamas desktop codebase has a way of
normalising those differences seamlessly so that developers using its
SDK don't have to worry about those discrepancies].

so if this was JUST ME - if it was alllll about me, meee, meee, i
would be the kind of hypothetical egomaniac who would hypothetically
say hypothetically "f*** it, i'm not going to bother with these
[hypothetical] mozilla [hypothetical] jerks, iiii'm gonna use that
pythonwebkit code, nyer nyer nyer, [hypothetically] sod the lot of
you, nyer nyer nyer" - don't for one second think or imagine that i
ACTUALLY would say that, ok!

i - personally - have the intelligence and the capability to do that
kind of thing - to take on code that other people simply don't even
grasp, and to join it to *other* code that serves some completely
alternative purpose, and make the whole thing work.

btw it may interest you to know that pythonwebkit actually uses the
xulrunner python-based IDL file parser! i modified it to be able to
understand webkit's IDL files instead of xulrunner's IDL files! i
also adapted python-gobject's codegen.py for outputting the back-end
part of the c-based python module, joined the two together.

... but you see where i'm going with this? i neither made demands
nor expected my ego to be evaluated as part of the equation.

>>  the end result will actually be an *improvement* in the quality of
>> the xulrunner and the firefox codebase because it will result in a
>> different kind of testing - from a different angle with different
>> code-paths of the exact same core components that are utilised in
>> firefox.
>
> That's at least a more reasonable argument.  I don't believe it,
> though.

https://bugzilla.mozilla.org/show_bug.cgi?id=728645

looking back at the file history of FocusManager.cpp the bugreports
indicate that there have been changes to the FocusManager API in
exactly the area which is involved.

now the problem that _i_ have, as an outsider, is that normally i
would deploy "git bisect" techniques to track down the commit revision
with a binary search in order to isolate the commits between xulrunner
9.0 and xulrunner 10.0.2 which resulted in the segfault.

this technique is, as you know, commonly deployed as a means to
substitute for lack of unit test regression which _should_ have been
being run on a regular basis [note, pointedly, that hulahop is
*entirely* excluded from that daily continuous unit test regression
testing, which is what this is all about].

i looked for *two days* for an understanding of the mozilla codebase
and organisation in order to be able to provide that valuable
information - the two consecutive commits that did and did not result
in that segfault. could i find _anything_ which guided me on how to
do that commit bisecting? naah. i cannot find _anything_ on how to
relate commits in mozilla-central to the release cycles tree... it's
just .... waargh! :)

but anyway, that aside: can you see that between [working] xulrunner
9.0 code and [non-working] xulrunner 10.0.2 code (when using
xpcom+hulahop), there were some commits made that dealt with segfaults
on certain platforms (macosx, windows), but that somehow xulrunner on
gnu/linux does *not* have such segfaults?

but the key question is this: what do you think would have happened
if there was daily automated regression tests being carried out using
xpcom+hulahop?

do you think that xulrunner 10.0.2 would have gone out the door at
the time that it did, if this segfault had been detected at the time?

> I think any future in our embedding API should be based around APIs
> designed to be an embedding API, not around a collection of internal
> functions gathered up and labeled as an API.

david - you've entirely - i mean so completely missed the point it
actually takes my breath away.

>  The problem with the
> latter is that it's hard to avoid changing them, because they
> weren't designed to be permanent.

david: please look up what middleware technologies such as CORBA,
COM, Objective-C, gobject introspection etc. are, have achieved and
enabled, both historically (if 20 years can be considered "historic"
in the context of computing) as well as recently.

i don't wish to offend you or anything but the fact remains that
you've demonstrated a complete lack of comprehension of what XPCOM
actually brings to the table.

the summary is this: Common Object Model middleware such as
microsoft's COM, CORBA and XPCOM *are* permanent. they're a permanent
API that's so absolute "set-in-stone permanent" that 20 YEARS later,
it's possible to connect to even _proprietary_ DLLs that have a COM
interface, for example, and still use them in a modern application,
today.

even advances and improvements in COM over the past 2 decades are
done in such a way that they're always, always backwards-compatible.
backwards-compatibility is DESIGNED into the fundamental API of a good
Common Object Model's infrastructure, right at the very very beginning
when it's first created.

so please excuse me, but i'm a bit taken aback that you're commenting
on this topic when you've demonstrated a complete lack of knowledge of
the fundamentals of Common Object Model architectures. you're not
alone, though, i have to say.

>  You're not going to turn the
> latter into the former by shaking bugs out through additional
> testing.

david - i'm not being rude or anything but you just don't get it.
please review and understand what XPCOM and technology similar to it
actually is before commenting further on this subject.

i look forward to hearing from you in the near future when you will
be able to say "omg, did i _really_ say that? that's very funny!" and
we can have a bit of a laugh and share a joke about how much you
didn't know about the immense power of Common Object Model technology
(in general).

you're not alone. very very VERY few people in the Free Software
Community understand the immense power and design freedom that Common
Object Model technology gives people. some people _are_ finally
reinventing some of the concepts of COM, in the form of gobject
introspection, but that is really a very recent addition which will
take some considerable time to percolate through the free software
community.

l.

[lkcl luke]

david, hi,

i've got over being taken aback by what you said about the APIs thing,
and it occurred to me that if i didn't provide further information and
explanation there was the risk that people reading my comments could
potentially view the response as being from a "har har i'm more
superior than you har har you're so ignorant har har you know nothing
i know everything now go away and educate yourself" complete fucking
twonce (viz: a twat _and_ a nonce, rolled into one obsequious
egoistical moron whose fingers should be broken to prevent them from
being able to type on the internet, ever again).

as i'm _not_ that kind of person, it's a responsibility and a duty for
me to clarify what i said.

you said:

"I think any future in our embedding API should be based around APIs
designed to be an embedding API, not around a collection of internal

functions gathered up and labeled as an API. The problem with the

latter is that it's hard to avoid changing them, because they
weren't designed to be permanent."

the thing is: xpcom isn't an API, it's middleware which *provides*
access to APIs. dynamically. changes to the APIs that are reflected
*through* XPCOM automatically are available.

thus, "XPCOM-the-API" requires no "labelling". it just... is. it
works, it's done. no further development required. that's the nature
of Common Object Models [as long as they've actually been designed
properly...]

thus also, any APIs that are *available* through XPCOM are *also* "not
your problem". they simply aren't. they're available - transparently
- end of story.

... do you see therefore what i was driving at?

now in addition to that, there's hulahop, which is the "next layer
up", and it's... if you will... "grounding" of both the
XPCOM-accessible NPAPI and the gecko engine into the "reality" of
python from a declarative programming perspective.

look closely at this:
http://pyxpcomext.mozdev.org/no_wrap/tutorials/hulahop/xpcom-hulahop.html

look first at the implementation of progresslistener.py. what do you
notice about it? it's nothing more than an object-orientated
"wrapper", in python, around the XPCOM-accessible
nsIWebProgressListener interface, isn't it?

so there _is_ no "API" to be "developed", here, is there?

then, come back to the main hula.py file. what do you notice? what
evidence of the existence of any mozilla-required "APIs" do you see?
what in that file requires any actual API "design"? i honestly cannot
see any.

these lines, for example:
# this is equivalent to ~/.firefox profiles subdirectory
hulahop.startup('/home/lkcl/test')

they are absolutely essential. there's no "design" required. ok,
perhaps you might pick a different name from "startup", but who
honestly cares about that?

then there's the ContentInvoker class. this is required as part of
the XPCOM API, which is set in stone, is not going to change. the
invocation and linking of ContentInvoker class instances is, again,
set in stone:

listener = xpcom.server.WrapObject(ContentInvoker(self),
interfaces.nsIDOMEventListener)

so that is inviolate, as well. no API development required.

the class Browser, which derives from the c-based python-gtk module
"WebView", is where there *might* be room for "API Development", but
there you're actually actively constrained by the _GTK_ API, or more
specifically by the python-gtk2 API, which defines how pygtk widget
APIs are written.

so absolutely no "development" of "APIs" is required *there*, either.

about the only function which looks slightly odd, to me, is this
"do_setup" function. i don't actually know the deep compelling
technical reason why that's there, and i mention that in the tutorial.
looking in the source code hulahop.c it's _still_ not blindingly
obvious, but i'm not going to worry about it. the main point is: it
appears to be a techical detail associated with the fundamentals of
python c-based modules, getting over some problem/issue that the
hulahop developers created, and, again, it has absolutely nothing to
do with xulrunner "APIs".

conceivably you could argue that the name of this function "do_setup"
could be changed, but what would be the point?

so, throughout the entire hulahop codebase there is nothing -
absolutely not one single thing - that requires "designing". the
hybrid combination of the various bits of technology *automatically*
defines the published and public API.

you create a widget, you get access to the NPAPI. wow! that's so
simple, it's actually difficult to make it complicated enough for
people to grasp the significance.

so do you see? not only is there no API designing required, but there
exists proven stable code that already *implements* the
already-automatically-technically-constrained "API". code which, if
you even tried to modify it would result in yet further (and
unnecessary) knock-on effects for the users of that code in an already
untenable situation.

anyway. i'll try tackling pyxpcom to see if i can get it
up-and-running with xulrunner-13.

l.

[Benjamin Smedberg]

On 2/25/2012 3:53 AM, luke.l...@gmail.com wrote:
>
>> We never solved it fully, and each embedder has had to
>> spend lots of time tweaking things.
> perhaps it might be worthwhile for the mozilla foundation to find - and fund - the people who _do_ understand these things, rather than totally abandon existing efforts and leave entire communities completely without support:
>
> http://lists.sugarlabs.org/archive/sugar-devel/2011-June/031865.html

There is no free lunch. As I noted, finding and funding embedding
support must necessarily come at the expense of something we could be
doing instead. Overall I'm not saying that we don't want Mozilla to be
embeddable, but that there are more important things to work on, both
strategically and technically. This is not a permanent calculus, and it
can change again in the future.

Part of this decision is merely making clear that there hasn't been a
community maintaining this code effectively. If you'd like to build that
community, I encourage you to do so!

> no it's not. if you have things like XPCOM which is a dynamic "object model" interface, you don't *need* "binary compatibility" - you just need to make sure that XPCOM works.
>
> XPCOM, which was based on (inspired by) microsoft's COM, is just as misunderstood, it seems, as COM is.
>
> the benefits of Common Object Model technology are just... i cannot emphasise enough how powerful they are. this is something that the Free Software Community as a whole simply does NOT get, to its detriment. microsoft dominated computing for nearly two decades thanks to COM/DCOM. apple dominates its sector through Objective-C, at the heart of which is, yes, you guessed it: a type of Dynamic Object Model system.

objective-c and COM are fundamentally different in how they achieve
backwards compatibility. XPCOM "working" correctly will prevent you from
crashing, but it only achieves the goal of being *compatible* if you
don't change the interfaces. We have tried this, and it's clear that our
current interfaces are not suited to freezing. The XPCOM object model in
gecko does not suit the needs of binary compatibility.

We are focused primarily on the *script* compatibility, and to the
extent that we want a good embedding API in the future, it will be
designed around embedding via script, and primarily JavaScript since
that is the language of the web. We do make strong compatibility
guarantees for JS, and we should be able to do that for embedders as well.

> so do you think it would be a good start for the mozilla foundation to
> adopt python-hulahop as the beginnings of reversing this trend towards
> isolation of the mozilla foundation from absolutely everything but
> javascript, javascript, javascript and firefox, firefox, firefox?

What kind of "adoption"? I don't think that python-hulahop is
strategically important to our core goals of innovation on the internet.
So I don't think that in terms of the Mozilla mission our core community
should make maintenance and development of that code a priority. I'd be
happy to provide a place for you or somebody else to build a community
around python-hulahop, though.

--BDS

[Benjamin Smedberg]

On 2/25/2012 7:05 AM, luke.l...@gmail.com wrote:
> the first question is this: how the heck is it possible that i could create, in under 3 weeks, an entire port of pyjamas-desktop to use the MSHTML IE engine, *without* requiring access to the proprietary MSHTML Trident Engine?

The browser engine provided by Microsoft has a stable COM API. It also
does not require the embedder to handle any kind of persistent data
access: history/cache/cookie stores are handled via a single operating
system store which is shared across processes.

> the key question is this: what is it about the microsoft technology that makes this possible? this is not a rhetorical question. this is not a sarcastic question. i genuinely require that you actually think, ben, and find out the answer (hint: you already implement one. or used to. until it was pulled).

No, we didn't.

The ActiveX control provided an emulation layer which satisifed some of
the behaviors (allowing gecko to run in a process hidden behind at least
the basic set of Microsoft interfaces). But because we don't have any
form of cross-process profile sharing, the embedder either had to
configure and run their own profile directory (and think about the
consequences of only being able to use that datastore from one process
at a time) or run without any data store, which is not a well-tested
gecko configuration and will cause many websites to fail because cookies
and the cache do not behave correctly.

In addition, the primary way to access DOM nodes through the MS API is
via IDispatch, and the bridge between IDispach and COM typeinfo was
never complete and didn't match the Microsoft implementation in key
ways. The closest cross-platform equivalent we have to IDispatch is the
NPRuntime, which is pretty well tested but isn't efficient and has
inherent cycle issues.

>
> and if microsoft can do it, why can't the mozilla foundation?

We *could* do it. But it would require significant engineering effort to
implement shared profile data, solve the IDispatch-or-equivalent problem
(especially if we needed it to be safe for object cycles via the cycle
collector), and then provide real QA and release resources for the
project. Since we are not prepared to do any of that at this time, it's
better to be perfectly clear about the situation then string embedders
along thinking that they have a better solution than they actually have.

--BDS

[lkcl luke]

allo ben, thanks for taking this up.

On Mon, Feb 27, 2012 at 1:53 PM, Benjamin Smedberg
<benj...@smedbergs.us> wrote:
> On 2/25/2012 3:53 AM, luke.l...@gmail.com wrote:
>>
>>

>>> We never solved it fully, and each embedder has had to
>>>       spend lots of time tweaking things.
>>
>>  perhaps it might be worthwhile for the mozilla foundation to find - and
>> fund - the people who _do_ understand these things, rather than totally
>> abandon existing efforts and leave entire communities completely without
>> support:
>>
>>  http://lists.sugarlabs.org/archive/sugar-devel/2011-June/031865.html
>

> There is no free lunch. As I noted, finding and funding embedding support
> must necessarily come at the expense of something we could be doing instead.
> Overall I'm not saying that we don't want Mozilla to be embeddable, but that
> there are more important things to work on, both strategically and
> technically. This is not a permanent calculus, and it can change again in
> the future.
>
> Part of this decision is merely making clear that there hasn't been a
> community maintaining this code effectively. If you'd like to build that
> community, I encourage you to do so!

well... there's the problem.

there's a community *depending* on this stuff *working*, but they are
all python programmers. they shouldn't _have_ to learn c or c++ -
massive amounts of it.

and, because this code doesn't actually need "development" - just
maintenance - there's no "community" to actually build!

there's a critical CRITICAL difference between middleware and the
code that *uses* that middleware. the maintenance and development
cycles are _completely_ different.

* code that uses middleware requires continuous upkeep, development
etc. in order to stay relevant. it's also orders of magnitude larger
than the middleware it uses

* middleware does NOT require constant development. it works or it
doesn't. if it doesn't work, it was designed WRONG in the first
place. middleware DOES however require maintenance to keep up-to-date
with its dependencies (mozilla-core in this case).

... how can a community be built around something that requires only
a tiny amount of occasional maintenance?

... it can't, can it? once every year to two years, a couple of
hard-core developers get together, bash out a few updates on the
dependencies, and then go away again? that's *not* a "community", is
it, ben.

you see what i'm getting at?

just because xpcom (and hulahop) appear not to have an "active
community" does *not* mean that they are "shit" or "unmaintained" or
"should be relegated to third party status" or "removed entirely from
the mozilla codebase".

that's the mistake that's been made.

>>  no it's not.  if you have things like XPCOM which is a dynamic "object
>> model" interface, you don't *need* "binary compatibility" - you just need to
>> make sure that XPCOM works.
>>
>>  XPCOM, which was based on (inspired by) microsoft's COM, is just as
>> misunderstood, it seems, as COM is.
>>
>>  the benefits of Common Object Model technology are just... i cannot
>> emphasise enough how powerful they are.  this is something that the Free
>> Software Community as a whole simply does NOT get, to its detriment.
>>  microsoft dominated computing for nearly two decades thanks to COM/DCOM.
>>  apple dominates its sector through Objective-C, at the heart of which is,
>> yes, you guessed it: a type of Dynamic Object Model system.
>

> objective-c and COM are fundamentally different in how they achieve
> backwards compatibility. XPCOM "working" correctly will prevent you from
> crashing, but it only achieves the goal of being *compatible* if you don't
> change the interfaces.

ah.... in static programming languages, yes. in dynamic programming
languages such as python where you use the trick of lazy / dynamic
binding (based on the name) - absolutely NOT. the changing of the
interfaces is COMPLETELY irrelevant.

> We have tried this, and it's clear that our current
> interfaces are not suited to freezing.

ben - python doesn't *care* about the interface GUIDs, and as long as
the interfaces are designed by adding extra default arguments (rather
than changing the number or order of the arguments), python doesn't
even care if you _add_ to the interfaces.

even if you change the type of one of the arguments from say a
nsHTMLImage to a nsHTMLElement python *still* doesn't care!

this is really really frustrating for me to have to be explaining
this stuff because you should _know_ how it works.

but there's more. do you remember this bugreport?
https://bugzilla.mozilla.org/show_bug.cgi?id=502234

do you remember that i said that adding that "optional_arg" concept
was a really bad idea [instead of doing proper IDispatch (coclass)
stuff where you can merge / look up multiple COM interfaces]?

the reason for that is clear when you look at microsoft's COM IDL
files for MSHTML. you don't need to look at _microsoft's_ COM IDL
file(s), you can look on the Wine web site, here:

http://source.winehq.org/source/include/mshtml.idl

take a look for HTMLElement1, 2, 3 and 4, then look at line 7414. do
you notice the use of that coclass thing? what that does is it
*merges* all of the functions - each of which have the same name but
have *different* parameters (usually different numbers of parameters)
- so that you achieve the same thing that c++ has through function
overloading.

what's significant about this is that the interface UUIDs for
HTMLElement1, 2, 3 and 4 have *NEVER CHANGED*. EVER.

if a new function was ever needed, or a change to a parameter was
ever needed, they just added an extra definition (HTMLElement5) and
the advantage of that is that older (static languages, c, c++, java)
*still work*, and dynamic languages get to take care of accessing the
right thing via "lazy dynamic binding".

do you see why this is so important?

> The XPCOM object model in gecko does
> not suit the needs of binary compatibility.

again: not in a dynamic programming language is this even remotely a problem.

> We are focused primarily on the *script* compatibility, and to the extent
> that we want a good embedding API in the future, it will be designed around
> embedding via script, and primarily JavaScript since that is the language of
> the web. We do make strong compatibility guarantees for JS, and we should be
> able to do that for embedders as well.

the static-language (c,c++,java) embedders are on their own, thanks
to XPCOM's design *not* having taken on the full features of COM which
was "inspired" by.

the dynamic language embedders, when you have good "lazy / dynamic"
bindings such as that which todd (or... wait.. it was mark hammond
wasn't it?) created, simply *don't* care.

>> so do you think it would be a good start for the mozilla foundation to
>> adopt python-hulahop as the beginnings of reversing this trend towards
>> isolation of the mozilla foundation from absolutely everything but
>> javascript, javascript, javascript and firefox, firefox, firefox?
>

> What kind of "adoption"? I don't think that python-hulahop is strategically
> important to our core goals of innovation on the internet.

ok - i'll pass that on to the pyjamas community. i'll let them know
that not only has the mozilla foundation badly let down the OLPC
community but is quite prepared to abandon the pyjamas community as
well.

but before i do that, i think it might be a good idea if you ran
pyjamas-desktop and actually saw and understood what it can do.
you'll need to use debian 6.0 and do "apt-get install pyjamas" - it
specifically has to be debian 6.0 because that was the last time it
was possible to get a stable build of xulrunner 1.9, xpcom and
python-hulahop all together in the same distribution.

> So I don't think
> that in terms of the Mozilla mission our core community should make
> maintenance and development of that code a priority. I'd be happy to provide
> a place for you or somebody else to build a community around python-hulahop,
> though.

the summary is: there *is* no expertise within the community that
*is* built around python-hulahop, and you've never run pyjamas-desktop
(not that i'm aware of) so it is very hard for you to comprehend (and
even harder for me to explain) what it is that i'm on about.

so that just leaves... me. the burden of responsibility for getting
this code working is entirely on me - and i'm not very happy about it.

i've managed to hack python-xpcom into submission: i had to remove
the pyloader module because it relies on "MOZILLA_INTERNAL_API" stuff,
uses internal strings when PyXPCOM.h declares external ones; i tried
getting the Makefile.in into line with one of the js loaders
(mozJSComponentLoader) that i found: adding LIBXUL_LIBRARY=1 and
STATIC_LIBRARY=1 into pyxpcom's pyloader Makefile.in had *zero*
effect, i ended up with missing functions at linker time (ones that
were in -lxul *despite* linking with -lxul) - in the end i went "fuck
it" and commented the code out.

then i moved on to hulahop, and immediately ran into missing header
files that "make -f client.mk install" hadn't bothered to install
(large numbers of js* files). i manually copied those over, and then
immediately ran into the build error below, at which point i decided
i'd had enough.

now, these are the kinds of things that will be blindingly-obvious to
you, because you're very very familiar with the ongoing day-to-day
development.

... but we "outsiders" are *not*, and it's fucking annoying to have to
get up-to-speed _just_ the once, knowing full well that by the time
it's done, there's a good chance that there will have been *another*
release - 14, 15, 16, 17, 18 - of xulrunner - leaving absolutely NO
chance of getting this into the mainline linux distributions.

do you see what i'm getting at?

even if i did this work, getting hulahop up-and-running with each
release, it would be pointless, because the mozilla foundation "rushes
ahead" with release, release, release, release.

if you want to help, you need to include hulahop in the release cycle.
bugs in hulahop because of changes made to xulrunner should be a
*critical* block on a xulrunner release.

l.

In file included from /usr/local/include/xulrunner-13.0a1/jscntxt.h:56:0,
from
/home/lkcl/src/sugar-hulahop/./src/hulahop-web-view.cpp:33:
/usr/local/include/xulrunner-13.0a1/jsgc.h: In static member function
'static js::gc::Chunk*
js::gc::Chunk::fromPointerToNext(js::gc::Chunk**)':
/usr/local/include/xulrunner-13.0a1/jsgc.h:787:76: warning: invalid
access to non-static data member 'js::gc::Chunk::info' of NULL object
/usr/local/include/xulrunner-13.0a1/jsgc.h:787:76: warning: (perhaps
the 'offsetof' macro was used incorrectly)
In file included from
/usr/local/include/xulrunner-13.0a1/jspropertycache.h:48:0,
from /usr/local/include/xulrunner-13.0a1/jscntxt.h:57,
from
/home/lkcl/src/sugar-hulahop/./src/hulahop-web-view.cpp:33:
/usr/local/include/xulrunner-13.0a1/vm/String.h: At global scope:
/usr/local/include/xulrunner-13.0a1/vm/String.h:67:6: error: multiple
definition of 'enum js::InternBehavior'
/usr/local/include/xulrunner-13.0a1/jsatom.h:555:6: error: previous
definition here
/usr/local/include/xulrunner-13.0a1/vm/String.h:69:23: error:
conflicting declaration 'DoNotInternAtom'
/usr/local/include/xulrunner-13.0a1/jsatom.h:557:23: error:
'js::DoNotInternAtom' has a previous declaration as
'js::InternBehavior js::DoNotInternAtom'
/usr/local/include/xulrunner-13.0a1/vm/String.h:70:18: error:
conflicting declaration 'InternAtom'
/usr/local/include/xulrunner-13.0a1/jsatom.h:558:18: error:
'js::InternAtom' has a previous declaration as 'js::InternBehavior
js::InternAtom'
/usr/local/include/xulrunner-13.0a1/vm/String.h:80:91: error: default
argument given for parameter 3 of 'JSAtom*
js_AtomizeString(JSContext*, JSString*, js::InternBehavior)'
/usr/local/include/xulrunner-13.0a1/jsatom.h:573:1: error: after
previous specification in 'JSAtom* js_AtomizeString(JSContext*,
JSString*, js::InternBehavior)'
In file included from
/usr/local/include/xulrunner-13.0a1/jspropertycache.h:48:0,

[lkcl luke]

allo ben,

ok, again, as this is quite long, let me do a summary:

* i'm again a bit caught out by having to correct some things which it
didn't occur to me that you might not be aware of, apologies for that
- it always catches me off-guard and i don't really know how to handle
it [socially]. so, apologies in advance.

* python's a scripted and dynamic language, and this is GUIs.

a) speed is therefore not a critical consideration
b) the design of pyjamas apps are always ALWAYS single-threaded
c) there IS only ever one profile (hulahop.setup("~/.mozilla/hula")
d) "lazy binding" in pyxpcom takes care of interface changes. we
simply do *not* care.

* there's an entire "cool!" suggestion for an embedded approach, i
won't spoil the surprise by mentioning it here :)

* pyxpcom and hulahop do *not* require any "development": they just
"work". they *only* require "maintenance". the resource allocation
is therefore radically different.

* unit tests and regression tests which can be automated, using
hulahop (and some tricks from the pyjamas-desktop codebase), give the
mozilla foundation an "extra string to the testing bow" of the entire
xulrunner runtime.

On Mon, Feb 27, 2012 at 2:06 PM, Benjamin Smedberg
<benj...@smedbergs.us> wrote:
> On 2/25/2012 7:05 AM, luke.l...@gmail.com wrote:
>>

>> the first question is this: how the heck is it possible that i could
>> create, in under 3 weeks, an entire port of pyjamas-desktop to use the
>> MSHTML IE engine, *without* requiring access to the proprietary MSHTML
>> Trident Engine?
>

> The browser engine provided by Microsoft has a stable COM API. It also does
> not require the embedder to handle any kind of persistent data access:
> history/cache/cookie stores are handled via a single operating system store
> which is shared across processes.
>>

>> the key question is this: what is it about the microsoft technology that
>> makes this possible?  this is not a rhetorical question.  this is not a
>> sarcastic question.  i genuinely require that you actually think, ben, and
>> find out the answer (hint: you already implement one.  or used to.  until it
>> was pulled).
>

> No, we didn't.

the question was generic - ok, you got it (the answer's ActiveX), but
you've slightly misunderstood... or perhaps jumped the gun, i'm not
sure which :)

> The ActiveX control provided an emulation layer which satisifed some of the
> behaviors (allowing gecko to run in a process hidden behind at least the
> basic set of Microsoft interfaces).

yes - you've got it. Microsoft created the (entire!) ActiveX
concept, in order to allow people to be able to embed COM-enabled
controls which conformed to the standard win32 GDI interface into
*any* application.

it didn't - doesn't - matter if that ActiveX component was written 2
or 20 years ago: you grab that DLL (and its associated .tlb) and you
can use it, now and forever.

the reason why i mentioned this was not because the mozilla
foundation created an ActiveX control which installs on the *windows*
platform....

.... i wanted to advocate that the mozilla foundation work with the
free software community to bring about ActiveX-like technology... for
*GNU/Linux* (!) in exactly the same way that it created the
absolutely astounding XPCOM technology.

it's the next logical step.

but if that's too much, then the logical alternative is to rip out
XPCOM and substitute COM for Unix, instead, and then take on some of
the source code from wine [which has had to create a perfect replica
of ActiveX].

don't believe there's

> But because we don't have any form of
> cross-process profile sharing, the embedder either had to configure and run
> their own profile directory (and think about the consequences of only being
> able to use that datastore from one process at a time)

yes - that's exactly what hulahop does. what's the problem? there's
only ever one process, and there's absolutely no problem with that.

the reason why in pyjamas-desktop that that's not a problem is
because... ok, *deep breath*, this is going to need some explaining.

when you compile a pyjamas application to run in a web browser, it
runs in the [single-process, no-threaded] javascript context. with
the exception of google's "gears" plugin, javascript execution in
browser engines is *entirely* "one jobbie". this makes life reaaally
easy from a programming perspective: it's a great way to do things.
no mutexes, no shared memory, naaathing.

if you want multiple "things" going on, you do it as timers, as ajax
etc. etc. but even there, only one "thing" can ever be executed (one
timer, one onload event etc.) this is _great_, because it means that
any "complexity" is done server-side. server-side is where you do
multi-processing, multi-threading, use Rabbit-MQ and other
asynchronous blah blah even multiple AJAX queries.

and the fantastic thing about this type of design, which is very
ideally close to the best MVC-like design principles, is that the
responsiveness of the UI front-end is *not* impeded.

it's great, in other words!

now here's the kicker: pyjamas desktop is ABSOLUTELY NO DIFFERENT.

let that sink in for a moment.

in case it's not obvious what the "big deal" is, i'll explicitly tell
you: a pyjamas-desktop back-end using e.g. hulahop *still* uses
nsITimer, *still* even uses nsIXMLHttpRequest, and, critically,
*still* talks to the EXACT same back-end server-side code that the
[javascript-compiled] pyjs version of the application does.

do you see the implications here?

we've just turned a web browser engine into a world-class GUI Widget
Set which *doesn't* suffer from "blocking" on the GUI front-end
because the development paradigm forces developers to follow
close-to-ideal pure MVC design principles.

the bottom line is that there *are* no processes (other that qty 1),
and if anyone actually ever tried to do something like use
g_object_timer or g_threads they would very very quickly run into
serious trouble with race conditions, risk of over-utilisation of CPU
resources and so on _and_ they wouldn't be able to compile the
application to javascript for running in web browsers (because
g_object_timer doesn't exist in web browsers).

the reason i know this is because i tried using g_object_timer in
early versions of the pyjd hulahop port, and yes, i ran into massive
problems :)

so can you see, the requirements are completely different, here, from
anything else you've previously encountered. pyjamas and
pyjamas-desktop is a _mature_ development environment. we've _been_
through the wringer on these issues.

> or run without any
> data store, which is not a well-tested gecko configuration and will cause
> many websites to fail because cookies and the cache do not behave correctly.

yes, i can see that would be the case.

> In addition, the primary way to access DOM nodes through the MS API is via
> IDispatch, and the bridge between IDispach and COM typeinfo was never
> complete and didn't match the Microsoft implementation in key ways. The
> closest cross-platform equivalent we have to IDispatch is the NPRuntime,
> which is pretty well tested but isn't efficient and has inherent cycle
> issues.

this is python. it's a scripted programming language. we don't
care. lazy dispatch, remember?

plus, i *know* that python-comtypes on win32 actually solved this,
gains access to the MSHTML COM Interface, fires up an IWebBrowser2
ActiveX plugin, and gets on with the job.

you remember i mentioned that "lazy dispatch" stuff? each and every
DOM access (function, property) goes via the "named" lookup, each and
every time. i believe it may be cached (in memory) which is part of
the lazy dispatcher's job.

and i _believe_ that mark hammond, who i believe also wrote the
python-comtypes stuff - also made sure that python-xpcom did likewise
[used lazy dispatch].

this is GUI stuff - it's not speed critical. it just has to work,
and it _does_ work. on pythonwebkit (which is a bit different, as it
has direct python bindings, none of this COM/XPCOM stuff
unfortunately), and when using hulahop, _and_ when using COM on win32.

so i know it can be done; i know also that hulahop actually works,
actually does the job - perfectly - for which it was accidentally
designed _way_ beyond what the hulahop creators actually needed it for
(!)

ironic, huh?

>>
>>
>> and if microsoft can do it, why can't the mozilla foundation?
>

> We *could* do it. But it would require significant engineering effort to
> implement shared profile data, solve the IDispatch-or-equivalent problem
> (especially if we needed it to be safe for object cycles via the cycle
> collector), and then provide real QA and release resources for the project.

ben - hulahop _works_. it doesn't *need* "development", and it
doesn't need shared profile data. one function call -
hulahop.setup("~/.mozilla/hulahopdir") - is all that's needed, and
this has been working - as needed and as designed - since it was
written back in 2006.

all that's needed is for hulahop (and pyxpcom) to be built on the
daily build cycles, and if it breaks, someone tracks down what the
problem is, reverts the commit that broke it until they also create a
patch that updates the code in hulahop (or pyxpcom). and/or makes
damn sure that xulrunner releases *don't* go out the door until
hulahop (and pyxpcom) are patched to work.

the reason for this is because both hulahop and pyxpcom are both
_mature_ products, they're _both_ middleware, and they do *not* need
"development" - just maintenance, to ensure that they're kept
up-to-date with their dependencies (which happen to be mozilla core
libraries unfortunately).

anyway: that aside - apart from hulahop+pyxpcom not being a massive
burden (because they don't need "development", only maintenance),
there really does exist technical development advantages to having a
2nd code-path for regression and unit testing, as that focus-related
bug hints at.

l.

[lkcl luke]

On Sat, Feb 25, 2012 at 4:59 PM, L. David Baron <dba...@dbaron.org> wrote:

> Has PyXPCOM been hooked up to the cycle collector yet, or do uses of
> APIs that depend on cycle collection just leak memory?

david, hi, this had me thinking again. pyxpcom was designed, eek...
a decade ago? but i'm not sure it's entirely relevant. the logic
goes as follows:

* python objects (in c) have their own obj-ref count concept
* XPCOM objects also have their own obj-ref count concept (AddRef)
which is exactly the same as COM, which "inspired" XPCOM.
* when a python object corresponding to an XPCOM object is created,
python takes care of adding *one* and *only* one XPCOM object into the
c data structure, and calls AddRef *once* and *only* once.
* also at the same time, the python object has its PyObject_Ref
increased (by one, obviously).
* when you need that exact same XPCOM object, if mark hammond didn't
program a way to look up the python object which was *already* created
to look after it, i'll fall off my bloody chair.
* so, assuming that mark is the competent programmer that he is, when
you look up an XPCOM object for which a python object already exists,
you DO NOT increase the refcount (AddRef) on the XPCOM object, you
increase the refcount on the *PYTHON* object.
* when the python object's refcount goes to zero, *python's*
garbage-collection takes care of destroying the python object, and in
the "destructor" function (in the c-based pyxpcom module), that code
also performs an XPCOM "Release" call.

so with this in mind, i do not see how the "cycle collector" has any
relevance, because pyxpcom is merely a direct reflector of the XPCOM
objects. but i believe that todd would welcome some input - to todd,
who is the de-facto maintainer of pyxpcom - because he, as the
de-facto maintainer, would be the person with the most experience in
actually sorting it out.

esp. if it's a global function that, if not called, results in XPCOM
objects not getting properly garbage-collected after the "Release" or
something - i don't know, because i don't know what this "cycle
collector" is, or what it does.

just fyi i went through some... interesting times with webkit's
memory management. they don't have anything like XPCOM: it's entirely
c++ classes. a massive suite of virtual base classes that derive from
a class named "DOMObject". mostly :)

at the ultimate lowest level they had to implement the usual "type
id" system which lets you externally identify an object even when it's
type-cast back down to the absolute lowest object in the hierarchy
(gobject has something similar, except they do it entirely in c!).
XPCOM and COM both have something similar, except they drop a GUID
into the type field.

this type id allows you to *BACK*-typecast a c++ object up to its
correct (real) class!

on top of this, they have c++ refcounting (virtual base class, again).

as the concept of XPCOM / COM was entirely missing, they have a
templated infrastructure which allows you to map
object-of-the-bound-type to
object-of-the-webkit-type-when-typecast-down-to-the-base-class-type.

then you have to _manually_ provide a massive suite of "upcast"
functions which check the type-id and call the next "upcast" function
in the chain, after doing a switch statement to get there!

all this to handle the one-for-one representation between (say)
python objects and webkit's c++ objects, or gobjects and webkit's c++
objects.

but even here, you can _still_ do and only need to do one and only
one refcount increase on a per-object basis, because once you've
gotten into the other side (the bindings), the refcounting of *that*
system takes over.

so there are close parallels - it's still complicated as hell though :)

anyway, do let todd know about this "cycle collection" because i have
no idea if he knows about it.

l.

[lkcl luke]

btw, ben, about speed of using XPCOM through pyxpcom+hulahop...

when i said we don't care [because working is a higher priority than
speed, and it's a GUI anyway], i think it helps to give some context.

for the pyjs (compiled) version of any pyjamas application, the fact
that python is a dynamic language, being translated to another dynamic
language, gives a performance hit that's so high it's difficult for
people to believe it could be that bad, just to obtain python feature
interoperability.

[with the end result that they go off and implement their *own*
python-to-javascript compiler, in some cases. this has actually
happened about four times, now!]

i spoke about this on b2g a couple of months back, when chris jones
(bless him) rather naively said that javascript was acceptable as the
de-facto "language of the web" implication being in absolutely all
circumstances because of projects such as dart, gwt and pyjamas.

if he'd ever run pyjs he'd've known that that simply isn't true,
because to even do something as simple as use the "add" operator, you
must check first if the two objects being added actually exist - which
must be done dynamically in case __getattr__ has to be used; then you
have to check the type of both objects (int? float? string?) and go
through a lookup table (!!) to find out if the two are compatible;
then you must coerce one type to the other; then _finally_ you can
actually add the two objects.

that very _very_ small and i mean small description should have you
going "oops..." and it should emphasise that maybe javascript as a
compiler target isn't such a good idea after all - not for dynamic
languages, anyway.

by contrast: for _static_ programming languages like java, c and c++,
it's great!

but for pyjamas, the performance hit is so high that when you run the
exact same application under pyjd it's a five-fold performance
improvement.

against that kind of background, and bearing in mind that that's still
using name-based lookups for absolutely each and every single access
to absolutely every single one of the functions, objects and
properties of DOM/HTML5, what we have right now looks very very good.

speed's *not* critical. we're using xulrunner+xpcom+hulahop as a GUI
widget set. a very powerful and flexible one. pyjd doesn't have to
be lightning-quick. i've installed it on 400mhz ARM9 CPUs, and it had
perfectly acceptable performance.

l.

[lkcl luke]

the policy of relegating pyxpcom to "third party status" within the
mozilla codebase has resulted in these kinds of things, below.

you will recognise this as the consequences of a decision some time in
the past few weeks (which i wasn't party to, because i am an
"outsider" having no funds or time to pay 24x7 attention to each and
every decision made by the people who _are_ funded by the mozilla
foundation)

basically that decision looks like it involved deliberate splitting of
the header files into "public api" and "not public api".

so, this header file nsIProxyObjectManager.h is treated as "internal api"...

... but somebody forgot that pyxpcom needs it... because they have
relegated pyxpcom to "third party status".

they forgot that pyxpcom *needs* to be built as an "internal"
component, yet the mozilla foundation has completely failed to provide
a system for building such components, yet at the same time has
basically dumped the responsibility onto other people.

if however pyxpcom was actually part of mozilla-central, then not only
would this problem simply not be present but also it would be a damn
sight easier to find issues.

temporarily i am forced to manually seek out, copy and make available
this "nsIProxyObjectManager.h" file each and every damn time i do a
rebuild and reinstall of a particular version of the xulrunner
runtime.

that's incredibly tiresome and it's part of the *additional* burden
that the mozilla foundation's decisions are placing onto external
developers.

... developers that are *not* funded by the mozilla foundation, it's
worth once again pointing out.

right now i cannot agree more that the OLPC team made the right
decision to remove all mozilla-foundation-funded technology from their
machines.

l.

c++ -o _xpcom.o -c -I../../../dist/system_wrappers -include
../../../../config/gcc_hidden.h -DOSTYPE=\"Linux2.6\" -DOSARCH=Linux
-DMOZ_NO_MOZALLOC -I/usr/include/python2.7
-I../../../../xpcom/src/module -I. -I../../../dist/include
-I../../../dist/include/nsprpub
-I/usr/local/lib/xulrunner-devel-13.0a1/include
-I/usr/local/lib/xulrunner-devel-13.0a1/include/nsprpub
-I/usr/include/nspr -fPIC -fno-rtti -fno-exceptions -Wall
-Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy
-Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof
-Wno-long-long -pedantic -DXULRUNNER_13_0A1 -std=gnu++0x -fshort-wchar
-fno-strict-aliasing -pipe -std=gnu++0x -DNDEBUG -DTRIMMED -Os
-freorder-blocks -fno-reorder-functions -DMOZILLA_CLIENT -include
/usr/local/lib/xulrunner-devel-13.0a1/include/mozilla-config.h
-Wp,-MD,.deps/_xpcom.pp ../../../../xpcom/src/module/_xpcom.cpp
../../../../xpcom/src/module/_xpcom.cpp:68:35: fatal error:
nsIProxyObjectManager.h: No such file or directory
compilation terminated.

[lkcl luke]

https://bugzilla.mozilla.org/show_bug.cgi?id=731121

where is the information required to fix this problem?

also, why was pyxpcom excluded from the update.... my god this was _months_ ago!
https://bugzilla.mozilla.org/show_bug.cgi?id=675221

why was pyxpcom not included in that update, when the information was
fresh in everyone's minds??

oh i know - because pyxpcom has been relegated to third party status.

ok.

how is this going to get fixed? (a so that this stops happening b
these specific coding errors so i can get some working code)

[Kyle Huey]

On Wed, Feb 29, 2012 at 4:30 PM, lkcl luke <luke.l...@gmail.com> wrote:

> <https://bugzilla.mozilla.org/show_bug.cgi?id=731121> oh i know - because

> pyxpcom has been relegated to third party status.
>

Yes, exactly.

how is this going to get fixed? (a so that this stops happening b
> these specific coding errors so i can get some working code)
>

It's not going to get fixed unless someone steps up to fix it. And as has
been stated several times, the core Mozilla community has no interest in
doing so.

- Kyle

[lkcl luke]

On Thu, Mar 1, 2012 at 1:07 AM, Kyle Huey <m...@kylehuey.com> wrote:
> On Wed, Feb 29, 2012 at 4:30 PM, lkcl luke <luke.l...@gmail.com> wrote:
>>
>> oh i know - because pyxpcom has been relegated to third party status.
>
>
> Yes, exactly.

now i am spending considerable time spending considerable core
mozilla community's time dealing with the mess caused by that
decision.

>> how is this going to get fixed? (a so that this stops happening b
>> these specific coding errors so i can get some working code)
>
>
> It's not going to get fixed unless someone steps up to fix it.

ok, this is inappropriate to be discussing when there is still
silence from ben when i've answered the issues he raised, pointed out
that they are moot, and am waiting for him to run and install
pyjamas-desktop.

all this is with a view to getting on the same page in order to
demonstrate what the benefits of doing so are...

... but for the sake of argument, even though it risks wasting
everybody's time, i'm going to provide an answer - because you asked.

a) who is going to pay for their time?

b) who is going to train them or answer their questions?

c) how much time is it going to take to train them and answer their questions?

... it's going to be the core mozilla developers, isn't it?

how long do you think it would take the core mozilla team to keep a
small codebase like pyxpcom up-to-date vs having to continuously
answer under-informed bugreports such as this one, here:

https://bugzilla.mozilla.org/show_bug.cgi?id=731121

boris, for example, who has been extremely helpful up until now, is
getting _really_ fed up - and that's just the one bugreport!

i'm going to keep on, and on, and on, and on, and on, and on at this
until i get answers, and pyxpcom works.

then, in a couple of months, i'm going to try again. and i'll find
more bugs. and i'll raise bugreports again and again until i get
answers.

and that loop will be repeated, working from an under-informed
position where i don't have the time or money to attend mozilla
conferences, or listen to mozilla phone calls (b2g), or trawl through
mozilla documentation, or trawl through millions of lines of mozilla
source code.

are you _getting_ it yet?

i don't _like_ having to do this kind of thing. it's not fun for me
to have to point these kinds of things out, but somebody has to.

>  And as has
> been stated several times,

... and as the points which _answer_ those statements have been
ignored several times...

> the core Mozilla community has no interest in
> doing so.

kyle, that's really not helpful right now.

this is complex: the core developers don't understand the issues, and
i'm endeavouring to help them to understand.

please don't interfere.

thank you.

l.

[Justin Lebar]

> i'm going to keep on, and on, and on, and on, and on, and on at this
> until i get answers, and pyxpcom works.

Luke, I think it is unlikely that you will accomplish your goals if
you make trolling your primary strategy.

We are pretty bad at ignoring trolls, here at Mozilla, but I expect
that if you troll hard enough, we will eventually learn to ignore you.

You've tried to argue that it's in our best interests to maintain the
python-xpcom bindings. Those in this thread have heard your argument,
and they disagree. You think that's our loss, and maybe you're right!

But just because you think we're wrong doesn't give you the right to
waste everyone's time. You are not entitled to anyone's attention
here or in bugzilla.

Maintaining xpcom bindings for Python is work! You're probably right
that it would be more work for you than it would be for a core xpcom
contributor. But it's work either way, and I think you are unlikely
to convince someone to do work by threatening to be a troll.

Regards,
-Justin

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

[lkcl luke]

On Thu, Mar 1, 2012 at 2:29 AM, Justin Lebar <justin...@gmail.com> wrote:
>> i'm going to keep on, and on, and on, and on, and on, and on at this
>> until i get answers, and pyxpcom works.
>
> Luke, I think it is unlikely that you will accomplish your goals if
> you make trolling your primary strategy.

i don't know what trolling is, and you are probably mistaking
trolling for pathological bug-fixing and 100% focus on getting a goal
completed irrespective of "trying to make friends".

but you are dragging the conversation off topic. i am still waiting
for ben to acknowledge the points made regarding his misunderstanding
of the situation regarding the continuous changes to XPCOM interfaces,
which are absolutely irrelevant to dynamic programming languages.

and, if that was the primary justification for destroying pyxpcom
then it is a mistake.

> You've tried to argue that it's in our best interests to maintain the
> python-xpcom bindings.  Those in this thread have heard your argument,

no they haven't. i'm not done yet.

there's more to this.

so far i've explained to chris that javascript as a compiler target
of a dynamic programming language results in an order of magniture
performance hit; to ben that he's mistaken about the binary
compatibility issue and so on.


> and they disagree.

how can you disagree with an incomplete perspective? i don't understand.

>  You think that's our loss, and maybe you're right!

maybe? there shouldn't be *any* "maybe". you're supposed to be
world leaders at the forefront of technology. there should _be_ no
"maybes".

> But just because you think we're wrong doesn't give you the right to
> waste everyone's time.  You are not entitled to anyone's attention
> here or in bugzilla.

you are assuming that this is ego-driven. where is the money that i
am being paid to do this work on behalf of others? where is it? can
you please tell me where the personal motivation is, please?

have you seen any?

mm?

you've already badly let down the OLPC community. that means that
it's now my responsibility to the pyjamas community to get this done,
and i'm PISSED that it's going to cost _me_ money and time when i've
been earning UNDER POVERTY LEVEL INCOME FOR FOUR FUCKING YEARS.

do you not fucking get it? we got EVICTED last year - myself, my
partner and my 2-year-old daughter, because i stick to free software
principles and have to turn down lucrative contracts where people in
the United States violate the Treaty of Rome's clauses on intellectual
property ownership.

you guys have absolutely no idea. i was _homeless_ for two years,
from about 2007 to 2009 for god's sake. and i _still_ kept to free
software principles and still kept working on free software.

> Maintaining xpcom bindings for Python is work!  You're probably right
> that it would be more work for you than it would be for a core xpcom
> contributor.  But it's work either way,

yes, it is. so go on. ban me. come on. go stick your fingers in
your ears. make the decision to revoke my bugzilla account (like the
webkit team did, after their deliberate bullying). see how far that
gets you.

the damage to the free software community of mark rowe's bullying was
- is - _immense_.

you want to do the same thing? go ahead. because if you don't start
taking responsibility for this instead of going "nur, nur it's not our
core focus", i _will_ absolutely pathologically drive you to that.

> and I think you are unlikely
> to convince someone to do work by threatening to be a troll.

i don't actually know what a troll is.

however i _will_ be absolutely pathological, persistent and get this done.

it would be very very good if you found someone _else_ to do the work.

anyway.

you've made me extremely angry, but let's forget that that happened, shall we?

now, back to the subject at hand.

here's where we are: i'm waiting for ben to acknowledge that the
points made about the dynamic programming language bindings to XPCOM
interfaces make no odds regarding this "binary compatibility"
argument.

he very kindly raised the point, which gives us an insight into why
the mistake was made.

now.

ben.

without prejudice.

it's okay to admit that you made a mistake.

nobody is going to think anything less of you if you say "ah, i made
an error of judgement because i didn't know how pyxpcom+hulahop works"

do you acknowledge that for dynamic programming languages which use
"lazy binding", binary compatibility with XPCOM interfaces is
irrelevant?

i'm keenly aware that all decisions have been made to destroy pyxpcom
based on the mistaken assumption that binary compatibility is
important to dynamic programming languages.

a simple yes would suffice, then we can move on to the next stage, ok?

l.

[lkcl luke]

... by the way: if you get me some money, so that i can actually focus
on this properly, you will find that my attitude COMPLETELY changes.

i won't have to worry about where next month's rent is coming from.

why are you surprised that i am angry about being backed into a corner
where i am forced to work on some free software - for the benefit of
OTHER PEOPLE?

i've BEEN through this. you remember samba? i broke the back of the
NT domains reverse-engineering... and the lead developers took the
credit, and got the contracts and funding. you've heard of the
openexchange project? libmapi? the same thing happened with the
Exchange 5.5. _again_ i got the ball rolling there - it cost me 12 to
14 hours a day for about 4 months, and i got _nothing_.

i'm sick of being taken advantage of, folks. it's not ok.

[cornel...@gmail.com]

Does anyone have an idea about how much work it would take to maintain python-xpcom?

Could it be automated using swig?

I've obvioulsly no idea what I'm talking about, but I use python-xpcom and hulahop a lot. I would really like to see this valuable piece of software working for the next 20 years or so. ;-)

[jameso...@gmail.com]

It seems that Luke has ended up in Ben's explicit or implicit killfile. I think that's unfortunate. Therefore, I will make an attempt to summarize what I see as the issues in this thread. I'm not going to add any new ideas, so feel free to skip everything after this paragraph if you feel you understand the issues. But if you do so, please take this as a vote to consider seeking better solutions than just orphaning pyxpcom.

As I understand the debate here, everyone acknowledges that maintaining pyxpcom would be several times easier if done by Mozilla developers as part of the release cycle, with automated testing, than by Luke from his position outside Mozilla. But, while Luke feels that such maintenance is worth the higher amount of time, Ben and Mozilla do not feel that it is worth the smaller amount of time.

But that's not the actual debate. The actual debate, the reason Luke is tearing his hair out and others have classed him as a troll, is that Luke (the person actually doing the work) believes that Ben (the person who doesn't want his team to be responsible for the work) is overestimating how much work it would be. That is because some of the initial reasons Ben gave for not wanting to do the work are inaccurate in Luke's estimation. Luke pointed this out, and evidently feels that Ben should acknowledge his mistakes. The Mozilla team seems to feel that Luke is just whining; as if A asks B to the dance, is told "I have to walk my dog", and then repeatedly offers B other dogwalking solutions despite being pointedly ignored.

The point is that, as with the dog, the question of exactly how difficult it would be for Mozilla to maintain pyxpcom doesn't actually matter. <arbitrary illustrative numbers>Whether the job is 10 times easier for Mozilla, as Luke believes, or only 2 times easier, as Ben believes (or believed last time he responded),</arbitrary illustrative numbers> the fact is that it is perfectly possible that Mozilla still decides it's not worth it and Luke still decides it is. And if the question of relative ease were resolved, the debate would immediately move on to discussing importance. (I happen to agree with Luke on the importance question, even though I'm not knowledgeable enough to take sides on the technical question. But that's beside the point.) Debating isn't going to resolve this.

How do such questions of comparative advantage usually get solved? Through some kind of trade. But it seems unlikely that money is going to change hands here. If Mozilla does this for Luke, is it likely that he will eventually someday find some way to repay it through coding? I'd say it's very likely, even though we can't point to an appropriate coding payback task today.

I can say that, I really can't see why it wouldn't be worthwhile to give Luke a full hearing, actually responding to his technical points. Yes, he's socially inept, and it's hard to keep an even temper when he's flying off the handle, no matter how justified he may be by his lights. But at the end of the day, it would be a huge, huge pity if his social ineptitude caused Mozilla to make the wrong decision. Even if he is a troll, he's also a damn good engineer, and that caliber of troll isn't so common that it's a huge task to respond to all of them. And if, on the other hand, he's even half right, then the work maintaining pyxpcom will be repaid several-fold by: broadening the community (with pyjamas and possibly sugar); an additional code-path to catch bugs sooner; having Luke himself, a smart engineer, helping you instead of annoyingly arguing with you; and finally, far from least of all, all the (in my opinion) extremely impressive things that could be done with the technology itself if it weren't for the festering bitrot.

OK, enough of my kibbutzing. I hope that my good-will contribution is appreciated. No offense intended to any of the parties involved.

Jameson

[jeff.va...@gmail.com]

On Thursday, March 1, 2012 3:57:53 AM UTC-6, Kees Bos wrote:
> Does anyone have an idea about how much work it would take to maintain python-xpcom?
>
> Could it be automated using swig?
>

The binding needs to be done at the right level. Ideally, one would like access to the DOM hierarchy at the level insert/delete/modify this, but not at the level of managing the memory and how things are inserted/deleted/modified. Then, one wouldn't care if memory management changed, or widget X requires that one now calls Y which now depends on Z' instead of Z. In other words, my opinion is that the language interface should be at the "do this for me level" and not at the object/memory management details (C or assembler level).

The problems with interfaces at lower levels include: supporting memory management in multiple systems/environments, the computational and memory costs of translating objects between two different environments, and increasing the overall burden on high level developers who want to bang out the next big thing.

(ok, wtf is up with my typing on google groups, some key combo keys submitting and/or dropping me from the reply -- something like ctrl + space)

I tried to use swig, but found boost::python more suitable for my personal projects. More recently, some have mentioned Lua lang bindings as a potential starting ground.

I would need to look a the xpcom code to even know what type of approach was reasonable for python xpcom.

This is tongue in cheek "I dare say that for many purposes most people using something like pyjamas desktop could just say **** it, and support only MSHTML (or whatever it is called) on Windows and if the user base supports it then spend the time to get webkit working on the "other" systems." (Note that unfortunately between Windows and webkit, most if not all general user platforms are covered).

I realize that it appears that the current user base for python xpcom is infinitesimal and irrelevant to some. However, why ignore or actively destroy ways that already support inter- language and inter-platform operations.

[jameso...@gmail.com]

[jeff.va...@gmail.com]

On Thursday, March 1, 2012 3:57:53 AM UTC-6, Kees Bos wrote:

> Does anyone have an idea about how much work it would take to maintain python-xpcom?
>
> Could it be automated using swig?
>

[Benjamin Smedberg]

On 3/1/2012 9:32 AM, jameso...@gmail.com wrote:
> It seems that Luke has ended up in Ben's explicit or implicit killfile.

This thread is now way off-topic. It was originally an announcement
about how we're removing *binary-stable* embedding APIs, and it has
morphed into a discussion about the relative merits of pyxpcom, which
didn't use these embedding APIs and was unaffected. We are all quite
clear that pyxpcom clients don't have to worry about binary changes
because they are insulated by pyxpcom middleware and have the benefits
of being a scripted language.

However, pyxpcom has *never* been built or maintained as part of the
regular Mozilla build/test infrastructure. It was originally in the CVS
tree because that's where you put extension projects. It has since been
moved into its own mercurial repository, and has a module owner, Todd
Whiteman (ActiveState), and mailing list:
http://listserv.activestate.com/mailman/listinfo/pyxpcom

As moderator of both the mozilla.dev.platform and mozilla.dev.embedding
forums, I request that any further discussion about the future of
pyxpcom and trying to build a community around it be moved to the
pyxpcom mailing list; it is completely offtopic for the dev.platform
list and has been thoroughly discussed in dev.embedding.

--BDS

[Bobby Holley]

>
> Does anyone have an idea about how much work it would take to maintain
> python-xpcom?
>

Lots.

I don't know much about pyxpcom, but I imagine it being pretty analogous to
XPConnect (our JS<->XPCOM bridge). XPConnect is probably the most notorious
module in Gecko - it's complicated, scary, and easy to get wrong. We
already sink a ton of engineering resources into it, and wouldn't if it
weren't so fundamental to our entire architecture.

More importantly, for a number of reasons the DOM is becoming more
intimately tied to JS: we're removing the abstraction layer that
sorta-kinda makes the DOM language-agnostic (it's too slow), and we're
writing new custom DOM bindings.

So if it isn't true already, pyxpcom won't be able to script the DOM
anymore, which makes it significantly less useful. Reversing this course
(and maintaining a separate set of python DOM bindings) is pretty much out
of the question: it incurs a performance penalty for the Web that we don't
want to pay, and it would take months and months of engineering by Boris,
Kyle, Peter, Blake, Ben, Johnny, Olli, and myself to make happen.

In a nutshell, script bindings aren't easy - this stuff is some of the most
complicated and delicate code in Gecko. So the issue isn't that Benjamin is
too lazy or proud to keep the relevant embedding APIs. The issue is that
there is a very small group of people in the world who have the knowledge
to build and maintain something like this, and they have other priorities.

bholley