Heads up: XPCOM component registration changing, changes required
Benjamin Smedberg
As noted in both the platform and Firefox meetings, I am working on a patch
which significantly changes the way that XPCOM components are registered. I
am doing this to solve two problems:
* Eliminate the Firefox restart which happens when Firefox is started after
an upgrade or after installing/uninstalling/enabling/disabling extensions
* Allow content processes (in Fennec, and then in Firefox) to start up
without reading or writing a cache of registration data (which we'd have to
sync between processes)
This is a backwards-incompatible change which will require code changes to
both binary and JS components.
== Changes to binary components ==
Binary commponents will change in the following way:
instead of exporting a function named "NSGetModule", binary components will
export a data symbol "NSModule". This will be a C++ structure contained
tables of CIDs, contracts, and category entries provided by the component,
as well as function pointers to actually create those CIDs.
It will be possible to make a single binary component which is compatible
with both Firefox 3.6 and Firefox 4 by exporting both NSModule and
NSGetModule. However, it is highly recommended that if extensions need
binary modules, they consider switching to ctypes, which will be a more
stable interface as the Mozilla platform continues to change.
Changing existing modules to the new data tables cannot be done
automatically. I have some emacs macros which helped me to reformat the data
"by hand", which I can post so that mailnews/seamonkey/etc will have an
easier time adapting to the new format. I welcome anyone with emacs or
python smarts turning my fragile macros into something more resilient.
== Changes to JS components ==
JavaScript components will change in the following way:
JS components will no longer register themself. Instead, JS components wil
register CIDs, contracts, and categories by adding lines to chrome.manifest.
See https://wiki.mozilla.org/XPCOM/Bootstrap#Registration for design docs
and examples. JS components will export a single toplevel function,
NSGetFactory. If components currently use XPCOMUtils.generateModule, this
change may be as simple as patching a few lines to switch to
XPCOMUtils.generateNSGetFactory
It will be possible to make a single JS component/extension which is
compatible with both Firefox 3.6 and Firefox 4. This may cause some
additional warnings about "unknown chrome registration directive" to appear
in the error console.
== APIs and version numbering ==
This change affects the following frozen APIs:
nsIModule (removed)
NSGetModule (standard component entry point, unused)
NS_InitXPCOM3 (removed, static component registration changed)
Because of this, we are considering bumping the gecko version number to
"2.0" when this lands, or tomorrow after the developer preview has been
tagged. If you see any reason we shouldn't do this, please let me know ASAP.
This may lead us to the odd position of building Firefox 4 on the
hg.mozilla.org/releases/mozilla-1.9.3 branch with a gecko version number of
2.0. If that hurts your brain too much, I'm sorry!
== Schedule ==
I am definitely trying to get this landed before the beta scheduled for late
June. Three of the six parts of this patch are complete and in the review
cycle, but I have yet to finish the manifest-reading and JS component
loading pieces. Since the two big patches are 18k lines apieces, it's
unlikely that this will land before I go on vacation next week. Assuming no
major issues come up, then, bhsieh or Mossop may land this next week, or I
will land it early the following week (Monday the 21st).
As soon as the final patches are posted (hopefully tomorrow), I'll
coordinate more detailed messaging and documentation for extension authors
with sheppy and the addons team.
See https://bugzilla.mozilla.org/show_bug.cgi?id=568691 for all the details.
--BDS
Joshua Cranmer
> It will be possible to make a single binary component which is
> compatible with both Firefox 3.6 and Firefox 4 by exporting both
> NSModule and NSGetModule. However, it is highly recommended that if
> extensions need binary modules, they consider switching to ctypes, which
> will be a more stable interface as the Mozilla platform continues to
> change.
So this would require #ifdef's to control which array format to use
depending on the branch you build?
> JS components will no longer register themself. Instead, JS components
> wil register CIDs, contracts, and categories by adding lines to
> chrome.manifest. See
> https://wiki.mozilla.org/XPCOM/Bootstrap#Registration for design docs
> and examples. JS components will export a single toplevel function,
> NSGetFactory. If components currently use XPCOMUtils.generateModule,
> this change may be as simple as patching a few lines to switch to
> XPCOMUtils.generateNSGetFactory
Will this allow me to have one component specify two separate
contract-IDs without writing custom code in JS?
Also, something I am currently relying in is that I am using an
_xpcom_factory hack to redirect instance creation:
function wfDatabase() {}
wfDatabase.prototype = {
classDescription: "Web Forums database",
contractID: "@mozilla.org/nsMsgDatabase/msgDB-webforum",
classID: Components.ID("{7519e8a6-f5e4-4b05-8cae-f2f4ad0ebfee}"),
_xpcom_factory: {
createInstance: function (outer, iid) {
if (outer)
throw Cr.NS_ERROR_NO_AGGREGATION;
return
Cc["@mozilla.org/nsMsgDatabase/msgDB-default"].createInstance(iid);
}
}
};
Would something like that still be possible in the new setup?
Benjamin Smedberg
> On 06/09/2010 11:04 PM, Benjamin Smedberg wrote:
>> It will be possible to make a single binary component which is
>> compatible with both Firefox 3.6 and Firefox 4 by exporting both
>> NSModule and NSGetModule. However, it is highly recommended that if
>> extensions need binary modules, they consider switching to ctypes, which
>> will be a more stable interface as the Mozilla platform continues to
>> change.
>
> So this would require #ifdef's to control which array format to use
> depending on the branch you build?
No, you will be able to compile a single extension using the new-format data
table and it will generate a generic module and implement nsIModule from that.
> Will this allow me to have one component specify two separate
> contract-IDs without writing custom code in JS?
You mean, can you register two CIDs for the same component.js? sure:
component cid1 components/mine.js
component cid2 components/mine.js
Or do you mean, can you register two contractids for the same CID? You can
do that too:
contract @j.cranmer/mything;1 cid1
contract @j.cranmer/myotherthing;1 cid1
> Also, something I am currently relying in is that I am using an
> _xpcom_factory hack to redirect instance creation:
> function wfDatabase() {}
> wfDatabase.prototype = {
> classDescription: "Web Forums database",
> contractID: "@mozilla.org/nsMsgDatabase/msgDB-webforum",
> classID: Components.ID("{7519e8a6-f5e4-4b05-8cae-f2f4ad0ebfee}"),
> _xpcom_factory: {
> createInstance: function (outer, iid) {
> if (outer)
> throw Cr.NS_ERROR_NO_AGGREGATION;
> return Cc["@mozilla.org/nsMsgDatabase/msgDB-default"].createInstance(iid);
> }
> }
> };
>
> Would something like that still be possible in the new setup?
I have no clue what this question is actually asking.
--BDS
Mark Finkle
> == Changes to JS components ==
>
> JavaScript components will change in the following way:
>
> JS components will no longer register themself. Instead, JS components wil
> register CIDs, contracts, and categories by adding lines to chrome.manifest.
> Seehttps://wiki.mozilla.org/XPCOM/Bootstrap#Registrationfor design docs
> and examples. JS components will export a single toplevel function,
> NSGetFactory. If components currently use XPCOMUtils.generateModule, this
> change may be as simple as patching a few lines to switch to
> XPCOMUtils.generateNSGetFactory
When overriding components, the new component simply uses the same
contract ID and is, hopefully, registered after the original
component. The order of registration is important and today we use a
components folder search scheme and the generated components.list file
to specify "order" of registration.
How will the order work in the new system? Apps, add-ons and xulrunner
(the platform) can all have component folders. They also all have
chrome.manifest files, so I assume the folder search order would be
the same as it is today. But what if platform and app components are
mixed in the same folder, as happens with Fennec currently. I guess
components.list could still be used, right?
Finkle
Ehsan Akhgari
runtime, by registering a new factory, etc?
--
Ehsan
<http://ehsanakhgari.org/>
> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning
>
Mook
> Followup to mozilla.dev.platform, please!
>
> As noted in both the platform and Firefox meetings, I am working on a
> patch which significantly changes the way that XPCOM components are
> registered. I am doing this to solve two problems:
>
> * Eliminate the Firefox restart which happens when Firefox is started
> after an upgrade or after installing/uninstalling/enabling/disabling
> extensions
> * Allow content processes (in Fennec, and then in Firefox) to start up
> without reading or writing a cache of registration data (which we'd have
> to sync between processes)
>
> This is a backwards-incompatible change which will require code changes
> to both binary and JS components.
(and bug 568691 doesn't have any - beside, bugs are the wrong place anyway):
- This means ("@foo" in Cc) will always be true (and not useful for
checking if a component is available), and to do a useful check it now
must be try { Cc["@foo"].createInstance() } catch (e) {} ? (This is
just pointing out more code that needs to be changed, possibly in
external code, and should not be taken as stop energy.)
> == Changes to binary components ==
- For an actual example of component registration, see
https://bugzilla.mozilla.org/attachment.cgi?id=450177&action=diff#a/xpfe/components/build/nsModule.cpp_sec1
(since there's no concise documentation of the WIP yet).
- Some XPCOM (C++, non-automated) tests are being removed with no
replacement. They were useful for making sure a port was working correctly.
- A note for people who might want to write components across
app/platform versions: still doable by hacking the factory to check app
version at that point. (That's compatible with Firefox 4 and Firefox
4+x, not just 3 and 4, and was the solution I settled on before anyway.)
- It would no longer be possible to add/remove category entries at
runtime. (They don't persist currently, bug 364864, but the patch
appears to remove them altogether.)
- This means you can't have components that don't register if you have
missing dependent system libraries. (Instead, they will register but
fail to be created.) This is merely a potential use case and doesn't
need to be addressed unless we find actual components doing so.
- (Re:ctypes: The sort of things we, as a XR app, end up doing isn't
doable from ctypes because it's all XPCOM and frequently running off the
main thread. Also, currently JS + off the main thread but not in
{Web|Chrome}Worker due to need of XPCOM = death in our experience,
possibly because we haven't found the right subset of safe things.)
- Something like
https://developer.mozilla.org/en/Using_Dependent_Libraries_In_Extension_Components
would still be doable, but each separate constructor must now be a stub
(instead of having one central stub as before).
- This would be a good time to also note for posterity the list of
shared libraries that are import-ABI-stable for components in the new
world. (Is it nspr4/plc4/js3250, i.e. same as before minus xpcom?
xpcomglue_s is a static lib, so that doesn't matter so much. I think
people were linking against libxul, which I always assumed was crazy...)
> == Changes to JS components ==
- I couldn't find an actual example of a JS component :( Sounds simple
enough, though...?
> == APIs and version numbering ==
>
> This change affects the following frozen APIs:
> nsIModule (removed)
> NSGetModule (standard component entry point, unused)
> NS_InitXPCOM3 (removed, static component registration changed)
nsICategoryManager
nsIComponentRegistrar (removes ability to register things at runtime)
nsIClassInfo (removing a flag)
> == Schedule ==
- Before landing this (in fact, long enough before to be able to give
sensible feedback - that means _more_ than a week), please have a
migration guide so that we can evaluate what sort of things might be
lacking.
- Documentation note:
https://developer.mozilla.org/en/Troubleshooting_XPCOM_components_registration
will need to change, since reading compreg.dat will no longer be useful.
(This would not block landing.)
== summary ==
Overall, this seems to have turned out to have less impact than I
initially thought it would from looking at the high level overview -
it's still a scary change, and now that there's at least more concrete
info I can think about it more. There are, of course, things I want
that are changing in ways I don't quite enjoy, but that's life, really.
--
Mook
Justin Wood (Callek)
> On 2010-06-09 8:04 PM, Benjamin Smedberg wrote:
>> Followup to mozilla.dev.platform, please!
>>
>> As noted in both the platform and Firefox meetings, I am working on a
>> patch which significantly changes the way that XPCOM components are
>> registered. I am doing this to solve two problems:
>>
>> * Eliminate the Firefox restart which happens when Firefox is started
>> after an upgrade or after installing/uninstalling/enabling/disabling
>> extensions
>> * Allow content processes (in Fennec, and then in Firefox) to start up
>> without reading or writing a cache of registration data (which we'd have
>> to sync between processes)
>>
>> This is a backwards-incompatible change which will require code changes
>> to both binary and JS components.
> Feedback I can think of now, since I can't find where else they would go
> (and bug 568691 doesn't have any - beside, bugs are the wrong place
> anyway):
>
> - This means ("@foo" in Cc) will always be true (and not useful for
> checking if a component is available), and to do a useful check it now
> must be try { Cc["@foo"].createInstance() } catch (e) {} ? (This is just
> pointing out more code that needs to be changed, possibly in external
> code, and should not be taken as stop energy.)
Hmm, and used in MANY test files in m-c for compat with different build
configs, including other apps.
--
~Justin Wood (Callek)
Neil
> - It would no longer be possible to add/remove category entries at
> runtime. (They don't persist currently, bug 364864, but the patch
> appears to remove them altogether.)
Does this affect jsd's init at startup, which we can't seem to turn off/on?
--
Warning: May contain traces of nuts.
Neil
> Also, something I am currently relying in is that I am using an
> _xpcom_factory hack to redirect instance creation:
> function wfDatabase() {}
> wfDatabase.prototype = {
> classDescription: "Web Forums database",
> contractID: "@mozilla.org/nsMsgDatabase/msgDB-webforum",
> classID: Components.ID("{7519e8a6-f5e4-4b05-8cae-f2f4ad0ebfee}"),
> _xpcom_factory: {
> createInstance: function (outer, iid) {
> if (outer)
> throw Cr.NS_ERROR_NO_AGGREGATION;
> return
> Cc["@mozilla.org/nsMsgDatabase/msgDB-default"].createInstance(iid);
> }
> }
> };
>
> Would something like that still be possible in the new setup?
As long as you know the CID of
"@mozilla.org/nsMsgDatabase/msgDB-default" it looks as if you can
register your contract ID as mapping to that CID too. And certainly the
C++ code still uses factory methods (bypassing nsIModule) so maybe the
JS glue will support something similar.
Ben Hearsum
> nsIModule (removed)
> NSGetModule (standard component entry point, unused)
> NS_InitXPCOM3 (removed, static component registration changed)
>
> Because of this, we are considering bumping the gecko version number to
> "2.0" when this lands, or tomorrow after the developer preview has been
> tagged. If you see any reason we shouldn't do this, please let me know
> ASAP.
>
> This may lead us to the odd position of building Firefox 4 on the
> hg.mozilla.org/releases/mozilla-1.9.3 branch with a gecko version number
> of 2.0. If that hurts your brain too much, I'm sorry!
Is there any reason we shouldn't branch to releases/mozilla-2.0.0 (or
mozilla-2.0) if we bump the Gecko version to that?
dbradley
> export a data symbol "NSModule". This will be a C++ structure contained
> tables of CIDs, contracts, and category entries provided by the component,
> as well as function pointers to actually create those CIDs.
Just curious why a data structure and not a new function that provides
access to the data structure?
Wouldn't this break if name mangling happened to change between
compiler versions.
I don't know if there is anything that currently does this, but this
would preclude dynamically generating the registration information.
David
johnjbarton
> Followup to mozilla.dev.platform, please!
>
> As noted in both the platform and Firefox meetings, I am working on a
> patch which significantly changes the way that XPCOM components are
> registered.
> == Changes to JS components ==
If you use JS components only from other JS code, eg in XUL window
overlay code, another way to adapt to this change is it convert your
component into a Components.utils module:
https://developer.mozilla.org/en/Using_JavaScript_code_modules
The converted code will work in 3.6 and I guess in 4.0. As a bonus you
get to delete the component registration code and the use of
.wrappedJSObject to access the object.
jjb
Benjamin Smedberg
> Will this affect the ability to dynamically replace components at
> runtime, by registering a new factory, etc?
Originally I removed nsIComponentReigstrar.registerFactory, yes. Since then
I have discovered tests and jetpacks rely on it, so I have added it back.
.registerFactoryLocation is removed.
Note that under my current patch, you intentionally cannot register
replacement CIDs ever (either statically or dynamically). You can only
register replacement contract IDs.
--BDS
Benjamin Smedberg
> - This means ("@foo" in Cc) will always be true (and not useful for
> checking if a component is available), and to do a useful check it now
> must be try { Cc["@foo"].createInstance() } catch (e) {} ? (This is just
> pointing out more code that needs to be changed, possibly in external
> code, and should not be taken as stop energy.)
Are you talking about binary components or JS components? Because binary
components must be dlopened to retrieve the registration data, they will
fail to load if there are missing dependencies and "@foo" in Cc should work
as it does today.
It is true that "@js-implemented-foo" in Cc may return true even if the JS
component is missing or has a syntax error, but I believe that this is
acceptable, since that is not a planned event, unlike missing load
dependencies which are fairly normal. This is one reason I kept registration
data for binary components within the component, rather than putting it all
in .manifest files.
> - Some XPCOM (C++, non-automated) tests are being removed with no
> replacement. They were useful for making sure a port was working correctly.
Which ones are you talking about specifically?
> - It would no longer be possible to add/remove category entries at
> runtime. (They don't persist currently, bug 364864, but the patch
> appears to remove them altogether.)
This is true and intentional.
> - This means you can't have components that don't register if you have
> missing dependent system libraries. (Instead, they will register but
> fail to be created.) This is merely a potential use case and doesn't
> need to be addressed unless we find actual components doing so.
This is not true, as noted above: the dlopen will fail.
> - Something like
> https://developer.mozilla.org/en/Using_Dependent_Libraries_In_Extension_Components
> would still be doable, but each separate constructor must now be a stub
> (instead of having one central stub as before).
I think it should be possible to do this without a separate stub for each
function, by doing the initialization in a module constructor. But yes, this
will get a bit more complicated.
> - This would be a good time to also note for posterity the list of
> shared libraries that are import-ABI-stable for components in the new
> world. (Is it nspr4/plc4/js3250, i.e. same as before minus xpcom?
> xpcomglue_s is a static lib, so that doesn't matter so much. I think
> people were linking against libxul, which I always assumed was crazy...)
NSPR and NSS are the only libraries in the Mozilla universe which are
import-stable. JS, despite appearances, is not import-stable; it just
changes relatively slowly.
>> == Changes to JS components ==
> - I couldn't find an actual example of a JS component :( Sounds simple
> enough, though...?
Yeah, that patch isn't finished yet ;-)
> nsIComponentRegistrar (removes ability to register things at runtime)
nsIComponentRegistrar.registerFactory is retained, but
.registerFactoryLocation is not. I may actually revert the interface changes
here and mark the obsolete functions as not-implemented, so that
NS_GetComponentRegistrar gives back a binary-compatible thing.
> nsIClassInfo (removing a flag)
This flag was an implmentation detail of the old generic factory, so it
shouldn't affect *binary* compatibility at all, merely requires a
source-code change.
>> == Schedule ==
> - Before landing this (in fact, long enough before to be able to give
> sensible feedback - that means _more_ than a week), please have a
> migration guide so that we can evaluate what sort of things might be
> lacking.
Given our current schedule, we don't have a lot of wiggle room. I'll be
trying to set up documentation guides ASAP, though.
> Overall, this seems to have turned out to have less impact than I
> initially thought it would from looking at the high level overview -
> it's still a scary change, and now that there's at least more concrete
> info I can think about it more. There are, of course, things I want that
> are changing in ways I don't quite enjoy, but that's life, really.
Yeah, it's a reward/risk tradeoff: in this case I think avoiding the EM
restart and making content processes work correctly is more important than
compatibility, but rest assured that we're not making the change lightly!
--BDS
johnjbarton
> Mook wrote:
>
>> - It would no longer be possible to add/remove category entries at
>> runtime. (They don't persist currently, bug 364864, but the patch
>> appears to remove them altogether.)
>
> Does this affect jsd's init at startup, which we can't seem to turn off/on?
>
Firebug sets
jsd.initAtStartup = false;
every time it starts. As soon as we access jsd we check the flag, and it
is false.
Perhaps the problem you are thinking about is
Bug 506149 - jsdService should allow off/on/off/on
which as far as I can tell means that once you start jsd you can't
completely turn it off. So if you have jsd.initAtStartup==true then you
can't do a runtime operation to recover.
jjb
Ehsan Akhgari
Do we have code which gets a component using its CID, rather than the
contract ID?
--
Ehsan
<http://ehsanakhgari.org/>
Benjamin Smedberg
> Does this affect jsd's init at startup, which we can't seem to turn off/on?
It does. jsdIDebuggerService no longer contains .initAtStartup, and it is
the responsibility of the client to set up observers an initialize JSD.
--BDS
Benjamin Smedberg
> Just curious why a data structure and not a new function that provides
> access to the data structure?
There's not a big difference. If there were a compelling need for a
function, I'd do that instead, but it's important that the module not do
anything with XPCOM when the function is called (it is currently loaded
while the component manager holds its registration mutex).
> Wouldn't this break if name mangling happened to change between
> compiler versions.
No, it's an `extern "C"` data symbol
> I don't know if there is anything that currently does this, but this
> would preclude dynamically generating the registration information.
That is true, and mostly intentional.
--BDS
Benjamin Smedberg
> Do we have code which gets a component using its CID, rather than the
> contract ID?
Yes, but that's generally considered a bug, and I don't think we do it in
any cases that app/extension authors actually care about.
--BDS
Benjamin Smedberg
> When overriding components, the new component simply uses the same
> contract ID and is, hopefully, registered after the original
> component. The order of registration is important and today we use a
> components folder search scheme and the generated components.list file
> to specify "order" of registration.
Huh, components.list happens to have ordering, but I never knew that anybody
was *using* it for ordering. I forgot to account for it, though, thanks for
the reminder!
> How will the order work in the new system? Apps, add-ons and xulrunner
> (the platform) can all have component folders. They also all have
> chrome.manifest files, so I assume the folder search order would be
> the same as it is today. But what if platform and app components are
> mixed in the same folder, as happens with Fennec currently. I guess
> components.list could still be used, right?
I forgot about it, but I think the best solution here is not to troll
components directories for DLLs at all. Instead, add another directive to
the .manifest
binary-component components/foo.dll
--BDS
Ehsan Akhgari
Fair enough. Fixing those instances is probably worth it anyway. Is
it something which can be detected using a static analysis script? If
so, I'll file a bug for that.
--
Ehsan
<http://ehsanakhgari.org/>
Benjamin Smedberg
Anyone using nsGetServiceByCID or nsCreateInstanceByCID. I'm not so worried
about people calling nsIComponentManager.getclassobject/createinstance or
nsiservicemanager.getservice, although you could check for callers of those
methods as well.
--BDS
Mike Shaver
<benj...@smedbergs.us> wrote:
>> How will the order work in the new system? Apps, add-ons and xulrunner
>> (the platform) can all have component folders. They also all have
>> chrome.manifest files, so I assume the folder search order would be
>> the same as it is today. But what if platform and app components are
>> mixed in the same folder, as happens with Fennec currently. I guess
>> components.list could still be used, right?
>
> I forgot about it, but I think the best solution here is not to troll
> components directories for DLLs at all. Instead, add another directive to
> the .manifest
>
> binary-component components/foo.dll
Yes, with ABI information as well please!
binary-component components/foo.dll abi=win
binary-component components/bar64.dll abi=win64
binary-component components/bar32.dll abi=win32 version=1.9.2.*
binary component components/bar32_ng.dll abi=win32 version=2.0.*
I think the manifest and registration stuff should be the same for JS
components and C++ ones, so this is a good change.
I don't think that we should be converting CID to contract ID
everywhere; when there are compelling and wide-enough needs to hook
into something, we should add the right hook, but the source is also a
good hook for people using things downstream, and having them patch
the CID in their code isn't the end of the world. Adding a contract
ID just makes it more likely that we'll have to support a pile of
random things replacing our code for different reasons, and then we'll
just be terrified to change anything related to it.
If anything, we should be going the other way: deCOMtaminating and
then adding hooks with semantics that are useful for extension, not
just where we happened to divide the effort between teams in 1997. :-P
Mike
Boris Zbarsky
> contract ID?
Yes. Such code is generally followed by a static_cast to a concrete
type, so overriding the CID would be Very Bad (as Google Gears
discovered when they did it with the cache).
-Boris
Benjamin Smedberg
> Yes, with ABI information as well please!
>
> binary-component components/foo.dll abi=win
> binary-component components/bar64.dll abi=win64
> binary-component components/bar32.dll abi=win32 version=1.9.2.*
> binary component components/bar32_ng.dll abi=win32 version=2.0.*
All of these directives will support the standard manifest flags documented
at https://developer.mozilla.org/en/Chrome_Registration#Manifest_Flags
If we need to add a modifier for the ABI, that should be fairly simple as well.
--BDS
Shawn Wilsher
> - This means you can't have components that don't register if you have
> missing dependent system libraries. (Instead, they will register but
> fail to be created.) This is merely a potential use case and doesn't
> need to be addressed unless we find actual components doing so.
Cheers,
Shawn
Shawn Wilsher
> Are you talking about binary components or JS components? Because binary
> components must be dlopened to retrieve the registration data, they will
> fail to load if there are missing dependencies and "@foo" in Cc should
> work as it does today.
>
> It is true that "@js-implemented-foo" in Cc may return true even if the
> JS component is missing or has a syntax error, but I believe that this
> is acceptable, since that is not a planned event, unlike missing load
> dependencies which are fairly normal. This is one reason I kept
> registration data for binary components within the component, rather
> than putting it all in .manifest files.
people towards? Try to open a library with ctypes, and it fails, so
they wouldn't want to register.
Cheers,
Shawn
Benjamin Smedberg
> But what if they use ctypes to do this since that's what we are pushing
> people towards? Try to open a library with ctypes, and it fails, so they
> wouldn't want to register.
Is this a hypothetical case? If we need to make decisions based on the
runtime availability of certain pieces of code, we shouldn't be using XPCOM
contracts, but some more purpose-built mechanism.
--BDS
Benjamin Smedberg
> But what if they use ctypes to do this since that's what we are pushing
> people towards? Try to open a library with ctypes, and it fails, so they
> wouldn't want to register.
Is this a hypothetical case? If we need to make decisions based on the
Shawn Wilsher
> Is this a hypothetical case? If we need to make decisions based on the
> runtime availability of certain pieces of code, we shouldn't be using
> XPCOM contracts, but some more purpose-built mechanism.
Cheers,
Shawn
Gijs Kruitbosch
But that's not good enough, because by then other JS components have already
loaded, and it becomes Hard to get the associated jsdscripts, set breakpoints,
yada yada. :-(
~ Gijs
Robert Kaiser
Is there any way in the new model to do what the comment and function
below do?
Robert Kaiser
Benjamin Smedberg
Yes, why wouldn't there be? The component manager hasn't changed at all,
just the registrar.
--BDS
Dave Townsend
> On 6/10/2010 2:30 AM, Mook wrote:
>> On 2010-06-09 8:04 PM, Benjamin Smedberg wrote:
>>> Followup to mozilla.dev.platform, please!
>>>
>>> As noted in both the platform and Firefox meetings, I am working on a
>>> patch which significantly changes the way that XPCOM components are
>>>
>>> * Eliminate the Firefox restart which happens when Firefox is started
>>> after an upgrade or after installing/uninstalling/enabling/disabling
>>> extensions
>>> * Allow content processes (in Fennec, and then in Firefox) to start up
>>> without reading or writing a cache of registration data (which we'd have
>>> to sync between processes)
>>>
>>> This is a backwards-incompatible change which will require code changes
>>> to both binary and JS components.
>> Feedback I can think of now, since I can't find where else they would go
>> (and bug 568691 doesn't have any - beside, bugs are the wrong place
>> anyway):
>>
>> - This means ("@foo" in Cc) will always be true (and not useful for
>> checking if a component is available), and to do a useful check it now
>> must be try { Cc["@foo"].createInstance() } catch (e) {} ? (This is just
>> pointing out more code that needs to be changed, possibly in external
>> code, and should not be taken as stop energy.)
>
> configs, including other apps.
The problem is that you can't use this test in cases where the
registration code determines whether to register the component or not.
When the components are physically present this should still work fine.
Mook
On 2010-06-10 7:49 AM, Benjamin Smedberg wrote:
> On 6/10/10 2:30 AM, Mook wrote:
>
>> - This means ("@foo" in Cc) will always be true (and not useful for
>> checking if a component is available), and to do a useful check it now
>> must be try { Cc["@foo"].createInstance() } catch (e) {} ? (This is just
>> pointing out more code that needs to be changed, possibly in external
>> code, and should not be taken as stop energy.)
>
> Are you talking about binary components or JS components? Because binary
> components must be dlopened to retrieve the registration data, they will
> fail to load if there are missing dependencies and "@foo" in Cc should
> work as it does today.
Binary; and yeah, you're right, it'll fail to get to the exported data,
so it's good. Thanks.
>
> It is true that "@js-implemented-foo" in Cc may return true even if the
> JS component is missing or has a syntax error, but I believe that this
> is acceptable, since that is not a planned event, unlike missing load
> dependencies which are fairly normal. This is one reason I kept
> registration data for binary components within the component, rather
> than putting it all in .manifest files.
Sounds like it's all working out then!
>
>> - Some XPCOM (C++, non-automated) tests are being removed with no
>> replacement. They were useful for making sure a port was working
>> correctly.
>
> Which ones are you talking about specifically?
xpcom/tests/{TestFactory,services/Myservice}. But then maybe I just
missed the replacements on my first skim - in which case that just needs
more time :)
>
>> - It would no longer be possible to add/remove category entries at
>> runtime. (They don't persist currently, bug 364864, but the patch
>> appears to remove them altogether.)
>
> This is true and intentional.
Ah; sad to see it go. There is one single instance this is used in
Songbird, and it happens to consist of static data anyway so can be
migrated to the new thing.
>
>> - This means you can't have components that don't register if you have
>> missing dependent system libraries. (Instead, they will register but
>> fail to be created.) This is merely a potential use case and doesn't
>> need to be addressed unless we find actual components doing so.
>
> This is not true, as noted above: the dlopen will fail.
Agreed; my mistake.
>
>> - Something like
>> https://developer.mozilla.org/en/Using_Dependent_Libraries_In_Extension_Components
>>
>> would still be doable, but each separate constructor must now be a stub
>> (instead of having one central stub as before).
>
> I think it should be possible to do this without a separate stub for
> each function, by doing the initialization in a module constructor. But
> yes, this will get a bit more complicated.
Hmm, yes, and have the constructors fail if they see that init had failed.
>
>> - This would be a good time to also note for posterity the list of
>> shared libraries that are import-ABI-stable for components in the new
>> world. (Is it nspr4/plc4/js3250, i.e. same as before minus xpcom?
>> xpcomglue_s is a static lib, so that doesn't matter so much. I think
>> people were linking against libxul, which I always assumed was crazy...)
>
> NSPR and NSS are the only libraries in the Mozilla universe which are
> import-stable. JS, despite appearances, is not import-stable; it just
> changes relatively slowly.
Ah; I thought it had users at one point.
>
>>> == Changes to JS components ==
>> - I couldn't find an actual example of a JS component :( Sounds simple
>> enough, though...?
>
> Yeah, that patch isn't finished yet ;-)
Thanks for putting the notice up this early, though - it's awesome to be
able to take the time to think about it.
>
>
>> nsIComponentRegistrar (removes ability to register things at runtime)
>
> nsIComponentRegistrar.registerFactory is retained, but
> .registerFactoryLocation is not. I may actually revert the interface
> changes here and mark the obsolete functions as not-implemented, so that
> NS_GetComponentRegistrar gives back a binary-compatible thing.
Yeah, was just noting for completeness - the only thing I have ever used
that was actually removed was autoRegister, and that was in a test anyway.
>
>> nsIClassInfo (removing a flag)
>
> This flag was an implmentation detail of the old generic factory, so it
> shouldn't affect *binary* compatibility at all, merely requires a
> source-code change.
Ah, right.
>
>>> == Schedule ==
>> - Before landing this (in fact, long enough before to be able to give
>> sensible feedback - that means _more_ than a week), please have a
>> migration guide so that we can evaluate what sort of things might be
>> lacking.
>
> Given our current schedule, we don't have a lot of wiggle room. I'll be
> trying to set up documentation guides ASAP, though.
Yeah, it's just that "jam in a whole new system that many extension /
application developers depend on in a short time frame" sounds scary.
But then I have always tended on the pessimistic side, so perhaps things
will work out :) Was just trying to make sure this has enough exposure
to not end up with some fatal flaw when it does end up freezing.
>
>> Overall, this seems to have turned out to have less impact than I
>> initially thought it would from looking at the high level overview -
>> it's still a scary change, and now that there's at least more concrete
>> info I can think about it more. There are, of course, things I want that
>> are changing in ways I don't quite enjoy, but that's life, really.
>
> Yeah, it's a reward/risk tradeoff: in this case I think avoiding the EM
> restart and making content processes work correctly is more important
> than compatibility, but rest assured that we're not making the change
> lightly!
Yeah; the people involved in the project are all too smart to do that :D
Just trying to bring in the perspective a downstream (previously
extension, now mostly app) developer to make sure the cases are covered,
since I assume you'd need to freeze this when Firefox 4 goes out.
Things are turning out good so far; I hope we haven't missed anything,
but I certainly can't think of any right now!
--
Mook
Robert Kaiser
Ah, OK, I feared getting the right one by CID could be impaired when two
components are registered for the same contract ID.
Robert Kaiser
Benjamin Smedberg
> Ah, OK, I feared getting the right one by CID could be impaired when two
> components are registered for the same contract ID.
I keep the registration of both CIDs intact.
--BDS
dbradley
> There's not a big difference. If there were a compelling need for a
> function, I'd do that instead, but it's important that the module not do
> anything with XPCOM when the function is called (it is currently loaded
> while the component manager holds its registration mutex).
Ah, yes, that makes sense. I had SOAP in the back of my mind. I think
it used to dynamically generate interfaces. I wasn't sure if anything
else had followed that pattern.
So is NS_IMPL_NSGETMODULE_WITH_CTOR_DTOR going away? I should probably
just look at the bug and the patch.
David
johnjbarton
Chromebug is able to get the jsdIScripts and set breakpoints in
components. You just have to turn jsd on at start up. I suppose there
maybe some js code that is missed, but if we discover it we just need to
prefix it with the start up code.
jjb
Neil
> You just have to turn jsd on at start up.
You mean manually, as opposed to using initAtStartup?
--
Warning: May contain traces of nuts.
johnjbarton
> johnjbarton wrote:
>
>> You just have to turn jsd on at start up.
>
> You mean manually, as opposed to using initAtStartup?
>
I mean call:
jsd.on();
I do it in app-startup but I suppose you could do it as soon as you have
the infrastructure to get the service. The ninjas who work on JS code
that runs before that don't use js debuggers anyway ;-).
jjb
Zack Weinberg
...
>
> Because of this, we are considering bumping the gecko version number
> to "2.0" when this lands, or tomorrow after the developer preview has
> been tagged. If you see any reason we shouldn't do this, please let
> me know ASAP.
Does this open the floodgates for changes to frozen interfaces?
zw
Robert Kaiser
I was about to say that it's too bad something like that comes so late
in the cycle, as I'd have welcomed us having the chance for someone
going through the source and merging all those nsI*2 interfaces back to
the original ones, which requires breaking the interface freeze.
Robert Kaiser
johnjbarton
> Followup to mozilla.dev.platform, please!
>
> As noted in both the platform and Firefox meetings, I am working on a
> patch which significantly changes the way that XPCOM components are
> registered.
Just one bit of feedback from Firebug team effort to adjust to this
change: for JS-only extensions (I guess the vast majority) the only
components one really needs are command-line handlers. It might help a
lot of extension developers to have a specific example for command line.
jjb
Neil
> On 6/11/2010 11:46 AM, Neil wrote:
>
>> johnjbarton wrote:
>>
>>> You just have to turn jsd on at start up.
>>
>> You mean manually, as opposed to using initAtStartup?
>
> I mean call:
> jsd.on();
> I do it in app-startup
That sounds reasonable but I've just seen a blog post saying you won't
be able to register for app-startup :-(
Justin Dolske
>> Because of this, we are considering bumping the gecko version number
>> to "2.0" when this lands, or tomorrow after the developer preview has
>> been tagged. If you see any reason we shouldn't do this, please let
>> me know ASAP.
>
> Does this open the floodgates for changes to frozen interfaces?
I think old dream of massive interface change and cleanup is not
something we actually want to do (for various reasons previously
discussed). A better question to ask (IMO) is if there are specific
changes you've been unable to make due to frozenness, and if the those
changes bring value worth the interface change cost to 3rd parties.
Anyway, not to over-rotate on "floodgates", but I'm not sure what the
switch to 2.0 means for frozenness... I guess they become unfrozen,
since we've been wanting to do that for a long time?
Justin
Zack Weinberg
>
> I think old dream of massive interface change and cleanup is not
> something we actually want to do (for various reasons previously
> discussed). A better question to ask (IMO) is if there are specific
> changes you've been unable to make due to frozenness, and if the
> those changes bring value worth the interface change cost to 3rd
> parties.
Ok, that's fair; two of personal interest are:
* the "with unfreezing" version of the patches in bug 558235
(changes nsIDOM{,SVG,NS}CSS2Properties and nsIDOMCSSStyleDeclaration)
* using real C++ bool for IDL bool (changes the world)
zw
Neil
How about using real C++ wchar_t for PRUnichar on MSVC?
Boris Zbarsky
> How about using real C++ wchar_t for PRUnichar on MSVC?
407 #if defined(HAVE_CPP_2BYTE_WCHAR_T) && defined(NS_WIN32)
408 typedef wchar_t PRUnichar;
409 #else
410 typedef PRUint16 PRUnichar;
411 #endif
is what nscore.h says. It also says:
389 #if defined(_MSC_VER) && (_MSC_VER>=1100)
...
398 #define HAVE_CPP_2BYTE_WCHAR_T
So this should already be happening, no?
-Boris
Mook
> On 6/16/10 5:08 AM, Neil wrote:
>> How about using real C++ wchar_t for PRUnichar on MSVC?
> So this should already be happening, no?
We still build with -Zc:wchar_t- so, no, we're using a typedef (not sure
if it's from a Windows header or a toolchain/CRT header). It's a short
instead of a native type.
With -Zc:wchar_t:
?test@@YA_WXZ (wchar_t __cdecl test(void))
With -Zc:wchar_t-:
?test@@YAGXZ (unsigned short __cdecl test(void))
Yeah, different name mangling too - since the current instructions for
building external components involves linking against libxpcomglue_s,
this is going to be loads of fun.
--
Mook
Ted Mielczarek
<mook.moz+nntp.n...@gmail.com.please-avoid-direct-mail>
wrote:
> On 2010-06-16 9:12 AM, Boris Zbarsky wrote:
>>
>> On 6/16/10 5:08 AM, Neil wrote:
>>>
>>> How about using real C++ wchar_t for PRUnichar on MSVC?
>
>> So this should already be happening, no?
>
> We still build with -Zc:wchar_t- so, no, we're using a typedef (not sure if
> it's from a Windows header or a toolchain/CRT header). It's a short instead
> of a native type.
This is https://bugzilla.mozilla.org/show_bug.cgi?id=508905 .
-Ted
Karsten Düsterloh
>> other JS components have already loaded, and it becomes Hard to get
>> the associated jsdscripts
>
> effort.
Where's that tracked? (Bug, Wiki?)
What's its status? Will it be ready when the old way breaks?
Or will we end up with broken Venkman for a long time?
Karsten
--
Feel free to correct my English. :)
Karsten Düsterloh
>> - It would no longer be possible to add/remove category entries at
>> runtime. (They don't persist currently, bug 364864, but the patch
>> appears to remove them altogether.)
>
> This is true and intentional.
Care to elaborate on that?
I'm not sure I get it right: If I want to change an entry, I'd need to
change the manifest and do a "cold" restart of the entire application?
Benjamin Smedberg
> Ok, that's fair; two of personal interest are:
>
> * the "with unfreezing" version of the patches in bug 558235
> (changes nsIDOM{,SVG,NS}CSS2Properties and nsIDOMCSSStyleDeclaration)
What kind of win do we get? I think this change is probably fine
immediately, but jst should make the final decision.
> * using real C++ bool for IDL bool (changes the world)
I don't think we want to make this change for FF4. Let's do it for gecko 2.1
--BDS
Zack Weinberg
> On 6/15/10 3:02 PM, Zack Weinberg wrote:
>
> > Ok, that's fair; two of personal interest are:
> >
> > * the "with unfreezing" version of the patches in bug 558235
> > (changes nsIDOM{,SVG,NS}CSS2Properties and
> > nsIDOMCSSStyleDeclaration)
>
> What kind of win do we get? I think this change is probably fine
> immediately, but jst should make the final decision.
It removes an extra vtable pointer from nsDOMCSSDeclaration and its
subclasses, and also it's easier to add CSS properties going forward.
dbaron said he thought these weren't worth being the thing that breaks
the frozen-interface guarantee, but given that we're already doing
that ...
> > * using real C++ bool for IDL bool (changes the world)
>
> I don't think we want to make this change for FF4. Let's do it for
> gecko 2.1
Fair.
zw
johnjbarton
>
> == Changes to JS components ==
>
>
> JS components will no longer register themself. Instead, JS components
> wil register CIDs, contracts, and categories by adding lines to
> chrome.manifest. See
> https://wiki.mozilla.org/XPCOM/Bootstrap#Registration for design docs
> and examples.
I tried to follow the example, creating these entries:
component {B5D5631C-4FE1-11DB-8373-B622A1EF5492}
components/chromebug_command_line.js
contract
@mozilla.org/commandlinehandler/general-startup;1?type=chromebug
{B5D5631C-4FE1-11DB-8373-B622A1EF5492}
category command-line-handler aaa-chromebug
{B5D5631C-4FE1-11DB-8373-B622A1EF5492}
component {287716D2-140B-11DE-912E-E0FC55D89593}
components/chromebug-startup-observer.js
contract @getfirebug.com/chromebug-startup-observer;1
{287716D2-140B-11DE-912E-E0FC55D89593}
category app-startup @@@Chromebug_Startup_Observer_Service
ser...@getfirebug.com/chromebug-startup-observer;1
I get no entries in compreg.dat with the string "chromebug". I get two
error messages
Error: ERROR addons.xpi: TypeError: newEntry is null
and of course:
Error: Warning: unrecognized command line flag -chromebug
What can I do to figure out why this is failing? chromebug is listed in
extensions.cache. There does not seem to be any xpti.dat file.
> JS components will export a single toplevel function,
> NSGetFactory.
Is this something I need to understand? It's not mentioned on the page
linked above.
> If components currently use XPCOMUtils.generateModule,
> this change may be as simple as patching a few lines to switch to
> XPCOMUtils.generateNSGetFactory
I don't use it.
jjb
Benjamin Smedberg
> I tried to follow the example, creating these entries:
>
> component {B5D5631C-4FE1-11DB-8373-B622A1EF5492}
> components/chromebug_command_line.js
> contract
> @mozilla.org/commandlinehandler/general-startup;1?type=chromebug
> {B5D5631C-4FE1-11DB-8373-B622A1EF5492}
> category command-line-handler aaa-chromebug
> {B5D5631C-4FE1-11DB-8373-B622A1EF5492}
The component/contract look fine: the category is incorrect: you want the
*contract* as the category value, so it should be:
category command-line-handler aaa-chromebug
@mozilla.org/commandlinehandler/general-startup;1?type=chromebug
>
> component {287716D2-140B-11DE-912E-E0FC55D89593}
> components/chromebug-startup-observer.js
> contract @getfirebug.com/chromebug-startup-observer;1
> {287716D2-140B-11DE-912E-E0FC55D89593}
> category app-startup @@@Chromebug_Startup_Observer_Service
> ser...@getfirebug.com/chromebug-startup-observer;1
This category also appears to be an incorrect value: the value should have a
comma after "service", i.e.:
category app-startup chromebugobserverservice
service,@getfirebug.com/chromebug-startup-observer;1
> I get no entries in compreg.dat with the string "chromebug". I get two
> error messages
compreg.dat is no longer used in the new branch. Just to be sure, are you
testing against my development branch or a tryserver build? This code has
not landed on trunk yet.
>> JS components will export a single toplevel function,
>> NSGetFactory.
>
> Is this something I need to understand? It's not mentioned on the page
> linked above.
Yes. Wherever you have a NSGetModule in your components, you need a
NSGetFactory now.
--BDS
johnjbarton
> On 6/30/10 1:54 AM, johnjbarton wrote:
> Just to be sure, are
> you testing against my development branch or a tryserver build? This
> code has not landed on trunk yet.
Oh. I'm sorry, I was just trying to get stuff to work in 4.0b1. Since my
addons would not install, I assumed that the problem was related to my
remaining components. We'll take this up again later then.
jjb
KWierso
what needs to be changed in extensions?
To get the components to work in both 3.6 and 4.0, do we just add the
JS components to chrome.manifest and leave the old registration in the
various files?
KWierso
I've converted my extension's components to use XPCOMUtils, since
that's the only thing I found documentation for with the new component
registration change. I made it so that it should be usable in both
Gecko 1.9.x and 2.0, like it shows in the documentation. (
https://developer.mozilla.org/en/XPCOM/XPCOM_changes_in_Gecko_1.9.3 )
The components still work fine in Firefox 3.6 (and yesterday's
nightly) with the change to using XPCOMUtils (it does throw some
messages about the unknown manifest code, but that's expected), but it
throws an error in today's Firefox 4.0 builds.
Here's the error:
Error: uncaught exception: [Exception... "Component returned failure
code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE)
[nsIJSCID.getService]" nsresult: "0x80570016
(NS_ERROR_XPC_GS_RETURNED_FAILURE)" location: "JS frame ::
chrome://rtse/content/rtse.js :: <TOP_LEVEL> :: line 27" data: no]
That's referring to this file:
http://pastebin.com/813PPHHp
Here's my component's code, for what it's worth:
http://pastebin.com/g8t38BHE
And my chrome.manifest:
http://pastebin.com/0Q9z4Xby
Is there anything obvious that I'm missing?
Benjamin Smedberg
> Here's the error:
> Error: uncaught exception: [Exception... "Component returned failure
> code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE)
> [nsIJSCID.getService]" nsresult: "0x80570016
> (NS_ERROR_XPC_GS_RETURNED_FAILURE)" location: "JS frame ::
> chrome://rtse/content/rtse.js ::<TOP_LEVEL> :: line 27" data: no]
...
> Is there anything obvious that I'm missing?
Offhand, that all looks correct. There is nothing else in the error console?
What is the value of Components.classes['@shawnwilsher.com/rtse;1'].number ?
--BDS
KWierso
>
> --BDS
I guess I should say that landing the patch that fixed extension
preferences fixed my extension. Apparently my Component messes with
some preferences during registration, and those preferences didn't
exist because of that bug.