Missing components in Thunderbird 3
Henrik Kaare Poulsen
I have developed an application for Thunderbird 2, and am now trying
to port to Thunderbird 3.
The application includes a standalone xpcom exe and an extension.
The application makes use of different components, such as:
@mozilla.org/addressbook;1
@mozilla.org/addressbook/carddatabase;1
@mozilla.org/addressbook/moz-abmdbcard;1
However, those components seems to be missing in Thunderbird 3.
In fact, for Thunderbird 3, nsIComponentRegistrar-
>EnumerateContractIDs lists only 70 components compared to Thunderbird
2's more than 900 components.
The problem occurs both in 3.0b1 download, in recent nightlies
download, as well as my own recent local builds of comm-central
mercurial source.
Does anybody know how to get access to those components ?
Thanks in advance
/Henrik
PS: In case anybody is interested, the application is blueZync for
synchronizing the addressbook:
http://kaarposoft.dk/bluezync/
Mark Banner
ordered]
On 01/18/2009 22:14, Henrik Kaare Poulsen wrote:
> However, those components seems to be missing in Thunderbird 3.
They are not "missing", they have either been removed or replaced in
development. The functions are still typically present, but have been
reworked on different interfaces (or removed if there were duplicates).
Between Thunderbird 2 and Thunderbird 3 we have reworked a lot of the
interfaces to provide a much better, generic, easier to use set. We
haven't made it as far as we would have liked, but what is now there is
certainly much better than what we had before.
> The application makes use of different components, such as:
> @mozilla.org/addressbook;1
> @mozilla.org/addressbook/moz-abmdbcard;1
For the basic address book (@mozilla.org/addressbook;1 -
nsIAddressBook), try looking at nsIAbManager in the new code.
For mdb cards (@mozilla.org/addressbook/moz-abmdbcard;1 - nsIAbMDBCard)
the functions/properties there are either on nsIAbCard, or not required
to be accessible.
> @mozilla.org/addressbook/carddatabase;1
This one should still be present. If I enter:
"@mozilla.org/addressbook/carddatabase;1" in Components.classes
into the error console in today's Thunderbird 3.0b2pre nightly build it
returns true which indicates it is there.
I suspect you are trying to save your own custom properties onto cards.
If so, you should now use the get/setProperty functions on nsIAbCard.
Once you have modified your card object, you can save it by using the
modifyCard function on the nsIAbDirectory where you got the card from.
> In fact, for Thunderbird 3, nsIComponentRegistrar-
>> EnumerateContractIDs lists only 70 components compared to Thunderbird
> 2's more than 900 components.
That does sound like you have some sort of issue, though maybe something
changed in the core code relating to how they are listed - we certainly
haven't removed that many components afaik.
Standard8
Joshua Cranmer
> The application makes use of different components, such as:
> @mozilla.org/addressbook;1
> @mozilla.org/addressbook/carddatabase;1
> @mozilla.org/addressbook/moz-abmdbcard;1
The address book has undergone extensive rewriting in Thunderbird 3.
Specifically, the nsIAbManager interface has been created and superseded
a few earlier interfaces for use in managing address book directories;
address book directories have gained a few new methods (superseding
usage of nsIAddrDatabase in a few cases); and the usage patterns for
nsIAbCard have drastically changed.
I also recommend eschewing usage of nsIAddrDatabase whenever possible
because it will be removed in the near (but not TB 3) future.
See
<http://mxr.mozilla.org/comm-central/source/mailnews/addrbook/public/>
for an up-to-date list of interfaces. Many, if not all, of the new
interfaces or methods come with documentation detailing their usage.
Henrik Kaare Poulsen
Thanks a lot for your answer!
I *do* understand, that many components have been replaced, removed or
reworked.
And I find most of the classes I need in the source tree.
However, the problem seems to be that they are *not* installed as
components...
So, if I take a build like
http://releases.mozilla.org/pub/mozilla.org/thunderbird/releases/3.0b1/linux-i686/en-US/thunderbird-3.0b1.tar.bz2
or
http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2009-01-18-03-comm-1.9.1/thunderbird-3.0b2pre.en-US.linux-i686.tar.bz2
the components are *not* in the components subdir.
Even in my own build of current comm-central mercurial source, the
components do not show up in the
dist/bin/components directory...
/henrik
Henrik Kaare Poulsen
Thanks for your answer.
And yes, from looking at the docs, it looks like I need to dig into
the sources...
However, as far as I can see, there are *NO* components installed,
which implements the interfaces from eg.
> <http://mxr.mozilla.org/comm-central/source/mailnews/addrbook/public/>
/henrik
Henrik Kaare Poulsen
build of Thunderbird 3.
The problem only occurs at runtime where the components are not found.
/Henrik
Mark Banner
> Hi Mark,
>
> Thanks a lot for your answer!
>
> I *do* understand, that many components have been replaced, removed or
> reworked.
> And I find most of the classes I need in the source tree.
>
> However, the problem seems to be that they are *not* installed as
> components...
If they weren't installed as components, then the Thunderbird
application would not run - or be extremely broken at least.
> So, if I take a build like
> http://releases.mozilla.org/pub/mozilla.org/thunderbird/releases/3.0b1/linux-i686/en-US/thunderbird-3.0b1.tar.bz2
> or
> http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2009-01-18-03-comm-1.9.1/thunderbird-3.0b2pre.en-US.linux-i686.tar.bz2
> the components are *not* in the components subdir.
>
> Even in my own build of current comm-central mercurial source, the
> components do not show up in the
> dist/bin/components directory...
Which files are you looking for? Thunderbird release builds (which both
of those are) are built with --enable-static, this will lump most of the
components into one library (or maybe the executable, I can't remember
which off hand).
Either way, your code should still be able to access them via the normal
methods (i.e. createInstance/getService).
Standard8
Henrik Kaare Poulsen
wrote:
> Which files are you looking for?
> Thunderbird release builds (which both
> of those are) are built with --enable-static, this will lump most of the
> components into one library (or maybe the executable, I can't remember
> which off hand).
Ahh - this might indeed be the problem.
If I grep for "addressbook" I only find the TBird exe.
However, in TBird 2, I find
components/libmail.so
components/libmailcomps.so
components/libimport.so
(and not the TBird 2 exe)
> Either way, your code should still be able to access them via the normal
> methods (i.e. createInstance/getService).
Apparently not...
If all components are compiled into the TBird exe, my standalone XPCOM
exe will not be able to get to them...
I can try to make a dynamic build, but in the future when users wants
to use my exe in their distro, we will end up with the same problem if
the distro packages the static version )-:
I am wondering if it would make sense to build the component libraries
by default even in a static build.
This would help everyone making standalone XPCOM exe's wishing to
access TBird.
/Henrik
Mark Banner
> If I grep for "addressbook" I only find the TBird exe.
> However, in TBird 2, I find
> components/libmail.so
> components/libmailcomps.so
> components/libimport.so
> (and not the TBird 2 exe)
FWIW Mac doesn't even have a "mail" lib.
>
>> Either way, your code should still be able to access them via the normal
>> methods (i.e. createInstance/getService).
>
> Apparently not...
> If all components are compiled into the TBird exe, my standalone XPCOM
> exe will not be able to get to them...
>
> I can try to make a dynamic build, but in the future when users wants
> to use my exe in their distro, we will end up with the same problem if
> the distro packages the static version )-:
> I am wondering if it would make sense to build the component libraries
> by default even in a static build.
For Thunderbird that would mean two things: doubling the size of the
download and package, as well as we'd have to stop one set of the
libraries being loaded as if you were able to build both at the moment,
then we would essentially try to load the components twice.
> This would help everyone making standalone XPCOM exe's wishing to
> access TBird.
If you follow the model already out there - xulrunner which is what
Firefox and Thunderbird are heading towards, then there would be a
common set of libraries. However this would still not include the
mailnews set, and we'd potentially have to do something on top of that
which would provide the shared libraries.
Given that we haven't achieved the xulrunner stage yet, sorting out a
shared library in the xulrunner style for mailnews is not going to be
something that is done quickly if we do want to do it.
FWIW I have also never heard of this request before.
For your application, you could always build your own copy of mailnews,
although not ideal it would mean that you could easily guarantee access
to the components for everyone.
The other question is does your application really need to be written in
c++? What about javascript?
There was some command-line argument for firefox where you could
essentially run an application using the firefox libraries by starting
it from the command line, a sort of intermediate xulrunner.
Unfortunately I can't remember the details at the moment and I'm not
sure it is supported in Thunderbird at the moment either.
Standard8
Henrik Kaare Poulsen
> c++? What about javascript?
My application is a plugin for a general synchronization framework
(OpenSync).
So it might be used in many different ways to sync with other
providers.
And as a plugin to OpenSync, it has to be C (or C++ maybe Python as
well).
> For your application, you could always build your own copy of mailnews,
> although not ideal it would mean that you could easily guarantee access
> to the components for everyone.
Yes, but I would need to be sure that I build exactly the same version
of the libs as the user is having from his distro to make sure that
data is not damaged.
This will be very hard to manage deployment-wise.
> If you follow the model already out there - xulrunner which is what
> Firefox and Thunderbird are heading towards, then there would be a
> common set of libraries. However this would still not include the
> mailnews set, and we'd potentially have to do something on top of that
> which would provide the shared libraries.
>
> Given that we haven't achieved the xulrunner stage yet, sorting out a
> shared library in the xulrunner style for mailnews is not going to be
> something that is done quickly if we do want to do it.
>
> FWIW I have also never heard of this request before.
My application can use the basic xpcom components from thunderbird,
lightning or xulrunner.
But since there is no GUI, there is not much point in using xulrunner.
Letting my exe initialize XPCOM directly worked fine for TBird 2.
I have not followed the discussions and arguments for the static
linking, and have very limited knowledge of the mozilla design and
buildprocess.
However, especially in the light of moving towards xulrunner, I think
accessible mailnews libraries would be the right way to go...
If I understand correctly, there would be no TBird exe anymore,
instead one just runs xulrunner with mailnews libraries and TBird
chrome.
So, whatever direction TBird will take, I certainly vote for making
the mailnews components accessible from 3rd party applications.
/Henrik
Dan Mosedale
>> The other question is does your application really need to be written in
>> c++? What about javascript?
>
> My application is a plugin for a general synchronization framework
> (OpenSync).
> So it might be used in many different ways to sync with other
> providers.
> And as a plugin to OpenSync, it has to be C (or C++ maybe Python as
> well).
>
It's worth keeping mind that the mail/news code currently expects to be
the only process accessing the stuff in your profile, and violating that
assumption (at least if you write to the profile) potentially risks
corruption. The safe way to do what you want to do, I think, would be
to run your code as part of the Thunderbird process (or have your code
talk via IPC of some sort to Thunderbird). As a bonus, this would
entirely avoid the linkage problems you're seeing now.
Dan
Henrik Kaare Poulsen
> It's worth keeping mind that the mail/news code currently expects to be
> the only process accessing the stuff in your profile, and violating that
> assumption (at least if you write to the profile) potentially risks
> corruption. The safe way to do what you want to do, I think, would be
> to run your code as part of the Thunderbird process (or have your code
> talk via IPC of some sort to Thunderbird). As a bonus, this would
> entirely avoid the linkage problems you're seeing now.
> Dan
Hi Dan,
Actually, my exe calls mail/news code to do the modifications.
But are you saying, that if I have two different processes (eg. 2 x
TBird or my exe + TBird) both accessing the profile, this might lead
to corruption?
I have been thinking about the IPC, but there are two issues at least:
1) Need to develop a lot of communication and packing/unpacking stuff
without adding functionality
2) Then it would not be possible to run without TBird open, and
probably difficult even with TBird open if the process would not be
initiated from TBird. And the plugin to OpenSync is supposed to be
stand-alone, so a user can sync e.g. KDE PIM to TBird from a KDE GUI
without opening TBird...
/Henrik
Dan Mosedale
>
> Actually, my exe calls mail/news code to do the modifications.
> But are you saying, that if I have two different processes (eg. 2 x
> TBird or my exe + TBird) both accessing the profile, this might lead
> to corruption?
Yes. I'm not sure what file-level-locking we do inside the profile (if
any); I bet bienvenu knows more than I do about this...
> I have been thinking about the IPC, but there are two issues at least:
> 1) Need to develop a lot of communication and packing/unpacking stuff
> without adding functionality
I could imagine using named shared memory segments to avoid
packing/unpacking. NSPR provides primitives for this.
> 2) Then it would not be possible to run without TBird open, and
> probably difficult even with TBird open if the process would not be
> initiated from TBird. And the plugin to OpenSync is supposed to be
> stand-alone, so a user can sync e.g. KDE PIM to TBird from a KDE GUI
> without opening TBird...
You could have two code paths, one for when Tb is running and one for
when it isn't. That's more work, of course.
If we don't currently do file-level locking on the files you need, we
might be able to introduce that. Again, I'm interested in bienvenu's
thoughts on this...
What set of files do you need to be able to touch?
Dan
Henrik Kaare Poulsen
I really, really appreciate your answers!
As an insignificant 3rd party developer at the outer rim of the
galaxy, it is great to receive such quick and instructive feedback!
> If we don't currently do file-level locking on the files you need, we
> might be able to introduce that. Again, I'm interested in bienvenu's
> thoughts on this...
>
> What set of files do you need to be able to touch?
The short answer:
For addressbook:
nsIAbCard and nsIAbMDBCard on @mozilla.org/addressbook/moz-abmdbcard;1
nsIAbDirectory on @mozilla.org/rdf/rdf-service;1
nsIAddrDatabase on @mozilla.org/carddatabase;1
For calendar:
mozIStorageConnection on @mozilla.org/storage/service;1
Currently, the implementation of my application is quite "convoluted",
with most of the code devoted to coding around mozilla "shortcomings",
and format conversion.
The long answer:
A) For sync of addressbook we need
1) Enumerate addressbooks, giving id + description for each
2) Open addresbook based on id
3) Enumerate cards in addressbook, giving id and either last
modification date or hash of data
4) Get card info, preferably in VCARD format
5) Add new card (returning id)
6) Delete card based on id
7) Modify card based on id and card info (preferably in VCARD format)
B) For sync of calendar:
The above but for calendar / event / VEVENT
I would think it makes a lot of sense to implement the above (in a
thread-safe and multi-process-safe way) as they will be needed for ANY
kind of sync application.
> > I have been thinking about the IPC, but there are two issues at least:
> > 1) Need to develop a lot of communication and packing/unpacking stuff
> > without adding functionality
>
> I could imagine using named shared memory segments to avoid
> packing/unpacking. NSPR provides primitives for this.
Well, yes, but if I really go for the IPC way, then the non-mozilla
end should be independent on mozilla/xpcom, so it should not use NSPR.
> > 2) Then it would not be possible to run without TBird open, and
> > probably difficult even with TBird open if the process would not be
> > initiated from TBird. And the plugin to OpenSync is supposed to be
> > stand-alone, so a user can sync e.g. KDE PIM to TBird from a KDE GUI
> > without opening TBird...
>
> You could have two code paths, one for when Tb is running and one for
> when it isn't. That's more work, of course.
Again, yes: you are right.
I would not mind the extra work except for one thing:
In my POW, it is just workarounds for non-optimal solutions in TBird
(which is NOT a critique, since I know you have or had a different
focus, but still...).
It seems that including synchronization in TBird is out of scope for
TBird 3 (ref https://bugzilla.mozilla.org/show_bug.cgi?id=303963) but
if we could get the "basics" right ref "Long answer" above, we would
have come a long way.
/Henrik
Dan Mosedale
>> What set of files do you need to be able to touch?
>
>
> A) For sync of addressbook we need
> 1) Enumerate addressbooks, giving id + description for each
> 2) Open addresbook based on id
> 3) Enumerate cards in addressbook, giving id and either last
> modification date or hash of data
> 4) Get card info, preferably in VCARD format
> 5) Add new card (returning id)
> 6) Delete card based on id
> 7) Modify card based on id and card info (preferably in VCARD format)
>
> B) For sync of calendar:
> The above but for calendar / event / VEVENT
When I asked this question, what I had in mind was the possibility of
having your out-of-process code continue to access the files directly as
they do today, and using OS-level file locks to arbitrate modifications
to the data. For this to work, we'd need to figure out
a) does the addrbook and calendar code already do anything like this?
b) if not, would it actually solve this problem and be worth trying?
c) who could do that work in the core (it could be you, if you're
interested :-)
>>> I have been thinking about the IPC, but there are two issues at least:
>>> 1) Need to develop a lot of communication and packing/unpacking stuff
>>> without adding functionality
>> I could imagine using named shared memory segments to avoid
>> packing/unpacking. NSPR provides primitives for this.
>
> Well, yes, but if I really go for the IPC way, then the non-mozilla
> end should be independent on mozilla/xpcom, so it should not use NSPR.
Linking with NSPR is much less painful than pulling in all of XPCOM
(XPCOM is built on top of NSPR).
>>> 2) Then it would not be possible to run without TBird open, and
>>> probably difficult even with TBird open if the process would not be
>>> initiated from TBird. And the plugin to OpenSync is supposed to be
>>> stand-alone, so a user can sync e.g. KDE PIM to TBird from a KDE GUI
>>> without opening TBird...
>> You could have two code paths, one for when Tb is running and one for
>> when it isn't. That's more work, of course.
>
> Again, yes: you are right.
> I would not mind the extra work except for one thing:
> In my POW, it is just workarounds for non-optimal solutions in TBird
> (which is NOT a critique, since I know you have or had a different
> focus, but still...).
Agreed; as davida likes to say, there is quite a bit of accumulated
technical debt in the code base that we need to pay down.
> It seems that including synchronization in TBird is out of scope for
> TBird 3 (ref https://bugzilla.mozilla.org/show_bug.cgi?id=303963) but
> if we could get the "basics" right ref "Long answer" above, we would
> have come a long way.
Unfortunately true. It would certainly be great if we could take small
steps to get low-hanging fruit in this area into the tree. However,
since we're getting into the Tb3 endgame, that's only likely to happen
if contributors step up to the plate and write patches.
Dan
Henrik Kaare Poulsen
I think that it is a waste of time for me to implement workarounds, it
would be far better to do it the right way in the mozilla codebase.
So, I am definetly not promising anything, but let's start the
discussion and see where it goes.
I think we need to start with
(A) Agreeing on interfaces (idl and semantics)
and then
(B) implementing.
So let's start with (A).
Any sync application is heavily dependent on a unique ID.
This has been discussed for half a year with no conclusion:
https://bugzilla.mozilla.org/show_bug.cgi?id=444093
How can we bring this to a conclusion?
Now for some interface suggestions
(btw: is this the right place to discuss this?
or should I submit each in bugzilla, or ???)
--------------
nsIAbCard.idl:
--------------
For a sync app, the most important is a unique ID and last modified
time.
unique ID:
- should it be a UUID, or could it be any string?
- should we just get it with getProperty, or add something like
attribute AString uuid
or
attribute AString uniqueID
last modified time:
we have kLastModifiedDateProperty
- is this just date, or does it include time?
- there is no clue in the .idl as to the format of this
- how to get/set it? there seems to be no getters/setters for any kind
of date format.
-------------------
nsIAbDirectory.idl:
-------------------
- Why do we have both nsIAbDirectory.idl and nsIAbCollection.idl?
What's the difference?
- We have
95 // will be used for LDAP replication
96 attribute unsigned long lastModifiedDate;
- - does this mean, that lastModifiedDate is used only for LDAP, not
MDB ?
- We need to add a method like:
nsISimpleEnumerator getModifiedCards(unsigned long sinceModifiedDate)
- - any objections?
- We need to add a method like:
nsIAbCard getCardFromUUID(in AString UUID);
- - any objections?
(I am leaving the discussion on Add/Delete cards, since I belive that
they could be implemented using the above...)
What do you all think about this?
/Henrik
Joshua Cranmer
> Any sync application is heavily dependent on a unique ID.
> This has been discussed for half a year with no conclusion:
> https://bugzilla.mozilla.org/show_bug.cgi?id=444093
> How can we bring this to a conclusion?
There has been a conclusion (for the most part), if you've read my
extremely length summary comment (comment 20). I've just been too
swamped with work lately to bring it to full closure.
Some recent work I have done this past weekend has included writing a
test suite for correct UUID semantics:
<http://hg.mozilla.org/users/Pidgeot18_gmail.com/ab_rewrite/file/bug444093/mailnews/addrbook/test/unit/test_uuid.js>.
Please read the entirety of comment 20 of bug 444093 before making
suggestions on the UUID properties. It is also helpful to read many of
the previous 20 comments as they have raised issues in terms of both
implementation and usage.
Henrik Kaare Poulsen
Thanks a lot for your quick reply!
On Jan 26, 7:51 pm, Joshua Cranmer <Pidgeo...@verizon.net> wrote:
> Henrik Kaare Poulsen wrote:
> > Any sync application is heavily dependent on a unique ID.
> > This has been discussed for half a year with no conclusion:
> >https://bugzilla.mozilla.org/show_bug.cgi?id=444093
> > How can we bring this to a conclusion?
>
> There has been a conclusion (for the most part), if you've read my
> extremely length summary comment (comment 20). I've just been too
> swamped with work lately to bring it to full closure.
> Please read the entirety of comment 20 of bug 444093 before making
> suggestions on the UUID properties. It is also helpful to read many of
> the previous 20 comments as they have raised issues in terms of both
> implementation and usage.
Well, I *did* read all the comments - including #20 - several times.
So my ignorance is not due to laziness but just to lack of proper
understanding.
I am sorry, but I am really a newbe here, and did not understand #20
as being a "conclusion"
(although a very enlightening comment indeed)
> Some recent work I have done this past weekend has included writing a
> test suite for correct UUID semantics:
> <http://hg.mozilla.org/users/Pidgeot18_gmail.com/ab_rewrite/file/bug44...>.
I cannot pretend to understand everything, but one thing puzzels me
indeed:
How can you test e.g. directory.uuid or card.localId ?
As far as I can see, they are not in the hg .idl files ???
Also, should there not be a card.uuid test ?
/Henrik
PS:
This is my first exposure to the mozilla project, including code and
ways of working.
Please just tell me to get lost if my questions / comments start to be
a pain in the ... .
Joshua Cranmer
> On Jan 26, 7:51 pm, Joshua Cranmer <Pidgeo...@verizon.net> wrote:
>> Some recent work I have done this past weekend has included writing a
>> test suite for correct UUID semantics:
>> <http://hg.mozilla.org/users/Pidgeot18_gmail.com/ab_rewrite/file/bug44...>.
>
> I cannot pretend to understand everything, but one thing puzzels me
> indeed:
>
> How can you test e.g. directory.uuid or card.localId ?
> As far as I can see, they are not in the hg .idl files ???
These are changed locally in that branch. The entire branch is a
collection of changesets that will eventually become a patch.
> Also, should there not be a card.uuid test ?
I haven't actually gotten around to writing that part yet, but I left a
placeholder via a comment (question 2.4, IIRC).
> This is my first exposure to the mozilla project, including code and
> ways of working.
> Please just tell me to get lost if my questions / comments start to be
> a pain in the ... .
At this rate, it's probably easiest if you get on IRC for general
development questions (irc.mozilla.org, mostly channel #maildev).
Henrik Kaare Poulsen
> These are changed locally in that branch. The entire branch is a
> collection of changesets that will eventually become a patch.
Aah, this sounds really great!
Guess there is not much point in me trying to do something as
suggested by Dan, then (-;
I'll just wait until those changes find their way into the hg trunk...
/Henrik