I have developed an application for Thunderbird 2, and am now trying to port to Thunderbird 3. The application includes a standalone xpcom exe and an extension.
The application makes use of different components, such as: @mozilla.org/addressbook;1 @mozilla.org/addressbook/carddatabase;1 @mozilla.org/addressbook/moz-abmdbcard;1
However, those components seems to be missing in Thunderbird 3.
In fact, for Thunderbird 3, nsIComponentRegistrar-
>EnumerateContractIDs lists only 70 components compared to Thunderbird
2's more than 900 components.
The problem occurs both in 3.0b1 download, in recent nightlies download, as well as my own recent local builds of comm-central mercurial source.
Does anybody know how to get access to those components ?
Thanks in advance /Henrik
PS: In case anybody is interested, the application is blueZync for synchronizing the addressbook: http://kaarposoft.dk/bluezync/
[note I've reordered some of Henrik's email to make the response better ordered]
On 01/18/2009 22:14, Henrik Kaare Poulsen wrote: > However, those components seems to be missing in Thunderbird 3.
They are not "missing", they have either been removed or replaced in development. The functions are still typically present, but have been reworked on different interfaces (or removed if there were duplicates).
Between Thunderbird 2 and Thunderbird 3 we have reworked a lot of the interfaces to provide a much better, generic, easier to use set. We haven't made it as far as we would have liked, but what is now there is certainly much better than what we had before.
> The application makes use of different components, such as: > @mozilla.org/addressbook;1 > @mozilla.org/addressbook/moz-abmdbcard;1
For the basic address book (@mozilla.org/addressbook;1 - nsIAddressBook), try looking at nsIAbManager in the new code.
For mdb cards (@mozilla.org/addressbook/moz-abmdbcard;1 - nsIAbMDBCard) the functions/properties there are either on nsIAbCard, or not required to be accessible.
> @mozilla.org/addressbook/carddatabase;1
This one should still be present. If I enter:
"@mozilla.org/addressbook/carddatabase;1" in Components.classes
into the error console in today's Thunderbird 3.0b2pre nightly build it returns true which indicates it is there.
I suspect you are trying to save your own custom properties onto cards. If so, you should now use the get/setProperty functions on nsIAbCard. Once you have modified your card object, you can save it by using the modifyCard function on the nsIAbDirectory where you got the card from.
> In fact, for Thunderbird 3, nsIComponentRegistrar- >> EnumerateContractIDs lists only 70 components compared to Thunderbird > 2's more than 900 components.
That does sound like you have some sort of issue, though maybe something changed in the core code relating to how they are listed - we certainly haven't removed that many components afaik.
Henrik Kaare Poulsen wrote: > The application makes use of different components, such as: > @mozilla.org/addressbook;1 > @mozilla.org/addressbook/carddatabase;1 > @mozilla.org/addressbook/moz-abmdbcard;1
The address book has undergone extensive rewriting in Thunderbird 3. Specifically, the nsIAbManager interface has been created and superseded a few earlier interfaces for use in managing address book directories; address book directories have gained a few new methods (superseding usage of nsIAddrDatabase in a few cases); and the usage patterns for nsIAbCard have drastically changed.
I also recommend eschewing usage of nsIAddrDatabase whenever possible because it will be removed in the near (but not TB 3) future.
FWIIW, I have updated my application source to compile against latest build of Thunderbird 3. The problem only occurs at runtime where the components are not found. /Henrik
> Even in my own build of current comm-central mercurial source, the > components do not show up in the > dist/bin/components directory...
Which files are you looking for? Thunderbird release builds (which both of those are) are built with --enable-static, this will lump most of the components into one library (or maybe the executable, I can't remember which off hand).
Either way, your code should still be able to access them via the normal methods (i.e. createInstance/getService).
On Jan 19, 10:53 am, Mark Banner <bugzi...@invalid.standard8.plus.com> wrote:
> Which files are you looking for?
I am looking for the files which contain the addressbook components.
> Thunderbird release builds (which both > of those are) are built with --enable-static, this will lump most of the > components into one library (or maybe the executable, I can't remember > which off hand).
Ahh - this might indeed be the problem.
If I grep for "addressbook" I only find the TBird exe. However, in TBird 2, I find components/libmail.so components/libmailcomps.so components/libimport.so (and not the TBird 2 exe)
> Either way, your code should still be able to access them via the normal > methods (i.e. createInstance/getService).
Apparently not... If all components are compiled into the TBird exe, my standalone XPCOM exe will not be able to get to them...
I can try to make a dynamic build, but in the future when users wants to use my exe in their distro, we will end up with the same problem if the distro packages the static version )-: I am wondering if it would make sense to build the component libraries by default even in a static build. This would help everyone making standalone XPCOM exe's wishing to access TBird.
> If I grep for "addressbook" I only find the TBird exe. > However, in TBird 2, I find > components/libmail.so > components/libmailcomps.so > components/libimport.so > (and not the TBird 2 exe)
FWIW Mac doesn't even have a "mail" lib.
>> Either way, your code should still be able to access them via the normal >> methods (i.e. createInstance/getService).
> Apparently not... > If all components are compiled into the TBird exe, my standalone XPCOM > exe will not be able to get to them...
> I can try to make a dynamic build, but in the future when users wants > to use my exe in their distro, we will end up with the same problem if > the distro packages the static version )-: > I am wondering if it would make sense to build the component libraries > by default even in a static build.
For Thunderbird that would mean two things: doubling the size of the download and package, as well as we'd have to stop one set of the libraries being loaded as if you were able to build both at the moment, then we would essentially try to load the components twice.
> This would help everyone making standalone XPCOM exe's wishing to > access TBird.
If you follow the model already out there - xulrunner which is what Firefox and Thunderbird are heading towards, then there would be a common set of libraries. However this would still not include the mailnews set, and we'd potentially have to do something on top of that which would provide the shared libraries.
Given that we haven't achieved the xulrunner stage yet, sorting out a shared library in the xulrunner style for mailnews is not going to be something that is done quickly if we do want to do it.
FWIW I have also never heard of this request before.
For your application, you could always build your own copy of mailnews, although not ideal it would mean that you could easily guarantee access to the components for everyone.
The other question is does your application really need to be written in c++? What about javascript?
There was some command-line argument for firefox where you could essentially run an application using the firefox libraries by starting it from the command line, a sort of intermediate xulrunner. Unfortunately I can't remember the details at the moment and I'm not sure it is supported in Thunderbird at the moment either.
> The other question is does your application really need to be written in > c++? What about javascript?
My application is a plugin for a general synchronization framework (OpenSync). So it might be used in many different ways to sync with other providers. And as a plugin to OpenSync, it has to be C (or C++ maybe Python as well).
> For your application, you could always build your own copy of mailnews, > although not ideal it would mean that you could easily guarantee access > to the components for everyone.
Yes, but I would need to be sure that I build exactly the same version of the libs as the user is having from his distro to make sure that data is not damaged. This will be very hard to manage deployment-wise.
> If you follow the model already out there - xulrunner which is what > Firefox and Thunderbird are heading towards, then there would be a > common set of libraries. However this would still not include the > mailnews set, and we'd potentially have to do something on top of that > which would provide the shared libraries.
> Given that we haven't achieved the xulrunner stage yet, sorting out a > shared library in the xulrunner style for mailnews is not going to be > something that is done quickly if we do want to do it.
> FWIW I have also never heard of this request before.
My application can use the basic xpcom components from thunderbird, lightning or xulrunner. But since there is no GUI, there is not much point in using xulrunner. Letting my exe initialize XPCOM directly worked fine for TBird 2.
I have not followed the discussions and arguments for the static linking, and have very limited knowledge of the mozilla design and buildprocess. However, especially in the light of moving towards xulrunner, I think accessible mailnews libraries would be the right way to go... If I understand correctly, there would be no TBird exe anymore, instead one just runs xulrunner with mailnews libraries and TBird chrome.
So, whatever direction TBird will take, I certainly vote for making the mailnews components accessible from 3rd party applications.
>> The other question is does your application really need to be written in >> c++? What about javascript?
> My application is a plugin for a general synchronization framework > (OpenSync). > So it might be used in many different ways to sync with other > providers. > And as a plugin to OpenSync, it has to be C (or C++ maybe Python as > well).
It's worth keeping mind that the mail/news code currently expects to be the only process accessing the stuff in your profile, and violating that assumption (at least if you write to the profile) potentially risks corruption. The safe way to do what you want to do, I think, would be to run your code as part of the Thunderbird process (or have your code talk via IPC of some sort to Thunderbird). As a bonus, this would entirely avoid the linkage problems you're seeing now. Dan
On Jan 19, 6:52 pm, Dan Mosedale <dm...@mozilla.org> wrote:
> It's worth keeping mind that the mail/news code currently expects to be > the only process accessing the stuff in your profile, and violating that > assumption (at least if you write to the profile) potentially risks > corruption. The safe way to do what you want to do, I think, would be > to run your code as part of the Thunderbird process (or have your code > talk via IPC of some sort to Thunderbird). As a bonus, this would > entirely avoid the linkage problems you're seeing now. > Dan
Hi Dan, Actually, my exe calls mail/news code to do the modifications. But are you saying, that if I have two different processes (eg. 2 x TBird or my exe + TBird) both accessing the profile, this might lead to corruption? I have been thinking about the IPC, but there are two issues at least: 1) Need to develop a lot of communication and packing/unpacking stuff without adding functionality 2) Then it would not be possible to run without TBird open, and probably difficult even with TBird open if the process would not be initiated from TBird. And the plugin to OpenSync is supposed to be stand-alone, so a user can sync e.g. KDE PIM to TBird from a KDE GUI without opening TBird...
> Actually, my exe calls mail/news code to do the modifications. > But are you saying, that if I have two different processes (eg. 2 x > TBird or my exe + TBird) both accessing the profile, this might lead > to corruption?
Yes. I'm not sure what file-level-locking we do inside the profile (if any); I bet bienvenu knows more than I do about this...
> I have been thinking about the IPC, but there are two issues at least: > 1) Need to develop a lot of communication and packing/unpacking stuff > without adding functionality
I could imagine using named shared memory segments to avoid packing/unpacking. NSPR provides primitives for this.
> 2) Then it would not be possible to run without TBird open, and > probably difficult even with TBird open if the process would not be > initiated from TBird. And the plugin to OpenSync is supposed to be > stand-alone, so a user can sync e.g. KDE PIM to TBird from a KDE GUI > without opening TBird...
You could have two code paths, one for when Tb is running and one for when it isn't. That's more work, of course.
If we don't currently do file-level locking on the files you need, we might be able to introduce that. Again, I'm interested in bienvenu's thoughts on this...
What set of files do you need to be able to touch?
I really, really appreciate your answers! As an insignificant 3rd party developer at the outer rim of the galaxy, it is great to receive such quick and instructive feedback!
> If we don't currently do file-level locking on the files you need, we > might be able to introduce that. Again, I'm interested in bienvenu's > thoughts on this...
> What set of files do you need to be able to touch?
The short answer:
For addressbook: nsIAbCard and nsIAbMDBCard on @mozilla.org/addressbook/moz-abmdbcard;1 nsIAbDirectory on @mozilla.org/rdf/rdf-service;1 nsIAddrDatabase on @mozilla.org/carddatabase;1
For calendar: mozIStorageConnection on @mozilla.org/storage/service;1
Currently, the implementation of my application is quite "convoluted", with most of the code devoted to coding around mozilla "shortcomings", and format conversion.
The long answer:
A) For sync of addressbook we need 1) Enumerate addressbooks, giving id + description for each 2) Open addresbook based on id 3) Enumerate cards in addressbook, giving id and either last modification date or hash of data 4) Get card info, preferably in VCARD format 5) Add new card (returning id) 6) Delete card based on id 7) Modify card based on id and card info (preferably in VCARD format)
B) For sync of calendar: The above but for calendar / event / VEVENT
I would think it makes a lot of sense to implement the above (in a thread-safe and multi-process-safe way) as they will be needed for ANY kind of sync application.
> > I have been thinking about the IPC, but there are two issues at least: > > 1) Need to develop a lot of communication and packing/unpacking stuff > > without adding functionality
> I could imagine using named shared memory segments to avoid > packing/unpacking. NSPR provides primitives for this.
Well, yes, but if I really go for the IPC way, then the non-mozilla end should be independent on mozilla/xpcom, so it should not use NSPR.
> > 2) Then it would not be possible to run without TBird open, and > > probably difficult even with TBird open if the process would not be > > initiated from TBird. And the plugin to OpenSync is supposed to be > > stand-alone, so a user can sync e.g. KDE PIM to TBird from a KDE GUI > > without opening TBird...
> You could have two code paths, one for when Tb is running and one for > when it isn't. That's more work, of course.
Again, yes: you are right. I would not mind the extra work except for one thing: In my POW, it is just workarounds for non-optimal solutions in TBird (which is NOT a critique, since I know you have or had a different focus, but still...).
It seems that including synchronization in TBird is out of scope for TBird 3 (ref https://bugzilla.mozilla.org/show_bug.cgi?id=303963) but if we could get the "basics" right ref "Long answer" above, we would have come a long way.
>> What set of files do you need to be able to touch?
> The long answer:
> A) For sync of addressbook we need > 1) Enumerate addressbooks, giving id + description for each > 2) Open addresbook based on id > 3) Enumerate cards in addressbook, giving id and either last > modification date or hash of data > 4) Get card info, preferably in VCARD format > 5) Add new card (returning id) > 6) Delete card based on id > 7) Modify card based on id and card info (preferably in VCARD format)
> B) For sync of calendar: > The above but for calendar / event / VEVENT
When I asked this question, what I had in mind was the possibility of having your out-of-process code continue to access the files directly as they do today, and using OS-level file locks to arbitrate modifications to the data. For this to work, we'd need to figure out
a) does the addrbook and calendar code already do anything like this? b) if not, would it actually solve this problem and be worth trying? c) who could do that work in the core (it could be you, if you're interested :-)
>>> I have been thinking about the IPC, but there are two issues at least: >>> 1) Need to develop a lot of communication and packing/unpacking stuff >>> without adding functionality >> I could imagine using named shared memory segments to avoid >> packing/unpacking. NSPR provides primitives for this.
> Well, yes, but if I really go for the IPC way, then the non-mozilla > end should be independent on mozilla/xpcom, so it should not use NSPR.
Linking with NSPR is much less painful than pulling in all of XPCOM (XPCOM is built on top of NSPR).
>>> 2) Then it would not be possible to run without TBird open, and >>> probably difficult even with TBird open if the process would not be >>> initiated from TBird. And the plugin to OpenSync is supposed to be >>> stand-alone, so a user can sync e.g. KDE PIM to TBird from a KDE GUI >>> without opening TBird... >> You could have two code paths, one for when Tb is running and one for >> when it isn't. That's more work, of course.
> Again, yes: you are right. > I would not mind the extra work except for one thing: > In my POW, it is just workarounds for non-optimal solutions in TBird > (which is NOT a critique, since I know you have or had a different > focus, but still...).
Agreed; as davida likes to say, there is quite a bit of accumulated technical debt in the code base that we need to pay down.
> It seems that including synchronization in TBird is out of scope for > TBird 3 (ref https://bugzilla.mozilla.org/show_bug.cgi?id=303963) but > if we could get the "basics" right ref "Long answer" above, we would > have come a long way.
Unfortunately true. It would certainly be great if we could take small steps to get low-hanging fruit in this area into the tree. However, since we're getting into the Tb3 endgame, that's only likely to happen if contributors step up to the plate and write patches.
I think that it is a waste of time for me to implement workarounds, it would be far better to do it the right way in the mozilla codebase. So, I am definetly not promising anything, but let's start the discussion and see where it goes.
I think we need to start with (A) Agreeing on interfaces (idl and semantics) and then (B) implementing.
So let's start with (A).
Any sync application is heavily dependent on a unique ID. This has been discussed for half a year with no conclusion: https://bugzilla.mozilla.org/show_bug.cgi?id=444093 How can we bring this to a conclusion?
Now for some interface suggestions (btw: is this the right place to discuss this? or should I submit each in bugzilla, or ???)
-------------- nsIAbCard.idl: --------------
For a sync app, the most important is a unique ID and last modified time.
unique ID: - should it be a UUID, or could it be any string? - should we just get it with getProperty, or add something like attribute AString uuid or attribute AString uniqueID
last modified time: we have kLastModifiedDateProperty - is this just date, or does it include time? - there is no clue in the .idl as to the format of this - how to get/set it? there seems to be no getters/setters for any kind of date format.
- Why do we have both nsIAbDirectory.idl and nsIAbCollection.idl? What's the difference? - We have 95 // will be used for LDAP replication 96 attribute unsigned long lastModifiedDate; - - does this mean, that lastModifiedDate is used only for LDAP, not MDB ? - We need to add a method like: nsISimpleEnumerator getModifiedCards(unsigned long sinceModifiedDate) - - any objections? - We need to add a method like: nsIAbCard getCardFromUUID(in AString UUID); - - any objections?
(I am leaving the discussion on Add/Delete cards, since I belive that they could be implemented using the above...)
Henrik Kaare Poulsen wrote: > Any sync application is heavily dependent on a unique ID. > This has been discussed for half a year with no conclusion: > https://bugzilla.mozilla.org/show_bug.cgi?id=444093 > How can we bring this to a conclusion?
There has been a conclusion (for the most part), if you've read my extremely length summary comment (comment 20). I've just been too swamped with work lately to bring it to full closure.
Please read the entirety of comment 20 of bug 444093 before making suggestions on the UUID properties. It is also helpful to read many of the previous 20 comments as they have raised issues in terms of both implementation and usage.
On Jan 26, 7:51 pm, Joshua Cranmer <Pidgeo...@verizon.net> wrote:
> Henrik Kaare Poulsen wrote: > > Any sync application is heavily dependent on a unique ID. > > This has been discussed for half a year with no conclusion: > >https://bugzilla.mozilla.org/show_bug.cgi?id=444093 > > How can we bring this to a conclusion?
> There has been a conclusion (for the most part), if you've read my > extremely length summary comment (comment 20). I've just been too > swamped with work lately to bring it to full closure. > Please read the entirety of comment 20 of bug 444093 before making > suggestions on the UUID properties. It is also helpful to read many of > the previous 20 comments as they have raised issues in terms of both > implementation and usage.
Well, I *did* read all the comments - including #20 - several times. So my ignorance is not due to laziness but just to lack of proper understanding. I am sorry, but I am really a newbe here, and did not understand #20 as being a "conclusion" (although a very enlightening comment indeed)
I cannot pretend to understand everything, but one thing puzzels me indeed:
How can you test e.g. directory.uuid or card.localId ? As far as I can see, they are not in the hg .idl files ???
Also, should there not be a card.uuid test ?
/Henrik
PS: This is my first exposure to the mozilla project, including code and ways of working. Please just tell me to get lost if my questions / comments start to be a pain in the ... .
Henrik Kaare Poulsen wrote: > On Jan 26, 7:51 pm, Joshua Cranmer <Pidgeo...@verizon.net> wrote: >> Some recent work I have done this past weekend has included writing a >> test suite for correct UUID semantics: >> <http://hg.mozilla.org/users/Pidgeot18_gmail.com/ab_rewrite/file/bug44...>.
> I cannot pretend to understand everything, but one thing puzzels me > indeed:
> How can you test e.g. directory.uuid or card.localId ? > As far as I can see, they are not in the hg .idl files ???
These are changed locally in that branch. The entire branch is a collection of changesets that will eventually become a patch.
> Also, should there not be a card.uuid test ?
I haven't actually gotten around to writing that part yet, but I left a placeholder via a comment (question 2.4, IIRC).
> This is my first exposure to the mozilla project, including code and > ways of working. > Please just tell me to get lost if my questions / comments start to be > a pain in the ... .
At this rate, it's probably easiest if you get on IRC for general development questions (irc.mozilla.org, mostly channel #maildev).
On Jan 27, 1:05 am, Joshua Cranmer <Pidgeo...@verizon.net> wrote:
> These are changed locally in that branch. The entire branch is a > collection of changesets that will eventually become a patch.
Aah, this sounds really great! Guess there is not much point in me trying to do something as suggested by Dan, then (-; I'll just wait until those changes find their way into the hg trunk...
