Re: Contacts sync

[Augustin Trancart]

Hi people \o

I'm currently working on syncing google contacts to FxOS device.

I made a privileged application, available in this repo [1] that consumes the Google Contact API.
It tries to follow these architectural recommandations from Francisco in [2].

At the moment, the sync process is manual. You need to click the "import" button to launch it. Making it recurrent is tracked in [3].

Edges are still quite rough, and you'll get NO visual feeback on how things are going :-) The WebIDE will give you some logs, which might or might not be of any help.
Also, the CSS probably needs some love. These things will be improved by [4].

There is no integration with the contact app at the moment. Some ideas have been given to me by Francisco in [1], but nothing implemented yet.

Most importantly: at the moment, it keeps your contact in sync with google in a one-way fashion, ignoring any modification you make locally. (you've been warned! On the other hand, it does not touch your google account, so it shouldn't screw anything there).

I have started to think about 2-ways sync, and the results of my thought are captured in [5].  I'm very interested in having feebacks about this algorithm! Any pair of eyes that is not mine will be welcome: I might have forgotten some situations or cases, and you might have a simpler idea :-) Fernando and Francisco, could you tell me what you think about it?

About compatibility: it works in 2.5+ with webcomponents activated. I might work it 2.2 if you activate them with webIDE, but AFAIK 2.2 does not have the manifest permission for wc, so I won't be able to submit this app in the marketplace for this version anyway. It works perfectly well in the simulator.

Testers, feedbacks and any other contributions are welcome. Thanks!

[1] https://github.com/Phoxygen/fxos-google-contacts-sync
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=859306#c122 (this one is about CardDAV sync, but I believe it also applies for any contact sync app)
[3] https://github.com/Phoxygen/fxos-google-contacts-sync/issues/10
[4] https://github.com/Phoxygen/fxos-google-contacts-sync/issues/13
[5] https://github.com/Phoxygen/fxos-google-contacts-sync/issues/19

Augustin Trancart
Phoxygen

PS: I haven't started CardDAV sync yet, but that's definitely something I want to do in the future!

[Fernando Jiménez Moreno]

Thank you Augustin!

I haven't tried or look at the code yet, but reading [1] I cannot avoid thinking that we really need to restart the work explained at [2] and do some changes in the mozContacts API if we want a proper contacts sync solution. As you are suggesting in your description, the mozContacts API has some limitations that makes contacts sync very hard to implement and also not very reliable. In addition to your comments, I would add the lack of concurrency control in the API. That has not been a significant issue for now cause most (if not all) of the write access to mozContacts has been done through actions triggered by (sometimes blocking) UI and we have an unique pre-installed contacts producer entry point. But now that we are talking about adding background synchronization and so adding extra contacts producers running in different processes, we may end up in a situation where concurrent access control is really needed to avoid conflicts and data loss.

IMHO we either need to move mozContacts to DataStore (which BTW runs in workers) or add some DataStore features like DataStore.sync() and revision per record to the mozContacts API.

Cheers,

/ Fernando

[1] https://github.com/Phoxygen/fxos-google-contacts-sync/issues/19
[2] https://wiki.mozilla.org/Gaia/Contacts/Data_Refactor

[Augustin Trancart]

On 26/11/2015 15:33, Francisco Jordano wrote:

Cannot access the repo to install it :(

Yeah, I forgot to make it public :-/ Should be ok now

Augustin Trancart
Phoxygen

[Augustin Trancart]

Hi Fernando,

This is definitely something we need to do. I thought of this as well: I can workaround much of the contacts API limitations including the lack of information on deleted elements, but I cannot fix the concurrency problem.

Is there already open bugs, code pointers etc I can use to poke that? I don't know if I'll be able to do that, but I'll need it anyway if I want it to be reliable in concurrent environment. The risk is low (in terms of occurrence), but the criticity is very high.

I'll need to finish to read [2] I guess :-)

Thanks!

Augustin Trancart
Phoxygen

On 27/11/2015 12:04, Fernando Jiménez Moreno wrote:

Thank you Augustin!

I haven't tried or look at the code yet, but reading [1] I cannot avoid thinking that we really need to restart the work explained at [2] and do some changes in the mozContacts API if we want a proper contacts sync solution. As you are suggesting in your description, the mozContacts API has some limitations that makes contacts sync very hard to implement and also not very reliable. In addition to your comments, I would add the lack of concurrency control in the API. That has not been a significant issue for now cause most (if not all) of the write access to mozContacts has been done through actions triggered by (sometimes blocking) UI and we have an unique pre-installed contacts producer entry point. But now that we are talking about adding background synchronization and so adding extra contacts producers running in different processes, we may end up in a situation where concurrent access control is really needed to avoid conflicts and data loss.

IMHO we either need to move mozContacts to DataStore (which BTW runs in workers) or add some DataStore features like DataStore.sync() and revision per record to the mozContacts API.

Cheers,

/ Fernando

[1] https://github.com/Phoxygen/fxos-google-contacts-sync/issues/19
[2] https://wiki.mozilla.org/Gaia/Contacts/Data_Refactor

[Michiel de Jong]

On a side note, I had a quick look at how we could make this cross-browser.

Code is here: view-source:https://google-contacts.5apps.com/

The OAuth flow works, but it requires a CORS proxy (or cross-origin XHR permissions [1]) to access the Google Contacts API. The CORS proxy I used in this example doesn't work because it forwards the 405 Method Not Allowed response during the OPTIONS preflight. I asked the 5apps guys if they can fix this - or we could use a different CORS proxy.

[1] Cross-origin XHR permissions:

* in Chrome, Opera, Firefox: available to WebExtensions https://developer.chrome.com/extensions/xhr

* in Cordova-Android and Cordova-iOS: http://plugreg.com/plugin/MobileChromeApps/cordova-plugin-chrome-apps-sockets-tcp

* in Firefox OS: available to privileged apps https://developer.mozilla.org/en-US/Apps/Build/App_permissions#systemXHR

* not sure about Windows Edge / Windows Phone, but did find this: https://msdn.microsoft.com/en-us/magazine/dn879349.aspx

* not sure about Safari

Cheers,

Michiel.

[Michiel de Jong]

On Wed, Dec 16, 2015 at 2:26 PM, Michiel de Jong <mbde...@mozilla.com> wrote:

On a side note, I had a quick look at how we could make this cross-browser.

Code is here: view-source:https://google-contacts.5apps.com/

The OAuth flow works, but it requires a CORS proxy (or cross-origin XHR permissions [1]) to access the Google Contacts API. The CORS proxy I used in this example doesn't work because it forwards the 405 Method Not Allowed response during the OPTIONS preflight. I asked the 5apps guys if they can fix this

And they did! So https://google-contacts.5apps.com/ works now. :)