Google Contacts connector in ASynK not working

[Sriram Karra]

Some of you may have noticed that sync to Google Contacts has stopped working in ASynK. It is due to a planned shutdown by Google of Username/Password method of authentication that ASynK uses.

I have been working lazily on oAuth support in ASynK, but it is not yet ready for prime time. But give sync totally does not work any more I will be working on that fix at higher priority. I will send further updates on this thread as oAuth becomes available.

Cheers,

-Karra

[Julien Cubizolles]

Sriram Karra <karr...@gmail.com> writes:

> Some of you may have noticed that sync to Google Contacts has stopped
> working in ASynK. It is due to a planned shutdown by Google of
> Username/Password method of authentication that ASynK uses.

Just noticed it today :-(

> I will send further updates on this thread as oAuth becomes available.

Thanks a lot for your work, I'll be ready to test it when you need.

Julien.

[caj...@gmail.com]

Ive been trying to implement a google contacts / CardDav sync for days now.

OAuth2 info is only provided for the service account from the api console, and u cant add contacts to that account. Delegating a domain-wide service account also doesnt work when trying to impersonate the Admin, i get access denied even though i followed all the instructions in the docs for domain-wide delegation

Only way ive seen it work is through the curl command in unix. But that requires user clicking (login) to aprove the app's access to his Contacts (or other scope)

Let me know if u get this working

Good Luck

[Sriram Karra]

On Thu, Jun 11, 2015 at 9:01 PM, <zvonimi...@mbit.io> wrote:

On Thursday, June 4, 2015 at 12:53:56 PM UTC+2, karra.etc wrote:
> Some of you may have noticed that sync to Google Contacts has stopped working in ASynK. It is due to a planned shutdown by Google of Username/Password method of authentication that ASynK uses.
>
>

I have been making a google contacts / carddav sync using OAuth2, and let me tell you, im not sure its possible for normal gmail accounts.

 

Only way that I've seen it working is using curl (but that requires the browser-click for authorization every time u make the request) but thats not what im looking for. (looking for automatic sync without user clicking)

g

It is fair to say that the simple workflow supported so far will not be possible. So I agree with what you are saying on that. That absolutely f-ing sucks. Is this called progress? *sigh*

But it's possible to build an app that can take user consent (with a click in the browser) and hook it up. 

 

Furthermore,u can't add contacts to a service account (in api console), and Google only provides OAuth2 information for the service account.

You are right. From my experiments so far Service Account is not the way to go for this app. For the reasons you have identified above. I am trying other things, which I will explain if it works. Again, I agree with your overall prognosis that the days of running an ASynK Google sync from a cron job or something similar are sort of over.

 

> I have been working lazily on oAuth support in ASynK, but it is not yet ready for prime time. But give sync totally does not work any more I will be working on that fix at higher priority. I will send further updates on this thread as oAuth becomes available.

Great, let me know if u manage to do it without the user clicking (the method described is making a google apps account (no gmail domain possible) and then delegating a service account and using it to access the contacts while impersonating the Admin of the Apps domain (when i try impersonating, i get access denied, as i mentioned before)

Will keep you posted. 

[zvone]

Tommorow i will be trying a headless browser framework method.

Something like phantomJS should do the trick with using only curl and login information

This might let u keep the app running with the same method of authentication, although u will probably need to change the contacts methods calls, like adding removing and so forth

I'l also keep you posted of any progress i make

--
You received this message because you are subscribed to a topic in the Google Groups "ASynK Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/asynk/WKvfdfZJ1eE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to asynk+un...@googlegroups.com.
To post to this group, send email to as...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/CAFkt3UO7T2zkacGWgLg%2B3PT8%3DHHmiC0DG-J5w8R7p7a6w8LGpw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

reinvent the wheel

[Julien Cubizolles]

Sriram Karra <karr...@gmail.com> writes:

> Some of you may have noticed that sync to Google Contacts has stopped
> working in ASynK. It is due to a planned shutdown by Google of
> Username/Password method of authentication that ASynK uses.

I noticed that some contacts synced between bbdb and google contacts
through asynk recently got lost on the bbdb side. Could it be related ?

Also it seems that you can access google contacts through CardDav, and
access these contacts seamlessly through the contact application on an
Andoid Phone. Do you think that could be used to replace a
Google-contacts-on-Android-phone <-> bbdb sync system ?

Julien.

[Sriram Karra]

Things are getting back under control. I have pushed fixes to a new branch called gdata-oauth2. Make sure you read the instructions here:  https://github.com/skarra/ASynK/blob/gdata-oauth2/README-google-oauth.md

Look forward to people trying it out.

[Sriram Karra]

Contacts should not get deleted. Can you try out the gdata-oauth2 branch that I just pushed and let me know how that works? 

[zvone]

Do i need to rebuild (./setup.py install) it from the new branch ? or just run it after checkout ?

i get "ImportError: No module named apiclient" after running it now, was working before

--
You received this message because you are subscribed to a topic in the Google Groups "ASynK Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/asynk/WKvfdfZJ1eE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to asynk+un...@googlegroups.com.
To post to this group, send email to as...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/CAFkt3UPZ5QaoJn8J0oNrn-e6Sr22_hB_THc%2Bps%2BufLkSW__M7Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

reinvent the wheel

[Sriram Karra]

Do a pip install googleapiclient

You received this message because you are subscribed to the Google Groups "ASynK Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to asynk+un...@googlegroups.com.

To post to this group, send email to as...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/CAD3cKMD9_6W7kjzSJpa6D3nTvM3ewRbXFMgqmMt2t8ZwuwPS4A%40mail.gmail.com.

[zvone]

Downloading/unpacking googleapiclient

  Real name of requirement googleapiclient is googleapiclient

  Could not find any downloads that satisfy the requirement googleapiclient

Cleaning up...

No distributions at all found for googleapiclient

Storing debug log for failure in /.pip/pip.log

More help please? im not so proficient building python

To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/CAFkt3UNU4_esi7zmNuAtEby%2BVr992%3D1X_z8M1%3DoJ4vg1o%3DYcig%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

reinvent the wheel

[Sriram Karra]

Sorry. Try this:

pip install google-api-python-client

To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/CAD3cKMCFbSvWFzcJ%3DOBSv5aQNok5GHG8B2GRrdPhtgPWC%3D2q3Q%40mail.gmail.com.

[zvone]

Managed to get it working, but it opens a browser and ask for permission (login) still..This is not what im looking for.

Wasnt AsynK meant to be an automatic cron job ?

gonna continue working on my phantomjs script which will do everything automatically with just the login info

To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/CAFkt3UNV_bHcwart6ervet6qHqY8uxRWLje-4wMDAE5SmO6BDQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

reinvent the wheel

[Sriram Karra]

Ok. Good luck.

To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/CAD3cKMBARFvVRvU1QDHwRbe-sBC8UP9KUwbdtCgnQ87%2Bd0RK2w%40mail.gmail.com.

[Sriram Karra]

Note that the user consent step is required only the first time. ASynK remembers the credentials (as allowed by oAuth) and can work in headless mode for subsequent runs.

[zvone]

great, didnt know that.

So it supports CalDav <> Gmail sync out of the box ?

--
You received this message because you are subscribed to a topic in the Google Groups "ASynK Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/asynk/WKvfdfZJ1eE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to asynk+un...@googlegroups.com.
To post to this group, send email to as...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/CAFkt3UN9Gr0LF9%2B7a4quWTsS5%2ByX94WNwzbvqrHQ2yFXqyrnDQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

reinvent the wheel

[Sriram Karra]

Yes, that is correct.

You received this message because you are subscribed to the Google Groups "ASynK Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to asynk+un...@googlegroups.com.

To post to this group, send email to as...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/CAD3cKMBMYLKOeR%3Dj%2BMuztrD3Ar2CAYTMxKLv7JbP9Zx-QkFLug%40mail.gmail.com.

[Julien Cubizolles]

Sriram Karra <karr...@gmail.com> writes:


> Contacts should not get deleted. Can you try out the gdata-oauth2 branch
> that I just pushed and let me know how that works?

Sorry for not answering sooner.

I just checked out the gdata-oauth2 branch, installed the
google-api-python-client python module.

I get the following error:

--8<---------------cut here---------------start------------->8---
Traceback (most recent call last):
File "/home/wilk/info/emacs/ASynK/asynk.py", line 321, in <module>
main()
File "/home/wilk/info/emacs/ASynK/asynk.py", line 318, in main
asynk.dispatch()
File "/home/wilk/info/emacs/ASynK/asynk/asynk_core.py", line 97, in dispatch
res = getattr(self, self.get_op())()
File "/home/wilk/info/emacs/ASynK/asynk/asynk_core.py", line 366, in op_sync
pname = self._load_profile()
File "/home/wilk/info/emacs/ASynK/asynk/asynk_core.py", line 584, in _load_profile
self._login()
File "/home/wilk/info/emacs/ASynK/asynk/asynk_core.py", line 85, in _login
coll.login()
File "/home/wilk/info/emacs/ASynK/asynk/state_collection.py", line 336, in login
self.get_username(), self.get_pwd())
File "/home/wilk/info/emacs/ASynK/asynk/pimdb_gc.py", line 66, in __init__
self.gc_init()
File "/home/wilk/info/emacs/ASynK/asynk/pimdb_gc.py", line 264, in gc_init
self.credentials = self._oauth_dance(storage)
File "/home/wilk/info/emacs/ASynK/asynk/pimdb_gc.py", line 242, in _oauth_dance
redirect_uri='http://localhost:%d' % port)
File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 137, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 2053, in flow_from_clientsecrets
client_type, client_info = clientsecrets.loadfile(filename, cache=cache)
File "/usr/local/lib/python2.7/dist-packages/oauth2client/clientsecrets.py", line 155, in loadfile
return _loadfile(filename)
File "/usr/local/lib/python2.7/dist-packages/oauth2client/clientsecrets.py", line 115, in _loadfile
raise InvalidClientSecretsError('File not found: "%s"' % filename)
oauth2client.clientsecrets.InvalidClientSecretsError: File not found: "/home/wilk/my_gmail_password"
--8<---------------cut here---------------end--------------->8---
The File not found is named after the gmail password I just typed.

Julien.

[Sriram Karra]

Julien,

What form of authentication did you use? I did not understand why you are using your gmail password here as the filename.

The point of oAuth is to do away with using your password in clear text. Did you follow all the instructions in the new README file?

Julien.

--
You received this message because you are subscribed to the Google Groups "ASynK Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to asynk+un...@googlegroups.com.

To post to this group, send an email to as...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/87ioadxict.fsf%40free.fr.

[Julien Cubizolles]

Sriram Karra <karr...@gmail.com> writes:

>
> The point of oAuth is to do away with using your password in clear text.
> Did you follow all the instructions in the new README file?

Obviously I didn't... I'll try it and let you know.

Julien.

[Julien Cubizolles]

Sriram Karra <karr...@gmail.com> writes:

> Julien,
>
> What form of authentication did you use? I did not understand why you are
> using your gmail password here as the filename.

It works! thanks a lot. I have not been able to use the .netrc file
though. I've created a .netrc file with:

--8<---------------cut here---------------start------------->8---
machine bbgcsynk
login my_gmail_login
password ~/.asynk/gmail-contacts.json
--8<---------------cut here---------------end--------------->8---

Is there a command line switch to use it? What should the machine
contain ? (I put the name of the asynk profile)

Julien.

[Sriram Karra]

On Thu, Jun 25, 2015 at 10:08 AM, Julien Cubizolles <j.cubi...@free.fr> wrote:

--8<---------------cut here---------------start------------->8---
machine bbgcsynk
        login my_gmail_login
        password ~/.asynk/gmail-contacts.json
--8<---------------cut here---------------end--------------->8---

Is there a command line switch to use it? What should the machine
contain ? (I put the name of the asynk profile)

Change the machine line to:

machine gc_bbgcsynk

If the credentials are for google contacts it should start with gc_ Similarly cd_ for carddave and ex_ for Exchange server

[Julien Cubizolles]

Sriram Karra <karr...@gmail.com> writes:


>> Is there a command line switch to use it? What should the machine
>> contain ? (I put the name of the asynk profile)

> Change the machine line to:
>
> machine gc_bbgcsynk

I changed that, and fixed the permissions to .netrc. But now, gmail
seems to refuse the connection: wether I use the .netrc file or disable
it and use the login password->json file method, I get the following:

--8<---------------cut here---------------start------------->8---
gdata.client.Unauthorized: Unauthorized - Server responded with: 401, <?xml version="1.0" encoding="UTF-8"?>
<errors xmlns="http://schemas.google.com/g/2005">
<error>
<domain>GData</domain>
<code>authError</code>
<location type="header">Authorization</location>
<internalReason>Invalid Credentials</internalReason>
</error>
</errors>

Exception AttributeError: "'GCPIMDB' object has no attribute 'server'" in <bound method GCPIMDB.__del__ of <pimdb_gc.GCPIMDB object at 0x7f777b021d10>> ignored
--8<---------------cut here---------------end--------------->8---

When I first set up the oauth credentials, the first try was a success.

Julien.

[Sriram Karra]

On Thu, Jun 25, 2015 at 6:12 PM, Julien Cubizolles <j.cubi...@free.fr> wrote:

I changed that, and fixed the permissions to .netrc. But now, gmail
seems to refuse the connection: wether I use the .netrc file or disable
it and use the login password->json file method, I get the following:

Can you send me two things:

1. Your ~/.netrc file portion that refers to this profile

2. ls -al ~/.netrc

3. debug logs for a dry run

You can email it directly ot me instead of the list, if you so wish.

[Sriram Karra]

On Thu, Jun 25, 2015 at 9:34 PM, Sriram Karra <karr...@gmail.com> wrote:

On Thu, Jun 25, 2015 at 6:12 PM, Julien Cubizolles <j.cubi...@free.fr> wrote:

I changed that, and fixed the permissions to .netrc. But now, gmail
seems to refuse the connection: wether I use the .netrc file or disable
it and use the login password->json file method, I get the following:

Can you send me two things:

1. Your ~/.netrc file portion that refers to this profile

2. ls -al ~/.netrc

3. debug logs for a dry run

4. ls -al ~/.asynk
 

[Julien Cubizolles]

Sriram Karra <karra.etc-Re5JQE...@public.gmane.org> writes:


>> 1. Your ~/.netrc file portion that refers to this profile

--8<---------------cut here---------------start------------->8---
machine gc_bbgcsynk
login j.cubizolles
password ~/.asynk/gmail-contacts.json
-8<---------------cut here---------------end--------------->8---

>> 2. ls -al ~/.netrc

--8<---------------cut here---------------start------------->8---
-rw------- 1 wilk wilk 80 juin 25 14:17 /home/wilk/.netrc
--8<---------------cut here---------------end--------------->8---

>> 3. debug logs for a dry run

--8<---------------cut here---------------start------------->8---
[11:26:15.829 DEBUG] Command line: "/home/wilk/info/emacs/ASynK/asynk.py --op sync --name bbgcsynk --dry-run"
[11:26:15.829 INFO] Parsing BBDB file /home/wilk/.bbdb...
[11:26:15.829 DEBUG] New BBContactsFolder: default
[11:26:15.829 INFO] Parsing BBDB Store with encoding utf-8...
[11:26:15.840 DEBUG] Assigning UUID 64de1254-1be5-11e5-9191-6045bdf646b4 for new contact: Hervé Leguil
[11:26:15.840 DEBUG] New BBContactsFolder: asynk
[11:26:15.965 DEBUG] New BBContactsFolder: eleves-14-15
[11:26:15.965 INFO] Parsing BBDB Store with encoding utf-8...Success
[11:26:15.966 INFO] Successfully parsed 536 entries.
[11:26:15.966 DEBUG] PIMDB gc does not have email_domains.
[11:26:15.966 DEBUG] PIMDB gc does not have notes_map
[11:26:15.966 DEBUG] PIMDB gc does not have phones_map
[11:26:15.966 INFO] Attempting to log into Google...
[11:26:15.968 INFO] Using pre-fetched access_token...
[11:26:15.968 DEBUG] Getting Group List to populate folders...
--8<---------------cut here---------------end--------------->8---

> 4. ls -al ~/.asynk

--8<---------------cut here---------------start------------->8---
total 60
drwxrwxr-x 4 wilk wilk 4096 juin 25 06:28 .
drwxr-xr-x 72 wilk wilk 4096 juin 26 11:29 ..
drwxrwxr-x 2 wilk wilk 4096 juin 25 06:28 backups
-rw-rw-r-- 1 wilk wilk 1205 août 13 2013 config.py
-rw-r--r-- 1 wilk wilk 743 mars 28 2014 config.pyc
-r-------- 1 wilk wilk 441 juin 25 06:12 gmail-contacts.json
-rw------- 1 wilk wilk 859 juin 25 06:28 j.cubizolles.dat
drwxrwxr-x 2 wilk wilk 4096 juin 26 11:26 logs
-rw-rw-r-- 1 wilk wilk 21626 juin 25 06:28 state.json
--8<---------------cut here---------------end--------------->8---

The error message is:

--8<---------------cut here---------------start------------->8---
Traceback (most recent call last):
File "/home/wilk/info/emacs/ASynK/asynk.py", line 321, in <module>
main()
File "/home/wilk/info/emacs/ASynK/asynk.py", line 318, in main
asynk.dispatch()
File "/home/wilk/info/emacs/ASynK/asynk/asynk_core.py", line 97, in dispatch
res = getattr(self, self.get_op())()
File "/home/wilk/info/emacs/ASynK/asynk/asynk_core.py", line 366, in op_sync
pname = self._load_profile()
File "/home/wilk/info/emacs/ASynK/asynk/asynk_core.py", line 584, in _load_profile
self._login()
File "/home/wilk/info/emacs/ASynK/asynk/asynk_core.py", line 85, in _login
coll.login()
File "/home/wilk/info/emacs/ASynK/asynk/state_collection.py", line 336, in login
self.get_username(), self.get_pwd())

File "/home/wilk/info/emacs/ASynK/asynk/pimdb_gc.py", line 68, in __init__
self.set_folders()
File "/home/wilk/info/emacs/ASynK/asynk/pimdb_gc.py", line 166, in set_folders
groups = self.list_folders(silent=True)
File "/home/wilk/info/emacs/ASynK/asynk/pimdb_gc.py", line 87, in list_folders
feed = self.get_groups_feed()
File "/home/wilk/info/emacs/ASynK/asynk/pimdb_gc.py", line 293, in get_groups_feed
feed = self.get_gdc().GetGroups()
File "/usr/lib/python2.7/dist-packages/gdata/contacts/client.py", line 218, in get_groups
return self.get_feed(uri, desired_class=desired_class, auth_token=auth_token, **kwargs)
File "/usr/lib/python2.7/dist-packages/gdata/client.py", line 640, in get_feed
**kwargs)
File "/usr/lib/python2.7/dist-packages/gdata/client.py", line 307, in request
response, Unauthorized)

gdata.client.Unauthorized: Unauthorized - Server responded with: 401, <?xml version="1.0" encoding="UTF-8"?>
<errors xmlns="http://schemas.google.com/g/2005">
<error>
<domain>GData</domain>
<code>authError</code>
<location type="header">Authorization</location>
<internalReason>Invalid Credentials</internalReason>
</error>
</errors>

--8<---------------cut here---------------end--------------->8---


Exception AttributeError: "'GCPIMDB' object has no attribute 'server'" in <bound method GCPIMDB.__del__ of <pimdb_gc.GCPIMDB object at 0x7fe68d1e6dd0>> ignored



Thanks for your help,

Julien.

[Sriram Karra]

Hm, Just looking at that I am not sure what got hosed with the set up.

Could you delete ~/.asynk/j.cubizoles.dat and run asynk.py? You will go through the oAuth flow. I want to see if the result is different then.

[Julien Cubizolles]

Sriram Karra <karr...@gmail.com> writes:


>> Can you send me two things:

I just sent you the information you asked for to your public.gmane.org
address.

Julien.

[Julien Cubizolles]

I deleted the dat file and everything is back to normal now. I just ran
asynk.py five times in a row without a glitch.

J. Cubizolles.

[Sriram Karra]

Hey I don't have a gmane address. Please send to karra.etc @gmail

--
You received this message because you are subscribed to the Google Groups "ASynK Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to asynk+un...@googlegroups.com.
To post to this group, send an email to as...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/87a8vmzsd0.fsf%40free.fr.

[Sriram Karra]

I have merged the gdata-oauth2 branch to master and release v2.2.0-rc2 as an experimental release. https://github.com/skarra/ASynK/releases/tag/v2.2.0-rc1

[zvone]

ERROR:root:Could not delete itemid: <hash> 

(Code: 405, Reason: Method Not Allowed)

any idea why local apache (running owncloud) rejects DELETE methods ?

--
You received this message because you are subscribed to a topic in the Google Groups "ASynK Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/asynk/WKvfdfZJ1eE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to asynk+un...@googlegroups.com.
To post to this group, send email to as...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/asynk/CAFkt3UOE58ziQHHtvUE27yqQP5nXYFg6RQnUF3JK-7EsX4d12A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

reinvent the wheel