OwnCloud with BBDB - Corruption of Address Book
32 views
Skip to first unread message
Fr. John Jenkins
Mar 6, 2017, 1:16:44 PM3/6/17
to ASynK Users
First of all I would like to thank you for this wonderful software which I used in the past to keep my BBDB in sync with Owncloud and via Carddav also with my phone.
Having moved now to another country I have a new email address and a new server. I installed the latest version of Owncloud 9.1 and proceeded to make a profile as usual. I then was able to sync my BBDB to Owncloud for the first time without difficulty [about 1100 addresses]. Since it was a new installation it simply went from BBDB -> OC
However on testing my sync I noticed that each further synchronization modifies far too many files - about 300 are deleted and then remade. Now there are about 300 addresses that are missing. After a subsequent sync it wants to add them again. What makes this alarming is that I haven't modified any address.
Finally, after trying to synchronize again the whole process seems to time out at the moment it says in the debug log "302 addresses need to be synched" -- however I haven't added any addresses to either store. Actually BBDB now only contains 302 addresses whereas before it contained 1100 or so. Thus I am also losing addresses.
ASynK doesn't seem to work with Owncloud 9.1, whereas before I had no problems.
Is ASynK simply not compatible with the latest version of Owncloud and should I roll back to an older version?
Thank you for any help you might provide. At the moment unfortunately I cannot use ASynK without corrupting the address books of both stores. it is such an essential piece of software for me that I would be willing to pay a bounty to fix this problem or make it compatible with Owncloud 9.1
JJ +
Having moved now to another country I have a new email address and a new server. I installed the latest version of Owncloud 9.1 and proceeded to make a profile as usual. I then was able to sync my BBDB to Owncloud for the first time without difficulty [about 1100 addresses]. Since it was a new installation it simply went from BBDB -> OC
However on testing my sync I noticed that each further synchronization modifies far too many files - about 300 are deleted and then remade. Now there are about 300 addresses that are missing. After a subsequent sync it wants to add them again. What makes this alarming is that I haven't modified any address.
Finally, after trying to synchronize again the whole process seems to time out at the moment it says in the debug log "302 addresses need to be synched" -- however I haven't added any addresses to either store. Actually BBDB now only contains 302 addresses whereas before it contained 1100 or so. Thus I am also losing addresses.
ASynK doesn't seem to work with Owncloud 9.1, whereas before I had no problems.
Is ASynK simply not compatible with the latest version of Owncloud and should I roll back to an older version?
Thank you for any help you might provide. At the moment unfortunately I cannot use ASynK without corrupting the address books of both stores. it is such an essential piece of software for me that I would be willing to pay a bounty to fix this problem or make it compatible with Owncloud 9.1
JJ +
Fr. John Jenkins
Mar 6, 2017, 1:38:36 PM3/6/17
to ASynK Users
Also I should point out that before synching I removed from BBDB all the fields that had an asynk:id as I knew that would probably cause difficulties. Thus the BBDB addressbook was entirely 'clean' when I began the process.
Sriram Karra
Mar 7, 2017, 4:01:12 AM3/7/17
to as...@googlegroups.com
John, I am sorry to hear about this. I have not tested ASynK with latest versions of OwnCloud, so I cannot comment on whether it is "officially" incompatible. As this is a side project that I maintain in my free time I have not invested in building robust end to end tests but rely on my users to inform me about issues with different stores as they encounter them. This is the first report I have received about problems with OwnCloud 9.1.
I will investigate this issue a little more in the coming weeks. In the mean time I guess you have no option but to roll back your version of ownCloud or just pause using ASynK, depending on your relative preference.
On Tue, Mar 7, 2017 at 12:08 AM, Fr. John Jenkins <jsrje...@gmail.com> wrote:
Also I should point out that before synching I removed from BBDB all the fields that had an asynk:id as I knew that would probably cause difficulties. Thus the BBDB addressbook was entirely 'clean' when I began the process.
--
Find ASynK useful? Consider leaving a tip at this Bitcoin address: 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
---
You received this message because you are subscribed to the Google Groups "ASynK Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to asynk+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Sriram Karra
Mar 7, 2017, 10:15:51 AM3/7/17
to as...@googlegroups.com
John, if you can create a test account for me on an OwnCloud 9.1 instance that would help me debug this issue. I have debugged other issues with OwnCloud using a similar approach with other users. It appears like installing a version of Owncloud 9.1 from scratch is going to be too painful for the present purpose.
Sriram Karra
Mar 9, 2017, 1:29:13 PM3/9/17
to as...@googlegroups.com
John, thanks for sending me the test credentials. I think I have isolated the issue. I have put a fix in this branch: https://github.com/skarra/ASynK/tree/issues/110
If you can validate it works for you as well, then I will test against some other older CardDAV servers and then merge it to master.
-Sriram
Fr. John Jenkins
Mar 10, 2017, 8:35:09 AM3/10/17
to as...@googlegroups.com
Dear Sriram,
Thanks for looking into this problem. I checked out the /issues/110
branch and I still get the same symptoms.
1/ sync from bbdb -> CardDav seems to work - 1016 entries created and
written to the server. The first sync seems perfect.
2/ sync after this first syncronization however begins to have the same
problem as noted before. It sees that 1016 entries in bbdb are the same,
but then sees that there are 1016 "NEW" entries in Carddav but
"deleted".
So it begins a "conflict resolution" : 1016 in bbdb need to be deleted.
Here is the log from debug. Afterwards it begins creating new entries in
BBDB and then deletes them. There seems to be something fundamentally
wrong in the way it classifies the CD entries as 'deleted'.
I am trying ASynK now with NextCloud which is a more up to date version
of Owncloud 9.1 thinking that there was something perhaps fixed in a
newer version. Unfortunately it is showing the same behavior as OC 9.1.
Also to not that ASynK hangs on "Writing sync state to the CardDAV
server", and stays this way for at least an hour (I interrupted it with
C-c after an hour of waiting).
Here is the log from the second sync:
Last synk for profile SSPXZA was at: 2017-03-10T07:23:01.93Z
[12:00:25.997 INFO] ==== bb =====
[12:00:25.997 INFO] New : 0
[12:00:25.997 INFO] Modified : 0
[12:00:25.997 INFO] Unchanged : 1016
[12:00:25.997 INFO] =====
[12:00:25.997 INFO] Total Entries : 1016
[12:00:25.997 INFO] Deleted : 0
[12:00:25.997 INFO] ==== cd =====
[12:00:25.997 INFO] New : 1016
[12:00:25.997 INFO] Modified : 0
[12:00:25.997 INFO] Unchanged : 0
[12:00:25.997 INFO] =====
[12:00:25.998 INFO] Total Entries : 1016
[12:00:25.998 INFO] Deleted : 1016
[12:00:25.998 INFO] Number of entries modified both places
(conflicts): 0
[12:00:25.998 INFO] conflict resolve direction : 1. db1id: bb,
db2id: cd
[12:00:25.998 INFO] After conflict resolution, size of bb mod :
0
[12:00:25.998 INFO] After conflict resolution, size of cd mod :
0
[12:00:25.999 INFO] After conflict resolution, size of bb del :
0
[12:00:25.999 INFO] After conflict resolution, size of cd del :
1016
[12:00:25.999 INFO]
=====================================================
[12:00:25.999 INFO] Sending New bb entries to cd
[12:00:25.999 INFO]
=====================================================
[12:00:25.999 INFO] No new entries that need to be synched
[12:00:25.999 INFO]
=====================================================
[12:00:25.999 INFO] Sending Modified bb entries to cd
[12:00:25.999 INFO]
=====================================================
[12:00:25.999 INFO] No modified entries that need to be synched
[12:00:26.000 INFO]
=====================================================
[12:00:26.000 INFO] Synching Deleted bb entries to cd
[12:00:26.000 INFO]
=====================================================
[12:00:26.000 INFO] No deleted entries that need to be synched.
[12:00:26.000 INFO]
=====================================================
[12:00:26.000 INFO] Sending New cd entries to bb
[12:00:26.000 INFO]
=====================================================
[12:00:26.000 INFO] 1016 new entries to be synched.
In die Thu, Mar 09 2017, Sriram Karra scripsit:
> >>> an email to asynk+un...@googlegroups.com.
Thanks for looking into this problem. I checked out the /issues/110
branch and I still get the same symptoms.
1/ sync from bbdb -> CardDav seems to work - 1016 entries created and
written to the server. The first sync seems perfect.
2/ sync after this first syncronization however begins to have the same
problem as noted before. It sees that 1016 entries in bbdb are the same,
but then sees that there are 1016 "NEW" entries in Carddav but
"deleted".
So it begins a "conflict resolution" : 1016 in bbdb need to be deleted.
Here is the log from debug. Afterwards it begins creating new entries in
BBDB and then deletes them. There seems to be something fundamentally
wrong in the way it classifies the CD entries as 'deleted'.
I am trying ASynK now with NextCloud which is a more up to date version
of Owncloud 9.1 thinking that there was something perhaps fixed in a
newer version. Unfortunately it is showing the same behavior as OC 9.1.
Also to not that ASynK hangs on "Writing sync state to the CardDAV
server", and stays this way for at least an hour (I interrupted it with
C-c after an hour of waiting).
Here is the log from the second sync:
Last synk for profile SSPXZA was at: 2017-03-10T07:23:01.93Z
[12:00:25.997 INFO] ==== bb =====
[12:00:25.997 INFO] New : 0
[12:00:25.997 INFO] Modified : 0
[12:00:25.997 INFO] Unchanged : 1016
[12:00:25.997 INFO] =====
[12:00:25.997 INFO] Total Entries : 1016
[12:00:25.997 INFO] Deleted : 0
[12:00:25.997 INFO] ==== cd =====
[12:00:25.997 INFO] New : 1016
[12:00:25.997 INFO] Modified : 0
[12:00:25.997 INFO] Unchanged : 0
[12:00:25.997 INFO] =====
[12:00:25.998 INFO] Total Entries : 1016
[12:00:25.998 INFO] Deleted : 1016
[12:00:25.998 INFO] Number of entries modified both places
(conflicts): 0
[12:00:25.998 INFO] conflict resolve direction : 1. db1id: bb,
db2id: cd
[12:00:25.998 INFO] After conflict resolution, size of bb mod :
0
[12:00:25.998 INFO] After conflict resolution, size of cd mod :
0
[12:00:25.999 INFO] After conflict resolution, size of bb del :
0
[12:00:25.999 INFO] After conflict resolution, size of cd del :
1016
[12:00:25.999 INFO]
=====================================================
[12:00:25.999 INFO] Sending New bb entries to cd
[12:00:25.999 INFO]
=====================================================
[12:00:25.999 INFO] No new entries that need to be synched
[12:00:25.999 INFO]
=====================================================
[12:00:25.999 INFO] Sending Modified bb entries to cd
[12:00:25.999 INFO]
=====================================================
[12:00:25.999 INFO] No modified entries that need to be synched
[12:00:26.000 INFO]
=====================================================
[12:00:26.000 INFO] Synching Deleted bb entries to cd
[12:00:26.000 INFO]
=====================================================
[12:00:26.000 INFO] No deleted entries that need to be synched.
[12:00:26.000 INFO]
=====================================================
[12:00:26.000 INFO] Sending New cd entries to bb
[12:00:26.000 INFO]
=====================================================
[12:00:26.000 INFO] 1016 new entries to be synched.
In die Thu, Mar 09 2017, Sriram Karra scripsit:
Sriram Karra
Mar 11, 2017, 5:27:13 AM3/11/17
to as...@googlegroups.com
The symptoms you describe sound typical of a run that is building on an earlier failed run, which sort of messed up the state.
Could you do the following:
(a) Start from a fresh BBDB file
(b) delete all entries in your carddav
(c) create a new asynk profile
Now run the sync. Even if it does not fully work, it will make it easier for me to debug.
-Sriram
>>> an email to asynk+unsubscribe@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>
--
Find ASynK useful? Consider leaving a tip at this Bitcoin address: 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
--- You received this message because you are subscribed to the Google Groups "ASynK Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to asynk+unsubscribe@googlegroups.com.
Fr. John Jenkins
Mar 12, 2017, 10:49:47 AM3/12/17
to as...@googlegroups.com
Dear Sriram,
My bbdb is clean, with no id's or extraneous fields. I can send you a
copy of it if this would help.
I made a new account on the carddav server: "testing"
I put a copy into the 'default place' "~/.bbdb" then did the following
to create a new profile. What follows is the response.
python ~/Source/ASynK/asynk.py --op=create-profile --name=testing --db bb cd --store ~/.bbdb http://cdte
st.sspx.ph/remote.php/carddav/ --folder default default
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...done
Applying user customizations from file /home/jenkins/.asynk/config.py...
Applying user customizations from file /home/jenkins/.asynk/config.py...done
[15:21:32.315 INFO] Debug logging to file: /home/jenkins/.asynk/logs/asynk_logs.2017-03-12.15-21-32.315277
[15:21:32.316 INFO] Deleting log files older than 7 days, if any...
[15:21:32.317 INFO] Deleting log files older than 7 days, if any...done
[15:21:32.388 INFO] Successfully added profile: testing
Now I start the sync with the following:
python ~/Source/ASynK/asynk.py --op=sync --name=testing
And what comes next:
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...done
Applying user customizations from file /home/jenkins/.asynk/config.py...
Applying user customizations from file /home/jenkins/.asynk/config.py...done
[15:25:10.787 INFO] Debug logging to file: /home/jenkins/.asynk/logs/asynk_logs.2017-03-12.15-25-10.787661
[15:25:10.788 INFO] Parsing BBDB file /home/jenkins/.bbdb...
[15:25:10.788 INFO] Parsing BBDB Store with encoding utf-8...
[15:25:11.143 INFO] Parsing BBDB Store with encoding utf-8...Success
[15:25:11.143 INFO] Successfully parsed 1018 entries.
Please enter username for cd2: testuser
Enter password for cd2: testpass
[15:25:17.939 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser...
Found current principal path: /remote.php/carddav/principals/testuser/
[15:25:49.224 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser... done.
[15:25:49.224 INFO] Deleting log files older than 7 days, if any...
[15:25:49.225 INFO] Deleting log files older than 7 days, if any...done
[15:25:49.225 INFO] Parsing BBDB file /home/jenkins/.bbdb...
[15:25:49.225 INFO] Parsing BBDB Store with encoding utf-8...
[15:25:49.595 INFO] Parsing BBDB Store with encoding utf-8...Success
[15:25:49.595 INFO] Successfully parsed 1018 entries.
Please enter username for cd2: testuser
Enter password for cd2: testpass
[15:25:55.372 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser...
Found current principal path: /remote.php/carddav/principals/testuser/
[15:26:04.182 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser... done.
[15:26:04.182 INFO] Deleting BBDB backup files older than 10 days, if any...
[15:26:04.235 INFO] Deleting BBDB backup files older than 10 days, if any...done
[15:26:04.235 INFO] Backedup BBDB Store (/home/jenkins/.bbdb) to file: /home/jenkins/.asynk/backups/bbdb_backup.testing.2017-03-12.15-26-04.235715
[15:26:04.236 INFO] Last synk for profile testing was at: 1980-01-01T00:00:00.00+00:00
[15:26:08.680 INFO] ==== bb =====
[15:26:08.680 INFO] New : 1018
[15:26:08.680 INFO] Modified : 0
[15:26:08.680 INFO] Unchanged : 0
[15:26:08.681 INFO] =====
[15:26:08.681 INFO] Total Entries : 1018
[15:26:08.681 INFO] Deleted : 0
[15:26:08.681 INFO] ==== cd =====
[15:26:08.681 INFO] New : 0
[15:26:08.681 INFO] Modified : 0
[15:26:08.681 INFO] Unchanged : 0
[15:26:08.681 INFO] =====
[15:26:08.681 INFO] Total Entries : 0
[15:26:08.681 INFO] Deleted : 0
[15:26:08.681 INFO] Number of entries modified both places (conflicts): 0
[15:26:08.681 INFO] conflict resolve direction : 1. db1id: bb, db2id: cd
[15:26:08.682 INFO] After conflict resolution, size of bb mod : 0
[15:26:08.682 INFO] After conflict resolution, size of cd mod : 0
[15:26:08.682 INFO] After conflict resolution, size of bb del : 0
[15:26:08.682 INFO] After conflict resolution, size of cd del : 0
[15:26:08.682 INFO] =====================================================
[15:26:08.682 INFO] Sending New bb entries to cd
[15:26:08.682 INFO] =====================================================
[15:26:08.682 INFO] 1018 new entries to be synched.
[15:26:13.390 INFO] Successfully created CardDAV entry for Grzegor Wegener (a733ac30116e82ce3ce8ffdd56fb26b1)
..... [a lot of entries created - this takes quite a while].
[16:33:08.788 INFO] Successfully created CardDAV entry for Gimnazjum i Liceum im. św. Tomasza z Akwinu (fd661ac154fe70da250d7ee291f5f632)
[16:33:08.789 INFO] Saving BBDB File /home/jenkins/.bbdb...
[16:33:08.890 INFO] Saving BBDB File /home/jenkins/.bbdb...done
[16:33:08.890 INFO] =====================================================
[16:33:08.890 INFO] Sending Modified bb entries to cd
[16:33:08.890 INFO] =====================================================
[16:33:08.890 INFO] No modified entries that need to be synched
[16:33:08.890 INFO] =====================================================
[16:33:08.890 INFO] Synching Deleted bb entries to cd
[16:33:08.890 INFO] =====================================================
[16:33:08.891 INFO] No deleted entries that need to be synched.
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] Sending New cd entries to bb
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] No new entries that need to be synched
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] Sending Modified cd entries to bb
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] No modified entries that need to be synched
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] Synching Deleted cd entries to bb
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] No deleted entries that need to be synched.
[16:33:09.069 INFO] Updating item inventory...
[16:33:09.167 INFO] Updating item inventory...done
So it seems to work well.
However then I do the sync again [just after doing this]:
jenkins@T450s% python ~/Source/ASynK/asynk.py --op=sync --name=testing [~]
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...done
Applying user customizations from file /home/jenkins/.asynk/config.py...
Applying user customizations from file /home/jenkins/.asynk/config.py...done
[16:43:42.766 INFO] Debug logging to file: /home/jenkins/.asynk/logs/asynk_logs.2017-03-12.16-43-42.766572
[16:43:42.767 INFO] Parsing BBDB file /home/jenkins/.bbdb...
[16:43:42.767 INFO] Parsing BBDB Store with encoding utf-8...
[16:43:43.048 INFO] Parsing BBDB Store with encoding utf-8...Success
[16:43:43.048 INFO] Successfully parsed 1018 entries.
Please enter username for cd2: testuser
Enter password for cd2: testpass
[16:43:49.469 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser...
Found current principal path: /remote.php/carddav/principals/testuser/
[16:43:55.442 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser... done.
[16:43:55.442 INFO] Deleting log files older than 7 days, if any...
[16:43:55.442 INFO] Deleting log files older than 7 days, if any...done
[16:43:55.442 INFO] Parsing BBDB file /home/jenkins/.bbdb...
[16:43:55.443 INFO] Parsing BBDB Store with encoding utf-8...
[16:43:55.729 INFO] Parsing BBDB Store with encoding utf-8...Success
[16:43:55.730 INFO] Successfully parsed 1018 entries.
Please enter username for cd2: testuser
Enter password for cd2: testpass
[16:44:00.554 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser...
Found current principal path: /remote.php/carddav/principals/testuser/
[16:44:05.064 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser... done.
[16:44:05.065 INFO] Deleting BBDB backup files older than 10 days, if any...
[16:44:05.065 INFO] Deleting BBDB backup files older than 10 days, if any...done
[16:44:05.065 INFO] Backedup BBDB Store (/home/jenkins/.bbdb) to file: /home/jenkins/.asynk/backups/bbdb_backup.testing.2017-03-12.16-44-05.065909
[16:44:05.066 INFO] Last synk for profile testing was at: 2017-03-12T14:33:08.96Z
and it hangs for quite awhile at this point.
I will write a follow up message when it resolves whatever it is doing.
JJ +
In die Sat, Mar 11 2017, Sriram Karra scripsit:
> >> >>> an email to asynk+un...@googlegroups.com.
My bbdb is clean, with no id's or extraneous fields. I can send you a
copy of it if this would help.
I made a new account on the carddav server: "testing"
I put a copy into the 'default place' "~/.bbdb" then did the following
to create a new profile. What follows is the response.
python ~/Source/ASynK/asynk.py --op=create-profile --name=testing --db bb cd --store ~/.bbdb http://cdte
st.sspx.ph/remote.php/carddav/ --folder default default
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...done
Applying user customizations from file /home/jenkins/.asynk/config.py...
Applying user customizations from file /home/jenkins/.asynk/config.py...done
[15:21:32.315 INFO] Debug logging to file: /home/jenkins/.asynk/logs/asynk_logs.2017-03-12.15-21-32.315277
[15:21:32.316 INFO] Deleting log files older than 7 days, if any...
[15:21:32.317 INFO] Deleting log files older than 7 days, if any...done
[15:21:32.388 INFO] Successfully added profile: testing
Now I start the sync with the following:
python ~/Source/ASynK/asynk.py --op=sync --name=testing
And what comes next:
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...done
Applying user customizations from file /home/jenkins/.asynk/config.py...
Applying user customizations from file /home/jenkins/.asynk/config.py...done
[15:25:10.787 INFO] Debug logging to file: /home/jenkins/.asynk/logs/asynk_logs.2017-03-12.15-25-10.787661
[15:25:10.788 INFO] Parsing BBDB file /home/jenkins/.bbdb...
[15:25:10.788 INFO] Parsing BBDB Store with encoding utf-8...
[15:25:11.143 INFO] Parsing BBDB Store with encoding utf-8...Success
[15:25:11.143 INFO] Successfully parsed 1018 entries.
Please enter username for cd2: testuser
Enter password for cd2: testpass
[15:25:17.939 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser...
Found current principal path: /remote.php/carddav/principals/testuser/
[15:25:49.224 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser... done.
[15:25:49.224 INFO] Deleting log files older than 7 days, if any...
[15:25:49.225 INFO] Deleting log files older than 7 days, if any...done
[15:25:49.225 INFO] Parsing BBDB file /home/jenkins/.bbdb...
[15:25:49.225 INFO] Parsing BBDB Store with encoding utf-8...
[15:25:49.595 INFO] Parsing BBDB Store with encoding utf-8...Success
[15:25:49.595 INFO] Successfully parsed 1018 entries.
Please enter username for cd2: testuser
Enter password for cd2: testpass
[15:25:55.372 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser...
Found current principal path: /remote.php/carddav/principals/testuser/
[15:26:04.182 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser... done.
[15:26:04.182 INFO] Deleting BBDB backup files older than 10 days, if any...
[15:26:04.235 INFO] Deleting BBDB backup files older than 10 days, if any...done
[15:26:04.235 INFO] Backedup BBDB Store (/home/jenkins/.bbdb) to file: /home/jenkins/.asynk/backups/bbdb_backup.testing.2017-03-12.15-26-04.235715
[15:26:04.236 INFO] Last synk for profile testing was at: 1980-01-01T00:00:00.00+00:00
[15:26:08.680 INFO] ==== bb =====
[15:26:08.680 INFO] New : 1018
[15:26:08.680 INFO] Modified : 0
[15:26:08.680 INFO] Unchanged : 0
[15:26:08.681 INFO] =====
[15:26:08.681 INFO] Total Entries : 1018
[15:26:08.681 INFO] Deleted : 0
[15:26:08.681 INFO] ==== cd =====
[15:26:08.681 INFO] New : 0
[15:26:08.681 INFO] Modified : 0
[15:26:08.681 INFO] Unchanged : 0
[15:26:08.681 INFO] =====
[15:26:08.681 INFO] Total Entries : 0
[15:26:08.681 INFO] Deleted : 0
[15:26:08.681 INFO] Number of entries modified both places (conflicts): 0
[15:26:08.681 INFO] conflict resolve direction : 1. db1id: bb, db2id: cd
[15:26:08.682 INFO] After conflict resolution, size of bb mod : 0
[15:26:08.682 INFO] After conflict resolution, size of cd mod : 0
[15:26:08.682 INFO] After conflict resolution, size of bb del : 0
[15:26:08.682 INFO] After conflict resolution, size of cd del : 0
[15:26:08.682 INFO] =====================================================
[15:26:08.682 INFO] Sending New bb entries to cd
[15:26:08.682 INFO] =====================================================
[15:26:08.682 INFO] 1018 new entries to be synched.
[15:26:13.390 INFO] Successfully created CardDAV entry for Grzegor Wegener (a733ac30116e82ce3ce8ffdd56fb26b1)
..... [a lot of entries created - this takes quite a while].
[16:33:08.788 INFO] Successfully created CardDAV entry for Gimnazjum i Liceum im. św. Tomasza z Akwinu (fd661ac154fe70da250d7ee291f5f632)
[16:33:08.789 INFO] Saving BBDB File /home/jenkins/.bbdb...
[16:33:08.890 INFO] Saving BBDB File /home/jenkins/.bbdb...done
[16:33:08.890 INFO] =====================================================
[16:33:08.890 INFO] Sending Modified bb entries to cd
[16:33:08.890 INFO] =====================================================
[16:33:08.890 INFO] No modified entries that need to be synched
[16:33:08.890 INFO] =====================================================
[16:33:08.890 INFO] Synching Deleted bb entries to cd
[16:33:08.890 INFO] =====================================================
[16:33:08.891 INFO] No deleted entries that need to be synched.
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] Sending New cd entries to bb
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] No new entries that need to be synched
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] Sending Modified cd entries to bb
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] No modified entries that need to be synched
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] Synching Deleted cd entries to bb
[16:33:08.891 INFO] =====================================================
[16:33:08.891 INFO] No deleted entries that need to be synched.
[16:33:09.069 INFO] Updating item inventory...
[16:33:09.167 INFO] Updating item inventory...done
So it seems to work well.
However then I do the sync again [just after doing this]:
jenkins@T450s% python ~/Source/ASynK/asynk.py --op=sync --name=testing [~]
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...
Applying base config from file /home/jenkins/Source/ASynK/config/config_v8.json...done
Applying user customizations from file /home/jenkins/.asynk/config.py...
Applying user customizations from file /home/jenkins/.asynk/config.py...done
[16:43:42.766 INFO] Debug logging to file: /home/jenkins/.asynk/logs/asynk_logs.2017-03-12.16-43-42.766572
[16:43:42.767 INFO] Parsing BBDB file /home/jenkins/.bbdb...
[16:43:42.767 INFO] Parsing BBDB Store with encoding utf-8...
[16:43:43.048 INFO] Parsing BBDB Store with encoding utf-8...Success
[16:43:43.048 INFO] Successfully parsed 1018 entries.
Please enter username for cd2: testuser
Enter password for cd2: testpass
[16:43:49.469 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser...
Found current principal path: /remote.php/carddav/principals/testuser/
[16:43:55.442 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser... done.
[16:43:55.442 INFO] Deleting log files older than 7 days, if any...
[16:43:55.442 INFO] Deleting log files older than 7 days, if any...done
[16:43:55.442 INFO] Parsing BBDB file /home/jenkins/.bbdb...
[16:43:55.443 INFO] Parsing BBDB Store with encoding utf-8...
[16:43:55.729 INFO] Parsing BBDB Store with encoding utf-8...Success
[16:43:55.730 INFO] Successfully parsed 1018 entries.
Please enter username for cd2: testuser
Enter password for cd2: testpass
[16:44:00.554 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser...
Found current principal path: /remote.php/carddav/principals/testuser/
[16:44:05.064 INFO] Logging into CardDAV server: http://cdtest.sspx.ph/remote.php/carddav/ - user: testuser... done.
[16:44:05.065 INFO] Deleting BBDB backup files older than 10 days, if any...
[16:44:05.065 INFO] Deleting BBDB backup files older than 10 days, if any...done
[16:44:05.065 INFO] Backedup BBDB Store (/home/jenkins/.bbdb) to file: /home/jenkins/.asynk/backups/bbdb_backup.testing.2017-03-12.16-44-05.065909
[16:44:05.066 INFO] Last synk for profile testing was at: 2017-03-12T14:33:08.96Z
and it hangs for quite awhile at this point.
I will write a follow up message when it resolves whatever it is doing.
JJ +
In die Sat, Mar 11 2017, Sriram Karra scripsit:
> >> >>> For more options, visit https://groups.google.com/d/optout.
> >> >>>
> >> >>
> >> >>
> >> >
> >>
> >
> > --
> > Find ASynK useful? Consider leaving a tip at this Bitcoin address:
> > 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
> > --- You received this message because you are subscribed to the Google
> > Groups "ASynK Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to asynk+un...@googlegroups.com.
> >> >>>
> >> >>
> >> >>
> >> >
> >>
> >
> > --
> > Find ASynK useful? Consider leaving a tip at this Bitcoin address:
> > 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
> > --- You received this message because you are subscribed to the Google
> > Groups "ASynK Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
Sriram Karra
Mar 13, 2017, 2:17:00 AM3/13/17
to as...@googlegroups.com
Let me try to recreate the problem with a larger numbers of contacts.
Just FYI - The CardDAV standard does not have batch operations, so creates and updates have to go across the network one contact at a time. Contrast this with the Google Contacts API that has batch operations and I can sync 100 contacts in a single op api call. It's super fast as a result...
> >> >>> an email to asynk+unsubscribe@googlegroups.com.
> >> >>> For more options, visit https://groups.google.com/d/optout.
> >> >>>
> >> >>
> >> >>
> >> >
> >>
> >
> > --
> > Find ASynK useful? Consider leaving a tip at this Bitcoin address:
> > 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
> > --- You received this message because you are subscribed to the Google
> > Groups "ASynK Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to asynk+unsubscribe@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> >
--
Find ASynK useful? Consider leaving a tip at this Bitcoin address: 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
---
You received this message because you are subscribed to the Google Groups "ASynK Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to asynk+unsubscribe@googlegroups.com.
Fr. John Jenkins
Mar 13, 2017, 2:34:38 AM3/13/17
to as...@googlegroups.com
Sriram,
After leaving the computer on all night to finish the script, it never
finished. It was stopped forever at the line:
[16:44:05.066 INFO] Last synk for profile testing was at: 2017-03-12T14:33:08.96Z
So this morning I stopped it with C-c and here was the output:
^CTraceback (most recent call last):
File "/home/jenkins/Source/ASynK/asynk.py", line 325, in <module>
main()
File "/home/jenkins/Source/ASynK/asynk.py", line 322, in main
asynk.dispatch()
File "/home/jenkins/Source/ASynK/asynk/asynk_core.py", line 97, in dispatch
res = getattr(self, self.get_op())()
File "/home/jenkins/Source/ASynK/asynk/asynk_core.py", line 401, in op_sync
result = sync.sync(self.get_sync_dir())
File "/home/jenkins/Source/ASynK/asynk/sync.py", line 295, in sync
sl1, sl2 = self.prep_lists(dirn)
File "/home/jenkins/Source/ASynK/asynk/sync.py", line 283, in prep_lists
return self._prep_lists_2_way(self.get_f1(), self.get_f2())
File "/home/jenkins/Source/ASynK/asynk/sync.py", line 179, in _prep_lists_2_way
f2.prep_sync_lists(f1.get_dbid(), f2sl)
File "/home/jenkins/Source/ASynK/asynk/folder_cd.py", line 66, in prep_sync_lists
curi = self.get_itemids(pname, destid)
File "/home/jenkins/Source/ASynK/asynk/folder_cd.py", line 119, in get_itemids
self._refresh_contacts()
File "/home/jenkins/Source/ASynK/asynk/folder_cd.py", line 308, in _refresh_contacts
items = sess.getPropertiesOnHierarchy(path, props)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/client/clientsession.py", line 226, in getPropertiesOnHierarchy
self.runSession(request)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/client/clientsession.py", line 817, in runSession
self.doSession(request)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/client/clientsession.py", line 865, in doSession
self.sendRequest(request)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/protocol/http/session.py", line 81, in sendRequest
self.doRequest(request)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/client/clientsession.py", line 950, in doRequest
self.readResponseData(request, response)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/client/clientsession.py", line 1004, in readResponseData
data = response.read()
File "/usr/lib/python2.7/httplib.py", line 588, in read
return self._read_chunked(amt)
File "/usr/lib/python2.7/httplib.py", line 646, in _read_chunked
value.append(self._safe_read(chunk_left))
File "/usr/lib/python2.7/httplib.py", line 703, in _safe_read
chunk = self.fp.read(min(amt, MAXAMOUNT))
File "/usr/lib/python2.7/socket.py", line 384, in read
data = self._sock.recv(left)
KeyboardInterrupt
In die Mon, Mar 13 2017, Sriram Karra scripsit:
> Let me try to recreate the problem with a larger numbers of contacts.
> Just FYI - The CardDAV standard does not have batch operations, so creates
> and updates have to go across the network one contact at a time. Contrast
> this with the Google Contacts API that has batch operations and I can sync
> 100 contacts in a single op api call. It's super fast as a result...
> On Sun, Mar 12, 2017 at 8:19 PM, Fr. John Jenkins <jen...@sspx.org.za>
> wrote:
> > Dear Sriram,
> >
> > My bbdb is clean, with no id's or extraneous fields. I can send you a
> > copy of it if this would help.
> >
> > I made a new account on the carddav server: "testing"
> >
> > I put a copy into the 'default place' "~/.bbdb" then did the following
> > to create a new profile. What follows is the response.
> >
> > python ~/Source/ASynK/asynk.py --op=create-profile --name=testing --db bb
> > cd --store ~/.bbdb http://cdte
> > st.sspx.ph/remote.php/carddav/ --folder default default
> > Applying base config from file /home/jenkins/Source/ASynK/
> > config/config_v8.json...
> > Applying base config from file /home/jenkins/Source/ASynK/
> > config/config_v8.json...done
> > > >> >>> an email to asynk+un...@googlegroups.com.
After leaving the computer on all night to finish the script, it never
finished. It was stopped forever at the line:
[16:44:05.066 INFO] Last synk for profile testing was at: 2017-03-12T14:33:08.96Z
^CTraceback (most recent call last):
File "/home/jenkins/Source/ASynK/asynk.py", line 325, in <module>
main()
File "/home/jenkins/Source/ASynK/asynk.py", line 322, in main
asynk.dispatch()
File "/home/jenkins/Source/ASynK/asynk/asynk_core.py", line 97, in dispatch
res = getattr(self, self.get_op())()
File "/home/jenkins/Source/ASynK/asynk/asynk_core.py", line 401, in op_sync
result = sync.sync(self.get_sync_dir())
File "/home/jenkins/Source/ASynK/asynk/sync.py", line 295, in sync
sl1, sl2 = self.prep_lists(dirn)
File "/home/jenkins/Source/ASynK/asynk/sync.py", line 283, in prep_lists
return self._prep_lists_2_way(self.get_f1(), self.get_f2())
File "/home/jenkins/Source/ASynK/asynk/sync.py", line 179, in _prep_lists_2_way
f2.prep_sync_lists(f1.get_dbid(), f2sl)
File "/home/jenkins/Source/ASynK/asynk/folder_cd.py", line 66, in prep_sync_lists
curi = self.get_itemids(pname, destid)
File "/home/jenkins/Source/ASynK/asynk/folder_cd.py", line 119, in get_itemids
self._refresh_contacts()
File "/home/jenkins/Source/ASynK/asynk/folder_cd.py", line 308, in _refresh_contacts
items = sess.getPropertiesOnHierarchy(path, props)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/client/clientsession.py", line 226, in getPropertiesOnHierarchy
self.runSession(request)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/client/clientsession.py", line 817, in runSession
self.doSession(request)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/client/clientsession.py", line 865, in doSession
self.sendRequest(request)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/protocol/http/session.py", line 81, in sendRequest
self.doRequest(request)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/client/clientsession.py", line 950, in doRequest
self.readResponseData(request, response)
File "/home/jenkins/Source/ASynK/lib/caldavclientlibrary/client/clientsession.py", line 1004, in readResponseData
data = response.read()
File "/usr/lib/python2.7/httplib.py", line 588, in read
return self._read_chunked(amt)
File "/usr/lib/python2.7/httplib.py", line 646, in _read_chunked
value.append(self._safe_read(chunk_left))
File "/usr/lib/python2.7/httplib.py", line 703, in _safe_read
chunk = self.fp.read(min(amt, MAXAMOUNT))
File "/usr/lib/python2.7/socket.py", line 384, in read
data = self._sock.recv(left)
KeyboardInterrupt
In die Mon, Mar 13 2017, Sriram Karra scripsit:
> Let me try to recreate the problem with a larger numbers of contacts.
> Just FYI - The CardDAV standard does not have batch operations, so creates
> and updates have to go across the network one contact at a time. Contrast
> this with the Google Contacts API that has batch operations and I can sync
> 100 contacts in a single op api call. It's super fast as a result...
> On Sun, Mar 12, 2017 at 8:19 PM, Fr. John Jenkins <jen...@sspx.org.za>
> wrote:
> > Dear Sriram,
> >
> > My bbdb is clean, with no id's or extraneous fields. I can send you a
> > copy of it if this would help.
> >
> > I made a new account on the carddav server: "testing"
> >
> > I put a copy into the 'default place' "~/.bbdb" then did the following
> > to create a new profile. What follows is the response.
> >
> > python ~/Source/ASynK/asynk.py --op=create-profile --name=testing --db bb
> > cd --store ~/.bbdb http://cdte
> > st.sspx.ph/remote.php/carddav/ --folder default default
> > Applying base config from file /home/jenkins/Source/ASynK/
> > config/config_v8.json...
> > Applying base config from file /home/jenkins/Source/ASynK/
> > Applying user customizations from file /home/jenkins/.asynk/config.py...
> > Applying user customizations from file /home/jenkins/.asynk/config.
> > py...done
> > [15:21:32.315 INFO] Debug logging to file: /home/jenkins/.asynk/logs/
> > asynk_logs.2017-03-12.15-21-32.315277
> > [15:21:32.316 INFO] Deleting log files older than 7 days, if any...
> > [15:21:32.317 INFO] Deleting log files older than 7 days, if any...done
> > [15:21:32.388 INFO] Successfully added profile: testing
> >
> > Now I start the sync with the following:
> >
> > python ~/Source/ASynK/asynk.py --op=sync --name=testing
> >
> > And what comes next:
> > Applying base config from file /home/jenkins/Source/ASynK/
> > config/config_v8.json...
> > Applying user customizations from file /home/jenkins/.asynk/config.
> > py...done
> > [15:21:32.315 INFO] Debug logging to file: /home/jenkins/.asynk/logs/
> > asynk_logs.2017-03-12.15-21-32.315277
> > [15:21:32.316 INFO] Deleting log files older than 7 days, if any...
> > [15:21:32.317 INFO] Deleting log files older than 7 days, if any...done
> > [15:21:32.388 INFO] Successfully added profile: testing
> >
> > Now I start the sync with the following:
> >
> > python ~/Source/ASynK/asynk.py --op=sync --name=testing
> >
> > And what comes next:
> > Applying base config from file /home/jenkins/Source/ASynK/
> > config/config_v8.json...
> > Applying base config from file /home/jenkins/Source/ASynK/
> > config/config_v8.json...done
> > Applying user customizations from file /home/jenkins/.asynk/config.py...
> > Applying user customizations from file /home/jenkins/.asynk/config.
> > py...done
> > Applying user customizations from file /home/jenkins/.asynk/config.
> > py...done
> > Applying base config from file /home/jenkins/Source/ASynK/
> > config/config_v8.json...done
> > Applying user customizations from file /home/jenkins/.asynk/config.py...
> > Applying user customizations from file /home/jenkins/.asynk/config.
> > py...done
> > Applying user customizations from file /home/jenkins/.asynk/config.
> > py...done
> > > >> >>> For more options, visit https://groups.google.com/d/optout.
> > > >> >>>
> > > >> >>
> > > >> >>
> > > >> >
> > > >>
> > > >
> > > > --
> > > > Find ASynK useful? Consider leaving a tip at this Bitcoin address:
> > > > 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
> > > > --- You received this message because you are subscribed to the Google
> > > > Groups "ASynK Users" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > an
> > > > email to asynk+un...@googlegroups.com.
> > > >> >>>
> > > >> >>
> > > >> >>
> > > >> >
> > > >>
> > > >
> > > > --
> > > > Find ASynK useful? Consider leaving a tip at this Bitcoin address:
> > > > 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
> > > > --- You received this message because you are subscribed to the Google
> > > > Groups "ASynK Users" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > an
> > > > For more options, visit https://groups.google.com/d/optout.
> > > >
> >
> > --
> > Find ASynK useful? Consider leaving a tip at this Bitcoin address:
> > 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
> > ---
> > You received this message because you are subscribed to the Google Groups
> > "ASynK Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to asynk+un...@googlegroups.com.
> > > >
> >
> > --
> > Find ASynK useful? Consider leaving a tip at this Bitcoin address:
> > 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
> > ---
> > You received this message because you are subscribed to the Google Groups
> > "ASynK Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
Sriram Karra
Mar 14, 2017, 3:26:02 AM3/14/17
to as...@googlegroups.com
Thanks John. Not sure why it did not timeout. Can you send me the debug logs (in ~/.asynk/logs/) offlist? You could mask your pwd and contact names that will be printed. Note that the debug logs contain much more logs (DEBUG level) than what is shown on the console (INFO level).
> > > >> >>> an email to asynk+unsubscribe@googlegroups.com.
> > > >> >>> For more options, visit https://groups.google.com/d/optout.
> > > >> >>>
> > > >> >>
> > > >> >>
> > > >> >
> > > >>
> > > >
> > > > --
> > > > Find ASynK useful? Consider leaving a tip at this Bitcoin address:
> > > > 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
> > > > --- You received this message because you are subscribed to the Google
> > > > Groups "ASynK Users" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > an
> > > > email to asynk+unsubscribe@googlegroups.com.
> > > > For more options, visit https://groups.google.com/d/optout.
> > > >
> >
> > --
> > Find ASynK useful? Consider leaving a tip at this Bitcoin address:
> > 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
> > ---
> > You received this message because you are subscribed to the Google Groups
> > "ASynK Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to asynk+unsubscribe@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> >
--
Find ASynK useful? Consider leaving a tip at this Bitcoin address: 1KarraEGixRyScZ9qFHdekxNkgpPTuzWBc
---
You received this message because you are subscribed to the Google Groups "ASynK Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to asynk+unsubscribe@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages