CalDAV issue with sabre/dav v2.0.2 and Thunderbird

[ch7...@gmail.com]

I was using CalDAV for quite a while as calendar storage with Thunderbird/Lightning.
I have multiple clients from which one client runs continuously showing the actual calendar all the time.
I set the synchronisation interval in Lightning to 5 minutes, so every calendar entry shows up after at least 5 minutes on this machine.
This worked without any problems with sabre/dav v1.8.7.
Now I upgraded to sabre/dav v2.0.2 and the synchronisation is no longer working.
I need to restart Thunderbird to get new/modified calendar entries show up.
Right now I am using Thunderbird/24.6.0 Lightning/2.6.6
Does anybody know how to fix this issue?

[Evert Pot]

Hi!

Few questions:

1. Did the database upgrade script run succesfully?
2. Are you using custom backends, or customizations in backends?
3. Did you make any changes to your 'server.php' or 'calendarserver.php' after upgrading?
4. Are you using the WebDAV-Sync plugin?

Let me know,
Evert

[ch7...@gmail.com]

Hi Evert,

1. I think the database upgrade script finished sucessfully.
I did not have the addressboks table in my old database, so I created one before using the migration script.

server:/home/apache/htdocs/SabreDAV-2.0.2/bin # php migrateto20.php sqlite:calendars.db
SabreDAV migrate script for version 2.0
Connecting to database: sqlite:calendars.db
Detected SQLite.
Upgrading 'calendars'
1.8 table schema detected
Note: there's no easy way to remove fields in sqlite.
The ctag field is no longer used, but it's kept in place
Upgraded 'calendars' to 2.0 schema.
Creating 'calendarchanges' table.
Upgrading 'addressbooks'
No records were found in the 'addressbooks' table.

We're going to rename the old table to addressbooks_old (just in case).
and re-create the new table.
Creation of 2.0 addressbooks table is complete
Creating 'addressbookchanges' table.
Creating calendarsubscriptions table.
Creating propertystorage table.
Upgrading cards table to 2.0 schema
There was no data in the cards table, so we're re-creating it
The old table will be renamed to cards_old, just in case.
Upgrade to 2.0 schema completed.

Afterwards I deleted the cards_old and addressbooks_old table from the database.
I also removed the ctag field from the database as the script reported " it's no longer used" (manually)
The removing of the unused stuff did not make any difference, I tried it also with the database still containing the unused tables and the ctag field

2. I did not modify anything. All files are exactly like in sabredav-2.0.2.zip

3. I merged my old calendarserver.php with the one from version 2.0.2
There are some syntax changes: array() is now [ ],
the leading backslash is missing in the new command of V2.0.2
and the calendarserver.php from version 2.0.2 has a "Calendar subscription support plugin", which I added.
I have both files attached.
My actual calendarserver.php differs only in three lines from the one in the examples directory of V2.0.2:
timezone, database and $aclPlugin->adminPrincipals[] = 'principals/root';

4. I would say no, unless it comes with version 2.0.2 as a default.
It's not in my calendarserver.php.

Best Regards
Cajus

[Evert Pot]

Hey Cajus,

That's a really high amount of detail, thanks for that!
I have a hunch what the problem may be, and it would definitely be a
sabredav bug.

Would you mind adding the following lines to your calendarserver.php:

$server->addPlugin(
new \Sabre\DAV\Sync\Plugin()
);

If it starts working after that change, I know where to look for the
problem.

Using the sync plugin is a good idea regardless, as it will make
everything faster and more efficient, but sabredav should work without
that change.

Evert

> --
> You received this message because you are subscribed to the Google
> Groups "SabreDAV Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to sabredav-discu...@googlegroups.com
> <mailto:sabredav-discu...@googlegroups.com>.
> To post to this group, send email to sabredav...@googlegroups.com
> <mailto:sabredav...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/sabredav-discuss.
> For more options, visit https://groups.google.com/d/optout.

[ch7...@gmail.com]

Hey Evert,

I tried it with or without the sync plugin, but there is no difference.

I saw, that you asked for adding

$server->addPlugin(
  new \Sabre\DAV\Sync\Plugin()
);

which has the "\" prefix in the new line.
One of the changes between the the calendarserver.php from version 1.8.7 and 2.0.2 is, that there are no prefixes in the version 2.0.2 any more.
Could this be an issue?
By the way, in case of the sync plugin, I tried it with and without the backslash, which doesn't make a difference ether.

Cajus

> <mailto:sabredav-discuss+unsub...@googlegroups.com>.

[Evert Pot]

Hi!

On Saturday, July 12, 2014 2:03:30 AM UTC-4, ch7...@gmail.com wrote:

Hey Evert,

I tried it with or without the sync plugin, but there is no difference.

I saw, that you asked for adding
$server->addPlugin(
  new \Sabre\DAV\Sync\Plugin()
);
which has the "\" prefix in the new line.
One of the changes between the the calendarserver.php from version 1.8.7 and 2.0.2 is, that there are no prefixes in the version 2.0.2 any more.
Could this be an issue?
By the way, in case of the sync plugin, I tried it with and without the backslash, which doesn't make a difference ether.

The initial \ is optional, and has the same effect, *if* the file does not start with 'namespace' somewhere at the top.
I usually add the \ for good measure.

So if that didn't fix your issue, it means we have some other problem.

So I will set up my thunderbird and lightning and see if I can reproduce the issue.

The problem description I have from you is:

1. A calendar is set up in lightning
2. Some other client modifies/deletes/creates an event
3. The event does not show up after a few minutes, but only after lightning restarts

Is that accurate?

Evert

[ch7...@gmail.com]

Hi Evert,

yes, your description is correct.
I would say
3. The event does not show up after a few minutes and not when the synchronize button is pressed, but only after thunderbird & lightning restarts
(pressing the synchronize button is faster than waiting ;-)
I don't think it makes a difference, but in fact I have 6 calendars hosted in SabreDAV, which are all showed by the clients.
(one calendar for each family member)

Cajus

[ch7...@gmail.com]

Hey Evert,

could you reproduce the issue?
There is no change even with the latest version of sabreDAV (2.0.3) and the latest version of Thunderbird (31.0) / Lightning (3.3)

Best Regards

Cajus

Am Montag, 14. Juli 2014 21:48:18 UTC+2 schrieb Evert Pot:

[Evert Pot]

Hi!

I haven't looked yet, but I opened a ticket here:

https://github.com/fruux/sabre-dav/issues/497

I'll do my best to look into it before the next release.

Evert

[ch7...@gmail.com]

Hi Evert,

as the bug is still open, I expect you did not look into it in the meantime!?
I saw a new version 2.0.5 and wanted to try it, but I am missing the "vendor" directory.
It vanished from everywhere, but calendarserver.php references it

// Files we need                                                                                                                                                    
require_once 'vendor/autoload.php';                                                                                                                                 

Did I miss something?

Cajus

[ch7...@gmail.com]

Forget about the missing "verdor" directory, I picked the source.zip....
Cajus

[ch7...@gmail.com]

Now I tried 2.0.5, but the bug is still there :-(

Cajus

[Evert Pot]

Hi!

So I just tried this out. Set up my lightning using some caldav calendar, made a change using iCal and waited.
After a while the event just appeared on lightning as well, so I'm having some issues replicating this issue :S

Evert

[ch7...@gmail.com]

Hi Evert,

I was afraid about this answer.
As I normally only have Thunderbird/Lightning as calendar apps, I looked for another Linux app being able to work with CalDAV.
I found KOrganizer and set up my calendars there.

When I add a calendar entry to Lightning, it will show up in KOrganizer on refresh.
When I add a calendar entry to KOrganizer, it will NOT show up in Lightning on refresh.
I checked the entry with the web-interface of caldav, it is there!
It will not show up in lightning unless I restart thunderbird :-(

I still have CalDAV 1.8.7 on my web server.
I switched back to 1.8.7, everything works like expected.
Back to 2.0.5 -> no refresh in Lightning any more!

Do you have any idea how to find the problem?

Cajus

[ch7...@gmail.com]

Hi Evert,

I tried to find out a little more about the difference between caldav-1.8.7 and caldav-2.0.5
I created empty databases for both versions.
From Thunderbird/Lightning on a Windows-Box I entered the calendar entries
and on Thunderbird/Lightning on my Linux machine I clicked the "synchronize" button and saw what's happening.
I let wireshark monitor the trafic.
When I enter the first calendar entry to an empty calendar, the"synchronize" works with both caldav versions.
But with the second calendar entry (and all following) caldav-2.0.5 refuses to synchronize.

I am not familiar with the protocol, which is used to exchange data and status information,
but I saw differences between caldav-1.8.7 and caldav-2.0.5.

I see:
TB/Lightning sends HTTP/XML PROPFIND to the caldav server
The caldav server answers with a HTTP/XML Multi-Status message
With caldav-1.8.7 I see in the XML-part of the  Status-Message <cs:getctag> 3 </cs:getctag>,
while the "3" counts up with each new calendar entry.
With caldav-2.0.5 I see in the XML-part of the  Status-Message <cs:getctag> http://sabre.io/ns/sync/0 </cs:getctag>
this value never changes, whether there are new entries in the calendar or not!
Trying to access http://sabre.io/ns/sync/0 results in "not found"
Could this be the reason for the missing sync feature?

Would it help, if I send the *. pcapng files to you? (I would prefer to send it as private email, not posting them here)

Best Regards
Cajus

Am Montag, 27. Oktober 2014 22:39:12 UTC+1 schrieb Evert Pot:

[Evert Pot]

Hi Cajus,

I just tested this again on the 2.0 branch. I'm running:

1. Latest sabre/dav 2.0
2. Thunderbird 31.2.0
3. Lightning 3.3.1

I have the SabreDAV 'sync' plugin enabled.

In my case the ctag never even gets requested. Because the server has sync-token support, lightning uses that instead of the ctag.
Creating a new event 'just works'. In my case the sync-token (and ctag) increases for every change.

Creating a new calendar sets the sync-token to 1 though, it doesn't start counting at 0.. so that's rather weird!

So two more questions:

1. Are you really sure that the sync plugin is turned on?
2. Could you share the contents of the relevant row in the 'calendars' table?

Evert

[ch7...@gmail.com]

Hi Evert,

finally I made it. Your last question was the answer.
Yes, the sync plugin is really turned on (in my calendarserver.php),
but my calendar table entries did not contain a value in the column "synctoken".
This might be the result of using a migrated database from CalDAV versions pre sync-support.
The migration script might add a value there.
For the creation of the test-database I also used a script, which was from the pre-sync-support CalDAV version.
I added a new calendar through the web interface and saw, that this row in the calendar table has a '1' in the synctoken column.
So I added some values to the synctoken columns of my normal calendars and everything 'just works'.
Thank you so much for your continuing help. You might close the corresponding bug.

Cajus

[Evert Pot]

Shit.. that is .. unfortunate.

I think I can do something in the migration script to initialize that column with a value if there was none.

So glad this was finally figured out though, and it also explains why I wasn't seeing this issue.