[CalDAV] update single recurrence element - howto workaround

[stefan.lendl]

Hi!

I would like to workaround to store the tree like structure the vcal-
standard has for updating single elements of a recurrence.

This could be done by just adding a new event for the updated
recurrence-element and adding an exception in the recurrence event.

The only problem I have with that is that after the call to
updateCalendarObject the CalDAV-Clients call only getCalendarObject
for the updated Object and then the edited single entry disapears
until a manual total reload.

Is it somehow possible to send back to the client some information
that I want a full reload or can I somehow send back the newly created
Entry with the other record on the next load?

Thanks,
Stefan

[Mr. Demeanour]

On 27/07/2011 15:49, stefan.lendl wrote:
> Hi!
>
> I would like to workaround to store the tree like structure the
> vcal- standard has for updating single elements of a recurrence.
>
> This could be done by just adding a new event for the updated
> recurrence-element and adding an exception in the recurrence event.

You shouldn't have to do both of these - it's one or the other (if I
understand correctly what you are saying).

>
> The only problem I have with that is that after the call to
> updateCalendarObject the CalDAV-Clients call only getCalendarObject
> for the updated Object and then the edited single entry disapears
> until a manual total reload.
>
> Is it somehow possible to send back to the client some information
> that I want a full reload or can I somehow send back the newly
> created Entry with the other record on the next load?

I believe the "right" way to do what you want is to store the
exceptional VEVENT in the same resource (VCALENDAR object) as the
original recurring event, with a RECURRENCE-ID to identify which
instance is to be overridden. Unless the client asks for
expand-recurrences, then the server should return the entire VCALENDAR
object, containing both the original recurring event and the exception.

And if the client *does* request expand-recurrences, then the question
is moot, I think.

Thunderbird/Lightning seems to do this.

You can also use an EXDATE property on the original recurring event; but
that just changes the date/time of some occurrence - you can't change
other properties like the subject using EXDATE.

--
MrD.

[Evert Pot]

>>
>> Is it somehow possible to send back to the client some information
>> that I want a full reload or can I somehow send back the newly
>> created Entry with the other record on the next load?
>
> I believe the "right" way to do what you want is to store the
> exceptional VEVENT in the same resource (VCALENDAR object) as the
> original recurring event, with a RECURRENCE-ID to identify which
> instance is to be overridden. Unless the client asks for
> expand-recurrences, then the server should return the entire VCALENDAR
> object, containing both the original recurring event and the exception.
>
> And if the client *does* request expand-recurrences, then the question
> is moot, I think.

Note that SabreDAV doesn't support expansion of recurrences yet.

>
> Thunderbird/Lightning seems to do this.
>
> You can also use an EXDATE property on the original recurring event; but
> that just changes the date/time of some occurrence - you can't change
> other properties like the subject using EXDATE.

I'm pretty sure you need both; An EXDATE to specify which instance of
the recurrence to exclude, and a new VEVENT object with a
Recurrence-ID matching that value from EXDATE exactly.

An easy way to test how it's done, is to fire up iCal and do it
through that client first. It will make it pretty obvious what the
correct way is.

Evert

[Stefan Lendl]

On 07/28/2011 01:17 AM, Evert Pot wrote:
> An easy way to test how it's done, is to fire up iCal and do it
> through that client first. It will make it pretty obvious what the
> correct way is.

Yes i did that. It doesn't do an EXDATE in that case but adds a new
Event with the same UID and some kind of foreign key (through the
recurrence id).

EXDATE is only used when deleting single instances.

My problem is, that the way it is done by iCal (and Thunderbird) is
quite a mess to implement with the backend (would have to introduce a
tree like structure). So I wanted to cheat and do it in another obvious
way: add an EXDATE and create a non-related Event.

That would work if I could somehow force the client to reload the
calendar (or at least send it the new event), but i had no luck with that.

[Evert Pot]

>
> My problem is, that the way it is done by iCal (and Thunderbird) is quite a
> mess to implement with the backend (would have to introduce a tree like
> structure). So I wanted to cheat and do it in another obvious way: add an
> EXDATE and create a non-related Event.

It may be worth trying to implement it the proper way =) Sure, it's a
tree-like structure, but only 1 level deep really..

>
> That would work if I could somehow force the client to reload the calendar
> (or at least send it the new event), but i had no luck with that.

iCal has 'push' support nowadays, but it works through xmpp, and I
have no idea about the details. Going through this effort will likely
be harder than just implement it the usual way though.

That's as far as I know the only way, otherwise you're going to have
to wait for the automatic refresh.

[Stefan Lendl]

On 07/28/2011 06:59 AM, Evert Pot wrote:
> It may be worth trying to implement it the proper way =) Sure, it's a
> tree-like structure, but only 1 level deep really..

Not really because it would be possible that someone edits one instance
and creates another recurrence event. And there again ....

But yeah you're right won't really have another option.

Thanks,
Stefan

[Mr. Demeanour]

On 28/07/2011 00:17, Evert Pot wrote:
>>>
>>> Is it somehow possible to send back to the client some information
>>> that I want a full reload or can I somehow send back the newly
>>> created Entry with the other record on the next load?
>>
>> I believe the "right" way to do what you want is to store the
>> exceptional VEVENT in the same resource (VCALENDAR object) as the
>> original recurring event, with a RECURRENCE-ID to identify which
>> instance is to be overridden. Unless the client asks for
>> expand-recurrences, then the server should return the entire VCALENDAR
>> object, containing both the original recurring event and the exception.
>>
>> And if the client *does* request expand-recurrences, then the question
>> is moot, I think.
>
> Note that SabreDAV doesn't support expansion of recurrences yet.
>
>>
>> Thunderbird/Lightning seems to do this.
>>
>> You can also use an EXDATE property on the original recurring event; but
>> that just changes the date/time of some occurrence - you can't change
>> other properties like the subject using EXDATE.
>
>
> I'm pretty sure you need both; An EXDATE to specify which instance of
> the recurrence to exclude, and a new VEVENT object with a
> Recurrence-ID matching that value from EXDATE exactly.

Ah. I need to do some (more) re-reading in that case.

>
> An easy way to test how it's done, is to fire up iCal and do it
> through that client first. It will make it pretty obvious what the
> correct way is.

Of course. But that assumes I'm an Apple weenie. In fact I have no Apple
equipment, and so I can't run iCal.

Isn't it dreadful that RFC 5545 is written so badly that it can only be
interpreted by reference to a specific piece of proprietary software.

--
MrD.

[Evert Pot]

>
> Of course. But that assumes I'm an Apple weenie. In fact I have no Apple
> equipment, and so I can't run iCal.

Sorry this was mostly intended for the OP.

[Mr. Demeanour]

On 28/07/2011 05:46, Stefan Lendl wrote:
> On 07/28/2011 01:17 AM, Evert Pot wrote:
>> An easy way to test how it's done, is to fire up iCal and do it
>> through that client first. It will make it pretty obvious what the
>> correct way is.
> Yes i did that. It doesn't do an EXDATE in that case but adds a new
> Event with the same UID and some kind of foreign key (through the
> recurrence id).
>
> EXDATE is only used when deleting single instances.

Oh yes, I believe that's correct; EXDATE is not an exception, it's an
*exclusion*.

--
MrD.

[stefan.lendl]

On Jul 28, 6:59 am, Stefan Lendl <st.le...@gmail.com> wrote:
> Not really because it would be possible that someone edits one instance
> and creates another recurrence event. And there again ....

I have to correct myself: It seems like the tree is forced by the
standard to have only 1 level (at least iCal and Thunderbird force it
that way).

So yes thx, i'll try to do the same things as iCal (even if this
change is painful again...)

[musonic]

I've been trying to think of a solution to a similar problem. I rewrote the backend so that whenever a client created or edited an event the server would parse the VCALENDAR or VEVENT object and put each individual property into it's own column in the db table. I thought that this would enable me to speed up searching and retrieving information to use in my web app and avoid having to interact with the server via HTTP each time whilst retaining that functionality so that users could still use their preferred client if they wished. 

However, I ran into difficulties when I started dealing with recurring events. I (mistankenly) thought, just like the OP, that an exception to a recurring event would be created as a new entry in the table. It became obvious when playing with iCal that this wasn't the case. 

So, my question is this... Am I wasting my time developing a backend that allows my app to communicate directly with the db rather than making HTTP requests (like any other client)? I don't know much about performance issues so I'm not sure if making lots of HTTP requests is going to slow everything down unacceptably. It just seems a bit silly to add this extra layer when, technically, my app is able to interact directly with the db. 

Finally, because expand-recurrences isn't yet supported, am I correct in saying that to retrieve and query recurring events I can use limit-recurrence-set and retrieve the full VCALENDAR and then parse it through the VObject/recurrenceIterator?

[Evert Pot]

Hi Musonic,

Please include the original email if you reply to the list. I use a
regular email client so it's hard to see what was discussed. Found it
in the archives though.

Since about a month, expand-recurrences is supported on the
'master' (future 1.6.0) branch.

As for interacting directly with the database.. For simple stuff I
don't think there's much wrong with that. However, if you are starting
to add additional business logic, you will after a while have to
implement this business logic on 2 layers.

In general I think it's bad design to treat your (simple) database as
the central point. There should be a layer in between that is shared.

Instead of HTTP requests though, you can interact with the SabreDAV
api directly. Instead of executing a real propfind request, you can
simply do:

$server->getProperties('principals/username', array('{DAV:}
displayname'));

Granted, that this side of the API is not as well documented, but for
many types of requests I'll be able to point you in the right
directions on how to construct the request.

You can actually also fake the http request in this, just construct a
'fake'

Sabre_HTTP_Request and
Sabre_HTTP_Response

You'll need to subclass Sabre_HTTP_Response so it doesn't
automatically send back everything to a browser, but this is
effectively how the unittests 'fake' all the requests.

This last method will allow you to do 'pretend' requests locally.

Up to you though, depending on your requirements you may feel it's not
worth the trouble :) The SabreDAV stack will likely be heavier than
your needs.
Evert

[musonic]

On Thursday, 9 February 2012 19:38:22 UTC, Evert Pot wrote:

Hi Musonic,

Please include the original email if you reply to the list. I use a
regular email client so it's hard to see what was discussed. Found it
in the archives though.

 Apologies for this. I didn't realise that when using the Google groups UI it didn't include the orignal thread by default.

Thanks for this. I've spent a lot of time digging around the library trying to figure out how to implement the API, but frankly it's proved a little beyond me! I've been able to use the example above, but I need to be able to dig much deeper. I tried, for example, using the getProperties method to try and get a calendarObject, but with no success. I can't figure out how the API accesses methods in the, for example, PDO class. I can see that the CalendarObject class calls these methods to interact with the db, but how do I use the API to access the CalendarObject or Calendar classes?  

You can actually also fake the http request in this, just construct a
'fake'

Sabre_HTTP_Request and
Sabre_HTTP_Response

You'll need to subclass Sabre_HTTP_Response so it doesn't
automatically send back everything to a browser, but this is
effectively how the unittests 'fake' all the requests.

This last method will allow you to do 'pretend' requests locally.

This sounds great, but I'm not sure I know exactly how to go about implementing it. It looks like I need to override the global _SERVER array when I instantiate the request class, but I'm really not very clear. 

Up to you though, depending on your requirements you may feel it's not
worth the trouble :) The SabreDAV stack will likely be heavier than
your needs.
Evert

I'm really keen to get my head around SabreDAV because my primary reason for using it rather than just rolling my own iCal-like data model to use with my local application is so that users can choose which client they want to access their information with.

I really appreciate all your help. 

[Evert Pot]

On Feb 10, 2012, at 6:19 PM, musonic wrote:

>
>
> On Thursday, 9 February 2012 19:38:22 UTC, Evert Pot wrote:
> Hi Musonic,
>
> Please include the original email if you reply to the list. I use a
> regular email client so it's hard to see what was discussed. Found it
> in the archives though.
>
> Apologies for this. I didn't realise that when using the Google groups UI it didn't include the orignal thread by default.
>

No worries, and sorry for the slow reply.. Had some family stuff to attend to.

The underlying design idea is that a 'user' of the API never interacts directly with the node objects directly.

You can access the CalendarObject directly, by calling:

$server->tree->getNodeForPath('calendars/[user]/[calendar]/[object]');

This will return a CalendarObject instance (or throw a NotFound exception).
However, if you do that, you circumvent some parts of the system that may be important. The ACL layer will be ignored, for instance.

But sometimes you need to, this part of the API isn't 100% feature complete.

So main access to the file tree is through the 'tree' object, the API for this can be found in Sabre_DAV_ObjectTree.

>
>
> You can actually also fake the http request in this, just construct a
> 'fake'
>
> Sabre_HTTP_Request and
> Sabre_HTTP_Response
>
> You'll need to subclass Sabre_HTTP_Response so it doesn't
> automatically send back everything to a browser, but this is
> effectively how the unittests 'fake' all the requests.
>
> This last method will allow you to do 'pretend' requests locally.
>
> This sounds great, but I'm not sure I know exactly how to go about implementing it. It looks like I need to override the global _SERVER array when I instantiate the request class, but I'm really not very clear.

You don't need to override it, you can just create a 'fake' one.
Here's an example of how you would create a 'fake' GET request:

// We're assuming you fully setup your '$server' object.

$serverArray = array(
'REQUEST_URI' => '/calendars/[user]/[calendar]/[calendarobject]',
'REQUEST_METHOD' => 'GET'
);

$request = new Sabre_HTTP_Request($serverArray);
$response = Sabre_HTTP_ResponseMock();

$server->httpRequest = $request;
$server->httpResponse = $response;

// Now to execute the request:
$server->exec();

After this, the full response can be found in your $response object.

Now the only missing bit is the Sabre_HTTP_ResponseMock class.
This isn't part of the main codebase, but can be found in:

tests/Sabre/HTTP
instead of:
lib/Sabre/HTTP

If you feel this is what you want, I can move the 'Mock' response class to the main codebase. Let me know if that's helpful. Otherwise you can just pull it from the tests directly, or easily create your own.

>
> Up to you though, depending on your requirements you may feel it's not
> worth the trouble :) The SabreDAV stack will likely be heavier than
> your needs.
> Evert
>
> I'm really keen to get my head around SabreDAV because my primary reason for using it rather than just rolling my own iCal-like data model to use with my local application is so that users can choose which client they want to access their information with.
> I really appreciate all your help.

Well I hope the explanation makes sense :)

Cheers,
Evert

[musonic]

Thanks so much for this, Evert. I think I'm beginning to get to grips with what I need to do to achieve what I'm aiming for. I'd actually already discovered how to access the object tree directly and realised that this was probably not the way to go!

So, I've moved the responseMock class into the main codebase and have played around with making "fake" requests. I've basically got it working, but have run into some problems with integrating the auth and ACL plugins. I already have an auth module for my app so I don't really feel I need/want to authorise users again when making a "local" request. However, if I simply disable the auth plugin and backend for these requests then the ACL won't work. Is there a way around this. I've tried sub-classing the auth PDO backend class and overriding the authenticate method and passing it the already logged in users username, but it still needs a password, and I don't want to be keeping passwords hanging around in sessions etc to pass between different areas of the app. Once the user is logged in we can keep info like username etc, but we shouldn't keep the password they typed in. 

If I was to basically disable the authenticate method for users that have already logged in, set the currentUser variable and return true from the method, would this a) work, and b) be secure?

Am I correct in saying that the ACL classes only need to find currentUser to be happy?

Thanks again for all your support - I don't know how you do it by yourself!!!!

[musonic]

One more thing... when I get the object returned from the mock response class, what's the best way to interpret the body resource?

[Evert Pot]

Hi!

On Feb 15, 7:40 pm, musonic <niclebreui...@gmail.com> wrote:
> Thanks so much for this, Evert. I think I'm beginning to get to grips with
> what I need to do to achieve what I'm aiming for. I'd actually already
> discovered how to access the object tree directly and realised that this
> was probably not the way to go!
> So, I've moved the responseMock class into the main codebase and have
> played around with making "fake" requests. I've basically got it working,
> but have run into some problems with integrating the auth and ACL plugins.
> I already have an auth module for my app so I don't really feel I need/want
> to authorise users again when making a "local" request. However, if I
> simply disable the auth plugin and backend for these requests then the ACL
> won't work. Is there a way around this. I've tried sub-classing the auth
> PDO backend class and overriding the authenticate method and passing it the
> already logged in users username, but it still needs a password, and I
> don't want to be keeping passwords hanging around in sessions etc to pass
> between different areas of the app. Once the user is logged in we can keep
> info like username etc, but we shouldn't keep the password they typed in.
> If I was to basically disable the authenticate method for users that have
> already logged in, set the currentUser variable and return true from the
> method, would this a) work, and b) be secure?
> Am I correct in saying that the ACL classes only need to find currentUser
> to be happy?
>
> Thanks again for all your support - I don't know how you do it by
> yourself!!!!

Thank you!

Instead of subclassing the PDO class, just create a new auth class
altogether.

Implement this interface:

https://github.com/evert/SabreDAV/blob/master/lib/Sabre/DAV/Auth/IBackend.php

And simply return 'true' in the authentication() method, and a correct
username for getUserName().

That's all you need.

Evert

[Evert Pot]

On Feb 15, 8:00 pm, musonic <niclebreui...@gmail.com> wrote:
> One more thing... when I get the object returned from the mock response
> class, what's the best way to interpret the body resource?
>

Depends on what the response format is.

For the multistatus response, Sabre_DAV_Client actually has a parser:

https://github.com/evert/SabreDAV/blob/master/lib/Sabre/DAV/Client.php

For an implementation I had a very similar problem.
This particular project had to add CalDAV support to their application
and work with any CalDAV server. If the user of this application
didn't have a CalDAV server, they would be able to use a built-in
CalDAV server, based on sabredav.

So for this client I actually subclassed:

Sabre_DAV_Client

and specifically the 'request' method. Instead of this method building
up the request using curl, I replaced the the thing to interact
directly with an instance of Sabre_DAV_Server.

This allowed me to easily use the propfind() method, and use the
parseMultiStatus() method for REPORT responses.

Don't know if you want to take it that far, but either way.. the
parseMutliStatus() method can also serve as an example for you to
interpret a {DAV:}multistatus response.

Evert

[musonic]

Brilliant. Worked a treat! :-) Thanks. 

[musonic]

This is really helpful, thanks. The only request I've made so far is a simple GET to retrieve a calendar object. The response content-type is "text/calendar" and the body is a resource of type stream.

I haven't had to work with streams before, but I've used stream_get_contents to read the body into a variable as a string and then used the VObject library to parse it. Does this sound like the best way?

[Evert Pot]

>
> This is really helpful, thanks. The only request I've made so far is a simple GET to retrieve a calendar object. The response content-type is "text/calendar" and the body is a resource of type stream.
> I haven't had to work with streams before, but I've used stream_get_contents to read the body into a variable as a string and then used the VObject library to parse it. Does this sound like the best way?

Yes!

[musonic]

So, I've been playing around and trying to implement the Client class, modified to make "fake" requests. I've overwritten the 'request' method and have been able to make simple GET requests successfully. I then tried to make a PUT request to create a new calendar object. Unfortunately this has not been successful. I built up a new VCALENDAR object using the VObject library and passed that to Request method as the third parameter. This threw an exception:

Sabre_DAV_Exception_UnsupportedMediaType This resource only supports valid iCalendar 2.0 data. Parse error: Invalid VObject, line 1 did not follow the icalendar/vcard format.

I was pretty confident that having built the object using VObject it would be valid, so I checked through the backtrace to see what was being passed around. This is where I found something weird. The method that is throwing the error is the 'validateICalendar' method in the Sabre_CalDAV_Plugin class. If I check what is being passed to this method before it tries to do anything at all I find that the parameter is:

resource(92) of type (stream)

This is fine because the method then converts it into a string. Unfortunately, if I check the variable after this I find that it is simply an empty string! No wonder it's being classed as invalid. 

So, I then tried to pass the VCALENDAR object as the second parameter when creating a new instance of the Sabre_HTTP_Request class. I tried passing it as a string, as an array and as an array with the key 'body'. I keep on hitting the same problem. When 'validateICalendar' tries to convert the stream, it just gets an empty string.

What am I doing wrong?!

On Thursday, 16 February 2012 11:07:23 UTC, Evert Pot wrote:

[Evert Pot]

Hi,

On Feb 18, 2012, at 12:02 PM, musonic wrote:

> So, I've been playing around and trying to implement the Client class, modified to make "fake" requests. I've overwritten the 'request' method and have been able to make simple GET requests successfully. I then tried to make a PUT request to create a new calendar object. Unfortunately this has not been successful. I built up a new VCALENDAR object using the VObject library and passed that to Request method as the third parameter. This threw an exception:
>
> Sabre_DAV_Exception_UnsupportedMediaType This resource only supports valid iCalendar 2.0 data. Parse error: Invalid VObject, line 1 did not follow the icalendar/vcard format.
>
> I was pretty confident that having built the object using VObject it would be valid, so I checked through the backtrace to see what was being passed around. This is where I found something weird. The method that is throwing the error is the 'validateICalendar' method in the Sabre_CalDAV_Plugin class. If I check what is being passed to this method before it tries to do anything at all I find that the parameter is:
>
> resource(92) of type (stream)
>
> This is fine because the method then converts it into a string. Unfortunately, if I check the variable after this I find that it is simply an empty string! No wonder it's being classed as invalid.
>
> So, I then tried to pass the VCALENDAR object as the second parameter when creating a new instance of the Sabre_HTTP_Request class. I tried passing it as a string, as an array and as an array with the key 'body'. I keep on hitting the same problem. When 'validateICalendar' tries to convert the stream, it just gets an empty string.
>
> What am I doing wrong?!

Not sure why this would happen.

Can you show some code? Just put it up on Gist (https://gist.github.com/)

Evert

[Nicholas Le Breuilly]

I suspected as much, but wasn't sure the body wasn't being set some other way! All sorted.
Thanks again,
Nic

> --
> You received this message because you are subscribed to the Google Groups "SabreDAV Discussion" group.
> To post to this group, send email to sabredav...@googlegroups.com.
> To unsubscribe from this group, send email to sabredav-discu...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sabredav-discuss?hl=en.
>