Improve Sakai CLE WebDAV support

[Nandana]

Hi,

I'm a master student from Universidad Politécnica de Madrid (UPM), Spain and I am interested in the project "Improve Sakai CLE WebDAV support". I would like to learn a bit more about this project and I started by trying out the WebDAV  feature in Sakai. It went well but I noticed a strange behavior when connected with Linux. It only worked with Linux after I connected with Windows once. I tried it with twice to make sure it was not a timing issue or something like that, and I could reproduce it for the second time too. I am not sure whether it is a known issue, and I blogged all the steps I followed here [1].

As the next step, I am planning to set a local demo instance [2] (I used Free Test Drive Sakai 2.7 from Unicon to just to check WebDAV features) and also take a glimpse at the existing Sakai CLE WebDAV code. Is this [3] the right place to look ?

I would like to know whether I am going in the right direction and highly appreciate any pointers about this project. Thanks a lot in advance.

Best Regards,

Nandana 

[1] - http://www.nandana.org/2011/03/sakai-cle-webdav-support.html

[2] - https://confluence.sakaiproject.org/display/DOC/Install+Guide+-+Demo+Install+(2.7)

[3] - https://source.sakaiproject.org/svn/dav/trunk/

[aewp2]

Hi Nandana,

thanks for your interest!

I need to ask Seth and Anthony to sign up here - please stand by. I'm
sure they'll be keen to discuss ideas with you.

Amyas

[Seth Theriault]

Hello,

> It only worked with Linux after I connected with
> Windows once. I tried it with twice to make sure it was not a timing issue
> or something like that, and I could reproduce it for the second time too. I
> am not sure whether it is a known issue, and I blogged all the steps I
> followed here [1].

As you have discovered, WebDAV support is inconsistent both in Sakai
and in the clients. Over the years, many people in the Sakai community
have tweaked the WebDAV instructions in Sakai as well as the code
itself to minimize these problems. While improving the code won't fix
all of the problems, it might provide a more maintainable foundation.

> As the next step, I am planning to set a local demo instance [2] (I used
> Free Test Drive Sakai 2.7 from Unicon to just to check WebDAV features) and
> also take a glimpse at the existing Sakai CLE WebDAV code. Is this [3] the
> right place to look ?

Yes, "trunk" is the latest Sakai CLE code (targeted now at the future
2.9 release), but you may wish to start with this branch of upcoming
2.8.0 release instead for exploration purposes:

https://source.sakaiproject.org/svn/dav/branches/sakai-2.8.x/

> I would like to know whether I am going in the right direction and
> highly appreciate any pointers about this project. Thanks a lot in advance.

You are definitely headed in the right direction. In Sakai, WeDAV
interfaces with something called the ContentHosting service, which
manages the content repository. They are closely aligned, so you
probably want to take a look at the CHS code and javadocs:

https://source.sakaiproject.org/svn/kernel/branches/kernel-1.2.x/api/src/main/java/org/sakaiproject/content/api/
https://source.sakaiproject.org/release/kernel/1.2.0/apidocs/index.html

I'll hunt around to see if I can find a more informative CHS reference
for you.

Seth

[Nandana Mihindukulasooriya]

Hi Seth,

Thanks a lot for your informative reply. Your pointers were very helpful and I started looking at WebDAV code and CHS code. I also took a quick look at RFC4918 to learn the terminology and understand the high level concepts of the spec, however I didn't pay much attention to low level technical details yet. I have few small questions and I hope you will be able to help me out.

First of all, what are the best places to look guidelines on getting started on Sakai development. I found Sakai Programmer's Cafe [1] very useful and was wondering whether there are other resources like that. For instance, I first got bit confused about the branching strategy when I wanted to checkout the complete 2.8.x branch to create the project using "mvn eclipse:eclipse" as each sub-project had their branch under them. It took me a while to realize that this is handled using svn:externals. Are there any other documentation like Sakai Programmer's Cafe where these nitty-gritty tips and tricks can be found.

Second question is more specifically on remote debugging. I checked out the 2.8.x branch, and was able to built the demo-pack using that branch. Then, I wanted to go through the code a bit with remote debug on and see the flow of basic WebDAV functionalities as I find that is the fastest way to grasp the new code. In Tomcat, I normally set the debug JVM properties and start it on debug mode [2]. So I modified the start-sakai.sh like this http://paste.pocoo.org/show/359391/, but I am getting this error http://paste.pocoo.org/show/359388/. 

Does the properties I set for remote debugging conflict with any JVM property set by Sakai ? What is the usual way that Sakai developers remote debug the server ?

Thanks a lot of your time in advance, and I totally appreciate your help. 

Best Regards,

Nandana 

[1] http://bit.ly/sakai-programmers-cafe

[2] http://wiki.apache.org/tomcat/FAQ/Developing#Q1

[Nandana Mihindukulasooriya]

Hi,

Second question is more specifically on remote debugging. I checked out the 2.8.x branch, and was able to built the demo-pack using that branch. Then, I wanted to go through the code a bit with remote debug on and see the flow of basic WebDAV functionalities as I find that is the fastest way to grasp the new code. In Tomcat, I normally set the debug JVM properties and start it on debug mode [2]. So I modified the start-sakai.sh like this http://paste.pocoo.org/show/359391/, but I am getting this error http://paste.pocoo.org/show/359388/. 

I was able to solve this problem and move on. I had to set another property "JPDA_OPTS" and it was not a Sakai related problem. Please ignore this question.

Best Regards,

Nandana

[Seth Theriault]

> First of all, what are the best places to look guidelines on getting started
> on Sakai development. I found Sakai Programmer's Cafe [1] very useful and
> was wondering whether there are other resources like that. For instance, I
> first got bit confused about the branching strategy when I wanted to
> checkout the complete 2.8.x branch to create the project using "mvn
> eclipse:eclipse" as each sub-project had their branch under them. It took me
> a while to realize that this is handled using svn:externals. Are there any
> other documentation like Sakai Programmer's Cafe where these nitty-gritty
> tips and tricks can be found.

To be truly honest, the Programmer's Cafe is probably your best
resource for a "guide" to Sakai development. I would encourage to also
follow the Sakai Development mailing list, sakai-
d...@collab.sakaiproject.org, where many CLE-related technical
questions get answered. Don't be scared off by variety of questions
posed, but do make sure your question is timely, clear, and hasn't
been answered somewhere else (like the Cafe pages!).

I am glad to hear you solved the remote debugging problem.

Seth

[Nandana Mihindukulasooriya]

Hi Seth,

Thanks for the reply and I subscribed to the dev list. I went through the code and wanted make sure I understood the high level picture correctly. I will write the idea I got in the most abstract and simplest terms. 

Sakai CLE has it's own WebDAV implementation which mainly consists of DavServlet & CHS code.

Sakai DavServlet [1] has been written based on some version (4.x?) of WebdavServlet [2] of Apache Tomcat, however it doesn't use (inherit or compose) Tomcat WebdavServlet. DavServelt takes care of most of the WebDAV control flow like authentication, method dispatch, locking, request parsing, and response generation. 

ContentHostingService [3] mainly deals with manging shared content (resources and collections) and their properties. A resource is a file and resources are organized in collections same as in a file system. CHS is capable on performing actions like list, access, create, copy, move, delete etc. mentioned in the WebDAV spec on resources and collections and also on their properties. CHS also manages permissions. Some of the above functionalities mentioned for DavServlet like locking could span across CHS too. DbContentService [4] is a ContentHostingService implementation with DB support. 

Is the above high level idea correct or have I got something wrong ? If above summery is correct, what would be the scope of this project ? Will we be mostly rewriting the DavServlet part or will it include CHS part too. I understand that this will depend on how Milton will fit in  and will change as we understand the best approaches to restructure things, but I am just trying to get a rough idea.

Now I started looking at Milton samples and demos to understand how it could be used to rewrite the WebDAV code of Sakai. According to Milton website, dotCMS [5] has followed a similar approach replacing the tomcat WebdavServlet with Milton library. I might be able to learn something from their codebase [6] too.

Thanks a lot for your feedback in advance.

Best Regards,
Nandana

[1] - https://source.sakaiproject.org/svn/dav/branches/sakai-2.8.x/dav/src/java/org/sakaiproject/dav/DavServlet.java

[2] - http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/catalina/servlets/WebdavServlet.java?view=markup

[3] - https://source.sakaiproject.org/svn/kernel/branches/kernel-1.2.x/api/src/main/java/org/sakaiproject/content/api/ContentHostingService.java

[4] - https://source.sakaiproject.org/svn/kernel/branches/kernel-1.2.x/kernel-impl/src/main/java/org/sakaiproject/content/impl/DbContentService.java

[5] - http://dotcms.com/products/technology/

[6] - https://svn.dotcms.org/trunk/src/com/dotmarketing/webdav/

[Seth Theriault]

Hello again,

> Is the above high level idea correct or have I got something wrong ? If
> above summery is correct, what would be the scope of this project ? Will we
> be mostly rewriting the DavServlet part or will it include CHS part too. I
> understand that this will depend on how Milton will fit in  and will change
> as we understand the best approaches to restructure things, but I am just
> trying to get a rough idea.

What you describe sounds pretty accurate. The main thrust here, as you
suggest, is to rewrite the DavServlet part so that the WebDAV support
itself is fairly solid and in less need of hands-on tinkering. If we
don't have to touch CHS, we will probably be a lot happier and more
successful in this endeavor.

Seth

[Nandana Mihindukulasooriya]

Dear Amyas/Seth,

My proposal for the project "Improve Sakai CLE WebDAV support" has been rejected. However, thanks a lot both of you for all the support you gave to come up with the proposal. I wish all the best for the selected students and I am sure Sakai will have 4 fabulous summer of code projects.

Best Regards,

Nandana

[にゃんこ]

You're not the only one that was rejected for that proposal :)

2011/4/25 Nandana Mihindukulasooriya <nanda...@gmail.com>

[aewp2]

Hi Nandana

thanks so much for taking the time to put together a great proposal,
we'd love to take up all such but as always we only have a limited
number of slots. In both this and the OAE mobile project we had an
extremely difficult time to choose between top-ranked candidates.

I and the mentor team have every confidence you will have no problem
finding alternative employment over the (northern hemisphere) summer,

Amyas