foldersync webdav problems

[Martin Bosner]

Hi,

i got a problem with foldersync (webdav) and seafile.

When i syncronize some hundret files it stops working after around 100 files and it starts showing this in the seafile dav logs:

[2014-05-07 00:11:44,369]:   - (anonymous) - "PROPFIND /Bilderupload Martin/" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:11:44,624]:   - m.bo...@email.com - "PROPFIND /Bilderupload Martin/" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.152sec -> 207 Multistatus
[2014-05-07 00:11:47,225]:   - (anonymous) - "PROPFIND /Bilderupload Martin/" length=249, depth=1, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:11:48,237]:   - m.bo...@email.com - "PROPFIND /Bilderupload Martin/" length=249, depth=1, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.895sec -> 207 Multistatus
[2014-05-07 00:11:55,924]:   - (anonymous) - "DELETE /Bilderupload Martin/20140430_084902.jpg.tacitpart" connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:12:13,006]:   - m.bo...@email.com - "DELETE /Bilderupload Martin/20140430_084902.jpg.tacitpart" depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=16.960sec -> 204 No Content
[2014-05-07 00:13:20,527]:   - (anonymous) - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="Apache-HttpClient/UNAVAILABLE (java 1.4)", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:13:20,808]:  Raising DAVError 404 Not Found: The specified resource was not found
[2014-05-07 00:13:20,808]:   - m.bo...@email.com - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="Apache-HttpClient/UNAVAILABLE (java 1.4)", elap=0.167sec -> 404 Not Found
[2014-05-07 00:13:26,183]:   - (anonymous) - "PUT /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=3752431, expect="100-continue", connection="Keep-Alive", agent="Apache-HttpClient/UNAVAILABLE (java 1.4)", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:13:31,630]:   - m.bo...@email.com - "PUT /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=3752431, expect="100-continue", connection="Keep-Alive", agent="Apache-HttpClient/UNAVAILABLE (java 1.4)", elap=0.498sec -> 201 Created
[2014-05-07 00:13:32,271]:   - (anonymous) - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:14:03,348]:   - (anonymous) - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:14:06,232]:   - m.bo...@email.com - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=33.848sec -> 207 Multistatus
[2014-05-07 00:14:34,395]:   - (anonymous) - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:14:36,896]:   - m.bo...@email.com - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=33.439sec -> 207 Multistatus
[2014-05-07 00:15:07,725]:   - m.bo...@email.com - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=33.211sec -> 207 Multistatus
[2014-05-07 00:15:08,595]:   - (anonymous) - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="Apache-HttpClient/UNAVAILABLE (java 1.4)", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:15:42,367]:   - m.bo...@email.com - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="Apache-HttpClient/UNAVAILABLE (java 1.4)", elap=33.661sec -> 207 Multistatus
[2014-05-07 00:15:48,172]:   - (anonymous) - "PUT /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=3752431, expect="100-continue", connection="Keep-Alive", agent="Apache-HttpClient/UNAVAILABLE (java 1.4)", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:16:08,039]:   - m.bo...@email.com - "PUT /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=3752431, expect="100-continue", connection="Keep-Alive", agent="Apache-HttpClient/UNAVAILABLE (java 1.4)", elap=17.307sec -> 204 No Content
[2014-05-07 00:16:08,734]:   - (anonymous) - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:16:39,788]:   - (anonymous) - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:16:43,182]:   - m.bo...@email.com - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=34.335sec -> 207 Multistatus
[2014-05-07 00:17:10,838]:   - (anonymous) - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:17:12,676]:   - m.bo...@email.com - "PROPFIND /Bilderupload Martin/20140430_084902.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="android-cloud-api - Android 4.4.2", elap=32.771sec -> 207 Multistatus
[2014-05-07 00:17:42,519]:   - (anonymous) - "PROPFIND /Bilderupload Martin/20140430_085133.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="Apache-HttpClient/UNAVAILABLE (java 1.4)", elap=0.000sec -> 401 Not Authorized
[2014-05-07 00:17:42,712]:  Raising DAVError 404 Not Found: The specified resource was not found
[2014-05-07 00:17:42,712]:   - m.bo...@email.com - "PROPFIND /Bilderupload Martin/20140430_085133.jpg.tacitpart" length=249, depth=0, connection="Keep-Alive", agent="Apache-HttpClient/UNAVAILABLE (java 1.4)", elap=0.086sec -> 404 Not Found

Is this a seafile problem or a foldersync problem?

Cheers

[JiaQiang Xu]

I can't see anything unusual from the webdav log. The "not found" message may be normally triggered by foldersync checking file existence.

[Martin Bosner]

Is there a reason why there are so many "401 Not Authorized" ?

[JiaQiang Xu]

That's the normal http authentication protocol. You can google it.

[René Rumpel]

Hi,

I have the same issue as Martin. Did you find any solution for this?

I have a clean setup of seafile using MySQL and with enabled WebDAV. After copying about 100 - 150 files the process gets slower and slower and finally stops.

René

[Jacek Szafarkiewicz]

I think it is problem with deleting files by webdav - it is very slow: few seconds for file.

The problem with process gets slower is it starts new thread after each file transfer and when it starts 30 threads (for 30 files in queue) and every thread try to access to disk, also every thread allocate some memory, when the ram is over, it go to the swap and again access disk.

[René Rumpel]

Hi Jacek,

as FolderSync uploads files with a ".tacitpart" extension and then renames them, your explanation does make sense.

--
You received this message because you are subscribed to a topic in the Google Groups "seafile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/seafile/r-kXWXSoOWM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to seafile+u...@googlegroups.com.
To post to this group, send email to sea...@googlegroups.com.
Visit this group at http://groups.google.com/group/seafile.
For more options, visit https://groups.google.com/d/optout.

[Martin Bosner]

Sorry for the late response.

I tested foldersync with some other webdav tools and it is working fine. The problem seems to be on the seafile webdav implementation. If you use the cli seafile check tools you should see problems in the library where you uploaded the files to. Could you confirm that? I am currently busy on a business trip but as soon as i am back i will check which commands which told me that there are errors in the libs.

Martin

[Markus Germeier]

Hi there,

Am Dienstag, 1. Juli 2014 09:30:58 UTC+2 schrieb René Rumpel:

I have a clean setup of seafile using MySQL and with enabled WebDAV. After copying about 100 - 150 files the process gets slower and slower and finally stops.

I have the same problem while trying to use FolderSync to upload my Titanium Backup files. The process gets slower and slower and then it seems like ist stops. The seaf-server process uses one CPU with 100% in that time. I am using the 3.1.5 server on linux.

After some digging around I found the problem and a hotfix. In seahub/thirdpart/wsgidav/addons/seafile/seafile_dav_provider.py there is a class "SeafileResource" with a method "getLastModified". This method calls "seafile_api.get_files_last_modified" which seems to take a *LONG* time if there are a lot of files in the directory. Changing that line to "mtimes = [ ]" (thus disabling the modification time lookup) "fixed" my syncing problems.

I hope this information is useful to other users and I would like to see a real fix in an upcoming version.

Keep up the good work!

Regards

   Markus

[Shuai Lin]

Thansk Markus, we'll take a look at this.

--
You received this message because you are subscribed to the Google Groups "seafile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seafile+u...@googlegroups.com.