Does syncing a file change only transmit the file delta's, or does it transmit the whole file?

578 views
Skip to first unread message

Belorion Cypher

unread,
Jan 16, 2014, 12:19:26 PM1/16/14
to sea...@googlegroups.com
In other words, if I make a small modification to a large file, does it only need to transmit information about the change to the server, or does it re-upload the entire file?


JiaQiang Xu

unread,
Jan 17, 2014, 9:56:11 AM1/17/14
to sea...@googlegroups.com
Seafile detects delta.


2014/1/17 Belorion Cypher <belo...@gmail.com>
In other words, if I make a small modification to a large file, does it only need to transmit information about the change to the server, or does it re-upload the entire file?


--
You received this message because you are subscribed to the Google Groups "seafile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seafile+u...@googlegroups.com.
To post to this group, send email to sea...@googlegroups.com.
Visit this group at http://groups.google.com/group/seafile.
For more options, visit https://groups.google.com/groups/opt_out.

Kristian Grizelj

unread,
Jan 17, 2014, 3:02:00 PM1/17/14
to sea...@googlegroups.com
Not true, seafile not suport delta.
Ok, partially true, suport only from client to server.
if you change the file to work coputer, then cashing chunk file and sends only the delta. Start the second (home computer) redownloading whole file. I do not even know recognize rename, move and copy. All that initiates a complete redownload.
Client does not know that have that file, not have created chunk cashes and downloading complete file.
Solution, do not ever stop seafile client, do not delete cashes chunk.

Nicolas Martinelli

unread,
Apr 18, 2014, 5:07:55 PM4/18/14
to sea...@googlegroups.com
Indeed, since Seafile does not keep the cache chunks locally (they are removed at restart, which is good), it has to download the same file even in the case of small modifications. In my opinion, this could be avoided by splitting any modified file into chunks on client side, then only download the modified ones. Of course, that adds extra load on client side, but greatly reduces the potential server load.

Seriously, this can really be a major issue for scalability: any slight change on a large file (or worse, renaming a folder) implies a complete re-download by every single client. Since Seafile does not support P2P transfers, all the load goes to the server. Imagine the following: 20 clients sharing a folder of 10 Go (which are not huge numbers). One client renames the folder... Boom, a minimum of 190 Go must be sent by the server. Even on a local network, that's not acceptable.

I hope this will be taken into consideration for future developments.

Regards,
Nicolas

JiaQiang Xu

unread,
Apr 21, 2014, 2:14:14 AM4/21/14
to sea...@googlegroups.com
We'll consider your proposal. Thanks!

blablup

unread,
Apr 27, 2014, 5:45:23 PM4/27/14
to sea...@googlegroups.com
Maybe it would be even better to save a db with all chunks available on the client and the file they are in (and maybe the offset) this would also reduce the load. The only thing you need is a db with all the chunks.

Stefan Seidel

unread,
Jul 26, 2015, 3:51:09 PM7/26/15
to seafile, kiko...@gmail.com
On Friday, January 17, 2014 at 9:02:00 PM UTC+1, Kristian Grizelj wrote:
Not true, seafile not suport delta.
Ok, partially true, suport only from client to server.
if you change the file to work coputer, then cashing chunk file and sends only the delta. Start the second (home computer) redownloading whole file. I do not even know recognize rename, move and copy. All that initiates a complete redownload.
Client does not know that have that file, not have created chunk cashes and downloading complete file.
Solution, do not ever stop seafile client, do not delete cashes chunk.


Dana petak, 17. siječnja 2014. 15:56:11 UTC+1, korisnik JiaQiang Xu napisao je:
Seafile detects delta.


From my experience, none of this is true. I have worked quite a bit on a 5MB ODG file (LibreOffice Drawing). Everytime I save the file, the whole 5MB are transferred to the server. And, not incidentally, the server disk usage also increases by 5MB. I have checked two variants of the file (I inserted a single space character). rsync --no-W -v file1 file2 transferred 90kb out, 14kb in. This is the number I want to see, but I see 5MB out.

So in conclusion, maybe seafile supports delta, but on my setup (Windows 8.1, client 4.2.8, server Ubuntu 12.04 Seafile 4.3.0, but was the same in previous 4.2.x version), it does not use/send any delta for my 5MB binary file, it sends the whole file for every single modification.

Stefan
Reply all
Reply to author
Forward
0 new messages