Re: Rsync'ing and Tarballs

[tbil...@gmail.com]

I thought I'd throw this one out at you guys.

Instead of having 211MB tarballs for each release, one may consider
setting up an rsync server. There's a tremendous amount of stuff
that's static across the tarballs. I'll bet the tool chain stuff has
changed since the first tarball.

Bitkeeper used to do something with hard links on a clone where hardly
any space was used until the file was overwritten. Something like that
on the multiple directories would cause only the delta in each
additional directory, but the whole tree would appear to be full.

This would make switching between lots of releases pretty straight
forward. It could save a lot of space on local copies too. Just hard
link clone locally and rsync the differences to the latest release.

Thoughts ?

Thanks

Tom

[andrew fresh]

On Fri, Nov 03, 2006 at 10:59:52PM -0000, tbil...@gmail.com wrote:
> Instead of having 211MB tarballs for each release, one may consider
> setting up an rsync server. There's a tremendous amount of stuff
> that's static across the tarballs. I'll bet the tool chain stuff has
> changed since the first tarball.

They already have SVN, so for updating the source, can't you just svn
sync (or whatever the command is) to the correct tag for the release?
That should just get you the files that have changed and be lots
smaller to transfer.

I don't have a dev environment for the OSD here, so I haven't done any
of that yet, but I do keep up with some other source trees similar to
doing that.

l8rZ,
--
andrew - ICQ# 253198 - JID: afr...@jabber.org

BOFH excuse of the day: The salesman drove over the CPU board.

[Xorlev]

This sounds pretty good. I think the main benefit here would be a
pristine source-tree for building. SVN works pretty well, but when
switching between tags and such a lot gets left behind from previous
builds and more often than not causes build problems.

By hardlinking to a base clone, rsync would remove the hardlinks of the
changed files before writing them, correct? This seems like it would
work well, but what about manual editing of hardlinked files? This
would change the clone as well. Or am I missing something here?

Xorlev

[critch]

Or maybe this is the time to split the portions out into a different
tree structure such that your build tools sit in one tree and the code
base in another base tree. Then only at certain points do you traverse
up and reupdate the build tools. This would potentially reduce the
regular subversion traffic to a much smaller set and therefore reduce
bandwidth.

--
critch <cri...@drunkenlogic.com>

[FireLord]

I agree since the svn already have more than one Gig of files in it,
imagine the overhead and band consumption if everyone syncs the whole
tree...

Anyway IMHO the taballs could still be built but kept on another server
than svn's...
Bye,
Riccardo