comparing directories
Patrick Proniewski
Short story: Is there a nice way to compare files from a remote
directory tree with files from a local directory tree?
Long story: I manage my config files for 2 FreeBSD servers using SVN.
I have a repository somewhere, a local working copy, and 2 remote
servers. Unfortunately it happens quite often that I'm tweaking
various config files on a remote server, and have no time for an
immediate SVN commit. So I need a way to synchronize production files
with working copy files, easily. I can't do it manually, there are 280
files, growing. A simple "compare server01:/etc with the local copy"
would be so great!
patpro
David Kelly
"svn status" from the command line will tell you what files are different vs the specified version in the repository.
Guess I don't understand your question as I think svn does everything you are asking. Just "svn commit" the master working files from one server then for each server "svn update" and only those files which are different will be updated.
--
David Kelly N4HHE, dke...@HiWAAY.net
========================================================================
Whom computers would destroy, they must first drive mad.
LuKreme
> Short story: Is there a nice way to compare files from a remote directory tree with files from a local directory tree?
within BBEdit? I would use rsync to do this.
> Long story: I manage my config files for 2 FreeBSD servers using SVN. I have a repository somewhere, a local working copy, and 2 remote servers. Unfortunately it happens quite often that I'm tweaking various config files on a remote server, and have no time for an immediate SVN commit. So I need a way to synchronize production files with working copy files, easily. I can't do it manually, there are 280 files, growing. A simple "compare server01:/etc with the local copy" would be so great!
Um… I think one of the advantages of Git is making this type of thing much more transparent. I'm sure it's possible with SVN, but that really doesn't have anything to do with BBEdit.
Or am I missing something?
--
Seeing, contrary to popular wisdom, isn't believing. It's where belief
stops, because it isn't needed any more. --Pyramids
David Kelly
On Dec 18, 2010, at 12:10 PM, LuKreme wrote:
>> Long story: I manage my config files for 2 FreeBSD servers using SVN. I have a repository somewhere, a local working copy, and 2 remote servers. Unfortunately it happens quite often that I'm tweaking various config files on a remote server, and have no time for an immediate SVN commit. So I need a way to synchronize production files with working copy files, easily. I can't do it manually, there are 280 files, growing. A simple "compare server01:/etc with the local copy" would be so great!
>
> Um… I think one of the advantages of Git is making this type of thing much more transparent. I'm sure it's possible with SVN, but that really doesn't have anything to do with BBEdit.
>
> Or am I missing something?
I think the O.P. is missing something when for some reason there is time to post an incomplete question to a mailing list but for some reason there isn't time to do an "svn commit". After which an "svn update" on the other server would put an end to the issue.
There isn't anything in this scenario favoring git. There is one master copy. One person doing the changes. Isn't even anything requiring svn. Svn is good for keeping an archive of changes, and its good for simultaneously making different changes on different machines then reconciling the two differing edits.
But if you have a collection of files on one machine that you wish to be identical on another then rsync(1) is your friend. Its included in MacOS X (or Xcode). Have to install it from /usr/ports/net/rsync on FreeBSD.
Rsync is smart enough not to blindly copy every file, only changed files, and then only the changed portions.
Patrick Proniewski
I'm subscribed to the Digest, I'm going to reply to every one of you
here.
> David Kelly <dke...@hiwaay.net> Dec 18 06:40AM -0600 ^
>
> "svn status" from the command line will tell you what files are
> different vs the specified version in the repository.
I know that, but my working copy is in sync with my repository, that's
not the issue. The problem is my production files are not always in
sync with my SVN repository (in general, they are newer).
> Guess I don't understand your question as I think svn does
> everything you are asking. Just "svn commit" the master working
> files from one server then for each server "svn update" and only
> those files which are different will be updated.
I'm not using SVN to maintain production file `in situ`. Files exist
at 3 different places: in production (on the server), in the
repository, and in my working copy. When I'm working on a server, I
make changes in various files, or create new ones, I wait few hours or
few days to make sure everything works fine, then I don't always
remember what files have changed. Comparing manually the production
directory tree and the working copy is a very painful and long
process. I want to make that automatic. Before creating a shell script
to do so, I would like to know if bbedit is able to compare a remote
directory with a local directory. Like it does with 2 local directories.
I'm already using BBEdit this way, for file-to-file comparison: I
change one file server-side, and if I want to commit immediately. I
compare this file with its local working copy using BBEdit "compare
two front documents", merge changes to the local copy, and commit.
> LuKreme <kre...@kreme.com> Dec 18 11:10AM -0700 ^
>
> On 18-Dec-2010, at 02:22, Patrick Proniewski wrote:
> > Short story: Is there a nice way to compare files from a remote
> directory tree with files from a local directory tree?
>
> within BBEdit? I would use rsync to do this.
It's a possibility, yes. If I have to go "the shell script way", it
will surely involve rsync. But the sole purpose of rsync would be to
create a local copy of the production tree. I still want to use bbedit
to compare file, because I want to review changes manually.
> David Kelly <dke...@hiwaay.net> Dec 18 09:26PM -0600 ^
>
> > Or am I missing something?
>
>
> I think the O.P. is missing something when for some reason there is
> time to post an incomplete question to a mailing list but for some
> reason there isn't time to do an "svn commit". After which an "svn
> update" on the other server would put an end to the issue.
Your making assumptions here ;)
As I wrote above, I don't use SVN to put files in production. By the
way, my problem is the production files are generally newer than those
in SVN, so an update is not an option for me.
regards,
patpro
David Kelly
On Dec 19, 2010, at 6:25 AM, Patrick Proniewski wrote:
> Hi all,
>
> I'm subscribed to the Digest, I'm going to reply to every one of you here.
>
>> David Kelly <dke...@hiwaay.net> Dec 18 06:40AM -0600 ^
>>
>> "svn status" from the command line will tell you what files are different vs the specified version in the repository.
>
> I know that, but my working copy is in sync with my repository, that's not the issue. The problem is my production files are not always in sync with my SVN repository (in general, they are newer).
Then commit the production files to the repository.
>> Guess I don't understand your question as I think svn does everything you are asking. Just "svn commit" the master working files from one server then for each server "svn update" and only those files which are different will be updated.
>
> I'm not using SVN to maintain production file `in situ`.
If you heard what I've been saying between the lines, you *should*. Let svn be your remote file sync mechanism.
> Files exist at 3 different places: in production (on the server), in the repository, and in my working copy. When I'm working on a server, I make changes in various files, or create new ones, I wait few hours or few days to make sure everything works fine, then I don't always remember what files have changed.
If the files were under svn then status, diff, or commit, will tell you which ones have been changed since the last commit.
> Comparing manually the production directory tree and the working copy is a very painful and long process. I want to make that automatic.
Svn will automate that.
You need to study the use of forks. Thinking you need a "beta" fork to run in parallel to your head "working copy" which will be very much like the difference between FreeBDD -CURRENT and -STABLE.
Perhaps each server has a customized version of the config files? Then there is another fork.
Rich Siegel
wrote:
>I'm not using SVN to maintain production file `in situ`. Files exist
>at 3 different places: in production (on the server), in the
>repository, and in my working copy. When I'm working on a server, I
>make changes in various files, or create new ones, I wait few hours or
>few days to make sure everything works fine, then I don't always
>remember what files have changed. Comparing manually the production
>directory tree and the working copy is a very painful and long
>process. I want to make that automatic.
At this exact moment, your best bet is probably to use something
like ExpanDrive to mount your remote server via ssh as a file
system; then you can use "Find Differences" to compare your
local directory with the server. Do note that MacFUSE+ssh (of
which ExpanDrive is one implementation) will perform very
slowly; but for this purpose it should serve.
I think that the points about changing your workflow are well
made, since the way you've described it today, your workflow is
calculated to drive you nuts, but that's really a different
issue. :-)
R.
--
Rich Siegel Bare Bones Software, Inc.
<sie...@barebones.com> <http://www.barebones.com/>
Someday I'll look back on all this and laugh... until they
sedate me.
David Kelly
On Dec 19, 2010, at 9:29 AM, Rich Siegel wrote:
> At this exact moment, your best bet is probably to use something like ExpanDrive to mount your remote server via ssh as a file system; then you can use "Find Differences" to compare your local directory with the server. Do note that MacFUSE+ssh (of which ExpanDrive is one implementation) will perform very slowly; but for this purpose it should serve.
NFS on FreeBSD works well with MacOS. Shares a lot of the same code. However, I grimace at even suggesting an NFS export from a public web server over the internet without an encrypted VPN tunnel.
Which brings us back to rsync which is a purpose made remote file compare and copy utility.
Ron Catterall
Ron
--
Ron Catterall Ph.D. D.Sc.
r...@catterall.net
http://catterall.net
Patrick Proniewski
> David Kelly <dke...@hiwaay.net> Dec 19 09:20AM -0600 ^
>
> > I'm not using SVN to maintain production file `in situ`.
>
> If you heard what I've been saying between the lines, you *should*.
> Let svn be your remote file sync mechanism.
>
> > Files exist at 3 different places: in production (on the server),
> in the repository, and in my working copy. When I'm working on a
> server, I make changes in various files, or create new ones, I wait
> few hours or few days to make sure everything works fine, then I
> don't always remember what files have changed.
>
> If the files were under svn then status, diff, or commit, will tell
> you which ones have been changed since the last commit.
So basically, you say I should turn my /boot, /etc and /usr/local/etc/
into working copies and clutter them with .svn directories? And what
about the 30 or 50 text files and 150MB of binary files that live here
and are not under version control?
> You need to study the use of forks. Thinking you need a "beta" fork
> to run in parallel to your head "working copy" which will be very
> much like the difference between FreeBDD -CURRENT and -STABLE.
I really don't see the point, unless I'm supposed to change the
current fork in production every now and then.
If I understand correctly:
- I turn /boot, /etc, /usr/local/etc into working copies
- each one follows the "beta" fork
where is my "head" fork? Why do I have a "head" fork anyway? May be
I'm missing something here, but forking looks really overkill for my
needs.
> Perhaps each server has a customized version of the config files?
> Then there is another fork.
Yes they have. They have nothing in common, except the "freebsd 8.1"
part. They serve very different purposes, on very different networks.
For now, I use a single working copy, with a directory for each host.
Simple enough.
> Rich Siegel <sie...@barebones.com> Dec 19 10:29AM -0500 ^
>
> On Sunday, December 19, 2010, Patrick Proniewski <pat...@patpro.net>
> wrote:
>
> >remember what files have changed. Comparing manually the production
> >directory tree and the working copy is a very painful and long
> >process. I want to make that automatic.
>
> At this exact moment, your best bet is probably to use something
> like ExpanDrive to mount your remote server via ssh as a file
> system; then you can use "Find Differences" to compare your
> local directory with the server. Do note that MacFUSE+ssh (of
> which ExpanDrive is one implementation) will perform very
> slowly; but for this purpose it should serve.
I've given this option a though, earlier, but dismissed the idea. I
don't want to add a kernel module on every Mac I'm working on, just
for that purpose.
> I think that the points about changing your workflow are well
> made, since the way you've described it today, your workflow is
> calculated to drive you nuts, but that's really a different
> issue. :-)
I'm not driven nuts yet ;)
And my workflow would be very nice if I had the ability to compare
remote/local directories the way I can compare local/local directories.
> David Kelly <dke...@hiwaay.net> Dec 19 10:57AM -0600 ^
>
> NFS on FreeBSD works well with MacOS. Shares a lot of the same code.
> However, I grimace at even suggesting an NFS export from a public
> web server over the internet without an encrypted VPN tunnel.
NFS could be ok. And I'm pretty sure I can make a launchd plist to
handle ssh tunnel creation on demand to access a remote NFS behind a
firewall via an encrypted connection. I've done that quite often with
other protocols (usenet, mysql, vnc, apple remote desktop, ...)
BBEdit can browse a remote directory tree via an ssh connection, too
bad it can't natively use the same transport to make directory tree
comparisons :/
> Ron Catterall <r...@catterall.net> Dec 19 11:12AM -0600 ^
>
> Just a thought - would Interarchy do the job?
May be the net disk feature would work, yes. But if I can do it for
free with NFS/ssh-tunnel it's better.
thanks,
patpro