[fossil-users] How can I determine if a repository has actually changed?

[Ron Aaron]

I've got a bunch of Fossil repositories which I back up by doing:

fossil pull
fossil config pull all

I am now also encrypting the repos after backing up, and putting the
encrypted files on "Ubuntu One" for off-site failsafe backup.

The problem I am trying to solve is that I do NOT want to "synchronize"
a repo which has not changed; that is, I do not want to update the
encrypted file in the Ubuntu One sync folder -- since that is wasteful
of my bandwidth for no purpose.

What I had *hoped* would work, was to do "fossil sha" on the
repositories, and "fossil sha" again after the sync, and only copy files
whose SHA1 sums are different, thus minimizing both my processing time
and the network bandwidth.

Unfortunately, my current testing shows that doing the "fossil pull;
fossil config pull all" modifies the repository even when I know for
sure that there are no actual changes since the last sync.

What would you recommend as a method for accomplishing what I am trying
to do?

Thanks
Ron
_______________________________________________
fossil-users mailing list
fossil...@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[Stephan Beal]

You can place the fsl files directly in your ubuntu1 folder (or dropbox,  or whatever) and serve them from there.

----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal

[Ron Aaron]

Certainly I could, but that means that my fsl files are put there as-is,
and I want them encrypted before putting up there. It also means that
the fsl files will always be synched, even if nothing actually changes,
which is what I want to avoid.

On 03/21/2012 06:32 PM, Stephan Beal wrote:
>
> You can place the fsl files directly in your ubuntu1 folder (or
> dropbox, or whatever) and serve them from there.
>

[Stephan Beal]

On Wed, Mar 21, 2012 at 6:06 PM, Ron Aaron <r...@ronware.org> wrote:

Certainly I could, but that means that my fsl files are put there as-is,
and I want them encrypted before putting up there.  It also means that
the fsl files will always be synched, even if nothing actually changes,
which is what I want to avoid.

i don't know about Ubuntu1, but dropbox synchronizes only the bytes which changed, so the sync is really fast. There is, however, still a couple caveats with this approach (sorry for my brevity earlier - i was on my phone):

- The temp journal files from sqlite3 can sometimes get synced. i "pause" dropbox when i'm actively using a dropboxed repo. i don't remember if U1 has a pause option.

- If you use the repo from multiple machines and one or both are intermittently offline, it can lead to file change conflcts. Dropbox keeps both copies, one renamed to something like "filename (hostname2's conflicted copy).fsl".

So... it's not an ideal solution, but it works fine for single-developer/single-machine repos.

--

[Leo Razoumov]

On Wed, Mar 21, 2012 at 13:25, Stephan Beal <sgb...@googlemail.com> wrote:
>
> i don't know about Ubuntu1, but dropbox synchronizes only the bytes which
> changed, so the sync is really fast. There is, however, still a couple
> caveats with this approach (sorry for my brevity earlier - i was on my
> phone):
>

True, but does not help if your file is encrypted. You change a single
byte of your plain-text-file and your encrypted version changes
entirely.

--Leo--

[Ron Aaron]

On 03/21/2012 08:06 PM, Leo Razoumov wrote:
> True, but does not help if your file is encrypted. You change a single
> byte of your plain-text-file and your encrypted version changes
> entirely.

Precisely so. And I don't want to encrypt and synch the file, unless it
has changed in a meaningful way (e.g. not just some synch timestamp inside)

[Leo Razoumov]

On Wed, Mar 21, 2012 at 14:53, Ron Aaron <r...@ronware.org> wrote:
> On 03/21/2012 08:06 PM, Leo Razoumov wrote:
>> True, but does not help if your file is encrypted. You change a single
>> byte of your plain-text-file and your encrypted version changes
>> entirely.
>
> Precisely so.  And I don't want to encrypt and synch the file, unless it
> has changed in a meaningful way (e.g. not just some synch timestamp inside)

Poor man's way of figuring it out is to capture the output from fossil
pull (or fossil push) command, parse it and if all numbers of
transfered artifacts and deltas are zero than nothing changed.

--Leo--

[Ron Aaron]

On 03/21/2012 09:18 PM, Leo Razoumov wrote:
> Poor man's way of figuring it out is to capture the output from fossil
> pull (or fossil push) command, parse it and if all numbers of
> transfered artifacts and deltas are zero than nothing changed.

That will not work in this case, because I do not do the external save
all the time. That is to say, that the encrypted off-site files are
only updated periodically, while the fossil repos they are made from are
updated possibly often.

So what I am looking for is a way to take a 'snapshot' of a repo, and
determine if the new version of that repo is actually different, even
though I may have done multiple "pulls" in between checks.

I hope I'm explaning what I want to do sufficiently well...

The main reason for these contortions is that I don't want to send a
500M file over my limited bandwidth if I don't need to.

[Stephan Beal]

On Wed, Mar 21, 2012 at 8:40 PM, Ron Aaron <r...@ronware.org> wrote:

So what I am looking for is a way to take a 'snapshot' of a repo, and

determine if the new version of that repo is actually different, even
though I may have done multiple "pulls" in between checks.

Doesn't the timeline reveal if anything meaningful was changed? Could you not query the timeline (e.g. via scripting fossil json timeline...)?

[Stephan Beal]

On Wed, Mar 21, 2012 at 10:17 PM, Stephan Beal <sgb...@googlemail.com> wrote:

Doesn't the timeline reveal if anything meaningful was changed? Could you not query the timeline (e.g. via scripting fossil json timeline...)?

Or, more simply:

~> echo $(fossil fossil timeline -n 1) | cut -d'[' -f 2 | cut -d']' -f1

47817fefdf

(the echo/subshell there is to strip the newlines from the output)

That can probably be done more elegantly.

[Leo Razoumov]

On Wed, Mar 21, 2012 at 17:17, Stephan Beal <sgb...@googlemail.com> wrote:
> On Wed, Mar 21, 2012 at 8:40 PM, Ron Aaron <r...@ronware.org> wrote:
>>
>> So what I am looking for is a way to take a 'snapshot' of a repo, and
>> determine if the new version of that repo is actually different, even
>> though I may have done multiple "pulls" in between checks.
>
>
> Doesn't the timeline reveal if anything meaningful was changed? Could you
> not query the timeline (e.g. via scripting fossil json timeline...)?
>

I think this is the winner. I cannot thing of any (non pathologically
esoteric) cases when a repo changes but the
last 20 commits stay the same.

--Leo--

[Ron Aaron]

Sorry, the "$2" needs to be a "$1" -- that was a finger-flub on my part

On 03/22/2012 09:13 AM, Ron Aaron wrote:
> OK, I've come up with a small bash script to get an 'id' which I can use
> to detect changes in a repo. Save the following to "fossilid" and make
> it executable:
>
> if [ ! -f "$1" ]
> then
> echo "fossilid needs the name of the repository to 'id'"
> exit 1
> fi
>
> configsha=`fossil config export all -R $2 - | grep -v '^#' | fossil sha
> - | cut -d' ' -f1`
> logsha=`fossil timeline -n 10 -R $1 | fossil sha - | cut -d' ' -f1`
>
> echo "$configsha $logsha"