Not found the right client (linux) yet - suggestions?

[Johannes S]

Hey guys,

I signed up for a trial of datastorage nearly 2 weeks ago.
Unfortunately I haven't found the right client for my needs, yet.
My Backup should be encrypted, it's not that I don't trust John, I
just want to know that my data stays private even if someone hacks
datastorageunit...And the backup must be reliable even if I interrupt
it, as I will run it as a cronjob on a not always online computer.

I tried rsync together with encfs like one already suggested. It
works, a bit slow, but it's ok. The problem I ran into: Renaming a big
folder in the encrypted directory (mounted) and interrupting the slow
renaimg process will cause the encrypted directory to become bigger
than all the files you see in cleartext (mounting the encfs again). I
suspect that encfs just forgets about the (interrupted) renamed files
and won't show them to you if mounted. However, the files are still
there, as the size of the unmounted encrypted directory remains the
same...(so now there is a missmatch between the size of all encrypted
files and those you see in cleartext when the encfs directory is
mounted). So running rsync with encfs might cause some extra space
used on the server if rsync is interrupted, as rsync can't delete
those files as they are not shown to any file manager etc. I don't
know what happens with interrupted delete commands (especially on
large files) or only partial transferred files, but I suspect the same
issue.

duplicity on the other hand fails to resume an interrupted backup over
ssh with: Invalid data - SHA1 hash mismatch. It's a known bug:
https://bugs.launchpad.net/duplicity/+bug/487720

so any other ideas? :)

best regards

johannes

[Martin Larsen]

Johannes S wrote:

> so any other ideas?:)
>

I am very fond of rsnapshot, but as it uses rsync for data transport, it
might have the same issues.

However, I don't quite understand what you mean with the size mismatch.
Can you make up an example?

You know it probably, but rsync has the --delete option to delete files
in the destination which are not present in the source.

Martin

[Johannes S]

Hi Martin,

thanks for your answer. rsnapshot probably will have the same issue,
as it is (in my opinion) an encfs-issue:

About the size mismatch:

say you encrypt your folder "datastorageunit" with encfs, so that the
clear text files are in the folder "decrypted"

if you rename a big folder (mine was about 4 gigabyte) in "decrypted"
and interrupt the renaming process (e.g. kill enc_fs, restart,
whatever) and remount, you'll get a result like this when you remount
the folder "decrypted":

the renamed folder still hast the same name, but a smaller size before
renaming. Also the whole folder "decrypted" is now smaller. But: the
folder "datastorageunit" holding all the encrypted files still hast
the same size.
So it seems as encfs looses "visible" or "clear text" data without
deleting any encrypted orphans in "datastorageunit"

I tested it a few times, "datastorageunit" still hat 4 Gigabytes,
while the folder "decrypted" only showed some 2,3 Gigabytes to me....

On the one hand it is good that no files are deleted just because of
an interruption, on the other hand, some files aren't accessable any
more in "cleartext" when you mount your encrypted directory...

[knnniggett]

I just ran into the same issue yesterday... after mounting my sshfs
volume, then mounting the encfs volume (both locally), I proceeded to
issue the command "mv CDROM_Images disk_images". After waiting about
a minute the command line returned w/o error as expected. In my case,
I did not interrupt the renaming process. When I went into the new
folder, it was entirely empty. Crap... that was about a 200GB
mistake. Fortunately, I keep a local backup so I haven't lost
anything other than upload time.

Until I read this post, I was thinking I caused the problem by
foolishly trying to do this operation locally rather than via a remote
ssh terminal.

I just checked my quota, and sure enough it has not decreased as one
might expect. Clearly this deserves some looking into. I'll read up
on the encfs website for any relevant info.

In the meantime, Johannes, have your tried Truecrypt as your
encryption method? The thing I don't like about it is that there is
no way to grow a Truecrypt volume (when using the latest version) once
you run out of space, but that may not be such a bad thing for others.

[Johannes S]

I opened a bug report concerning encfs:

http://code.google.com/p/encfs/issues/detail?id=61

Unfortunately I thought I could solve the duplicity issue by a bash
script checking this and that...but it's just too difficult. Pay
attention to the fact that everything will seem to be fine eben if you
interrupt duplicity (over ssh) unless you verifiy the backup or try to
restore it! Every volume duplicity gets interrupted won't be fine,
even if duplicity says it will continue. Duplicity continues...but
with a new volume, the old one left behind damaged...

About truecrpyt: Thought about that too, but what happens if a mounted
truecrpyt volume gets unclean unmounted (e.g. connection error)? My
fear is that it won't probably won't mount anymore. A big disadvantage
is that creating the truecrypt volume on the server and filling it up
with data needs double bandwidth and so a huge amount of time

[knnniggett]

I did some reading and although similar, I now realize our issues are
different.

Apparently there is a known recursive directory rename bug (https://
bugs.launchpad.net/encfs/+bug/183358) that was fixed in version
1.4.1.1.
Of course I'm using version 1.4.1, the version just prior to the bug
fix, and unfortunately newer versions are not available in the CentOS
(rpmforge) repository. Looks like I've found my reason to compile the
latest from source. I'm kicking myself for not doing this sooner.

I have starred the bug report you created as I am interested to know
what becomes of it.

[knnniggett]

This won't resolve the issue, but have you tried to execute this
command to help identify where the problem areas are?

encfsctl showcruft /path/to/encrypted/folder

On Feb 26, 5:17 pm, Johannes S <schwarz...@googlemail.com> wrote: