Error when trying to restore a file: OSError: [Errno 2] No such file or directory: 'hardlinks/[...]

[Daniel Hahler]

I am getting the following error when trying to restore a file:

% ~/src/backshift/backshift --list-backup --backup-id 1356222925.74_lenny_slash_sun-dec-23-01-35-25-2012_a3b6e3afe260f5b0 --save-directory /mnt/ds/encrypted/backup/lenny.backshift
Traceback (most recent call last):
  File "/home/daniel/src/backshift/backshift", line 29, in <module>
    top()
  File "/home/daniel/src/backshift/backshift", line 25, in top
    main.main()
  File "/home/daniel/src/backshift/main.py", line 558, in main
    perform_list_backup(repo, options.backup_id, options.starting_directory)
  File "/home/daniel/src/backshift/main.py", line 493, in perform_list_backup
    repo.list_backup(backup_id, starting_directory)
  File "/home/daniel/src/backshift/repo_mod.py", line 486, in list_backup
    hardlink_data = hardlinks_mod.Restore_hardlinks(backup_id)
  File "/home/daniel/src/backshift/hardlinks_mod.py", line 140, in __init__
    for deviceno_string in os.listdir(hardlinks_path):
OSError: [Errno 2] No such file or directory: 'hardlinks/1356222925.74_lenny_slash_sun-dec-23-01-35-25-2012_a3b6e3afe260f5b0'

Is this because --list-backups shows "None" in the last column? Can I only restore files from those backups?


Cheers,
Daniel.

[Dan Stromberg]

If backups fail part way through, they'll tend to leave behind no list of hardlinks, which puzzles subsequent restores.

But if you mkdir the directory it's complaining about (under your backshift repo), you can restore from that backup.

--list-backups shows a None completion time if a backup never finished.

HTH :)

--
Dan Stromberg

[Daniel Hahler]

I've tried that now (creating the missing hardlinks directory), and got the following error:

drwxr-xr-x daniel/daniel 0 2011-08-13 19:14 home/daniel/.local/
-rw-rw-r-- daniel/daniel 2578390 2012-01-10 17:07 home/daniel/vaIo6ZySr_k.flv
drwxr-xr-x daniel/daniel       0 2012-12-22 17:01 home/daniel/.local/share/

Traceback (most recent call last):
  File "/home/daniel/src/backshift/backshift", line 29, in <module>
    top()
  File "/home/daniel/src/backshift/backshift", line 25, in top
    main.main()
  File "/home/daniel/src/backshift/main.py", line 558, in main
    perform_list_backup(repo, options.backup_id, options.starting_directory)
  File "/home/daniel/src/backshift/main.py", line 493, in perform_list_backup
    repo.list_backup(backup_id, starting_directory)

  File "/home/daniel/src/backshift/repo_mod.py", line 487, in list_backup
    self.traverse(backup_id, starting_directory, display_tvf, hardlink_data)
  File "/home/daniel/src/backshift/repo_mod.py", line 539, in traverse
    directory_content = db_mod.open(files_path, 'rb', backend_open=compressed_file_mod.Compressed_file)
  File "/home/daniel/src/backshift/dohdbm.py", line 42, in open
    return Dohdbm(filename, flag, mode, backend_open)
  File "/home/daniel/src/backshift/dohdbm.py", line 86, in __init__
    self.file_ = self.backend_open(filename, 'rb')
  File "/home/daniel/src/backshift/compressed_file_mod.py", line 38, in __init__
    self.memory_file = stringio.StringIO(compressed_string_mod.decompress_string(data, zero_length_ok))
  File "/home/daniel/src/backshift/compressed_string_mod.py", line 87, in decompress_string
    raise ValueError('Did not get a valid compression type from compressed_data')
ValueError: Did not get a valid compression type from compressed_data

The command was:

~/src/backshift/backshift --list-backup --backup-id 1356222925.74_lenny_slash_sun-dec-23-01-35-25-2012_a3b6e3afe260f5b0 --save-directory /mnt/ds/encrypted/backup/lenny.backshift --starting-directory /home/daniel 2>&1 | less

I have also tried it without using "--starting-directory", which resulted in the same error.

Restoring files appears to be not that easy with backshift..


Thanks,
Daniel.

[Dan Stromberg]

I've seen errors similar to this when I had chunks that were of zero length.

You, on the other hand, appear to have a files database (dohdbm) that is zero length.

In backshift, both chunks and files ("files" are directory metadata really) are saved as a compressed file, where the first byte is an indicator of what type of compression was used.

If a file is 0 length, it naturally messes with decompression, as there's no indicator of what type of decompression to use or any data there to decompress.  It's an example of something that should never exist in normal operation of the system - unless a backup was terminated early, or a filesystem became corrupted.

If you absolutely must restore from a partial save, I'd suggest something like:

find /mnt/ds/encrypted/backup/lenny.backshift -type f -size 0 -print0 | xargs -0 rm -fv

This should eliminate your 0 length files, which have no data in them anyway.  You won't get back all of your data, because the files metadata for the directory or directories in question won't have pointers to where it existed, but it should get you back most of what you backed up.

However, it's best if you can find another backup-id that was successfully completed, if possible, and restore from that instead.  It's always challenging getting data back from a crashed or not-yet-completed backup (shown as a None completion time in backshift --list-backups).

HTH

--
Dan Stromberg

[Daniel Hahler]

I have found an older, completed backup and could restore from there.

But isn't it possible to restore a single file only, without producing the tar for the whole parent directory?
I want to get a single file out of my home directory. Do I really have to restore everything below it (temporarily) to get the file I need from the tar?

btw: one of the great things with rdiff-backup is that there's rdiff-backup-fs (a fuse filesystem that allows to browse your backups by date).
Is something like this planned / possible with backshift?


Cheers,
Daniel.

[Dan Stromberg]

On Sun, Jan 6, 2013 at 2:37 PM, Daniel Hahler <google-gr...@thequod.de> wrote:

I have found an older, completed backup and could restore from there.

Ah, good.

 

But isn't it possible to restore a single file only?

Yes and no.

backshift will produce a tar archive to stdout for the entire hierarchy, but you can "backshift --produce-tar | tar xvf - some.file" just the part you want via a pipeline.  So backshift produces a bigger tar archive than you need, but you don't actually need to write more than what you really require to disk.

 

I want to get a single file out of my home directory. Do I really have to restore everything below it (temporarily) to get the file I need from the tar?

I agree, that's an annoyance. 

btw: one of the great things with rdiff-backup is that there's rdiff-backup-fs (a fuse filesystem that allows to browse your backups by date).
Is something like this planned / possible with backshift?

I've been thinking about doing this for a long time - since before backshift was started.

I think it'd be a very desirable feature. 

--
Dan Stromberg

[Daniel Hahler]

Well, the produce-tar command now only resulted in a symlink-like file:

% ~/src/backshift/backshift --backup-id 1355584195.88_lenny_slash_sat-dec-15-16-09-55-2012_f9965ce076b92662 --save-directory /mnt/ds/encrypted/backup/lenny.backshift --starting-directory /home/daniel --produce-tar | tar xvf - home/daniel/.zsh_history > zsh_history_from_backup
% cat zsh_history_from_backup
home/daniel/.zsh_history

The file only contains the original filename?!


Regards,
Daniel.

[Dan Stromberg]

On Mon, Jan 7, 2013 at 2:41 PM, Daniel Hahler <google-gr...@thequod.de> wrote:

Well, the produce-tar command now only resulted in a symlink-like file:

% ~/src/backshift/backshift --backup-id 1355584195.88_lenny_slash_sat-dec-15-16-09-55-2012_f9965ce076b92662 --save-directory /mnt/ds/encrypted/backup/lenny.backshift --starting-directory /home/daniel --produce-tar | tar xvf - home/daniel/.zsh_history > zsh_history_from_backup
% cat zsh_history_from_backup
home/daniel/.zsh_history

The file only contains the original filename?! 

You probably have a /home/daniel/home/daniel/.zsh_history .

It's usually best to cd / before doing an extraction, though not always.

tar's stdout is just a list of files, but the files should be getting created, just not in the expected spot.

--
Dan Stromberg

[Daniel Hahler]

You probably have a /home/daniel/home/daniel/.zsh_history .

It's usually best to cd / before doing an extraction, though not always.

tar's stdout is just a list of files, but the files should be getting created, just not in the expected spot.

Of course.

It worked as expected - and it's good that I did not "cd /" before, because I wanted to merge the backup with the current one.


Thanks for your support,
Daniel.