swift (using ovh) backend fails to create usable fs

[Florian Ludwig]

Hi there,

I am trying to use s3ql 2.29 with swift backend. I always end up with a corrupted backend. (logs below) Downloading the files created by mkfs and using local:// as a backend also does result in the same error message so I assume the upload part is the problem, not the downloading part.

Any ideas what could be the culprit?

Thank you very much in advance!

Florian

# mkfs.s3ql --authfile cred --force --backend-options disable-expect100,no-feature-detection swiftks://auth.cloud.ovh.net/DE1:s3ql-test

Before using S3QL, make sure to read the user's guide, especially

the 'Important Rules to Avoid Losing Data' section.

Purging existing file system data..

Please note that the new file system may appear inconsistent

for a while until the removals have propagated through the backend.

Enter encryption password: 

Confirm encryption password: 

Generating random encryption key...

Creating metadata tables...

Dumping metadata...

Dumping metadata...

..objects..

..blocks..

..inodes..

..inode_blocks..

..symlink_targets..

..names..

..contents..

..ext_attributes..

Compressing and uploading metadata...

Wrote 154 bytes of compressed metadata.

Cycling metadata backups...

Backing up old metadata...

Please store the following master key in a safe location. It allows 

decryption of the S3QL file system in case the storage objects holding 

this information get corrupted:

---BEGIN MASTER KEY---

Jkud 6wSU Y4UB BFj0 TMFU ixz0 JI2n 0Dcf RNnY jT/5 w/M=

---END MASTER KEY---

# mount.s3ql --authfile cred --backend-options disable-expect100 swiftks://auth.cloud.ovh.net/DE1:s3ql-test /test/container/

Using 8 upload threads.

Autodetected 1048520 file descriptors available for cache entries

Detected Swift features for storage.de1.cloud.ovh.net:443: copy via COPY, Bulk delete 1000 keys at a time, maximum meta value length is 255 bytes

ERROR: Backend data corrupted, or file system revision needs upgrade.

[Florian Ludwig]

Interestingly it does work without encryption (--plain).

[Nikolaus Rath]

On Jul 26 2018, Florian Ludwig <f.lu...@greyrook.com> wrote:
> Hi there,
>
> I am trying to use s3ql 2.29 with swift backend. I always end up with a
> corrupted backend. (logs below)

My guess is that the backend does not store/return object metadata
correctly.

> Downloading the files created by mkfs and
> using local:// as a backend also does result in the same error message

The file format is different, this can't work.

> so I
> assume the upload part is the problem, not the downloading part.

No, it may be either.

> Any ideas what could be the culprit?

You could try to uncomment the catching of the exception in
src/s3ql/common.py. This will not fix the problem, but may narrow it
down. Comment out the following lines:

diff --git a/src/s3ql/common.py b/src/s3ql/common.py
--- a/src/s3ql/common.py
+++ b/src/s3ql/common.py
@@ -279,9 +279,9 @@
except DanglingStorageURLError as exc:
raise QuietError(str(exc), exitcode=16)

- except CorruptedObjectError:
- raise QuietError('Backend data corrupted, or file system '
- 'revision needs upgrade.', exitcode=17)
+# except CorruptedObjectError:
+# raise QuietError('Backend data corrupted, or file system '
+# 'revision needs upgrade.', exitcode=17)

except NoSuchObject:
encrypted = False
@@ -309,16 +309,16 @@
compress = getattr(options, 'compress', ('lzma', 2))

with ComprencBackend(fs_passphrase, compress, backend) as tmp_backend:
- try:
+# try:
if encrypted:
data_pw = tmp_backend['s3ql_passphrase']
else:
data_pw = None
# Try to read metadata to detect old file system revision
tmp_backend.fetch('s3ql_metadata')
- except CorruptedObjectError:
- raise QuietError('Backend data corrupted, or file system '
- 'revision needs upgrade.', exitcode=17)
+# except CorruptedObjectError:
+# raise QuietError('Backend data corrupted, or file system '
+# 'revision needs upgrade.', exitcode=17)

return lambda: ComprencBackend(data_pw, compress, options.backend_class(options))

Best,
-Nikolaus



--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

»Time flies like an arrow, fruit flies like a Banana.«

[Florian Ludwig]

Hi,

thanks you for the pointer. After debugging through the code I found the culprit.... My --authfile did have an "fs-passphrase" entry which contained a placeholder.

:]

Now everything looks good.

[drobert...@gmail.com]

Hi,

I have been using S3QL on OVH swift storage with OK results.. my feeling is AWS S3 is more "solid" but OVH has been OK, for the price and I continue to use it - though it is hard to compare as I have quite different data in each provider.

One suggestion from OVH support that has helped in my case has been to use the option:

 --backend-options tcp-timeout=150

with fsck, mount etc