Meta key data is missing

[Brian Pribis]

We had a very old version of s3ql installed.  1.17.  I've been able to incrementally upgrade to 2.12 without any problems.  But trying to upgrade to 2.13 fails.

The output for upgrading looks like:

sudo bin/s3qladm --authfile /etc/s3ql/authinfo2 upgrade s3://data-boxcarcloud-com/harold

Getting file system parameters..

Using cached metadata.

I am about to update the file system to the newest revision.

You will not be able to access the file system with any older version

of S3QL after this operation.

You should make very sure that this command is not interrupted and

that no one else tries to mount, fsck or upgrade the file system at

the same time.

Please enter "yes" to continue.

> yes

Upgrading from revision 21 to 22...

..processed 146/7371 objects (2.0%, 0 bytes rewritten)..Uncaught top-level exception:

Traceback (most recent call last):

  File "bin/s3qladm", line 26, in <module>

    s3ql.adm.main(sys.argv[1:])

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/adm.py", line 94, in main

    return upgrade(options)

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/common.py", line 549, in wrapper

    return fn(*a, **kw)

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/adm.py", line 339, in upgrade

    update_obj_metadata(backend, backend_factory, db, options.threads)

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/adm.py", line 431, in update_obj_metadata

    t.join_and_raise()

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/common.py", line 503, in join_and_raise

    raise EmbeddedException(exc_info, self.name)

s3ql.common.EmbeddedException: caused by an exception in thread Thread-5.

Original/inner traceback (most recent call last): 

Traceback (most recent call last):

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/backends/comprenc.py", line 390, in _convert_legacy_metadata0

    meta_new['data'] = meta['data']

  File "/usr/local/lib/python3.4/dist-packages/dugong-3.7.5-py3.4.egg/dugong/__init__.py", line 1637, in __getitem__

    return self._store[key.lower()][1]

KeyError: 'data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/common.py", line 482, in run

    self.run_protected()

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/common.py", line 533, in run_protected

    self.target(*self.args, **self.kwargs)

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/adm.py", line 477, in upgrade_loop

    meta = backend._convert_legacy_metadata(meta)

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/backends/comprenc.py", line 341, in _convert_legacy_metadata

    meta = self._convert_legacy_metadata0(meta)

  File "/home/brian/s3/s3ql-release-2.13/src/s3ql/backends/comprenc.py", line 392, in _convert_legacy_metadata0

    raise CorruptedObjectError('meta key data is missing')

s3ql.backends.common.CorruptedObjectError: meta key data is missing

[Nikolaus Rath]

On Nov 11 2019, Brian Pribis <brian....@boxcarpress.com> wrote:
> We had a very old version of s3ql installed. 1.17. I've been able to
> incrementally upgrade to 2.12 without any problems. But trying to upgrade
> to 2.13 fails.
> The output for upgrading looks like:

[..]

> Upgrading from revision 21 to 22...
> ..processed 146/7371 objects (2.0%, 0 bytes rewritten)..Uncaught top-level

[...]

> meta_new['data'] = meta['data']
> File
> "/usr/local/lib/python3.4/dist-packages/dugong-3.7.5-py3.4.egg/dugong/__init__.py",
> line 1637, in __getitem__
> return self._store[key.lower()][1]
> KeyError: 'data'

Looks like there is an object stored that doesn't have the expected
format. It's hard to say more than that. Maybe modify the code to print
the object name? If it's just one object, you may get away with just
removing that. If it affects multiple objects, then something else is
wrong..

Best,
-Nikolaus

--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

»Time flies like an arrow, fruit flies like a Banana.«

[Brian Pribis]

On Tue, Nov 12, 2019 at 10:27 AM Nikolaus Rath <Niko...@rath.org> wrote:

> KeyError: 'data'

Looks like there is an object stored that doesn't have the expected
format. It's hard to say more than that. Maybe modify the code to print
the object name? If it's just one object, you may get away with just
removing that. If it affects multiple objects, then something else is
wrong..

But all the other conversions went well.  Like 2.10 to 2.12 went off without a hitch.  Just this one from 2.12 to 2.13 suddenly is a problem.  

I can also mount and fsck under 2.12, if that helps.

If I have to print the filename, where is the best place to do that?

Thanks for your help.