Recovery s3ql metadata v 3.3.2

[Denzil Barrios]

Please, can you help me. For some strange reason, the metadata was lost, I have the local database and the cache, and the files in the bucket, but I can't do a fsck.s3ql to correct and mount. I have this installed on an ubuntu 20.04. I tried to recover it with version 5.11, but it tells me that it does not recognize the bucket file system

This is my log:

2023-07-24 09:51:10.785 3821413:MainThread s3ql.fsck.main: Checking DB integrity...
2023-07-24 09:51:13.681 3821413:MainThread s3ql.fsck.check: Creating temporary extra indices...
2023-07-24 09:51:14.074 3821413:MainThread s3ql.fsck.check_lof: Checking lost+found...
2023-07-24 09:51:14.092 3821413:MainThread s3ql.fsck.check_cache: Checking for dirty cache objects...
2023-07-24 09:51:14.092 3821413:MainThread s3ql.fsck.check_names_refcount: Checking names (refcounts)...
2023-07-24 09:51:14.417 3821413:MainThread s3ql.fsck.check_contents_name: Checking contents (names)...
2023-07-24 09:51:14.531 3821413:MainThread s3ql.fsck.check_contents_inode: Checking contents (inodes)...
2023-07-24 09:51:14.589 3821413:MainThread s3ql.fsck.check_contents_parent_inode: Checking contents (parent inodes)...
2023-07-24 09:51:14.650 3821413:MainThread s3ql.fsck.check_objects_refcount: Checking objects (reference counts)...
2023-07-24 09:51:14.776 3821413:MainThread s3ql.fsck.check_objects_id: Checking objects (backend)...
2023-07-24 09:51:33.512 3821413:MainThread s3ql.fsck.check_objects_size: Checking objects (sizes)...
2023-07-24 09:51:33.522 3821413:MainThread s3ql.fsck.check_blocks_obj_id: Checking blocks (referenced objects)...
2023-07-24 09:51:33.540 3821413:MainThread s3ql.fsck.check_blocks_refcount: Checking blocks (refcounts)...
2023-07-24 09:51:33.692 3821413:MainThread s3ql.fsck.check_blocks_checksum: Checking blocks (checksums)...
2023-07-24 09:51:33.692 3821413:MainThread s3ql.fsck.check_inode_blocks_block_id: Checking inode-block mapping (blocks)...
2023-07-24 09:51:33.722 3821413:MainThread s3ql.fsck.check_inode_blocks_inode: Checking inode-block mapping (inodes)...
2023-07-24 09:51:33.770 3821413:MainThread s3ql.fsck.check_inodes_refcount: Checking inodes (refcounts)...
2023-07-24 09:51:34.099 3821413:MainThread s3ql.fsck.check_inodes_size: Checking inodes (sizes)...
2023-07-24 09:51:34.339 3821413:MainThread s3ql.fsck.check_ext_attributes_name: Checking extended attributes (names)...
2023-07-24 09:51:34.339 3821413:MainThread s3ql.fsck.check_ext_attributes_inode: Checking extended attributes (inodes)...
2023-07-24 09:51:34.339 3821413:MainThread s3ql.fsck.check_symlinks_inode: Checking symlinks (inodes)...
2023-07-24 09:51:34.339 3821413:MainThread s3ql.fsck.check_loops: Checking directory reachability...
2023-07-24 09:51:42.716 3821413:MainThread s3ql.fsck.check_unix: Checking unix conventions...
2023-07-24 09:51:46.521 3821413:MainThread s3ql.fsck.check_foreign_keys: Checking referential integrity...
2023-07-24 09:51:46.794 3821413:MainThread s3ql.fsck.check: Dropping temporary indices...
2023-07-24 09:51:46.817 3821413:MainThread s3ql.metadata.dump_and_upload_metadata: Dumping metadata...
2023-07-24 09:51:46.817 3821413:MainThread s3ql.metadata.dump_metadata: ..objects..
2023-07-24 09:51:46.889 3821413:MainThread s3ql.metadata.dump_metadata: ..blocks..
2023-07-24 09:51:47.029 3821413:MainThread s3ql.metadata.dump_metadata: ..inodes..
2023-07-24 09:51:47.558 3821413:MainThread s3ql.metadata.dump_metadata: ..inode_blocks..
2023-07-24 09:51:47.665 3821413:MainThread s3ql.metadata.dump_metadata: ..symlink_targets..
2023-07-24 09:51:47.666 3821413:MainThread s3ql.metadata.dump_metadata: ..names..
2023-07-24 09:51:47.794 3821413:MainThread s3ql.metadata.dump_metadata: ..contents..
2023-07-24 09:51:48.219 3821413:MainThread s3ql.metadata.dump_metadata: ..ext_attributes..
2023-07-24 09:51:48.220 3821413:MainThread s3ql.metadata.upload_metadata: Compressing and uploading metadata...
2023-07-24 09:51:52.569 3821413:MainThread s3ql.metadata.upload_metadata: Wrote 13.9 MiB of compressed metadata.
2023-07-24 09:51:52.570 3821413:MainThread s3ql.metadata.upload_metadata: Cycling metadata backups...
2023-07-24 09:51:52.570 3821413:MainThread s3ql.metadata.cycle_metadata: Backing up old metadata...
2023-07-24 09:52:14.355 3821413:MainThread s3ql.fsck.main: Cleaning up local metadata...
2023-07-24 09:52:15.151 3821413:MainThread s3ql.fsck.main: Completed fsck of s3c://ewr1.vultrobjects.com:443/moodle-data-simatecgt/
2023-08-06 23:35:50.208 1645:MainThread s3ql.backends.common.wrapped: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.open_read (attempt 3)...
2023-08-06 23:36:10.324 1645:MainThread s3ql.backends.common.wrapped: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.open_read (attempt 4)...
2023-08-06 23:36:30.505 1645:MainThread s3ql.backends.common.wrapped: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.open_read (attempt 5)...
2023-08-06 23:36:50.860 1645:MainThread s3ql.backends.common.wrapped: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.open_read (attempt 6)...
2023-08-06 23:37:11.624 1645:MainThread s3ql.backends.common.wrapped: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.open_read (attempt 7)...
2023-08-06 23:37:33.563 1645:MainThread s3ql.backends.common.wrapped: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.open_read (attempt 8)...
2023-08-06 23:37:56.807 1645:MainThread s3ql.backends.common.wrapped: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.open_read (attempt 9)...
2023-08-06 23:38:24.411 1645:MainThread s3ql.backends.common.wrapped: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying 

later

2023-08-08 06:36:15.331 14271:MainThread s3ql.backends.common.wrapped: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.open_read (attempt 228)...
2023-08-08 06:42:43.501 14271:MainThread s3ql.fsck.main: Starting fsck of s3c://ewr1.vultrobjects.com:443/moodle-data-simatecgt/
2023-08-08 06:42:43.865 14271:MainThread s3ql.fsck.main: Using cached metadata.
2023-08-08 06:42:43.881 14271:MainThread s3ql.fsck.main: Remote metadata is outdated.
2023-08-08 06:42:43.881 14271:MainThread s3ql.fsck.main: Checking DB integrity...
2023-08-08 06:42:48.800 14271:MainThread s3ql.fsck.check: Creating temporary extra indices...
2023-08-08 06:42:49.163 14271:MainThread s3ql.fsck.check_lof: Checking lost+found...
2023-08-08 06:42:49.172 14271:MainThread s3ql.fsck.log_error: Cleaning up interrupted upload of object 5039817
2023-08-08 06:42:49.172 14271:MainThread s3ql.fsck.log_error: Cleaning up interrupted upload of object 5039818
2023-08-08 06:42:49.173 14271:MainThread s3ql.fsck.log_error: Cleaning up interrupted upload of object 5039819
2023-08-08 06:42:49.173 14271:MainThread s3ql.fsck.log_error: Cleaning up interrupted upload of object 5039820
2023-08-08 06:42:49.173 14271:MainThread s3ql.fsck.log_error: Cleaning up interrupted upload of object 5039821
2023-08-08 06:42:49.173 14271:MainThread s3ql.fsck.log_error: Cleaning up interrupted upload of object 5039822
2023-08-08 06:42:49.173 14271:MainThread s3ql.fsck.log_error: Cleaning up interrupted upload of object 5039823
2023-08-08 06:42:49.173 14271:MainThread s3ql.fsck.log_error: Cleaning up interrupted upload of object 5039824
2023-08-08 06:42:49.173 14271:MainThread s3ql.fsck.log_error: Cleaning up interrupted upload of object 5039825
2023-08-08 06:42:49.181 14271:MainThread s3ql.fsck.check_cache: Checking for dirty cache objects...
2023-08-08 06:42:49.350 14271:MainThread s3ql.fsck.log_error: Writing dirty block 0 of inode 10719722 to backend
2023-08-08 06:42:50.160 14271:MainThread s3ql.fsck.log_error: Writing dirty block 0 of inode 10719399 to backend
2023-08-08 06:42:51.719 14271:MainThread s3ql.fsck.log_error: Writing dirty block 0 of inode 10719811 to backend
2023-08-08 06:42:53.550 14271:MainThread s3ql.fsck.check: Dropping temporary indices...
2023-08-08 06:42:53.572 14271:MainThread root.excepthook: Uncaught top-level exception:
Traceback (most recent call last):
  File "/usr/bin/fsck.s3ql", line 11, in <module>
    load_entry_point('s3ql==3.3.2', 'console_scripts', 'fsck.s3ql')()
  File "/usr/lib/s3ql/s3ql/fsck.py", line 1279, in main
    fsck.check(check_cache)
  File "/usr/lib/s3ql/s3ql/fsck.py", line 83, in check
    self.check_cache(check_cache == 'keep')
  File "/usr/lib/s3ql/s3ql/fsck.py", line 207, in check_cache
    raise RuntimeError('Strange file in cache directory: %s' % filename)
RuntimeError: Strange file in cache directory: 10195866-0.tmp

Thank in advance

[Nikolaus Rath]

On Aug 10 2023, Denzil Barrios <mcd...@gmail.com> wrote:
> Please, can you help me. For some strange reason, the metadata was
> lost,

What do you mean with "the metadata was lost"? I see no indication of
that in the logs that you provided.

> I
> have the local database and the cache, and the files in the bucket, but I
> can't do a fsck.s3ql to correct and mount. I have this installed on an
> ubuntu 20.04. I tried to recover it with version 5.11, but it tells me that

> it does not recognize the bucket file system.

Yes, you need to use your original S3QL version for any recovery
attempt.

>
> This is my log:
>
> 2023-07-24 09:51:10.785 3821413:MainThread s3ql.fsck.main: Checking DB
> integrity...

[...]

> 2023-08-08 06:36:15.331 14271:MainThread s3ql.backends.common.wrapped:
> Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying
> Backend.open_read (attempt 228)...

Which backend are you using? Basically, there's nothing S3QL can do if
the server does not respond.

> 2023-08-08 06:42:43.501 14271:MainThread s3ql.fsck.main: Starting fsck of
> s3c://ewr1.vultrobjects.com:443/moodle-data-simatecgt/

It looks like there is something missing between the last log entry and
this one. Did you abort and start fsck anew?

> 2023-08-08 06:42:53.572 14271:MainThread root.excepthook: Uncaught
> top-level exception:
> Traceback (most recent call last):
> File "/usr/bin/fsck.s3ql", line 11, in <module>
> load_entry_point('s3ql==3.3.2', 'console_scripts', 'fsck.s3ql')()
> File "/usr/lib/s3ql/s3ql/fsck.py", line 1279, in main
> fsck.check(check_cache)
> File "/usr/lib/s3ql/s3ql/fsck.py", line 83, in check
> self.check_cache(check_cache == 'keep')
> File "/usr/lib/s3ql/s3ql/fsck.py", line 207, in check_cache
> raise RuntimeError('Strange file in cache directory: %s' % filename)
> RuntimeError: Strange file in cache directory: 10195866-0.tmp

I'd suggest to just move this file out of the way and try again.


Best,
-Nikolaus

[Denzil Barrios]

Hi, thanks for your help,

I copied exactly everything to another bucket, because this one has connectivity problems and I moved the .tmp file out of the cache directory, but when I run the fsck.s3ql command, it shows that the metadata does not exist:

# fsck.s3ql --cachedir /var/cache/s3ql --authfile /etc/s3ql.authinfo s3c://ewr1.vultrobjects.com:443/moodle-data-simatecgt/

ERROR: Uncaught top-level exception:

Traceback (most recent call last):

  File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 320, in open_read
    resp = self._do_request('GET', '/%s%s' % (self.prefix, key))
  File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 528, in _do_request
    self._parse_error_response(resp)
  File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 562, in _parse_error_response
    raise get_S3Error(tree.findtext('Code'), tree.findtext('Message'), resp.headers)
s3ql.backends.s3c.NoSuchKeyError: NoSuchKey: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/fsck.s3ql", line 11, in <module>
    load_entry_point('s3ql==3.3.2', 'console_scripts', 'fsck.s3ql')()

  File "/usr/lib/s3ql/s3ql/fsck.py", line 1141, in main
    backend = get_backend(options)
  File "/usr/lib/s3ql/s3ql/common.py", line 248, in get_backend
    return get_backend_factory(options)()
  File "/usr/lib/s3ql/s3ql/common.py", line 323, in get_backend_factory
    tmp_backend.fetch('s3ql_metadata')
  File "/usr/lib/s3ql/s3ql/backends/common.py", line 293, in fetch
    return self.perform_read(do_read, key)
  File "/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped
    return method(*a, **kw)
  File "/usr/lib/s3ql/s3ql/backends/common.py", line 256, in perform_read
    fh = self.open_read(key)
  File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 194, in open_read
    fh = self.backend.open_read(key)
  File "/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped
    return method(*a, **kw)
  File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 322, in open_read
    raise NoSuchObject(key)
s3ql.backends.common.NoSuchObject: Backend does not have anything stored under key 's3ql_metadata'

There is no metadata, because the file does not exist in the bucket, and I need to recreate it again to retrieve the information from the bucket.
I don't know how to create a new metadata from the database, the cache, and the files in the bucket.

Thank you very much for the help

[Nikolaus Rath]

On Aug 11 2023, Denzil Barrios <mcd...@gmail.com> wrote:
> Hi, thanks for your help,
>
> I copied exactly everything to another bucket, because this one has
> connectivity problems and I moved the .tmp file out of the cache directory,
> but when I run the fsck.s3ql command, it shows that the metadata does not
> exist:
>
> # fsck.s3ql --cachedir /var/cache/s3ql --authfile /etc/s3ql.authinfo s3c://
> ewr1.vultrobjects.com:443/moodle-data-simatecgt/
>
> ERROR: Uncaught top-level exception:
> Traceback (most recent call last):
> File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 320, in open_read
> resp = self._do_request('GET', '/%s%s' % (self.prefix, key))
> File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 528, in _do_request
> self._parse_error_response(resp)
> File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 562, in
> _parse_error_response
> raise get_S3Error(tree.findtext('Code'), tree.findtext('Message'),
> resp.headers)
> s3ql.backends.s3c.NoSuchKeyError: NoSuchKey: None

That's quite obviously a different problem than you had before, because
fsck.s3ql made much more progress than:

>> > 2023-08-08 06:42:53.572 14271:MainThread root.excepthook: Uncaught
>> > top-level exception:
>> > Traceback (most recent call last):
>> > File "/usr/bin/fsck.s3ql", line 11, in <module>
>> > load_entry_point('s3ql==3.3.2', 'console_scripts', 'fsck.s3ql')()
>> > File "/usr/lib/s3ql/s3ql/fsck.py", line 1279, in main
>> > fsck.check(check_cache)
>> > File "/usr/lib/s3ql/s3ql/fsck.py", line 83, in check
>> > self.check_cache(check_cache == 'keep')
>> > File "/usr/lib/s3ql/s3ql/fsck.py", line 207, in check_cache
>> > raise RuntimeError('Strange file in cache directory: %s' % filename)
>> > RuntimeError: Strange file in cache directory: 10195866-0.tmp
>>
>> I'd suggest to just move this file out of the way and try again.

I'd suggest you run fsck.s3ql just as before, after having moved the
file out of the way.


Also, you did not answer any of the questions from my previous email,
which is making it difficut to help you.

Best,
-Nikolaus

[Denzil Barrios]

Thanks for your help, here I give you the asnswer of your cuestions:

1. What do you mean with "the metadata was lost"? I see no indication of that in the logs that you provided.

*** The fist log was about the last time my Vultr bucket works, it was only to give you some information, but in the new one I haven't the metadata file, it doesn't exist. And this is the s3ql response:

root@lms-srv02:~# fsck.s3ql --cachedir /var/cache/s3ql --authfile /etc/s3ql.authinfo s3c://sjc1.vultrobjects.com:443/moodle-data-simatecgt
ERROR: Uncaught top-level exception:

Traceback (most recent call last):

  File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 320, in open_read
    resp = self._do_request('GET', '/%s%s' % (self.prefix, key))
  File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 528, in _do_request
    self._parse_error_response(resp)
  File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 562, in _parse_error_response
    raise get_S3Error(tree.findtext('Code'), tree.findtext('Message'), resp.headers)
s3ql.backends.s3c.NoSuchKeyError: NoSuchKey: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/fsck.s3ql", line 11, in <module>
    load_entry_point('s3ql==3.3.2', 'console_scripts', 'fsck.s3ql')()

  File "/usr/lib/s3ql/s3ql/fsck.py", line 1141, in main
    backend = get_backend(options)
  File "/usr/lib/s3ql/s3ql/common.py", line 248, in get_backend
    return get_backend_factory(options)()
  File "/usr/lib/s3ql/s3ql/common.py", line 323, in get_backend_factory
    tmp_backend.fetch('s3ql_metadata')
  File "/usr/lib/s3ql/s3ql/backends/common.py", line 293, in fetch
    return self.perform_read(do_read, key)
  File "/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped
    return method(*a, **kw)
  File "/usr/lib/s3ql/s3ql/backends/common.py", line 256, in perform_read
    fh = self.open_read(key)
  File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 194, in open_read
    fh = self.backend.open_read(key)
  File "/usr/lib/s3ql/s3ql/backends/common.py", line 108, in wrapped
    return method(*a, **kw)
  File "/usr/lib/s3ql/s3ql/backends/s3c.py", line 322, in open_read
    raise NoSuchObject(key)
s3ql.backends.common.NoSuchObject: Backend does not have anything stored under key 's3ql_metadata'

2. Which backend are you using? Basically, there's nothing S3QL can do if the server does not respond.

*** I changed my last backend just because it failed for 3 days and when I had access the provider said that I have to change the information to a new bucket in another location. Thats why I copied everything to my new backend and now it is   s3c://sjc1.vultrobjects.com:443/xxx also I rename cache directories, database and params with the name of the new bucket.

3. It looks like there is something missing between the last log entry and this one. Did you abort and start fsck anew?

*** I didn't have access at about 3 days I was trying to connect with fsck several times but the server not respond, but now in the new bucket when I execute the fsck command it show me:

 "s3ql.backends.common.NoSuchObject: Backend does not have anything stored under key 's3ql_metadata'" . 

I searched this file several times but it doesn't exist, I believe it was lost because of the fsck conexion errors,  thats why I need to recreate a metadata file.

I hope with this information you understand better my situation, I dont  have so much time to make it works. I really need your help.

Thanks in advance. 

El viernes, 11 de agosto de 2023 a la(s) 03:02:52 UTC-6, Nikolaus Rath escribió:

[Nikolaus Rath]

On Aug 11 2023, Denzil Barrios <mcd...@gmail.com> wrote:
> 2. Which backend are you using? Basically, there's nothing S3QL can do if
> the server does not respond.
> *** I changed my last backend just because it failed for 3 days and when I
> had access the provider said that I have to change the information to a new
> bucket in another location. Thats why I copied everything to my new backend
> and now it is s3c://sjc1.vultrobjects.com:443/xxx also I rename cache
> directories, database and params with the name of the new bucket.

This is most likely where your problems are coming from. How did you
copy the data from the old bucket into the new bucket? Is there any
chance you can just work with the old bucket again?

> I searched this file several times but it doesn't exist, I believe it was
> lost because of the fsck conexion errors,

This seems unlikely. Are you sure this was copied from the old into the
new bucket?

>> > 2023-08-08 06:42:53.572 14271:MainThread root.excepthook: Uncaught
>> > top-level exception:
>> > Traceback (most recent call last):
>> > File "/usr/bin/fsck.s3ql", line 11, in <module>
>> > load_entry_point('s3ql==3.3.2', 'console_scripts', 'fsck.s3ql')()
>> > File "/usr/lib/s3ql/s3ql/fsck.py", line 1279, in main
>> > fsck.check(check_cache)
>> > File "/usr/lib/s3ql/s3ql/fsck.py", line 83, in check
>> > self.check_cache(check_cache == 'keep')
>> > File "/usr/lib/s3ql/s3ql/fsck.py", line 207, in check_cache
>> > raise RuntimeError('Strange file in cache directory: %s' % filename)
>> > RuntimeError: Strange file in cache directory: 10195866-0.tmp
>>
>> I'd suggest to just move this file out of the way and try again.

Did you try this?


Best,
-Nikolaus

[Denzil Barrios]

Hello, I answer you between the lines +++++

El viernes, 11 de agosto de 2023 a la(s) 12:39:51 UTC-6, Nikolaus Rath escribió:

On Aug 11 2023, Denzil Barrios <mcd...@gmail.com> wrote:
> 2. Which backend are you using? Basically, there's nothing S3QL can do if
> the server does not respond.
> *** I changed my last backend just because it failed for 3 days and when I
> had access the provider said that I have to change the information to a new
> bucket in another location. Thats why I copied everything to my new backend
> and now it is s3c://sjc1.vultrobjects.com:443/xxx also I rename cache
> directories, database and params with the name of the new bucket.

This is most likely where your problems are coming from. How did you
copy the data from the old bucket into the new bucket? Is there any
chance you can just work with the old bucket again?

++++++

I accessed the old bucket using a tool called s3browser, downloaded everything and uploaded it to the new bucket.
When I try to access the old bucket, it tells me a 403 forbidden error, it's because the bucket is bad and it doesn't allow me to connect with s3ql.

It is not possible for me to use the old bucket, because the access is unstable with any tool like s3browser, it takes me a long time to download all the content. 

++++++

 

> I searched this file several times but it doesn't exist, I believe it was
> lost because of the fsck conexion errors,

This seems unlikely. Are you sure this was copied from the old into the
new bucket?

++++++

Yes, I uploaded everything to the new bucket, from the old bucket

My problem is, I don't have the s3ql_metadata file locally or remotely in any bucket, and I can't readably access my information in the new bucket,  all the objects are there, but, show me the metadata doesn't exist in the backend when use fsck.s3ql

++++++

 

>> > 2023-08-08 06:42:53.572 14271:MainThread root.excepthook: Uncaught
>> > top-level exception:
>> > Traceback (most recent call last):
>> > File "/usr/bin/fsck.s3ql", line 11, in <module>
>> > load_entry_point('s3ql==3.3.2', 'console_scripts', 'fsck.s3ql')()
>> > File "/usr/lib/s3ql/s3ql/fsck.py", line 1279, in main
>> > fsck.check(check_cache)
>> > File "/usr/lib/s3ql/s3ql/fsck.py", line 83, in check
>> > self.check_cache(check_cache == 'keep')
>> > File "/usr/lib/s3ql/s3ql/fsck.py", line 207, in check_cache
>> > raise RuntimeError('Strange file in cache directory: %s' % filename)
>> > RuntimeError: Strange file in cache directory: 10195866-0.tmp
>>
>> I'd suggest to just move this file out of the way and try again.

Did you try this?

++++++++

Yes, I tried, but it was when the old bucket did allow access, the way I solved it was by moving the .tmp somewhere else, and when I tried again use fsck.s3ql, the old bucket was inaccessible (forbidden 403), That's why I had to transfer information between buckets

+++++++++ 

Do you think it is possible to rebuild the s3ql_metadata to upload it to the backend (bucket)?

I really appreciate the time you take to help me.

Blessings,

- Denzil

Best,
-Nikolaus

[Nikolaus Rath]

On Aug 11 2023, Denzil Barrios <mcd...@gmail.com> wrote:
> On Aug 11 2023, Denzil Barrios <mcd...@gmail.com> wrote:
>> 2. Which backend are you using? Basically, there's nothing S3QL can do if
>> the server does not respond.
>> *** I changed my last backend just because it failed for 3 days and when
> I
>> had access the provider said that I have to change the information to a
> new
>> bucket in another location. Thats why I copied everything to my new
> backend
>> and now it is s3c://sjc1.vultrobjects.com:443/xxx also I rename cache
>> directories, database and params with the name of the new bucket.
>
> This is most likely where your problems are coming from. How did you
> copy the data from the old bucket into the new bucket? Is there any
> chance you can just work with the old bucket again?
>
>
> ++++++
> I accessed the old bucket using a tool called s3browser, downloaded
> everything and uploaded it to the new bucket.
> When I try to access the old bucket, it tells me a 403 forbidden error,
> it's because the bucket is bad and it doesn't allow me to connect with s3ql.
> It is not possible for me to use the old bucket, because the access is
> unstable with any tool like s3browser, it takes me a long time to download
> all the content.
> ++++++

Did s3browser also copy the metadata for each object?

>> I searched this file several times but it doesn't exist, I believe it was
>> lost because of the fsck conexion errors,
>
> This seems unlikely. Are you sure this was copied from the old into the
> new bucket?
>
> ++++++
> Yes, I uploaded everything to the new bucket, from the old bucket
>
> My problem is, I don't have the s3ql_metadata file locally or remotely in
> any bucket, and I can't readably access my information in the new bucket,
> all the objects are there, but, show me the metadata doesn't exist in the
> backend when use fsck.s3ql

How about any of the metadata backups, s3ql_metadata_*? If you have
those, you could just re-upload that as s3ql_metadata.

> Do you think it is possible to rebuild the s3ql_metadata to upload it to
> the backend (bucket)?

Yes. You would just need to change the fsck.s3ql code to not attempt to look
for the object. The object will then be created from the local database.

However, I do not understand how this object can just "go missing". I
expect something else went wrong, and therefore just making this object
re-appear will not help much. For example, I wouldn't be surprised if a
large number of data objects was also inaccessible.

Best,
-Nikolaus