Directory structure of S3QL backend files

[Moon silvery]

Hello,

I love the design of S3QL so much that I want to use it in my homelab to replace my whole storage pool. But I have encountered a problem that when storing over 10 million files in a mount point, S3QL creates so many objects in a single bucket, without any prefix to divide these files into sub-directories. This behavior is causing my object storage backend (self-hosted MINIO) creating over 10 million files under a single directory, which is very slow on any operation that need list folder (any filesystem could not handle so many files in a single directory).
I have checked the document carefully, but I could not find any instruction on how could I change this behavior. What I want is let S3QL stores objects with prefix, like when it want to store s3ql_data_1197116, the directory structure would be s3ql_data_1/s3ql_data_1197/s3ql_data_1197116, so that any single directory would have no more than 1k files/sub-directories.
Could you please help to take a look and share some idea? I know this could be a backend specific problem, that AWS or other cloud provider do not have this issue. But I indeed appreciate the design of S3QL and want to use it in every place I could. Thanks a lot!

Best regards,
Moon

[Nikolaus Rath]

Have you considered using the local:// S3QL backend instead of going
through Minio?

Best,
-Nikolaus

--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

»Time flies like an arrow, fruit flies like a Banana.«