Active Storage Mirror Service

[Hany Elbanna]

in: http://edgeguides.rubyonrails.org/active_storage_overview.html#mirror-service

"You can start mirroring to the new service, copy existing files from the old service to the new, then go all-in on the new service."

I have a bucket on S3 and try to copy its content to local to setup an on-premise server.

Active storage adds files to S3 bucket direct with names like "BvWAx6tgrodtsYMB6oxKSTRz"
and its variant in variants folder
then BvWAx6tgrodtsYMB6oxKSTRz folder with the same name
then the variant file with new name like "cf64579b5ef85d267d1782150b3e470a308a085a4763b"

But in local it creates folder with first two characters from the file name "Bv"
inside it another folder with the second two characters "WA"
then the file with its name "BvWAx6tgrodtsYMB6oxKSTRz"

and its and its variant in "va" folder
inside it "ri" folder
inside it "variants" folder
inside it a folder with the file name "BvWAx6tgrodtsYMB6oxKSTRz"
then the variant file with variant file name "cf64579b5ef85d267d1782150b3e470a308a085a4763b"

How can I make the local structure like S3 structure to let me copy files after some time to on-premise server.

I see it done automatic if I set Mirror Service, but I need it for files before mirror.

Thanks

[Walter Lee Davis]

I think you're going to get this behavior any time you save the files on a disk (rather than an object store, like S3). Disks suffer tremendous performance degradation when you put more than a few thousand items in any "folder". So any local-file storage scheme will use this sort of segmentation scheme to ensure that each folder has comparatively few children. This goes all the way back to Paperclip (the original file store).

It seems as though ActiveStorage knows how to translate the path back to its S3 version, so why do you need it to be that way on your disk now? If you have a lot of assets, this will only end badly for you if you try to replicate on a disk what the S3 service does with a database. (I would be willing to bet that the actual S3 file storage is implemented in a similar "sharing" fashion, it's just the virtual file path that you appear to see that has everything in one bucket.)

Walter

>
>
>
> Thanks
>
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to rubyonra...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/ad5d3368-15d6-4ff3-ae13-c93b8db1e4e6%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.