Migrate from MogileFS to S3

Roger Tong

unread,

Jul 8, 2020, 6:06:39 AM7/8/20

to mogile

Hi,

Our project is using MogileFS for a couple of years. Actually, we are now using a mix of MogileFS and Amazon S3 to store user uploaded files.

To make it simple, we plan to store all files in one place(Amazon S3) which means we need to migrate current files on MogileFS to S3.

Is there any recommendation/tools to finish such migration?

Thanks and best regards,

Roger

dormando

unread,

Jul 8, 2020, 2:57:47 PM7/8/20

to mogile

There're tools/tooling under https://github.com/mogilefs/MogileFS-Utils
which can iterate through all of the keys and/or fids in storage.

So you could build something based off of that.

feel like I wrote an rsync-like thing at some point but I Don't see it.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "mogile" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to mogile+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/mogile/7439384a-f0b6-4f24-ac7c-231c4ea99c5fo%40googlegroups.com.
>
>

Roger Tong

unread,

Jul 9, 2020, 2:03:44 AM7/9/20

to mogile

Thanks for your quick response. We'll try it to build our migration tool.

On Thursday, July 9, 2020 at 2:57:47 AM UTC+8, Dormando wrote:

There're tools/tooling under https://github.com/mogilefs/MogileFS-Utils
which can iterate through all of the keys and/or fids in storage.

So you could build something based off of that.

feel like I wrote an rsync-like thing at some point but I Don't see it.

On Wed, 8 Jul 2020, Roger Tong wrote:

> Hi,
> Our project is using MogileFS for a couple of years. Actually, we are now using a mix of MogileFS and Amazon S3 to store user uploaded files.
> To make it simple, we plan to store all files in one place(Amazon S3) which means we need to migrate current files on MogileFS to S3.
> Is there any recommendation/tools to finish such migration?
>
> Thanks and best regards,
> Roger
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "mogile" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to mog...@googlegroups.com.

Reply all

Reply to author

Forward