filer leveldb to redis?

[Andy Chen]

hi, chris:

      now i run weed filer server with leveldb, how change to redis storage?Automatic conversion? or any method?

[Chris Lu]

The storage is different and not easily interchangeable.

Btw: why switch to Redis?

Chris

On Tue, Apr 24, 2018 at 9:33 PM Andy Chen <nine5...@gmail.com> wrote:

hi, chris:

      now i run weed filer server with leveldb, how change to redis storage?Automatic conversion? or any method?

--
You received this message because you are subscribed to the Google Groups "Seaweed File System" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andy Chen]

Because leveldb has a single point of failure, I want to switch to redis as the master-slave backup.

The program I can think of:

1. Export the file list (weed export).

2. Then it is stored in redis (Analyze the file name and storage path). 

3. Restart filer server with redis.

I have seen that when using redis storage, is the file path used as the key(eg: /5ec14fe2d3c91325c71a8f0786c9e826), in the actual location of the volume as the value(eg: 3,06f33770e2), is not it? This program is feasible? Number of files already have 200 million

[Chris Lu]

weed export is iterating the files in one volume.  What you need is to iterate through the leveldb and persist each entry to redis.

The redis is a flat-namespace store, different from the level db implementation.

In leveldb, there is an auto-incremented directory id, and:

/*
The entry in level db has this format:
  key: genKey(dirId, fileName)
  value: []byte(fid)
And genKey(dirId, fileName) use first 4 bytes to store dirId, and rest for fileName
*/

It'll be good to know your use case, and how many directories are there.

Chris

To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+unsubscribe@googlegroups.com.

[王雪峰]

do we have an easy way? I also want to migrate leveldb to redis.