Releasing New Filer Soon!

ChrisLu

unread,

May 27, 2018, 2:33:19 PM5/27/18

to Seaweed File System

I am about to release a new filer implementation in the coming days.

This moves SeaweedFS from a distributed file object store closer to a real and scalable file system! And still much simpler than other file systems!

The new design has these improvements:

1. much simpler to implement for different storage

2. added with more meta data on files and directories

3. added directory listing in both Cassandra and Redis

4. “weed mount” can work both read and write

Breaking changes:

1. the filer storage will not be compatible with old filer store.

2. configuration is moved to a "filer.toml" file under current directory, or "$HOME/.seaweedfs/", or "/etc/seaweedfs/" folder.

The changes are in this branch:

https://github.com/chrislusf/seaweedfs/tree/filer2_development

The attached is an example filer.toml file

Chris

# A sample TOML config file for SeaweedFS filer store

[memory]
# local in memory, mostly for testing purpose
enabled = false

[leveldb]
# local on disk, mostly for simple single-machine setup, fairly scalable
enabled = false
dir = "."              # directory to store level db files

####################################################
# multiple filers on shared storage, fairly scalable
####################################################

[mysql]
#
# need to choose or create a database.
# need to manually create a table "filemeta".
#
# CREATE TABLE IF NOT EXISTS filemeta (
#   dirhash     BIGINT        COMMENT 'first 64 bits of MD5 hash value of directory field',
#   name        VARCHAR(1000) COMMENT 'directory or file name',
#   directory   VARCHAR(4096) COMMENT 'full path to parent directory',
#   meta        BLOB,
#   PRIMARY KEY (dirhash, name)
# ) DEFAULT CHARSET=utf8;
#
enabled = true
hostname = "localhost"
port = 3306
username = "root"
password = ""
database = ""              # create or use an existing database
connection_max_idle = 2
connection_max_open = 100

[postgres]
# CREATE TABLE IF NOT EXISTS filemeta (
#   dirhash     BIGINT,
#   name        VARCHAR(1000),
#   directory   VARCHAR(4096),
#   meta        bytea,
#   PRIMARY KEY (dirhash, name)
# );
enabled = false
hostname = "localhost"
port = 5432
username = "postgres"
password = ""
database = ""              # create or use an existing database
sslmode = "disable"
connection_max_idle = 100
connection_max_open = 100

[cassandra]
# CREATE TABLE filemeta (
#    directory varchar,
#    name varchar,
#    meta blob,
#    PRIMARY KEY (directory, name)
# ) WITH CLUSTERING ORDER BY (name ASC);
enabled = false
keyspace="seaweedfs"
hosts=[
   "localhost:9042",
]

[redis]
enabled = true
address  = "localhost:6379"
password = ""
db = 0

Qin Liu

unread,

May 29, 2018, 1:34:23 AM5/29/18

to Seaweed File System

Hi Chris,

Does the new filler support full POSIX file system semantics? In particular, does it or will it support mmap?

在 2018年5月28日星期一 UTC+8上午2:33:19，ChrisLu写道：

ChrisLu

unread,

May 29, 2018, 3:13:45 AM5/29/18

to Seaweed File System

No. This is only through enhancing the FUSE support. mmap would not be possible.

Chris

ChrisLu

unread,

May 31, 2018, 2:32:05 AM5/31/18

to Seaweed File System

I just built the binaries for 0.90beta, with the new filer and improved "weed mount".