Releasing New Filer Soon!

50 views
Skip to first unread message

ChrisLu

unread,
May 27, 2018, 2:33:19 PM5/27/18
to Seaweed File System
I am about to release a new filer implementation in the coming days.
This moves SeaweedFS from a distributed file object store closer to a real and scalable file system! And still much simpler than other file systems!

The new design has these improvements:

1. much simpler to implement for different storage
2. added with more meta data on files and directories
3. added directory listing in both Cassandra and Redis
4. “weed mount” can work both read and write

Breaking changes:
1. the filer storage will not be compatible with old filer store.
2. configuration is moved to a "filer.toml" file under current directory, or "$HOME/.seaweedfs/", or "/etc/seaweedfs/" folder.

The changes are in this branch:


The attached is an example filer.toml file

Chris


# A sample TOML config file for SeaweedFS filer store

[memory]
# local in memory, mostly for testing purpose
enabled = false

[leveldb]
# local on disk, mostly for simple single-machine setup, fairly scalable
enabled = false
dir = "." # directory to store level db files

####################################################
# multiple filers on shared storage, fairly scalable
####################################################

[mysql]
#
# need to choose or create a database.
# need to manually create a table "filemeta".
#
# CREATE TABLE IF NOT EXISTS filemeta (
# dirhash BIGINT COMMENT 'first 64 bits of MD5 hash value of directory field',
# name VARCHAR(1000) COMMENT 'directory or file name',
# directory VARCHAR(4096) COMMENT 'full path to parent directory',
# meta BLOB,
# PRIMARY KEY (dirhash, name)
# ) DEFAULT CHARSET=utf8;
#
enabled = true
hostname = "localhost"
port = 3306
username = "root"
password = ""
database = "" # create or use an existing database
connection_max_idle = 2
connection_max_open = 100

[postgres]
# CREATE TABLE IF NOT EXISTS filemeta (
# dirhash BIGINT,
# name VARCHAR(1000),
# directory VARCHAR(4096),
# meta bytea,
# PRIMARY KEY (dirhash, name)
# );
enabled = false
hostname = "localhost"
port = 5432
username = "postgres"
password = ""
database = "" # create or use an existing database
sslmode = "disable"
connection_max_idle = 100
connection_max_open = 100

[cassandra]
# CREATE TABLE filemeta (
# directory varchar,
# name varchar,
# meta blob,
# PRIMARY KEY (directory, name)
# ) WITH CLUSTERING ORDER BY (name ASC);
enabled = false
keyspace="seaweedfs"
hosts=[
"localhost:9042",
]

[redis]
enabled = true
address = "localhost:6379"
password = ""
db = 0


Qin Liu

unread,
May 29, 2018, 1:34:23 AM5/29/18
to Seaweed File System
Hi Chris,

Does the new filler support full POSIX file system semantics? In particular, does it or will it support mmap?

在 2018年5月28日星期一 UTC+8上午2:33:19,ChrisLu写道:

ChrisLu

unread,
May 29, 2018, 3:13:45 AM5/29/18
to Seaweed File System
No. This is only through enhancing the FUSE support. mmap would not be possible.

Chris

ChrisLu

unread,
May 31, 2018, 2:32:05 AM5/31/18
to Seaweed File System
I just built the binaries for 0.90beta, with the new filer and improved "weed mount".

Reply all
Reply to author
Forward
0 new messages