LISAFS project

[Ayush Ranjan]

Hi all,

We've been working on redesigning the protocol with which the sentry communicates with the gofer process. The old protocol (currently default) is 9P. Lisafs protocol is set to replace it in the near future. Lisafs protocol implementation is checked in and runsc's fsgofer supports lisafs.

The goal of lisafs is to make the sentry to gofer communication more economical - make few round trips which helps reduce the scheduling overhead incurred by the futex operations that switch between sentry and gofer.

Apart from the improved RPC latency, lisafs implementation is also faster and more memory efficient than 9P implementation.

We are looking for volunteers to enable lisafs and give us feedback about code stability and performance gains. It is only supported with VFS2 which is default. You can enable it by:

`sudo runsc install -- --lisafs`

Or add "--lisafs" to the runtime configuration in /etc/docker/daemon.json. Or use runsc do:

`sudo runsc --lisafs do echo 123`

Please also note that this is not production ready yet. We have worked to get as much confidence from our internal testing as possible. All syscall tests and integration tests pass with lisafs enabled.

You can file bugs or send feedback using this link. We look forward to hearing from you!

[Ayush Ranjan]

Hi all!

LISAFS has been enabled by default in runsc as of 006ba8495554 ("Enable lisafs by default for runsc.").

If you encounter any issues, you can disable LISAFS with --lisafs=false runsc flag. Please be sure to inform us of the issue, if you do find the need to disable LISAFS using this link.

[Wenhui Zhang]

Hi, 

Is that any performance comparison data on 9P vs Lisafs please?

[Ayush Ranjan]

Hi,

Yes you can find some data in go/lisafs-internal-evaluation.

--
You received this message because you are subscribed to a topic in the Google Groups "gVisor Users [Public]" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gvisor-users/aCUsTK3UPwI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gvisor-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gvisor-users/1430fbe0-7be0-4f7b-90b4-e35e45e12cd1n%40googlegroups.com.

[Wenhui Zhang]

Hi, 

Sorry that I tried to search for lisafs-internal-evaluation folder and file in the gvisor repo, and did not find it there.

Could you help point out the location/pointer to the “go/lisafs-internal-evaluation” please?

Wenhui

[Ayush Ranjan]

Whoops, my mistake. That is an internal link.

While running filesystem intensive workloads with runsc (like bazel build), we found that LISAFS substantially cuts down gofer memory usages (30%-60% reduction), GC cycles and allocations. It also cuts down sentry allocations by ~15%. In terms of performance, we found that it reduces runsc's overhead as compared to native by ~15% for bazel build. Of course, these numbers really depend on what workload you are running.

[Wenhui Zhang]

Hi, Ayush:

Thanks so much for your swift reply, the perf  improvement is indeed impressive;)

Thanks for the details.

Wenhui 

On Nov 6, 2022, at 8:12 PM, Ayush Ranjan <ayush...@google.com> wrote:



[Ayush Ranjan]

The 9P gofer protocol is now being deprecated, it has been the default since August. This means that the runsc flag --lisafs will not have any effect. If you are still using 9P (by disabling lisafs), please migrate to lisafs. If you encounter issues, let us know.