Rutabaga channel types

[Alyssa Ross]

Currently, crosvm supports two different Rutabaga channels. Giving an
empty channel name will get you RUTABAGA_CHANEL_TYPE_WAYLAND, and the
channel name "mojo" will get you RUTABAGA_CHANNEL_TYPE_CAMERA. Other
channel names are not permitted, nor is it possible to directly specify
a channel type.

This makes it difficult to use Rutabaga channels for anything that is
not Wayland or "camera". Even though, as far as I can tell, which
channel is being used does not affect crosvm's functionality at all,
it'd still be necessary for me to modify crosvm to get it to accept my
channel type. So, at the very least I think that crosvm should allow
the user to configure arbitrary Rutabaga channel types by just supplying
the channel type u32, because really only the guest, and the thing that
starts crosvm need to agree on what the semantics of a channel type are,
not crosvm itself. The need for crosvm to map channel names to channel
types ends up nudging towards this situation where crosvm gets an
artificial need to understand the semantics of channel types, so I
wonder if it would be better to deprecate the mapping of "mojo" to 2,
and just have the user pass in 2 directly when they're running the mojo
backend and a VM that will know how to talk to it.

In conclusion:

• Should it be possible to run crosvm with arbitrary Rutabaga
channel type <-> socket mappings?
• Should the existing channel name concept be dropped, so channel type
semantics can be kept out of crosvm entirely and left to the two
endpoints of the channel communication?

[Gurchetan Singh]

On Mon, Oct 16, 2023 at 1:38 AM Alyssa Ross <h...@alyssa.is> wrote:

Currently, crosvm supports two different Rutabaga channels.  Giving an
empty channel name will get you RUTABAGA_CHANEL_TYPE_WAYLAND, and the
channel name "mojo" will get you RUTABAGA_CHANNEL_TYPE_CAMERA.  Other
channel names are not permitted, nor is it possible to directly specify
a channel type.

This makes it difficult to use Rutabaga channels for anything that is
not Wayland or "camera". 

What are you thinking of using the Rutabaga channel besides Wayland/camera? Typically, unless you're passing around virtio-gpu buffers, you probably can use vsock.

Though, if vsock is not an option,  adding more non-{Wayland}/{camera} channel types is fine [even if they are only used with your particular project].  We can just say they are "unstable" for a while with a comment or something if it's experimental code.

Even though, as far as I can tell, which
channel is being used does not affect crosvm's functionality at all,
it'd still be necessary for me to modify crosvm to get it to accept my
channel type.  So, at the very least I think that crosvm should allow
the user to configure arbitrary Rutabaga channel types by just supplying
the channel type u32, because really only the guest, and the thing that
starts crosvm need to agree on what the semantics of a channel type are,
not crosvm itself.  The need for crosvm to map channel names to channel
types ends up nudging towards this situation where crosvm gets an
artificial need to understand the semantics of channel types, so I
wonder if it would be better to deprecate the mapping of "mojo" to 2,
and just have the user pass in 2 directly when they're running the mojo
backend and a VM that will know how to talk to it.

In conclusion:

 • Should it be possible to run crosvm with arbitrary Rutabaga
   channel type <-> socket mappings?
 • Should the existing channel name concept be dropped, so channel type
   semantics can be kept out of crosvm entirely and left to the two
   endpoints of the channel communication?

I'm not sure how things would work without a defined channel name.  For example, if QEMU passes "--wayland_socket={blah},channel_type=2" and crosvm does "--wayland_socket={blah},channel_type=1", how would Sommelier (which is design to work with both), which channel_type corresponds to the Wayland socket?  So don't we need RUTABAGA_CHANEL_TYPE_* for such examples?

 

[Alyssa Ross]

Gurchetan Singh <gurchet...@chromium.org> writes:

> On Mon, Oct 16, 2023 at 1:38 AM Alyssa Ross <h...@alyssa.is> wrote:
>
>> Currently, crosvm supports two different Rutabaga channels. Giving an
>> empty channel name will get you RUTABAGA_CHANEL_TYPE_WAYLAND, and the
>> channel name "mojo" will get you RUTABAGA_CHANNEL_TYPE_CAMERA. Other
>> channel names are not permitted, nor is it possible to directly specify
>> a channel type.
>>
>> This makes it difficult to use Rutabaga channels for anything that is
>> not Wayland or "camera".
>
>
> What are you thinking of using the Rutabaga channel besides Wayland/camera?
> Typically, unless you're passing around virtio-gpu buffers, you probably
> can use vsock.

Well, my guest already has a virtio-gpu device, and doesn't have a vsock
device, so that was my initial motivation for looking into Rutabaga
channels instead of vsock.

But also, in the near future I'll be looking into transporting certain
XDG Desktop Portals between host and guest. Some portal backend
interfaces require sending file descriptors from the backend to the
frontend/application, which something that can transport Wayland would
be quite suited for. (Whether it makes sense to actually transport the
file descriptors like that will be portal-dependent, but I think it's
very likely to be useful to be able to do it for some of them.)

It would still need to be defined somewhere that 1 is the channel type
for Wayland, it's just that crosvm itself doesn't need to know that.

If I'm doing something similar over vsock, my host program and guest
program need to agree on a port, but the VMM doesn't maintain a list of
vsock service ports, and prevent me from using ones it doesn't know
about.

Having a registry of well-known channel types makes sense, I just don't
think it should live in crosvm's virtio-gpu crate, and crosvm shouldn't
enforce it, because what the channels mean in a question of policy that
in the end doesn't matter to crosvm itself.

(It would also probably be good for QEMU to support setting the channel
type.)

[Gurchetan Singh]

On Tue, Oct 17, 2023 at 1:41 AM Alyssa Ross <h...@alyssa.is> wrote:

Gurchetan Singh <gurchet...@chromium.org> writes:

> On Mon, Oct 16, 2023 at 1:38 AM Alyssa Ross <h...@alyssa.is> wrote:
>
>> Currently, crosvm supports two different Rutabaga channels.  Giving an
>> empty channel name will get you RUTABAGA_CHANEL_TYPE_WAYLAND, and the
>> channel name "mojo" will get you RUTABAGA_CHANNEL_TYPE_CAMERA.  Other
>> channel names are not permitted, nor is it possible to directly specify
>> a channel type.
>>
>> This makes it difficult to use Rutabaga channels for anything that is
>> not Wayland or "camera".
>
>
> What are you thinking of using the Rutabaga channel besides Wayland/camera?
> Typically, unless you're passing around virtio-gpu buffers, you probably
> can use vsock.

Well, my guest already has a virtio-gpu device, and doesn't have a vsock
device, so that was my initial motivation for looking into Rutabaga
channels instead of vsock.

FWIW, both Crostini and Android guests use vsock quite a bit.  It might be useful to look at what Crostini does for your particular use case and see if you can just copy that, or make it more generic.    

 

But also, in the near future I'll be looking into transporting certain
XDG Desktop Portals between host and guest.  Some portal backend
interfaces require sending file descriptors from the backend to the
frontend/application, which something that can transport Wayland would
be quite suited for.  (Whether it makes sense to actually transport the
file descriptors like that will be portal-dependent, but I think it's
very likely to be useful to be able to do it for some of them.)

Would that be possible with XWayland + Sommelier?

The registry of channel types lives in the rutabaga layer:

https://github.com/google/crosvm/blob/main/rutabaga_gfx/ffi/src/include/rutabaga_gfx_ffi.h#L89

(the camera one isn't really exposed yet since it's quite CrOS specific)

Though, crosvm needs to convert from its command line to the channel type.  

https://chromium.googlesource.com/chromiumos/platform2/+/refs/heads/main/vm_tools/concierge/arc_vm.cc#377

https://chromium.googlesource.com/chromiumos/platform2/+/refs/heads/main/vm_tools/concierge/vm_builder.cc#145

 

The command line takes the form of "--wayland-sock={XDG_RUNTIME_DIR}/wayland-0 --wayland-sock=/usr/run/mojo.socket, mojo".  The first is considered the display socket and doesn't have to be explicitly named.  And the second has to be explicitly named.

I think what you are suggesting is just "--wayland-sock={XDG_RUNTIME_DIR}/wayland-0,1 --wayland-sock=/usr/run/mojo.socket, 2"?

Isn't that more or less the same as the current situation, just replacing strings (or the absence of strings) with a number?  

Though I do agree the naming could be better.  

[Alyssa Ross]

Gurchetan Singh <gurchet...@chromium.org> writes:

>> But also, in the near future I'll be looking into transporting certain
>> XDG Desktop Portals between host and guest. Some portal backend
>> interfaces require sending file descriptors from the backend to the
>> frontend/application, which something that can transport Wayland would
>> be quite suited for. (Whether it makes sense to actually transport the
>> file descriptors like that will be portal-dependent, but I think it's
>> very likely to be useful to be able to do it for some of them.)
>>
>
> Would that be possible with XWayland + Sommelier?

I don't think XWayland has anything to do with it. Desktop Portals are
D-Bus interfaces that let a sandboxed application ask for things like
access to a file that isn't already in its sandbox. Unless I want to
write a Wayland protocol for it and tunnel it over Wayland, I need a
separate channel of some sort (whether that ends up being Rutabaga or
vsock), because the unnamed Wayland channel is already taken by the
connection to the compositor.

That's exactly my point. The only reason crosvm needs to know what the
channel is for is to map from name to type. If it just accepted channel
types directly, we could remove the requirement for channel types to be
registered, which is a hurdle to jump through to use Rutabaga that
doesn't need to be there. No other socket protocol (vsock, TCP/IP,
etc.) enforces registration of what are essentially port numbers like
this, so Rutabaga channels shouldn't either.

Whatever program starts crosvm already has to know the specifics of the
protocol it's using Rutabaga for, because it has to know the path to its
socket, so it would make a lot more sense to just teach it what the
channel type number is, and have it pass that to crosvm.

Think about somebody trying to use Rutabaga for some new purpose outside
Google. They get crosvm — maybe they don't even build it themselves but
get it from a distro package manager, and they want to start it up with
some custom channel for their new purpose. They discover that, instead
of just specifying a port number, like they're used to with vsock, they
have to:

- Find the crosvm repo
- Clone it
- Read and understand the relevant code
- Modify it to accept their new channel type
- Learn the Chromium contribution process
- Create a Google account
- Read and sign a CLA (which there are very valid reasons not to want to do)
- Push their patch to Gerrit
- Wait for it to be reviewed and submitted
- Build a new crosvm, or, even worse, wait for their distro package to
be updated

That's a lot of work that could be totally unnecessary, and all that
would have to change is for crosvm to accept numeric channel types on
its command line.

[Gurchetan Singh]

On Wed, Oct 18, 2023 at 12:22 AM Alyssa Ross <h...@alyssa.is> wrote:

Gurchetan Singh <gurchet...@chromium.org> writes:

>> But also, in the near future I'll be looking into transporting certain
>> XDG Desktop Portals between host and guest.  Some portal backend
>> interfaces require sending file descriptors from the backend to the
>> frontend/application, which something that can transport Wayland would
>> be quite suited for.  (Whether it makes sense to actually transport the
>> file descriptors like that will be portal-dependent, but I think it's
>> very likely to be useful to be able to do it for some of them.)
>>
>
> Would that be possible with XWayland + Sommelier?

I don't think XWayland has anything to do with it.  Desktop Portals are
D-Bus interfaces that let a sandboxed application ask for things like
access to a file that isn't already in its sandbox.  Unless I want to
write a Wayland protocol for it and tunnel it over Wayland, I need a
separate channel of some sort (whether that ends up being Rutabaga or
vsock), because the unnamed Wayland channel is already taken by the
connection to the compositor.

Interesting - crostini also uses D-Bus, but via vsock.  I feel like there is a need to put these vsock proxies somewhere so there could be some sort of standardization there.

The rule of thumb is typically if your interface needs graphics buffers (like camera/wayland), virtio-gpu would be the solution.  Otherwise, use vsock.

Even with camera, there is interest in defining a formal virtio-camera device too in some quarters, so there is not much consensus there.  Camera/mojo in cross-domain was grandfathered in since virtio-wl supported it.

Understood.  Essentially the possibilities for the crosvm command line are:

1) "--wayland-sock={XDG_RUNTIME_DIR}/wayland-0 --wayland-sock=/usr/run/mojo.socket, mojo"

2)  "--sock={XDG_RUNTIME_DIR}/wayland-0,1 --sock=/usr/run/mojo.socket, 2"

3) "--wayland-sock={XDG_RUNTIME_DIR}/wayland-0 --mojo-socket=/usr/run/mojo.socket"

I'm not too opinionated on which one is used, but (1) is currently used.  If you want to change (1) and associated launchers, or just fix vhost-user-gpu to use a forward-looking approach, I doubt anybody will stop you.

Though, when to use vsock versus virtio-gpu seems like the more interesting design point.

[Alyssa Ross]

On Wed, Oct 18, 2023 at 06:29:10PM -0700, Gurchetan Singh wrote:
> On Wed, Oct 18, 2023 at 12:22 AM Alyssa Ross <h...@alyssa.is> wrote:
>
> > Gurchetan Singh <gurchet...@chromium.org> writes:
> >
> > >> But also, in the near future I'll be looking into transporting certain
> > >> XDG Desktop Portals between host and guest. Some portal backend
> > >> interfaces require sending file descriptors from the backend to the
> > >> frontend/application, which something that can transport Wayland would
> > >> be quite suited for. (Whether it makes sense to actually transport the
> > >> file descriptors like that will be portal-dependent, but I think it's
> > >> very likely to be useful to be able to do it for some of them.)
> > >>
> > >
> > > Would that be possible with XWayland + Sommelier?
> >
> > I don't think XWayland has anything to do with it. Desktop Portals are
> > D-Bus interfaces that let a sandboxed application ask for things like
> > access to a file that isn't already in its sandbox. Unless I want to
> > write a Wayland protocol for it and tunnel it over Wayland, I need a
> > separate channel of some sort (whether that ends up being Rutabaga or
> > vsock), because the unnamed Wayland channel is already taken by the
> > connection to the compositor.
> >
>
> Interesting - crostini also uses D-Bus, but via vsock. I feel like there
> is a need to put these vsock proxies somewhere so there could be some sort
> of standardization there.

Plausibly — it's early days of investigating this for me, so I'm still
not sure what my requirements are. One likely constraint is avoiding
allowing guests to speak D-Bus to the host or other guests directly,
because that's a large attack surface, and instead turning it into
simpler application-specific host<->guest protocols.

> The rule of thumb is typically if your interface needs graphics buffers
> (like camera/wayland), virtio-gpu would be the solution. Otherwise, use
> vsock.

What reason is there to use vsock over virtio-gpu? Performance?

D-Bus can run over TCP/IP or vsock fine, unless you need to transfer
file descriptors, and then it can't for obvious reasons.

Taking the ReadClipboard() method of the Clipboard portal as an example
— the application is given a pipe containing the clipboard contents to
read from, like in Wayland. Surely it makes sense to use the same
mechanism used for the Wayland clipboard to implement this?

Okay, I'll put it on my list.

[Gurchetan Singh]

+Fergus Dall might know more about how vsock is utilized and how, and also DBus + XDG portals.  vsock is very much the default for sending bytes, with virtio-gpu used as needed.

 

D-Bus can run over TCP/IP or vsock fine, unless you need to transfer
file descriptors, and then it can't for obvious reasons.

Taking the ReadClipboard() method of the Clipboard portal as an example
— the application is given a pipe containing the clipboard contents to
read from, like in Wayland.  Surely it makes sense to use the same
mechanism used for the Wayland clipboard to implement this?

Sure, but do try to upstream whatever method you use.  If you are using virtio-gpu, you could upstream here: https://github.com/flatpak/xdg-desktop-portal/.  That'll avoid a scenario of another person using vsock to solve this problem (if another person is interested in it). 

It is possible to use vsock for pipe write + read.  But if it uses zero copy mechanism (wayland keymaps), that's something vsock doesn't offer.

If doing an impl, you may even consider building a shared library from Sommelier's virtualization directory.

https://chromium.googlesource.com/chromiumos/platform2/+/HEAD/vm_tools/sommelier/virtualization/