raw socket support ?

[frederic...@gmail.com]

Hi,

Correct me if I'm wrong but it seems OSv does not support raw sockets:
fd=socket(AF_PACKET,SOCK_RAW,htons(ETH_P_ALL));
Is there any way to send/receive network packets directly to/from the network driver in OSv ?
Has anyone used OSv to implement network functions that work at L2 or L3 levels rather than L4 (TCP or UDP) ?

Thanks

[Rick Payne]

You could use pfil_add_hook() to hook packets at the IP level.

I ended up writing something to allow me to get packets at the ethernet
level as there's no equivilent hook for that.

What are you trying to implement?

Cheers,
Rick

[frederic...@gmail.com]

Thanks for the advice.
I'm trying to port the Click router framework (https://github.com/kohler/click), a packet processing framework on OSv. Packet processing is done in user space either through a packet socket or the pcap library.

[Vinicius Fulber Garcia]

Hi,
 I have a similar problem, I need to get ethernet frames in userspace (a raw sockets bound in an interface or something similar to this). Did you get any advance on it guys?

Thanks!

[Waldek Kozaczuk]

Hi,

Unfortunately, I have no idea what it would take to add raw sockets support. Please be aware that we maintain another IPV6 branch - https://github.com/cloudius-systems/osv/tree/ipv6 - which besides IPV6 might have better networking support but I doubt it supports raw sockets.

I am also adding Charles Meyers from Spirent who wrote original IPV6 support to this thread. He may have some thoughts on this matter. Also, Spirent has also its own OSv fork - https://github.com/SpirentOrion/osv - which has extra stuff/fixes to networking stack (I would like to port some of those to the mainline OSv at some point).

Waldek 

--
You received this message because you are subscribed to the Google Groups "OSv Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/b3788d18-74a7-4d7f-91ad-7998992fbdcan%40googlegroups.com.

[Pekka Enberg]

Hi,

On Wed, Jul 15, 2020 at 6:13 PM Waldek Kozaczuk <jwkoz...@gmail.com> wrote:

Hi,

Unfortunately, I have no idea what it would take to add raw sockets support. Please be aware that we maintain another IPV6 branch - https://github.com/cloudius-systems/osv/tree/ipv6 - which besides IPV6 might have better networking support but I doubt it supports raw sockets.

I am also adding Charles Meyers from Spirent who wrote original IPV6 support to this thread. He may have some thoughts on this matter. Also, Spirent has also its own OSv fork - https://github.com/SpirentOrion/osv - which has extra stuff/fixes to networking stack (I would like to port some of those to the mainline OSv at some point).

The TCP/IP stack supports raw sockets (it's the FreeBSD stack after all).

One potential issue is that the Linux socket() compatibility layer is incorrect. I see that linux_socket() (called by socket() function) has some support for raw sockets:

https://github.com/cloudius-systems/osv/blob/master/bsd/sys/compat/linux/linux_socket.cc#L619 

However, it's bit picky on the "domain" and "protocol" and will ignore what Frederic attempted to do:

       fd=socket(AF_PACKET,SOCK_RAW,htons(ETH_P_ALL));

I assume if you fix up linux_socket() to do what Linux does, things will work fine.

- Pekka

[Vinicius Fulber Garcia]

Hi,

 I think that the type of raw sockets (SOCK_RAW) works well in OSv, it is possible to hook L3 packets (assuming an INET domain) with no problems. I guess that BSD does not support the PACKET domain, thus the OSv does not support too. If I remember well, we can hook frames in BSD by using libpcap, which in turn uses bpf (alternatively, we can use bpf directly). However, the bsd/sys/net in OSv does not have the FreeBSD's bpf implementation. Maybe one interesting way to enable L2 packet hooking is by migrating the bpf from sys/net of FreeBSD to the bsd/sys/net of OSv (what do you think about that guys?). Another possibility is to create a kind of PACKET domain support (similar to NETLINK support provided by Charles Meyers in the Spirent fork), but in that case, we will need to hook the bytes of frames straightforward from the mbuffs.

Regards,

Vinicius

[Waldek Kozaczuk]

Honestly, I am not familiar enough in this subject domain so I cannot give any authoritative advice. But it seems to me that the latter idea of introducing and supporting LINUX_AF_PACKET may be less effort and more in line with what Charles did. I am not familiar with BPF so I do not know if it would even make sense for OSv nor how much effort it would be to support it.

Waldek

[Rick Payne]

Another alternative (for me and maybe others) would be to have a
standard way to hook packets direct from the virtio interface.
Especially if we could ensure that we don't dhcp on that interface.

For instance, setup 2 interfaces from the host - 1 for OSv-y stuff, and
one just for an application which takes over the interface at a very
low level.

Rick

> --
> You received this message because you are subscribed to the Google
> Groups "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to osv-dev+u...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/osv-dev/397b053e-b06b-467e-89b8-75d9dd6d117an%40googlegroups.com
> .

[Nadav Har'El]

On Thu, Jul 23, 2020 at 12:05 PM Rick Payne <ri...@rossfell.co.uk> wrote:

Another alternative (for me and maybe others) would be to have a
standard way to hook packets direct from the virtio interface.

We did have something like that in the past, and I remember Vlad used it to write

some sort of super-efficient HTTP server or something like that - Vlad, do you remember

where it is? I can't recall now :-(

We also had until recently something more general, "assigned virtio", where the application

gets access directly to the viritio rings (and needs to work with them - the kernel doesn't

touch them any more). Waldek recently removed it but I guess you can see how it worked

before that, and if needed we can bring it back.

To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/6a3ce5daecbcb84bcdee0738242937e11d2f9d33.camel%40rossfell.co.uk.

[Rick Payne]

On Thu, 2020-07-23 at 12:38 +0300, Nadav Har'El wrote:
> We also had until recently something more general, "assigned virtio",
> where the application
> gets access directly to the viritio rings (and needs to work with
> them - the kernel doesn't
> touch them any more). Waldek recently removed it but I guess you can
> see how it worked
> before that, and if needed we can bring it back.

Ah yes, I sort of remember that, and I see commit 65c558ce17f89d4ae.
I'll have a look, thanks.

Rick

[Vinicius Fulber Garcia]

Very nice ideas!

Take control of the VirtIO interfaces, as Rick and Nadav suggested, can be a good solution for me too. In my particular case, I have a management interface that DHCPs, but the other interfaces (that are used for the data plane) are isolated and manually configured. I will check this old commit too, thanks a lot.

Regards,

Vinícius 

PS: Just out of curiosity, this " super-efficient HTTP server " was committed at some point? If it was, I will search for that in old commits too. :-)

[Vladislav Zolotarov]

On 7/23/20 5:38 AM, Nadav Har'El wrote:

On Thu, Jul 23, 2020 at 12:05 PM Rick Payne <ri...@rossfell.co.uk> wrote:

Another alternative (for me and maybe others) would be to have a
standard way to hook packets direct from the virtio interface.

We did have something like that in the past, and I remember Vlad used it to write

some sort of super-efficient HTTP server or something like that - Vlad, do you remember

where it is? I can't recall now :-(

Me neither. Sorry. Too long ago... ;)

[Pekka Enberg]

On Fri, Jul 24, 2020 at 3:35 AM Vladislav Zolotarov <vl...@scylladb.com> wrote:

On 7/23/20 5:38 AM, Nadav Har'El wrote:

On Thu, Jul 23, 2020 at 12:05 PM Rick Payne <ri...@rossfell.co.uk> wrote:

Another alternative (for me and maybe others) would be to have a
standard way to hook packets direct from the virtio interface.

We did have something like that in the past, and I remember Vlad used it to write

some sort of super-efficient HTTP server or something like that - Vlad, do you remember

where it is? I can't recall now :-(

Me neither. Sorry. Too long ago... ;)

I think Nadav is thinking of "OSv memcached", which bypasses the TCP/IP stack.

You can find it here:

https://github.com/cloudius-systems/osv/wiki/osv-memcached

https://github.com/vladzcloudius/osv-memcached

https://github.com/cloudius-systems/osv-apps/tree/master/osv-memcached

It's also discussed in the USENIX ATC paper.

- Pekka 

[Vinicius Fulber Garcia]

Thanks Pekka,

I will explore the memcached server to see how it works! In a fast first look, it seems to be very useful for solving the problem too. :-)

--
You received this message because you are subscribed to a topic in the Google Groups "OSv Development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/osv-dev/1Y9mGIipxfs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to osv-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/CAGUyND8769mKj44YNdosBA2fcA_zpdMRxLXjF5VYxmg%3DkCyTAg%40mail.gmail.com.

--

M.Sc. Vinícius Fülber Garcia

Ph.D. Student in Computer Science

Federal University of Paraná

http://www.inf.ufpr.br/vfgarcia/

[Rick Payne]

On Fri, 2020-07-24 at 08:38 +0300, Pekka Enberg wrote:
>
> I think Nadav is thinking of "OSv memcached", which bypasses the
> TCP/IP stack.

It uses the pfil_add_hook feature, which means it gets to see the
packets before they're passed to the netstack. You get the full packet
but only if its already destined for the IP stack (ie. you can't
intercept l2 packets).

Rick