Cipsov4 pass through labeling not working for RHEL7 hosts

[Joshua Koontz]

I have two hosts running on Openstack, both configured with RHEL 7.3 and the MLS policy type. On each hosts I have two interfaces:

eth0: 10.10.10.12 and 10.10.10.13
eth1: 192.168.200.12 and 192.168.200.13

Here are the netlabel rules on each host:

map del default
map add default address:0.0.0.0/0 protocol:unlbl

cipsov4 add pass doi:5 tags:5
map add default address:127.0.0.0/8 protocol:cipsov4,5
map add default address:10.10.10.12 protocol:cipsov4,5
# or map add default address:10.10.10.13 protocol:cipsov4,5

# Trusted Network
map add default address:192.168.200.0/24 protocol:cipsov4,5

unlbl accept on
unlbl add interface:eth0 address:10.10.10.0/24 label:system_u:object_r:netlabel_peer_t:s0 -s15:c0.c1023

I have sshd configured to be served out from xinetd with the LABELED flags setting. This has been tested and works for unlbl rules assigned to eth1. If I have the rules configured as above with cipsov4 pass through, the attempt to ssh hangs with no meaningful data on either side of the connection. I've turned off the SELinux dontaudit rules, but don't see any meaningful data there.

[Paul Moore]

On Tue, Dec 6, 2016 at 7:48 AM, Joshua Koontz <jbko...@gmail.com> wrote:
> I have two hosts running on Openstack, both configured with RHEL 7.3 and the MLS policy type. On each hosts I have two interfaces:
>
> eth0: 10.10.10.12 and 10.10.10.13
> eth1: 192.168.200.12 and 192.168.200.13
>
> Here are the netlabel rules on each host:
>
> map del default
> map add default address:0.0.0.0/0 protocol:unlbl
>
> cipsov4 add pass doi:5 tags:5
> map add default address:127.0.0.0/8 protocol:cipsov4,5
> map add default address:10.10.10.12 protocol:cipsov4,5
> # or map add default address:10.10.10.13 protocol:cipsov4,5
>
> # Trusted Network
> map add default address:192.168.200.0/24 protocol:cipsov4,5
>
> unlbl accept on

> unlbl add interface:eth0 address:10.10.10.0/24 label:system_u:object_r:netlabel_peer_t:s0-s15:c0.c1023

>
>
> I have sshd configured to be served out from xinetd with the LABELED flags setting. This has been tested and works for unlbl rules assigned to eth1. If I have the rules configured as above with cipsov4 pass through, the attempt to ssh hangs with no meaningful data on either side of the connection. I've turned off the SELinux dontaudit rules, but don't see any meaningful data there.

Hi Joshua,

A few quick questions ... Have you ever been able to successfully
configure a NetLabel labeled network, or have you always been using
unlabeled networks up to this point? Have you tried running in
permissive mode? Have you run any sort of network sniffer on the
hosts to see if the network traffic is being sent and if it is labeled
correctly?

--
paul moore
www.paul-moore.com

[Joshua Koontz]

Hi Paul,

Yes, I've been able to assign Netlabel labeled networks in RHEL 6.8+. I've tried on these hoses in both enforcing and permissive mode, but don't see any AVC denials. I have turned off the dontaudit rules as well and don't see anything beyond { rlimitinh siginh noatsecure } for ssh.

I've run tcpdump on the interfaces.
Sending host:
# tcpdump -nvv -i eth4
tcpdump: listening on eth4, link-type EN10MB (Ethernet), capture size 65535 bytes
07:22:43.520591 IP (tos 0x0, ttl 64, id 65251, offset 0, flags [DF], proto TCP (6), length 76, options (unknown 134,EOL))
192.168.200.8.33774 > 192.168.200.7.ssh: Flags [S], cksum 0x1190 (incorrect -> 0x200c), seq 184895250, win 27200, options [mss 1360,sackOK,TS val 480885233 ecr 0,nop,wscale 7], length 0
07:22:44.523315 IP (tos 0x0, ttl 64, id 65252, offset 0, flags [DF], proto TCP (6), length 76, options (unknown 134,EOL))
192.168.200.8.33774 > 192.168.200.7.ssh: Flags [S], cksum 0x1190 (incorrect -> 0x1c21), seq 184895250, win 27200, options [mss 1360,sackOK,TS val 480886236 ecr 0,nop,wscale 7], length 0
07:22:46.527317 IP (tos 0x0, ttl 64, id 65253, offset 0, flags [DF], proto TCP (6), length 76, options (unknown 134,EOL))
192.168.200.8.33774 > 192.168.200.7.ssh: Flags [S], cksum 0x1190 (incorrect -> 0x144d), seq 184895250, win 27200, options [mss 1360,sackOK,TS val 480888240 ecr 0,nop,wscale 7], length 0
07:22:48.527309 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.200.7 tell 192.168.200.8, length 28
07:22:48.528058 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.200.7 is-at fa:16:3e:b9:b5:f1, length 28
07:22:50.535313 IP (tos 0x0, ttl 64, id 65254, offset 0, flags [DF], proto TCP (6), length 76, options (unknown 134,EOL))
192.168.200.8.33774 > 192.168.200.7.ssh: Flags [S], cksum 0x1190 (incorrect -> 0x04a5), seq 184895250, win 27200, options [mss 1360,sackOK,TS val 480892248 ecr 0,nop,wscale 7], length 0
07:22:58.543321 IP (tos 0x0, ttl 64, id 65255, offset 0, flags [DF], proto TCP (6), length 76, options (unknown 134,EOL))
192.168.200.8.33774 > 192.168.200.7.ssh: Flags [S], cksum 0x1190 (incorrect -> 0xe55c), seq 184895250, win 27200, options [mss 1360,sackOK,TS val 480900256 ecr 0,nop,wscale 7], length 0
^C
7 packets captured
7 packets received by filter
0 packets dropped by kernel

Receiving host:
# tcpdump -nvv -i eth4
tcpdump: listening on eth4, link-type EN10MB (Ethernet), capture size 65535 bytes
07:22:48.243301 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.200.7 tell 192.168.200.8, length 28
07:22:48.243313 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.200.7 is-at fa:16:3e:b9:b5:f1, length 28
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel

## ssh attempt ###
$ ssh -vv 192.168.200.7
OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.200.7 [192.168.200.7] port 22.
^C

Is there a better network sniffer you would recommend that would be show the label on the packets?

Thanks
Josh

[Paul Moore]

On Wed, Dec 7, 2016 at 7:29 AM, Joshua Koontz <jbko...@gmail.com> wrote:
> Hi Paul,
>
> Yes, I've been able to assign Netlabel labeled networks in RHEL 6.8+. I've tried on these hoses in both enforcing and permissive mode, but don't see any AVC denials. I have turned off the dontaudit rules as well and don't see anything beyond { rlimitinh siginh noatsecure } for ssh.
>
> I've run tcpdump on the interfaces.
> Sending host:
> # tcpdump -nvv -i eth4
> tcpdump: listening on eth4, link-type EN10MB (Ethernet), capture size 65535 bytes
> 07:22:43.520591 IP (tos 0x0, ttl 64, id 65251, offset 0, flags [DF], proto TCP (6), length 76, options (unknown 134,EOL))
> 192.168.200.8.33774 > 192.168.200.7.ssh: Flags [S], cksum 0x1190 (incorrect -> 0x200c), seq 184895250, win 27200, options [mss 1360,sackOK,TS val 480885233 ecr 0,nop,wscale 7], length 0
> 07:22:44.523315 IP (tos 0x0, ttl 64, id 65252, offset 0, flags [DF], proto TCP (6), length 76, options (unknown 134,EOL))
> 192.168.200.8.33774 > 192.168.200.7.ssh: Flags [S], cksum 0x1190 (incorrect -> 0x1c21), seq 184895250, win 27200, options [mss 1360,sackOK,TS val 480886236 ecr 0,nop,wscale 7], length 0
> 07:22:46.527317 IP (tos 0x0, ttl 64, id 65253, offset 0, flags [DF], proto TCP (6), length 76, options (unknown 134,EOL))
> 192.168.200.8.33774 > 192.168.200.7.ssh: Flags [S], cksum 0x1190 (incorrect -> 0x144d), seq 184895250, win 27200, options [mss 1360,sackOK,TS val 480888240 ecr 0,nop,wscale 7], length 0
> 07:22:48.527309 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.200.7 tell 192.168.200.8, length 28
> 07:22:48.528058 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.200.7 is-at fa:16:3e:b9:b5:f1, length 28
> 07:22:50.535313 IP (tos 0x0, ttl 64, id 65254, offset 0, flags [DF], proto TCP (6), length 76, options (unknown 134,EOL))
> 192.168.200.8.33774 > 192.168.200.7.ssh: Flags [S], cksum 0x1190 (incorrect -> 0x04a5), seq 184895250, win 27200, options [mss 1360,sackOK,TS val 480892248 ecr 0,nop,wscale 7], length 0
> 07:22:58.543321 IP (tos 0x0, ttl 64, id 65255, offset 0, flags [DF], proto TCP (6), length 76, options (unknown 134,EOL))
> 192.168.200.8.33774 > 192.168.200.7.ssh: Flags [S], cksum 0x1190 (incorrect -> 0xe55c), seq 184895250, win 27200, options [mss 1360,sackOK,TS val 480900256 ecr 0,nop,wscale 7], length 0

IPv4 option 134 is the CIPSO option, so it looks like you are sending
labeled traffic - that's good.

> Receiving host:
> # tcpdump -nvv -i eth4
> tcpdump: listening on eth4, link-type EN10MB (Ethernet), capture size 65535 bytes
> 07:22:48.243301 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.200.7 tell 192.168.200.8, length 28
> 07:22:48.243313 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.200.7 is-at fa:16:3e:b9:b5:f1, length 28
> ^C
> 2 packets captured
> 2 packets received by filter
> 0 packets dropped by kernel

Unfortunately it doesn't look like the other system is receiving the
packets; it looks like something is preventing the packets from making
it from one system to the other. I see that there was an ARP
request/reply between the two machines so they must be on the same
segment (no routers to worry about dropping the packets), are there
any other boxes in between the two systems which might be applying
some sort of filtering? Are either of the systems running host-based
firewalls? Do you control the OpenStack instance? It might be that
the hypervisor is blocking/filtering the traffic; what
distribution/kernel is running on the hypervisor?

> Is there a better network sniffer you would recommend that would be show the label on the packets?

I added CIPSO decoding to wireshark some time ago, you could try that,
but really just look for IPv4 option 134, that's the key.

--
paul moore
www.paul-moore.com

[Joshua Koontz]

The OpenStack instance is controlled by another division in our company, but they are helping us troubleshoot now.  They moved both instances to the same hypervisor, but I didn't see any change in behavior.  They are working to attach a network sniffer to the virtual interface on the hypervisor.  Here is some version information about the Openstack environment:

uname –r:

3.10.0-327.28.2.el7.x86_64

cat /etc/redhat-release:

Red Hat Enterprise Linux Server release 7.2 (Maipo)

yum list installed |grep kvm: 

libvirt-daemon-kvm.x86_64        1.2.17-13.el7_2.5       

qemu-kvm-common-rhev.x86_64      10:2.3.0-31.el7_2.13 

qemu-kvm-rhev.x86_64             10:2.3.0-31.el7_2.13

--
You received this message because you are subscribed to a topic in the Google Groups "netlabel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/netlabel/yOk2z8LSRac/unsubscribe.
To unsubscribe from this group and all its topics, send an email to netlabel+unsubscribe@googlegroups.com.
To post to this group, send email to netl...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Paul Moore]

On Thu, Dec 8, 2016 at 4:54 PM, Joshua Koontz <jbko...@gmail.com> wrote:
> The OpenStack instance is controlled by another division in our company, but
> they are helping us troubleshoot now. They moved both instances to the same
> hypervisor, but I didn't see any change in behavior. They are working to
> attach a network sniffer to the virtual interface on the hypervisor. Here
> is some version information about the Openstack environment:
>
> uname –r:
> 3.10.0-327.28.2.el7.x86_64
>
> cat /etc/redhat-release:
> Red Hat Enterprise Linux Server release 7.2 (Maipo)
>
> yum list installed |grep kvm:
> libvirt-daemon-kvm.x86_64 1.2.17-13.el7_2.5
> qemu-kvm-common-rhev.x86_64 10:2.3.0-31.el7_2.13
> qemu-kvm-rhev.x86_64 10:2.3.0-31.el7_2.13

Okay, I think we've found your problem. The issue is that the RHEL
based hypervisor understands the CIPSO labels but doesn't have the
necessary NetLabel/CIPSO configuration in place to interpret the
packet labels and enforce the security policy, so it drops the
traffic, as it should. If you want to send CIPSO labeled traffic
through a kernel which has NetLabel/CIPSO support, you will need to
configure the kernel for the CIPSO DOIs you are sending (and ensure
the necessary SELinux policy configuration).

--
paul moore
www.paul-moore.com