SELinux support

[Santi]

Hi,

I have been trying to figure out best way to label the objects in my Centos image. Currently file_contexts are broken until I run restorecon at /.

This is what I have in config.sh:

#=====================================  
# Enable Selinux permissive
#-------------------------------------  
/usr/bin/sed -i -e "s/SELINUX=[^\n]*/SELINUX=permissive/g" /etc/selinux/config

and I have these SELinux packages in config.kiwi:

<package name="selinux-policy-targeted"/>

<package name="policycoreutils"/>

<type image="oem" boot="oemboot/rhel-07.0" filesystem="ext3" installiso="true" installpxe="true" bootloader="grub2"
         kernelcmdline="selinux=1 console=ttyS0,115200 console=tty0" firmware="efi" hybrid="true">

After boot, context is completely off.

[root@localhost ~]# ps -efZ |grep tomcat
system_u:system_r:kernel_t:s0   tomcat   11969     1 48 17:22 ?        00:03:48 /usr/bin/java -Dlog4j.configurationFile=file:/etc/opt/xxx/xxx/log4j2_tomcat.xml -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start


[root@localhost efivar]# ps -efZ |grep http
system_u:system_r:kernel_t:s0   root     11911     1  0 17:22 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND
system_u:system_r:kernel_t:s0   apache   12072 11911  0 17:22 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND
system_u:system_r:kernel_t:s0   apache   12073 11911  0 17:22 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND
system_u:system_r:kernel_t:s0   apache   12074 11911  0 17:22 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND

Now I run, /usr/sbin/restorecon -iR / and reboot and things are correctly labeled:

[root@localhost ~]# ps -efZ |grep http

system_u:system_r:httpd_t:s0    root       689     1  0 19:15 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND

system_u:system_r:httpd_t:s0    apache     830   689  0 19:15 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND

system_u:system_r:httpd_t:s0    apache     834   689  0 19:15 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND

system_u:system_r:httpd_t:s0    apache     835   689  0 19:15 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND

unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 root 1332 1293  0 19:16 pts/0 00:00:00 grep --color=auto http

[root@localhost ~]# ps -efZ |grep tomcat

system_u:system_r:tomcat_t:s0   tomcat     788     1 32 19:15 ?        00:00:18 /usr/bin/java -Dlog4j.configurationFile=file:/etc/opt/xxx/xxx/log4j2_tomcat.xml -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start

I tried adding /usr/sbin/restorecon -iR / in config.sh at the very end but it didn't help.  How to have the fc enforced correctly in the first boot.

[Santi]

I am comparing kiwi installation script with kickstart script for SELinux option.
In kickstart, SELinux option is present before I pick any of the packages.. something like:

{

#version=RHEL7
# System authorization information
auth --enableshadow --passalgo=sha512

# Use network installation
url --url="http://buildlogs.centos.org/centos/7/os/x86_64-20140614/"
# Use text mode install
text
# Keyboard layouts
keyboard --vckeymap=us --xlayouts='us'
# System language
lang en_US.UTF-8

# Network information
network  --bootproto=dhcp --device=eth0 --ipv6=auto --activate
network  --hostname=centos7previewkickstarttest
# Root password
rootpw some-password
# Do not configure the X Window System
skipx
# System timezone
timezone Europe/Amsterdam --isUtc
#user --groups=wheel --name=useraccount --password=some-password --gecos="User"
# Skip EULA
eula --agreed
# Disable firewall
firewall --disabled
# Don't run the Setup Agent on first boot
firstboot --disabled
# Selinux (ENFORCING|permissive|disabled)
selinux --enforcing  <-------------------------------------------------
}

Is something similar required for kiwi? 

[Marcus Schäfer]

Hi,

I'm not an selinux expert, thus others might have better ideas
how to label the objects correctly. Here are my two cents on this
topic

1. When building rhel images my templates always provides the
selinux config as an overlay file:

root/etc/selinux/config

and I setup: SELINUX=permissive, which causes warnings and the
auto relabeling on first boot.

> I have been trying to figure out best way to label the objects in my
> Centos image. Currently file_contexts are broken until I run restorecon
> at /. This is what I have in config.sh:

2. yes 'restorecon -iR /' fixes the situation in the booted system but
I could not find a way to call this in the context of an image.
When called in config.sh it has no effect later. So my question would be
what does restorecon exactly and does it require the target system
selinux system to be up and running.

Feedback from selinux experts would be very much appreciated

Thanks

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer
-------------------------------------------------------
Marcus Schäfer (Res. & Dev.) SUSE Linux GmbH
Tel: 0911-740 53 0 Maxfeldstrasse 5
FAX: 0911-740 53 479 D-90409 Nürnberg
HRB: 21284 (AG Nürnberg) Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
http://www.suse.de
-------------------------------------------------------

[Santi]

Thank you.

I was looking for more info around this and came across this document (http://www.redhat.com/f/pdf/whitepapers/Filesystem_Labeling_SELinux.pdf) and in page 4, it states this:

======

So, when are files labeled? During an SELinux system installation, the setfiles(8) utility typically is used to label all of the files in filesystems that support EA security labeling. Package management tools such as RPM also may label files during installation, while system administrators often need to set security contexts manually with chcon(1) or setfilecon(1). 

======
Also, I checked dmesg output and this is what I see:

[root@localhost ~]# dmesg |grep -i selinux

[    0.000000] Command line: BOOT_IMAGE=/boot/linux ramdisk_size=512000 ramdisk_blocksize=4096 cdinst=1 selinux=1 console=ttyS0,115200 console=tty0 showopts

[    0.000000] Kernel command line: BOOT_IMAGE=/boot/linux ramdisk_size=512000 ramdisk_blocksize=4096 cdinst=1 selinux=1 console=ttyS0,115200 console=tty0 showopts

[    0.032853] SELinux:  Initializing.

[    0.036370] SELinux:  Starting in permissive mode

[    0.319749] EVM: security.selinux

[    5.992824] SELinux:  Registering netfilter hooks

[  212.181345] dracut[1969] -rwxr-xr-x   1 root     root       147120 Apr  7 14:40 usr/lib64/libselinux.so.1

[  217.629865] SELinux: 2048 avtab hash slots, 107131 rules.

[  217.670720] SELinux: 2048 avtab hash slots, 107131 rules.

[  217.773478] SELinux:  8 users, 86 roles, 4820 types, 285 bools, 1 sens, 1024 cats

[  217.773488] SELinux:  83 classes, 107131 rules

[  217.786025] SELinux:  Completing initialization.

[  217.786034] SELinux:  Setting up existing superblocks.

[  217.786051] SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts

[  217.786062] SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts

[  217.786071] SELinux: initialized (dev bdev, type bdev), uses genfs_contexts

Whereas, If I install an image which has /.autorelabel file, then I get this text:

*** Warning -- SELinux targeted policy relabel is required.

*** Ralabeling could take  a very long time, depending on file

*** system size and speed of hard drive.

Once relabeling finishes, then system reboots and dmesg shows the above  mentioned message... so It seems like file contexts are written when image is built.

So does it matter where do I put <package name="selinux-policy-targeted"/> in the config.kiwi file? 

Thanks

[Santi]

I am looking at virt-builder documentation: http://libguestfs.org/virt-builder.1.html#selinux

Virt-builder does not know how to give new files a label, so there are two possible strategies it can use to ensure correct labelling:

Using --selinux-relabel

This runs fixfiles(8) just before finalizing the guest, which sets SELinux labels correctly in the disk image.

Sometimes fixfiles is not possible during installation, in which case this option falls back on:

Does this make sense?

[Marcus Schäfer]

Hi,

> I am looking at virt-builder documentation:
> http://libguestfs.org/virt-builder.1.html#selinux

Thanks, that's valuable information. I have opened an issue for it here

https://github.com/SUSE/kiwi/issues/52

At the moment I have no free slot to work on this though, thus marked
with the "help wanted" label. I'll come back to this one asap

Regards,
Marcus
--
Public Key available via: https://keybase.io

[Santosh]

Files are getting labelled by using of this command:
/usr/sbin/setfiles /etc/selinux/targeted/contexts/files/file_contexts / -v

Now the issue is, where to put the command:
I first put this command into config.sh, at the very bottom. All files are getting labeled correctly in the beginning but for some reason /etc/ld.so.cache loses its label at the end of prepare stage. I realized that KIWIRoot.pm -> setup() routine does something extra after calling config.sh.

To avoid it, I created one file called fcontext.sh and put it at the same directory where config.sh is and added this section at the very end of sub() :

    #========================================
    # call fcontext.sh SELinux file context labeling script
    #----------------------------------------
    if ((! $initCache) && (-e "$imageDesc/fcontext.sh")) {
        $kiwi -> info ("Calling fcontext script: fcontext.sh");
        KIWIQX::qxx (" cp $imageDesc/fcontext.sh $root/tmp ");
        KIWIQX::qxx (" chmod u+x $root/tmp/fcontext.sh ");
        my ($code,$data) = KIWIGlobals -> instance() -> callContained (
            $root,"/tmp/fcontext.sh"
        );
        if ($code != 0) {
            $kiwi -> failed ();
            $kiwi -> info   ($data);
            return;
        } else {
            $kiwi -> loginfo ("fcontext.sh: $data");
        }
        KIWIQX::qxx (" rm -f $root/tmp/fcontext.sh ");
        $kiwi -> done ();
    }

Now all files have correct context. Does this make sense?

[Marcus Schäfer]

Hi,

> Files are getting labelled by using of this command:
> /usr/sbin/setfiles /etc/selinux/targeted/contexts/files/file_contexts /
> -v
> Now the issue is, where to put the command:

Support for SELinux file security contexts has been added to
the v8 version of kiwi. I will submit a new python3-kiwi package
today

> I first put this command into config.sh, at the very bottom. All files
> are getting labeled correctly in the beginning but for some reason
> /etc/ld.so.cache loses its label at the end of prepare stage. I
> realized that KIWIRoot.pm -> setup() routine does something extra after
> calling config.sh.

yes that could be the case. Actually the right place for the labeling
would be right before the data is synced to the image. It's done that
way in the v8 kiwi version. In v7 it imho could be done in

KIWIBoot.pm

#==========================================
# Copy root tree to disk
#------------------------------------------
# ... Add SELinux file security contexts setup here
#
$kiwi -> info ("Copying system image tree on disk");

I'd rather suggest to give v8 a try

> To avoid it, I created one file called fcontext.sh and put it at the
> same directory where config.sh is and added this section at the very
> end of sub() :
> #========================================
> # call fcontext.sh SELinux file context labeling script
> #----------------------------------------
> if ((! $initCache) && (-e "$imageDesc/fcontext.sh")) {
> $kiwi -> info ("Calling fcontext script: fcontext.sh");
> KIWIQX::qxx (" cp $imageDesc/fcontext.sh $root/tmp ");
> KIWIQX::qxx (" chmod u+x $root/tmp/fcontext.sh ");
> my ($code,$data) = KIWIGlobals -> instance() -> callContained (
> $root,"/tmp/fcontext.sh"
> );
> if ($code != 0) {
> $kiwi -> failed ();
> $kiwi -> info ($data);
> return;
> } else {
> $kiwi -> loginfo ("fcontext.sh: $data");
> }
> KIWIQX::qxx (" rm -f $root/tmp/fcontext.sh ");
> $kiwi -> done ();
> }
> Now all files have correct context. Does this make sense?

You can go that route, but I'm not in favour of adding yet another
script hook.

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc

[Santosh]

Certainly, I will give it a try.

Just for FYI: I found 3 policies when I compile the source. (namely, mls, sandbox and targeted though I haven't tried MLS or sandbox)

-rw-r--r--. 1 root root 3102676 Apr 22 18:10 selinux-policy-mls-3.13.1-60.el7.centos.3.noarch.rpm

-rw-r--r--. 1 root root  386884 Apr 22 18:10 selinux-policy-sandbox-3.13.1-60.el7.centos.3.noarch.rpm

-rw-r--r--. 1 root root 4124168 Apr 22 18:10 selinux-policy-targeted-3.13.1-60.el7.centos.3.noarch.rpm

By doing this, 

security_context = '/etc/selinux/targeted/contexts/files/file_contexts'

We are enforcing targeted policy only. Do you plan on making this configurable?

Thanks

[Marcus Schäfer]

Hi,

> By doing this,
> security_context = '/etc/selinux/targeted/contexts/files/file_contexts'
> We are enforcing targeted policy only. Do you plan on making this
> configurable?

We can add an attribute in the XML to select the security policy.
If you think this needs to be done please open an issue for it

Thanks

[Santosh]

just did the image creation with 8.13.2 and image creation failed with selinux errors:

[ INFO    ]: 10:01:11 | Syncing system to image

[ INFO    ]: 10:01:11 | --> Syncing EFI boot data to EFI partition

[ ERROR   ]: 10:01:11 | KiwiCommandError: rsync: stderr: rsync: rsync_xal_set: lsetxattr(""/tmp/kiwi_filesystem.awnry426/."","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/tmp/kiwi_filesystem.awnry426/EFI"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/tmp/kiwi_filesystem.awnry426/EFI/BOOT"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/tmp/kiwi_filesystem.awnry426/EFI/BOOT/.bootx64.efi.5fZ4YB"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/tmp/kiwi_filesystem.awnry426/EFI/BOOT/.earlyboot.cfg.XA8kZT"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/tmp/kiwi_filesystem.awnry426/EFI/centos"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/tmp/kiwi_filesystem.awnry426/EFI/centos/.gcdx64.efi.1FUBZb"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/tmp/kiwi_filesystem.awnry426/EFI/centos/.grubx64.efi.dKon1t"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/tmp/kiwi_filesystem.awnry426/EFI/centos/fonts"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/tmp/kiwi_filesystem.awnry426/EFI/centos/fonts/.unicode.pf2.5BUJ5L"","security.selinux") failed: Operation not supported (95)

rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1]

, stdout: (no output on stdout)

[ INFO    ]: 10:01:11 | Cleaning up InstallImageBuilder instance

[ INFO    ]: 10:01:11 | Cleaning up FileSystemFat16 instance

[ INFO    ]: 10:01:11 | Cleaning up BootImageKiwi instance

[ INFO    ]: 10:01:12 | Cleaning up Disk instance

[ INFO    ]: 10:01:12 | Cleaning up LoopDevice instance

I temporarily commented the code out to let the build process go through.

Searching internet, I found a couple of hits around this.. though this one looks very interesting.

https://lists.samba.org/archive/rsync/2009-April/023150.html

Please review.

Thanks

[Marcus Schäfer]

Hi,

> just did the image creation with 8.13.2 and image creation failed with
> selinux errors:
> [ INFO ]: 10:01:11 | Syncing system to image
> [ INFO ]: 10:01:11 | --> Syncing EFI boot data to EFI partition
> [ ERROR ]: 10:01:11 | KiwiCommandError: rsync: stderr: rsync:
> rsync_xal_set:
> lsetxattr(""/tmp/kiwi_filesystem.awnry426/."","security.selinux")
> failed: Operation not supported (95)

This points to the selinux setup on your building host right ?
It does not allow xattr when kiwi runs the rsync. I think you
need to open the host for this kind of operations.

[Santosh]

Thanks. I used ext3 for target-dir and I didn't get SELinux errors. It seems SELinux and brtfs may have some issues. Quick google search results in many pages!!

 mount |egrep "btrfs|ext3"

/dev/sda2 on / type btrfs (rw,relatime,space_cache)

/dev/sda2 on /.snapshots type btrfs (rw,relatime,space_cache)

/dev/sdc on /tmp type ext3 (rw,nosuid,noexec,relatime,data=ordered)

/dev/sdb on /mnt/disksdb type ext3 (rw,relatime,data=ordered)

kiwi --profile ovf --type vmx --debug --logfile ovf.logfile system build --description appliance --target-dir=/mnt/disksdb

and I got this in the log:

[ INFO    ]: 13:56:51 | Processing SELinux file security contexts

[ DEBUG   ]: 13:56:51 | EXEC: [chroot /mnt/disksdb/build/image-root setfiles /etc/selinux/targeted/contexts/files/file_contexts / -v]

[ INFO    ]: 13:57:01 | Syncing system to image

No more rsync_xal_set error :-)

[Santosh]

The last comment is for ovf where I am not using EFI and I think that's the reason SELinux file context labeling is working fine as underlying filesystem is not fat or dfat variant. Whereas

in case of iso, I am using efi and it is fat filesytem which may not be supporting extended attrs that selinux requires and due to that, context labeling is failing.

In kiwi7, where I added one config file and was executing at the very end of prepare stage, I may not have the fat filesystem then and that's why it worked there.

that's my theory.. What do you think?

Thanks a lot.

[Santosh]

I compared the steps between kiwi 7 and kiwi 8.

kiwi 7 did cp -a whereas kiwi 8 is doing rsync -a.

I am also referring to the doc mentioned at https://ibboard.co.uk/Linux/httpd-from-vfat-with-selinux.html which states that "but FAT file systems cannot store context data"

It seems cp as well as rsync cannot copy extended attributes to fat16 as fat16 doesn't support them but return code of cp is 0 where as rsync fails with error.

To test it further,

I created a 200MB partition and created FAT16

NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT

sda      8:0    0   32G  0 disk

|-sda1   8:1    0    2M  0 part

|-sda2   8:2    0  200M  0 part /boot/efi

`-sda3   8:3    0 31.8G  0 part /

sdb      8:16   0  200M  0 disk /mnt/dosfs <--------

sr0     11:0    1 1024M  0 rom

/dev/sdb on /mnt/dosfs type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)

and in rsync, I got the same error:

[root@localhost ~]# rsync -a -v -H -X -A --one-file-system /boot/efi /mnt/dosfs/

sending incremental file list

efi/

efi/EFI/

efi/EFI/BOOT/

efi/EFI/BOOT/bootx64.efi

efi/EFI/BOOT/grub.cfg

rsync: rsync_xal_set: lsetxattr(""/mnt/dosfs/efi"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/mnt/dosfs/efi/EFI"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/mnt/dosfs/efi/EFI/BOOT"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/mnt/dosfs/efi/EFI/BOOT/.bootx64.efi.nd5jjU"","security.selinux") failed: Operation not supported (95)

rsync: rsync_xal_set: lsetxattr(""/mnt/dosfs/efi/EFI/BOOT/.grub.cfg.dqdM91"","security.selinux") failed: Operation not supported (95)

sent 737170 bytes  received 67 bytes  1474474.00 bytes/sec

total size is 736809  speedup is 1.00

rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]

 getfattr -n security.selinux EFI/  <---------------------getfattr errors

EFI/: security.selinux: Operation not supported

[root@localhost ~]# cp -ar /boot/efi /mnt/dosfs/  <------no error but getfattr throws error

[root@localhost ~]#

getfattr -n security.selinux EFI/

EFI/: security.selinux: Operation not supported.

At this time I am kinda convinced that for EFI partition, we will have to ignore the error. Please share your thought.

[Marcus Schäfer]

Hi,

> It seems cp as well as rsync cannot copy extended attributes to fat16
> as fat16 doesn't support them but return code of cp is 0 where as rsync
> fails with error.

This is interesting, thanks for investigating this. I see rsync to warn
here on my system but not to error out as it seems the case on your system.
In any case we should not sync with xattr if the underlaying filesystem
does not support it.

I will add that functionality to the DataSync class

Thanks, expect new packages today

[Santosh]

Thanks. I am trying to test it but unable to find python3-xattr. 

python3-xattr needed by python3-kiwi-8.14.1-1.1.x86_64

Please tell me the repository you used to get this.

[Marcus Schäfer]

Hi,

> Thanks. I am trying to test it but unable to find python3-xattr.Â

> python3-xattr needed by python3-kiwi-8.14.1-1.1.x86_64
> Please tell me the repository you used to get this.

Sorry my fault, I forgot to click the publish button.
The repo state is currently rebuild, you should have it in a minute

Regards,
Marcus
--
Public Key available via: https://keybase.io

[Santosh]

thanks so much. 

[Santosh]

I can confirm that iso image is building fine with selinux file context. Thanks for all your prompt response. Much appreciated.

[Marcus Schäfer]

Hi,

> I can confirm that iso image is building fine with selinux file
> context. Thanks for all your prompt response. Much appreciated.

Thanks much for your feedback and debugging.
You helped a lot to improve kiwi :)

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc