does tidy cross file system boundary?

[Gregory Matthews]

Can someone answer the following question:

does the cfengine "tidy" promise cross file systems?

If a file system is mounted inside a directory that is subject to a
"tidy" promise, is it recursed into?

We have an ongoing incident where this is under suspicion so would be
great to an answer!

GREG
--
Greg Matthews 07795 952895
Scientific Computing Operations Manager
Diamond Light Source Ltd Oxfordshire UK

--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

[Gregory Matthews]

I should add "if xdev is not used"

so if xdev is not mentioned when recursing, does the tidy descend into
mounted filesystems?

G

[Nick Anderson]

Gregory Matthews writes:

Can someone answer the following question:

does the cfengine "tidy" promise cross file systems?

Hi Greg,

Per the documentation for the depth_search body xdev, Description: true/false exclude directories that are on different devices and the attribute is False by default. So, I would not expect the agent to traverse file systems if xdev is not explicitly used.

If a file system is mounted inside a directory that is subject to a "tidy" promise, is it recursed into?

It should unless xdev is true, as in the case of depth_search recurse(d) from the stdlib:

cf-locate  -f -p "depth_search recurse\(d\)"

body depth_search recurse(d)
# @brief Search files and direcories recursively, up to the specified depth
# Directories on different devices are included.
#
# @param d The maximum search depth
{
      depth => "$(d)";
      xdev  => "true";
}

But, it's good to test things:

bundle agent test_xdev_init
{
  files:
      "/tmp/xdev-device-test-tree/subdir1/stuff/file"
        create => "true",
        comment => "a file to test with";

      "/tmp/xdev-device-test-tree/subdir/xdev/."
        create => "true",
        comment => "We need a directory where we can mount a file system.";
}
bundle agent test_xdev_device
{
  commands:
      "dd if=/dev/zero of=/tmp/xdev-device bs=1024 count=10240"
        contain => in_shell,
        unless => fileexists( "/tmp/xdev-device" ),
        comment => "This file will contain a file system and be mounted to test xdev";

      "mkfs.ext3 /tmp/xdev-device"
        contain => in_shell,
        comment => "Here we give the device a file system",
        classes => results( "bundle", "xdev_device_filesystem" ),
        if => fileexists( "/tmp/xdev-device" ),
        unless => fileexists( "/tmp/xdev-device.has_filesystem" );

  files:
    xdev_device_filesystem_repaired::
      "/tmp/xdev-device.has_filesystem"
        create => "true";

    xdev_device_mount_repaired::
      "/tmp/xdev-device.is_mounted"
        create => "true";

  commands:
      "mount /tmp/xdev-device /tmp/xdev-device-test-tree/subdir/xdev"
        classes => results( "bundle", "xdev_device_mount" ),
        contain => in_shell,
        unless => fileexists( "/tmp/xdev-device.is_mounted" );

  files:

      "/tmp/xdev-device-test-tree/subdir/xdev/file-on-xdev"
        create => "true",
        if => fileexists( "/tmp/xdev-device.is_mounted" );
}
bundle agent test_xdev
{
  files:
      "/tmp/xdev-device-test-tree/."
        depth_search => inf_no_xdev,
        file_select => plain,
        delete => tidy,
        if => fileexists( "/tmp/xdev-device.is_mounted" );
}

body depth_search inf_no_xdev
{
        depth => "inf";
        xdev  => "true";
}

bundle agent __main__
{
  methods:

      "test_xdev_init";
      "test_xdev_device";
      "test_xdev";
}

exec 2>&1

test_xdev()
{
  echo "Running test policy with CFEngine $(cf-agent -V | head -n 1)"
  echo "xdev configuration: "
  grep "xdev  =>" /tmp/test-xdev.cf

  cf-agent -KIf /tmp/test-xdev.cf # Run wiht -v to see log messages like Skipping 'FILE' on differnt device when xdev is TRUE!

  test -f /tmp/xdev-device-mountpoint/./subdir/xdev/file-on-xdev && echo "File on another file system exists"
  test -f /tmp/xdev-device-mountpoint/./subdir/xdev/file-on-xdev || echo "File on another filesystem is abent"

  echo "Showing mount and file tree after policy run"
  mount | grep xdev
  find /tmp/xdev-device-test-tree

  echo "Cleanup"
  umount /tmp/xdev-device-test-tree/subdir/xdev/
  rm -rf /tmp/xdev-device*
  echo "DONE\n\n"
}

sed -i 's/.*xdev  =>.*/        #xdev  => "false";/g' /tmp/test-xdev.cf
test_xdev
sed -i 's/.*xdev  =>.*/        xdev  => "false";/g' /tmp/test-xdev.cf
test_xdev
sed -i 's/.*xdev  =>.*/        xdev  => "true";/g' /tmp/test-xdev.cf
test_xdev
:

Running test policy with CFEngine CFEngine Core 3.17.0
xdev configuration: 
        #xdev  => "false";
    info: Created directory for '/tmp/xdev-device-test-tree/subdir1/stuff/file'
    info: Created file '/tmp/xdev-device-test-tree/subdir1/stuff/file', mode 0600
    info: files promise '/tmp/xdev-device-test-tree/subdir1/stuff/file' repaired
    info: Created directory '/tmp/xdev-device-test-tree/subdir/xdev/.'
    info: files promise '/tmp/xdev-device-test-tree/subdir/xdev' repaired
    info: Executing 'no timeout' ... 'dd if=/dev/zero of=/tmp/xdev-device bs=1024 count=10240'
  notice: Q: "...dd if=/dev/zero": 10240+0 records in
Q: "...dd if=/dev/zero": 10240+0 records out
Q: "...dd if=/dev/zero": 10485760 bytes (10 MB, 10 MiB) copied, 0.0225902 s, 464 MB/s
    info: Last 3 quoted lines were generated by promiser 'dd if=/dev/zero of=/tmp/xdev-device bs=1024 count=10240'
    info: Completed execution of 'dd if=/dev/zero of=/tmp/xdev-device bs=1024 count=10240'
    info: Executing 'no timeout' ... 'mkfs.ext3 /tmp/xdev-device'
  notice: Q: "...mkfs.ext3 /tmp/": mke2fs 1.45.5 (07-Jan-2020)
Q: "...mkfs.ext3 /tmp/": Discarding device blocks:                   done                            
Q: "...mkfs.ext3 /tmp/": Creating filesystem with 2560 4k blocks and 2560 inodes
Q: "...mkfs.ext3 /tmp/": Allocating group tables: 0/1         done                            
Q: "...mkfs.ext3 /tmp/": Writing inode tables: 0/1         done                            
Q: "...mkfs.ext3 /tmp/": Creating journal (1024 blocks): done
Q: "...mkfs.ext3 /tmp/": Writing superblocks and filesystem accounting information: 0/1         done
    info: Last 7 quoted lines were generated by promiser 'mkfs.ext3 /tmp/xdev-device'
    info: Completed execution of 'mkfs.ext3 /tmp/xdev-device'
    info: Executing 'no timeout' ... 'mount /tmp/xdev-device /tmp/xdev-device-test-tree/subdir/xdev'
    info: Completed execution of 'mount /tmp/xdev-device /tmp/xdev-device-test-tree/subdir/xdev'
    info: Created file '/tmp/xdev-device.has_filesystem', mode 0600
    info: files promise '/tmp/xdev-device.has_filesystem' repaired
    info: Created file '/tmp/xdev-device.is_mounted', mode 0600
    info: files promise '/tmp/xdev-device.is_mounted' repaired
    info: Created file '/tmp/xdev-device-test-tree/subdir/xdev/file-on-xdev', mode 0600
    info: files promise '/tmp/xdev-device-test-tree/subdir/xdev/file-on-xdev' repaired
    info: Deleted file '/tmp/xdev-device-test-tree/./subdir/xdev/file-on-xdev'
    info: Deleted file '/tmp/xdev-device-test-tree/./subdir1/stuff/file'
    info: files promise '/tmp/xdev-device-test-tree/.' repaired
File on another file system exists
File on another filesystem is abent
Showing mount and file tree after policy run
/tmp/xdev-device on /tmp/xdev-device-test-tree/subdir/xdev type ext3 (rw,relatime)
/tmp/xdev-device-test-tree
/tmp/xdev-device-test-tree/subdir
/tmp/xdev-device-test-tree/subdir/xdev
/tmp/xdev-device-test-tree/subdir/xdev/lost+found
/tmp/xdev-device-test-tree/subdir1
/tmp/xdev-device-test-tree/subdir1/stuff
Cleanup
DONE


Running test policy with CFEngine CFEngine Core 3.17.0
xdev configuration: 
        xdev  => "false";
    info: Created directory for '/tmp/xdev-device-test-tree/subdir1/stuff/file'
    info: Created file '/tmp/xdev-device-test-tree/subdir1/stuff/file', mode 0600
    info: files promise '/tmp/xdev-device-test-tree/subdir1/stuff/file' repaired
    info: Created directory '/tmp/xdev-device-test-tree/subdir/xdev/.'
    info: files promise '/tmp/xdev-device-test-tree/subdir/xdev' repaired
    info: Executing 'no timeout' ... 'dd if=/dev/zero of=/tmp/xdev-device bs=1024 count=10240'
  notice: Q: "...dd if=/dev/zero": 10240+0 records in
Q: "...dd if=/dev/zero": 10240+0 records out
Q: "...dd if=/dev/zero": 10485760 bytes (10 MB, 10 MiB) copied, 0.0291186 s, 360 MB/s
    info: Last 3 quoted lines were generated by promiser 'dd if=/dev/zero of=/tmp/xdev-device bs=1024 count=10240'
    info: Completed execution of 'dd if=/dev/zero of=/tmp/xdev-device bs=1024 count=10240'
    info: Executing 'no timeout' ... 'mkfs.ext3 /tmp/xdev-device'
  notice: Q: "...mkfs.ext3 /tmp/": mke2fs 1.45.5 (07-Jan-2020)
Q: "...mkfs.ext3 /tmp/": Discarding device blocks:                   done                            
Q: "...mkfs.ext3 /tmp/": Creating filesystem with 2560 4k blocks and 2560 inodes
Q: "...mkfs.ext3 /tmp/": Allocating group tables: 0/1         done                            
Q: "...mkfs.ext3 /tmp/": Writing inode tables: 0/1         done                            
Q: "...mkfs.ext3 /tmp/": Creating journal (1024 blocks): done
Q: "...mkfs.ext3 /tmp/": Writing superblocks and filesystem accounting information: 0/1         done
    info: Last 7 quoted lines were generated by promiser 'mkfs.ext3 /tmp/xdev-device'
    info: Completed execution of 'mkfs.ext3 /tmp/xdev-device'
    info: Executing 'no timeout' ... 'mount /tmp/xdev-device /tmp/xdev-device-test-tree/subdir/xdev'
    info: Completed execution of 'mount /tmp/xdev-device /tmp/xdev-device-test-tree/subdir/xdev'
    info: Created file '/tmp/xdev-device.has_filesystem', mode 0600
    info: files promise '/tmp/xdev-device.has_filesystem' repaired
    info: Created file '/tmp/xdev-device.is_mounted', mode 0600
    info: files promise '/tmp/xdev-device.is_mounted' repaired
    info: Created file '/tmp/xdev-device-test-tree/subdir/xdev/file-on-xdev', mode 0600
    info: files promise '/tmp/xdev-device-test-tree/subdir/xdev/file-on-xdev' repaired
    info: Deleted file '/tmp/xdev-device-test-tree/./subdir/xdev/file-on-xdev'
    info: Deleted file '/tmp/xdev-device-test-tree/./subdir1/stuff/file'
    info: files promise '/tmp/xdev-device-test-tree/.' repaired
File on another file system exists
File on another filesystem is abent
Showing mount and file tree after policy run
/tmp/xdev-device on /tmp/xdev-device-test-tree/subdir/xdev type ext3 (rw,relatime)
/tmp/xdev-device-test-tree
/tmp/xdev-device-test-tree/subdir
/tmp/xdev-device-test-tree/subdir/xdev
/tmp/xdev-device-test-tree/subdir/xdev/lost+found
/tmp/xdev-device-test-tree/subdir1
/tmp/xdev-device-test-tree/subdir1/stuff
Cleanup
DONE


Running test policy with CFEngine CFEngine Core 3.17.0
xdev configuration: 
        xdev  => "true";
    info: Created directory for '/tmp/xdev-device-test-tree/subdir1/stuff/file'
    info: Created file '/tmp/xdev-device-test-tree/subdir1/stuff/file', mode 0600
    info: files promise '/tmp/xdev-device-test-tree/subdir1/stuff/file' repaired
    info: Created directory '/tmp/xdev-device-test-tree/subdir/xdev/.'
    info: files promise '/tmp/xdev-device-test-tree/subdir/xdev' repaired
    info: Executing 'no timeout' ... 'dd if=/dev/zero of=/tmp/xdev-device bs=1024 count=10240'
  notice: Q: "...dd if=/dev/zero": 10240+0 records in
Q: "...dd if=/dev/zero": 10240+0 records out
Q: "...dd if=/dev/zero": 10485760 bytes (10 MB, 10 MiB) copied, 0.0216883 s, 483 MB/s
    info: Last 3 quoted lines were generated by promiser 'dd if=/dev/zero of=/tmp/xdev-device bs=1024 count=10240'
    info: Completed execution of 'dd if=/dev/zero of=/tmp/xdev-device bs=1024 count=10240'
    info: Executing 'no timeout' ... 'mkfs.ext3 /tmp/xdev-device'
  notice: Q: "...mkfs.ext3 /tmp/": mke2fs 1.45.5 (07-Jan-2020)
Q: "...mkfs.ext3 /tmp/": Discarding device blocks:                   done                            
Q: "...mkfs.ext3 /tmp/": Creating filesystem with 2560 4k blocks and 2560 inodes
Q: "...mkfs.ext3 /tmp/": Allocating group tables: 0/1         done                            
Q: "...mkfs.ext3 /tmp/": Writing inode tables: 0/1         done                            
Q: "...mkfs.ext3 /tmp/": Creating journal (1024 blocks): done
Q: "...mkfs.ext3 /tmp/": Writing superblocks and filesystem accounting information: 0/1         done
    info: Last 7 quoted lines were generated by promiser 'mkfs.ext3 /tmp/xdev-device'
    info: Completed execution of 'mkfs.ext3 /tmp/xdev-device'
    info: Executing 'no timeout' ... 'mount /tmp/xdev-device /tmp/xdev-device-test-tree/subdir/xdev'
    info: Completed execution of 'mount /tmp/xdev-device /tmp/xdev-device-test-tree/subdir/xdev'
    info: Created file '/tmp/xdev-device.has_filesystem', mode 0600
    info: files promise '/tmp/xdev-device.has_filesystem' repaired
    info: Created file '/tmp/xdev-device.is_mounted', mode 0600
    info: files promise '/tmp/xdev-device.is_mounted' repaired
    info: Created file '/tmp/xdev-device-test-tree/subdir/xdev/file-on-xdev', mode 0600
    info: files promise '/tmp/xdev-device-test-tree/subdir/xdev/file-on-xdev' repaired
    info: Deleted file '/tmp/xdev-device-test-tree/./subdir1/stuff/file'
    info: files promise '/tmp/xdev-device-test-tree/.' repaired
File on another file system exists
File on another filesystem is abent
Showing mount and file tree after policy run
/tmp/xdev-device on /tmp/xdev-device-test-tree/subdir/xdev type ext3 (rw,relatime)
/tmp/xdev-device-test-tree
/tmp/xdev-device-test-tree/subdir
/tmp/xdev-device-test-tree/subdir/xdev
/tmp/xdev-device-test-tree/subdir/xdev/file-on-xdev
/tmp/xdev-device-test-tree/subdir/xdev/lost+found
/tmp/xdev-device-test-tree/subdir1
/tmp/xdev-device-test-tree/subdir1/stuff
Cleanup
DONE

So, it seems to be working here

[Nick Anderson]

heh correction in my first sentence:

> Per the documentation for the depth_search body xdev, Description: true/false exclude directories that are on different devices and the attribute is False by default. So, I would not expect the agent to traverse file systems if xdev is not explicitly used.

That should say I would expect the agent to traverse the file system unless xdev is set to exclude files from other devices.

[Aleksey Tsalolikhin]

The body "tidy" is defined in the CFEngine Standard Library.  You can find it here:  https://github.com/cfengine/masterfiles/blob/9750119782ace17cd35c7a2d6b9b8b073ad318e1/lib/files.cf#L1819-L1825

xdev is not defined, so per 

https://docs.cfengine.com/docs/3.15/reference-promise-types-files.html#xdev

I would expect it to traverse filesystem boundaries. 

The documentation states:

xdev

Description: true/false exclude directories that are on different devices

Type: boolean

Default value: false

-- 

Founder

Vertical Sysadmin, Inc.

Achieve real learning.

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/help-cfengine/87k0sif7zf.fsf%40northern.tech.
--
Nick Anderson | Doer of Things | (+1) 785-550-1767 | https://northern.tech

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/help-cfengine/87k0sif7zf.fsf%40northern.tech.

[Nick Anderson]

Aleksey Tsalolikhin writes:

The body "tidy" is defined in the CFEngine Standard Library. You can find it here: https://github.com/cfengine/masterfiles/blob/9750119782ace17cd35c7a2d6b9b8b073ad318e1/lib/files.cf#L1819-L1825

xdev is not defined, so per

https://docs.cfengine.com/docs/3.15/reference-promise-types-files.html#xdev

Note, xdev is not a valid attribute in a delete body, it's a depth_search attribute.

Listing 1: Shell command with stderr

exec 2>&1
echo "Valid attribtes for delete bodies (like tidy):"
cf-promises --syntax-description json | jq ".bodyTypes.delete.attributes|keys[]"

echo "Valid attribtes for depth_search bodies (like recurse):"
cf-promises --syntax-description json | jq ".bodyTypes.depth_search.attributes|keys[]"
:

Valid attribtes for delete bodies (like tidy):
"dirlinks"
"inherit_from"
"meta"
"rmdirs"
Valid attribtes for depth_search bodies (like recurse):
"depth"
"exclude_dirs"
"include_basedir"
"include_dirs"
"inherit_from"
"meta"
"rmdeadlinks"
"traverse_links"
"xdev"

[Aleksey Tsalolikhin]

Thanks, Nick, quite right.

https://github.com/cfengine/masterfiles/blob/9750119782ace17cd35c7a2d6b9b8b073ad318e1/lib/files.cf#L1771-L1779 is where depth_search is defined; xdev is set to true (as you mentioned).

One might think that means crossing filesystem boundaries is enabled, but it's actually the opposite.  :) 

Greg asked,

> does the cfengine "tidy" promise cross file systems?

"tidy" doesn't control recursion, only deletion.  Recursion is controlled by the "depth_search" attribute.  So we would need to look at the whole promise to state whether CFEngine is crossing filesystem boundaries.

As a side note --  I've noticed that when something mysterious is happening, people might say "is CFEngine doing it?" as an initial reaction.

When I was doing CFEngine consulting full-time in a large enterprise, this would frequently come up.  Often as a statement, not a question ("CFEngine is doing it!"), haha. I would do things like scan the CFE Enterprise promiselog and come back with "no, CFEngine didn't change anything on your system at the time you specified"; or just say, turn off CFEngine. Oh, you're still having your issue?  Well, okay, it's not CFEngine then, is it?  Turn CFEngine back on, please.  :)

That latter is applicable to all, whether running CFEngine Enterprise or Community Edition.

Of course very rarely it was CFEngine doing it. So you always have to look. Fun!

That "xdev" attribute is a little tricky because it's sort of a double negative situation. 

Best,

Aleksey

-- 

Founder

Vertical Sysadmin, Inc.

Achieve real learning.

[Nick Anderson]

Docs for this have been updated to clarify the behavior.

-**Description:** true/false exclude directories that are on different
-devices
+**Description:** When **true** files and directories on different devices from the promiser will be excluded from `dept
h_search` results.

doc builds in flight, so you should see the changes live shortly.

[Ted Zlatanov]

On Thu, 14 Jan 2021 10:37:42 -0800 (PST) "'Nick Anderson' via help-cfengine" <help-c...@googlegroups.com> wrote:

'Avh> Docs for this have been updated to clarify the behavior.
'Avh> -**Description:** true/false exclude directories that are on different
'Avh> -devices
'Avh> +**Description:** When **true** files and directories on different devices
'Avh> from the promiser will be excluded from `dept
'Avh> h_search` results.

'Avh> doc builds in flight, so you should see the changes live shortly.

So this is also incorrect in other place in the stdlib?

For example `body depth_search recurse_with_base` has xdev=true and says
it will include other devices.

Ted

[Aleksey Tsalolikhin]

Thanks, Nick!  Good of you to improve that text!

I find it's super clear the way the "find" man page has it:

       -xdev  Don't descend directories on other filesystems.

Consider stealing that wording?  :)  I still have to think a bit to parse "exclude", whereas the above wording is mechanical and everything is laid out already.  

Nice attention to detail as always, Ted.  :)  Thanks for the catch!

I love to see the tool (and supporting tooling/documentation) continue getting better still.

Best,

Aleksey

-- 

Founder

Vertical Sysadmin, Inc.

Achieve real learning.

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/help-cfengine/1remmru2.fsf%40lifelogs.com.

[Nick Anderson]

Ted Zlatanov writes:

So this is also incorrect in other place in the stdlib?

For example `body depth_search recurse_with_base` has xdev=true and says it will include other devices.

Ted, thanks for pointing that out!

Indeed, recurse_with_base has xdev true and the inline docs say:

nil

The docs for depth_search recurse also say included.

nil

I believe it is safer to update the documentation than to alter the implementation.

I will update the inline docs for currently maintained branches momentarily.

[Ted Zlatanov]

On Fri, 15 Jan 2021 09:46:35 -0600 "'Nick Anderson' via help-cfengine" <help-c...@googlegroups.com> wrote:

NA> Indeed, recurse_with_base has xdev true and the inline docs say:
NA> The docs for depth_search recurse also say /included/.
NA> I believe it is safer to update the documentation than to alter the
NA> implementation.

Yeah, absolutely. Thank you so much for fixing the docs.

Ted

[Nick Anderson]

I won't open another pr for change to the already changed places (docs, core, and now MPF), but I won't object to someone else opening pull requests to further clarifying the docs.

[Nick Anderson]

https://tracker.mender.io/browse/CFE-3541

[Gregory Matthews]

thanks all...

this was a bit hair-raising - we had a cleanup script that was migrated
from tmpwatch to cfengine "tidy" but someone had a CIFS mount in the
same area and the tidy traversed into the mounted file system and
removed a ton of stuff... ugh.

The doc for xdev was not sufficiently clear hence my badly worded
question. In the meantime we did our own testing and added an "xdev =>
true".

G

[Aleksey Tsalolikhin]

Ouch! Sorry to hear that, Greg. Glad you got to the bottom of it. And Nick's fixed the doc now as well (though that's closing the barn door after the horse got out).

Best,

Aleksey

-- 

Founder

Vertical Sysadmin, Inc.

Achieve real learning.

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/help-cfengine/1fd388b4-117d-e61d-6799-d621384c4ca0%40diamond.ac.uk.