"commands: arglist" mismatch between behaviour and documentation?

[t.d...@servicemusic.org.uk]

Using 3.18.5 community edition on RHEL9; also 3.15 on RHEL7 and RHEL8

Documentation https://docs.cfengine.com/docs/3.18/reference-promise-types-commands.html gives the example:

-------------------------------------------------

commands: "/bin/echo one" args => "two three", arglist => { "four", "five" };

So in the example above the command would be:

/bin/echo one two three four five

-------------------------------------------------

It also states:

-------------------------------------------------

That's particularly useful when there are embedded spaces and quotes in your arguments...

-------------------------------------------------

So I'm trying to build something, specifically requiring that "embedded spaces" claim.  But I hit a problem, so strip it down to a proof-of-concept script:

-------------------------------------------------

#! /bin/bash
echo "argcount: $#"
for (( i=1 ; i<=$#; i++))
do    
  echo "${i}:${!i}"
done

-------------------------------------------------

At the command line I verify the script:

-------------------------------------------------

$ ./modules/printargs A B "C with spaces" D
argcount: 4
1:A
2:B
3:C with spaces
4:D
$

-------------------------------------------------

(Four arguments, the third being three words with embedded spaces.)

I then run it from within our CFEngine framework:

-------------------------------------------------

  commands:
    any::

      "$(gcom.modulesdir)/printargs"
        arglist => { "A", "B", "C with spaces", "D" };

-------------------------------------------------

But instead of receiving four arguments, it receives six: here's the CFE "inform" output:
-------------------------------------------------

    info: Executing 'no timeout' ... '[...]/modules/printargs A B C with spaces D'
  notice: Q: ".../printargs A B ": argcount: 6
Q: ".../printargs A B ": 1:A
Q: ".../printargs A B ": 2:B
Q: ".../printargs A B ": 3:C
Q: ".../printargs A B ": 4:with
Q: ".../printargs A B ": 5:spaces
Q: ".../printargs A B ": 6:D
    info: Last 7 quoted lines were generated by promiser

-------------------------------------------------
That useful "embedded spaces" claim in the documentation seems to be being ignored.

Is the bug/problem:

• in the code
• or in the documentation
• or in my interpretation of the documentation

Help appreciated

-- David Lee

[Vratislav Podzimek]

>  * in the code
>  * or in the documentation
>  * or in my interpretation of the documentation
> Help appreciated
Definitely looks like a bug in the CFEngine code to me. I'm just surprised there's no failing test
for this. Please report the issue at https://northerntech.atlassian.net/browse/CFE

Thanks!

--
Vratislav

[t.d...@servicemusic.org.uk]

I appear no longer to have login access to be able to report issues.  (Previous issues with my involvement include CFE-3020, CFE-1655, CFE-2908, etc.)  I'm trying to regain that access, but it is not proving easy!  If that can be restored then I can report the issue.

Otherwise, could you log the issue for me?

Thanks.

-- David Lee

[Mike Weilgart]

Hi David,

I thought this sounded familiar, and indeed, I reported the same bug five years ago!  https://northerntech.atlassian.net/browse/CFE-2869

And it was reported a year before that as well: https://northerntech.atlassian.net/browse/CFE-2724

And David, I see that you've now successfully reported the issue as https://northerntech.atlassian.net/browse/CFE-4253, also.

To Northern Tech: I suggest that the code should be fixed for a future CFEngine version, AND that the documentation should be fixed for all existing CFEngine versions.  Should be modified to better describe what the code actually does, and then a note should be inserted in a little text box for the benefit of those rereading the docs who've read it before, saying something like, "NOTE: This documentation previously implied that the elements of arglist were each treated as separate arguments and could contain spaces or quotes; however the behavior of the code did not match this documentation, so the documentation has been updated.  A future CFEngine version may modify this behavior; see (link) for the bug tracking info."

Or just quietly update the doc to be more precisely descriptive, since it doesn't actually *say* how the arglist is currently processed, it just implies it with the "particularly useful" statement.

The documentation fix should be fairly trivial and I submit there's really no good reason to leave inaccurate documentation there to trip people up, even if the coding fix takes a while to prioritize and accomplish.

...Hmm, interesting, I think there is currently no way to have explicit control over how CFEngine handles the whitespace splitting in a commands promise...maybe an attribute for this could be added, with the default set to be the current behavior, for backward compatibility.  Maybe call it "argsplitting" and default it to "whitespace" (IF that is an accurate description of current code behavior; I think it is), and have another option "none" which would result in *args* being treated as one single argument; and *arglist* being treated as exactly as many arguments as appear as elements in that list.

Best,

--Mike Weilgart

[Lars Erik Wik]

Hi all

Thanks for shedding a light on this issue again. I've linked the issues and pushed the CFE-2724 ticket to the top of the backlog. As a temporary workaround you can use nested quoting, e.g.:

```
$ cat ~/test.cf
bundle agent __main__
{
  files:
    "/tmp/test.py"
      content => "import sys; print(*(arg for arg in sys.argv), sep='\n')";
  commands:
    "/usr/bin/python3 /tmp/test.py"
      args => "two `three four`",
      arglist => { "five", "'six seven'" };
}
$ /var/cfengine/bin/cf-agent -Kf ~/test.cf 
  notice: Q: "...in/python3 /tmp": /tmp/test.py
Q: "...in/python3 /tmp": two
Q: "...in/python3 /tmp": three four
Q: "...in/python3 /tmp": five
Q: "...in/python3 /tmp": six seven
```

[Mike Weilgart]

Thanks, Lars!

Side note about nested quoting, probably the cleanest and most flexible syntax I've seen for nested quoting when it would otherwise get really hairy, is that used by Postgres: https://www.postgresql.org/docs/current/sql-syntax-lexical.html#SQL-SYNTAX-DOLLAR-QUOTING

--Mike Weilgart

--
You received this message because you are subscribed to a topic in the Google Groups "help-cfengine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/help-cfengine/70qc5E83BMA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to help-cfengin...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/help-cfengine/7f369e0e-4b13-460a-a613-5538835a628dn%40googlegroups.com.

[t.d...@servicemusic.org.uk]

Lars and Mike,

Thanks for your replies on this.  Appreciated and useful!

Given an incoming, pre-existing "@(lines)" slist that includes elements which potentially contain embedded spaces, and following Lars' useful suggestion, I have inserted:

  vars:

    any::

      "lines_q" slist => maplist("'$(this)'", "@(lines)");

which simply puts additional quotes around each.  I then use that new "@(lines_q)" from that point onwards.

(As Mike hints, this isn't a full, general-purpose solution.  But for the present it looks sufficient in this particular, relatively simple, instance.)

-- David Lee

[Lars Erik Wik]

Glad to hear! Thanks for sharing your solution with the clever use of maplist.

[Lars Erik Wik]

FYI, this bug will be fixed in CFEngine 3.24.  For more info check out this blog post https://cfengine.com/blog/2024/change-in-behavior-arglist/ Thanks for reporting it.

[Mike Weilgart]

Awesome!  Thanks for the fix and the blog post!  :)

--Mike

To view this discussion on the web visit https://groups.google.com/d/msgid/help-cfengine/323c2810-00b8-4b17-818e-2f0a90548c70n%40googlegroups.com.

[t.d...@servicemusic.org.uk]

Lars,

Many thanks!

-- David Lee

-- Diamond Light Source