Curly braces dont work in shell script

[Dashmeet Anand]

Hi team,

Whenever I try to use curly braces in shell script
for e.g. command like this 
find $VAR -type f -mtime +60 -exec rm -f {} \\;

I get the following error from the test script.

(venv) jenkins_yaml % jenkins-jobs test config_jenkins_casc.yaml
WARNING:jenkins_jobs.cli.subcommand.test:(Deprecated) The default output behavior of `jenkins-jobs test` when given the --output flag will change in JJB 3.0. Instead of writing jobs to OUTPUT/jobname; they will be written to OUTPUT/jobname/config.xml. The new behavior can be enabled by the passing `--config-xml` parameter.
INFO:jenkins_jobs.cli.subcommand.base:Updating jobs in [PosixPath('config_jenkins_casc.yaml')] ([])
Traceback (most recent call last):
  File "venv/bin/jenkins-jobs", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "jenkins-job-builder/jenkins_jobs/cli/entry.py", line 179, in main
    jjb.execute()
  File "jenkins-job-builder/jenkins_jobs/cli/entry.py", line 158, in execute
    ext.obj.execute(self.options, self.jjb_config)
  File "jenkins-job-builder/jenkins_jobs/cli/subcommand/test.py", line 64, in execute
    builder, xml_jobs, xml_views = self.make_jobs_and_views_xml(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/cli/subcommand/base.py", line 121, in make_jobs_and_views_xml
    jobs = filter_matching(roots.generate_jobs(), glob_list)
                           ^^^^^^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/roots.py", line 57, in generate_jobs
    expanded_jobs += job.top_level_generate_items()
  File "jenkins-job-builder/jenkins_jobs/root_base.py", line 117, in top_level_generate_items
    expanded_contents = self._expand_contents(contents, item_params)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/job.py", line 63, in _expand_contents
      This job will reconfigure jenkins node templates and global environment variables
    expanded_contents = super()._expand_contents(contents, params)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/root_base.py", line 96, in _expand_contents
    expanded_contents = self._expander.expand(contents, params)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/expander.py", line 141, in expand
    return expander(obj, params, key_pos, value_pos)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/expander.py", line 35, in expand_dict
    expanded_value = expander.expand(
                     ^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/expander.py", line 141, in expand
    return expander(obj, params, key_pos, value_pos)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/expander.py", line 45, in expand_list
    items = [
            ^
  File "jenkins-job-builder/jenkins_jobs/expander.py", line 46, in <listcomp>
    expander.expand(item, params, None, obj.value_pos[idx])
  File "jenkins-job-builder/jenkins_jobs/expander.py", line 141, in expand
    return expander(obj, params, key_pos, value_pos)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/expander.py", line 35, in expand_dict
    expanded_value = expander.expand(
                     ^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/expander.py", line 141, in expand
    return expander(obj, params, key_pos, value_pos)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/expander.py", line 63, in __call__
    return self._formatter.format(str(obj), **params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 190, in format
    return self.vformat(format_string, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "jenkins-job-builder/jenkins_jobs/formatter.py", line 79, in vformat
    return super().vformat(format_string, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 194, in vformat
    result, _ = self._vformat(format_string, args, kwargs, used_args, 2)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 234, in _vformat
    obj, arg_used = self.get_field(field_name, args, kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 299, in get_field
    obj = self.get_value(first, args, kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daanand/Learning/jenkins_yaml/jenkins-job-builder/jenkins_jobs/formatter.py", line 114, in get_value
    return super().get_value(key, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 254, in get_value
    return args[key]

I've confirmed that it only happens when I try to use curly braces. I found this by trial and error and removing each line in my code.

I would also like to highlight that the error is not very useful and does not pin point the issue.

Any assistance is appreciated. 

[Vsevolod Fedorov]

Hi, Anand.

JJB uses curly braces for variables. So, it assumes that there are variables inside, and tries to expand them. To avoid that, you can just escape them by doubling, like this:

- shell: |

    find $VAR -type f -mtime +60 -exec rm -f {{}} \\;

    echo "This is expanded JJB variable 'var': {var}"

But I suggest to wrap bash scripts, json and other texts often containing curly braces with `!j2:` prefix. This way, JJB will expect Jinja2 template inside prefixed text, which uses double curly braces for variables:

- shell: !j2: |

    find $VAR -type f -mtime +60 -exec rm -f {} \\;

    echo "This is expanded JJB variable 'var': {{ var }}"

By the way, can you provide JJB source snippet which causes this failure? This looks like a JJB bug - it should not fail in those cases, but show proper error. And which JJB version do you use?

Vsevolod.

--
You received this message because you are subscribed to the Google Groups "jenkins-job-builder" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkins-job-bui...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkins-job-builder/a65ae7e6-d680-4979-a269-5ec05dd669f7n%40googlegroups.com.

[Dashmeet Anand]

Thanks for the response.

JJB version

Name: jenkins-job-builder
Version: 6.2.1.dev1
Summary: Manage Jenkins jobs with YAML
Home-page: https://jenkins-job-builder.readthedocs.io/en/latest/
Author: OpenStack Infrastructure Team
Author-email: jenkins-j...@googlegroups.com
License: Apache License, Version 2.0
Location: /Users/daanand/Learning/jenkins_yaml/jenkins-job-builder
Requires: fasteners, Jinja2, pbr, python-jenkins, PyYAML, six, stevedore
Required-by:


When you mention source snippet what exactly can I provide you?