Changelog?

42 views
Skip to first unread message

Dan Mick

unread,
Feb 9, 2024, 3:09:12 PM2/9/24
to jenkins-job-builder
Hi; we've been using jjb for years, and recently our CI started breaking; it appears as though some version has started interpreting things in our yml files as variables to expand.  All I know so far is that 4.3.0 didn't have this behavior and 6.0 does.

I can't find a place in the docs that talks about when tokens are expanded or how (other than inside templates, which these config files are not using), and I also can't find a changelog.  Can anyone point me at:

1) a definition of variable syntax and replacement rules, and/or
2) a changelog describing at least major breaking changes?

Dan Mick

unread,
Feb 9, 2024, 3:21:46 PM2/9/24
to jenkins-j...@googlegroups.com
I at least was able to test all the releases on pypi.org, and know that
it's 6.0.0 that's done it. Constructs like "${BRANCH}" in a builders
section are throwing

While formatting string '${BRANCH}': Missing parameter: 'BRANCH'
label: '${BRANCH}'
> --
> You received this message because you are subscribed to the Google
> Groups "jenkins-job-builder" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jenkins-job-bui...@googlegroups.com
> <mailto:jenkins-job-bui...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkins-job-builder/f027126e-0856-4889-b716-a333d715d4d6n%40googlegroups.com <https://groups.google.com/d/msgid/jenkins-job-builder/f027126e-0856-4889-b716-a333d715d4d6n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Tony Chia

unread,
Feb 9, 2024, 4:56:27 PM2/9/24
to Dan Mick, jenkins-j...@googlegroups.com

Here is what i could find from my mailbox



======================================================

[jjb] Jenkins Job Builder 6.0.0 is now available
======================================================

Vsevolod Fedorov
Thu, Jan 25, 2:35 AM
to jenkins-job-builder

Hi Everyone,

Jenkins Job Builder 6.0.0 is released.

There are breaking changes since previous 5.1.0 release:
Jobs are now expanded the same way as job templates;
Macros without parameters are now expanded the same way as macros with parameters;
Tags !include-raw: and !include-raw-escape: should now be used the same way in jobs and macros without parameters as they are used in job templates and macros with parameters.
See also these stories:
Expand jobs the same way as job templates are expanded (Story 2010963);
Expand macros even if called without arguments (Story 2010588).
To move to 6.0.0 release, make these adjustments to your JJB sources:
For every job and macro without parameters:
Duplicate curly braces: {...} -> {{...}};
Replace tags !include-raw: -> !include-raw-escape:
See this example: job-and-macro-expansions.yaml
Also, global defaults are now used when expanding job elements the same way as they are used for expanding job templates.
See this example:
concat_defaults003_job.yaml
concat_defaults003_job.xml

Commit list:

b669001e Vsevolod Fedorov Merge global defaults for jobs also
18efe506 Vsevolod Fedorov Unify variable and tags expansion inside jobs and macros
081fcaa0 Vsevolod Fedorov Add example for variable expansion inside macros and jobs

Vsevolod.


--


======================================================

[jjb] Jenkins Job Builder 5.1.0 is now available

======================================================
Hi Everyone,

Jenkins Job Builder 5.1.0 is released.

Changes since previous 5.0.4 release:
Restored macros support for notifications. It was lost with 5.0.0 release;
Added macro call context to errors;
Fixed bug: Folder defined at defaults is ignored (Story 2010984);
Fixed bug: Wrong files adding by tag !include-raw (include-raw-escape, etc) (Story 2010711) (by Maxim Trunov);
Fixed bug: On multibranch projects ignore-tags-older-than and ignore-tags-newer-than are inverted (Story 2004614) (by Michal Szelag);
Fixed legacy plugin version comparison (Story 2010990).
This also closed:
Story 2009943: PostBuildScript Plugin Version Format Change in 3.1.0-375.v3db_cd92485e1 Breaks Job Builder Version Compares;
Story 2009819: Slack Plugin Version Format Change in 602.v0da_f7458945d Breaks Job Builder Version Compares;
Removed cap on setuptools version (Story 2010842);
Added support for Python 3.11.

Commit list:

35a75d28 Vsevolod Fedorov Add support for Python 3.11
67645a46 Vsevolod Fedorov Fix legacy plugin version comparison; Remove cap on setuptools version
e2cf69b4 Vsevolod Fedorov Fix support for obsolete format of pre-scm-buildstep
e2735811 Vsevolod Fedorov Add missing tests for some wrappers
429118b4 Michal Szelag Fixing ignore-tags-newer-than inverted with ignore-tags-older-than.
d741ea02 Vsevolod Fedorov Task: 47875 Story: 2010711 Search for included files relative to the current job first
d9c10ec7 Vsevolod Fedorov Tests: Add ability to take several input files in one test
af9e394c Vsevolod Fedorov Add Loader.source_dir attribute
c1ac0e03 Vsevolod Fedorov Story: 2010984 Fix folder tests and defaults for folders
36ea2227 Vsevolod Fedorov Improve loader tests
a1d4f91d Vsevolod Fedorov Add context for macro calls
d17820a4 Vsevolod Fedorov Restore macros support for notifications
a82fa9d3 Vsevolod Fedorov Update documentation for lazy-loaded files
1a65a1d4 Vsevolod Fedorov Handle nested brace inside variable
21f0506e Vsevolod Fedorov Add tests for 'email' reporter
2c4de9ab Vsevolod Fedorov Replace obsolete tag with new one in example
99c9c860 Vsevolod Fedorov Move some parameter tests to proper directory
53bdf1ee Vsevolod Fedorov Update documentation links
940a4b65 Aleksey Tsalolikhin fix/2010888 Add default "current-parameters" value in "trigger-builds"


======================================================
[jjb] Jenkins Job Builder 5.0.4 is now available
======================================================
External
Inbox

Vsevolod Fedorov <vfed...@networkoptix.com>
Fri, Aug 18, 2023, 12:58 AM
to jenkins-job-builder, Aleksey

Hi Everyone,

Jenkins Job Builder 5.0.4 is released.

It has fixed dimension parameter overrides bug

Commit list:

d62faa15 Vsevolod Fedorov Fix parameters overrides in dimensions
specification

Vsevolod.




======================================================
[jjb] Jenkins Job Builder 5.0.3 is now available
======================================================
External

Inbox

Vsevolod Fedorov <vfed...@networkoptix.com>
Tue, Jul 4, 2023, 3:49 AM
to jenkins-job-builder, Paride, Oleg

Hi Everyone,

Jenkins Job Builder 5.0.3 is released.

Commit list:

0f29e2ea Vsevolod Fedorov Expand axis params before enumerating values
6d71ade6 Oleg Stiplin Add tokenCredentialId parameter support
to generic-webhook-trigger



======================================================
[jjb] Jenkins Job Builder 5.0.1 is now available
======================================================

Inbox

Vsevolod Fedorov <vfed...@networkoptix.com> Unsubscribe
Thu, Apr 13, 2023, 12:21 AM
to jenkins-job-builder

Hi Everyone,

Jenkins Job Builder 5.0.1 is released. Thanks to everyone who contributed to this release.

Shortened commit list:
60e8395c Vsevolod Fedorov Add source location and context to error messages
5ebd23af Vsevolod Fedorov Add support for Python 3.9 and 3.10
cd8c8bfb Piotr Falkowski Added configuration for Suppress SCM Triggering
d839dc76 Ettore Leandro Tognoli Add discord-notifier publisher
f0b52ec4 Thomas Bechtold Fix --enabled-only option when updating jobs
a701920d Vsevolod Fedorov STORY: 2010534 Fix bug: j2-yaml is not allowed inside macro body
af9e03ec Vsevolod Fedorov Rewrite YAML parser

In this release:

YAML parser/expander is rewritten. It causes some backward-compatibility breaking changes in this release. See cited message below.
This rewrite has fixed following known bugs:
https://storyboard.openstack.org/#!/story/2006270 Default value does not propertly unescape curly braces
https://storyboard.openstack.org/#!/story/2008510 Different behaviour on defaults list and inline variables for Jenkins Job Builder
https://storyboard.openstack.org/#!/story/2010428 Bug: TypeError: argument of type 'Jinja2Loader' is not iterable
https://storyboard.openstack.org/#!/story/2006725 yaml.load without Loader= is deprecated
Error handling is improved: now JJB shows tracebacks with error locations. See these tests for examples:

Vsevolod.
On 2/7/23 10:31, Vsevolod Fedorov wrote:
Hi, everybody.

I have submitted a review for rewriting YAML parser: https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965
This will allow more fine-tuned control over YAML parsing, YAML objects and parameter expansion logic.

Please add yourselves to reviewers, leave your comments and votes. I will also answer questions from this list, personal email and JJB IRC channel, at OFTC, #openstack-jjb.
But this breaks some backward compatibility, so I encourage you to check it against your Jenkins codebase.

From commit message:

Broken backward compatibility:

More aggressive parameter expansion. This may lead to parameters expanded in places where they were not expanded before.
See changes in this test for examples:
Top-level elements, which is not known to parser (such as 'job', 'view', 'project' etc), are now lead to parse failures.
Prepend them with underscore to be ignored by parser. For example:
Files included using '!include-raw:' elements and having formatting it path ('lazy-loaded' in previous implementation) are now expanded too.
Use '!include-raw-escape:' for them instead.
See changes in these tests for examples:
Parameters with template value using itself were substituted as is. For example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to recursive parameter error.
See changes in this test for example:
When job group includes a job which was never declared, it was just ignored. Now it fails: job is missing.
See changes in this test for example:
Any comments and suggestions are welcomed.


Vsevolod.

--

======================================================
[jjb] Jenkins Job Builder 4.2.0 is now available
======================================================
Inbox

Vsevolod Fedorov <vfed...@networkoptix.com> Unsubscribe
Tue, Feb 28, 2023, 3:16 AM
to jenkins-job-builder

Hi Everyone,

Jenkins Job Builder 4.2.0 is released. Thanks to everyone who
contributed to this release.

Shortened commit list:

5cea04aa Piotr Falkowski New permissions option for GitHub PR
Comment Build plugin
5e61fbaf Thomas Bechtold Add config option --enabled-only
69587283 Piotr Falkowski Support for GitHub Checks plugin
571df29f Vsevolod Fedorov Requirements: Fix setuptools version
456e6451 Mateusz Rygiel Add properties for Gitlab trigger
927bb191 Piotr Falkowski Support for follow-symlinks in archive
publisher
288308b4 Piotr Falkowski Support for Build Failure Analyzer plugin
f4d64f9f Vsevolod Fedorov Move tests to pytest
da50ce73 Ruan Bahia fix branch-api valid time periods
f0aeddf5 sp-ricard-valverde fix(scm): add clean options to after and
before
c12f334a Pavel Sedlák Add sandbox option to dsl builder
b84edf58 Vsevolod Fedorov ModuleError exception: Fix unresolved
module name case
76cc236d Johannes Carlsson Add support for abort-builds
e064158d Johannes Carlsson Add head-filter-regex to gerrit_scm
787164e0 Ken Dreyer wrappers: omit blank settings for
ssh-user-private-key credential
ce865d4b Ken Dreyer wrappers: rename credentials-binding examples

for more details on commits.




To unsubscribe from this group and stop receiving emails from it, send an email to jenkins-job-bui...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkins-job-builder/7be60c52-f9f1-483e-9ba8-d46990079c7b%40gmail.com.


--
Tony Chia
Sr Build and Release Engineer 

Langer, Christoph

unread,
Feb 10, 2024, 4:25:46 AM2/10/24
to Tony Chia, Dan Mick, jenkins-j...@googlegroups.com
Yep, that's what you have to do. Replace single { with double {{ and
also } -> }}. That did it for me.

Cheers
Christoph
> <mailto:vfed...@networkoptix.com>>
> Fri, Aug 18, 2023, 12:58 AM
> to jenkins-job-builder, Aleksey
>
> Hi Everyone,
>
> Jenkins Job Builder 5.0.4 is released.
>
> It has fixed dimension parameter overrides bug
> (https://storyboard.openstack.org/#!/story/2010883
> <https://storyboard.openstack.org/#!/story/2010883>).
>
> Commit list:
>
> d62faa15 Vsevolod Fedorov Fix parameters overrides in dimensions
> specification
>
> Vsevolod.
>
>
>
>
> ======================================================
> [jjb] Jenkins Job Builder 5.0.3 is now available
> ======================================================
> External
>
> Inbox
>
> Vsevolod Fedorov <vfed...@networkoptix.com
> <mailto:vfed...@networkoptix.com>>
> Tue, Jul 4, 2023, 3:49 AM
> to jenkins-job-builder, Paride, Oleg
>
> Hi Everyone,
>
> Jenkins Job Builder 5.0.3 is released.
>
> Commit list:
>
> 0f29e2ea Vsevolod Fedorov Expand axis params before enumerating values
> 6d71ade6 Oleg Stiplin Add tokenCredentialId parameter support
> to generic-webhook-trigger
>
>
>
> ======================================================
> [jjb] Jenkins Job Builder 5.0.1 is now available
> ======================================================
>
> Inbox
>
> Vsevolod Fedorov <vfed...@networkoptix.com
> <mailto:vfed...@networkoptix.com>> Unsubscribe
> https://opendev.org/jjb/jenkins-job-builder/src/branch/master/tests/yamlparser/error_fixtures <https://opendev.org/jjb/jenkins-job-builder/src/branch/master/tests/yamlparser/error_fixtures>
>
> Vsevolod.
> On 2/7/23 10:31, Vsevolod Fedorov wrote:
> Hi, everybody.
>
> I have submitted a review for rewriting YAML parser:
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965
> <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965>
> This will allow more fine-tuned control over YAML parsing, YAML objects
> and parameter expansion logic.
>
> Please add yourselves to reviewers, leave your comments and votes. I
> will also answer questions from this list, personal email and JJB IRC
> channel, at OFTC, #openstack-jjb.
> But this breaks some backward compatibility, so I encourage you to check
> it against your Jenkins codebase.
>
> From commit message:
>
> Broken backward compatibility:
>
> More aggressive parameter expansion. This may lead to parameters
> expanded in places where they were not expanded before.
> See changes in this test for examples:
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/inter-parameter-expansion.yaml <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/inter-parameter-expansion.yaml>
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/inter-parameter-expansion.xml <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/inter-parameter-expansion.xml>
> Top-level elements, which is not known to parser (such as 'job', 'view',
> 'project' etc), are now lead to parse failures.
> Prepend them with underscore to be ignored by parser. For example:
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/loader/fixtures/custom_retain_anchors_include001.yaml <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/loader/fixtures/custom_retain_anchors_include001.yaml>
> Files included using '!include-raw:' elements and having formatting it
> path ('lazy-loaded' in previous implementation) are now expanded too.
> Use '!include-raw-escape:' for them instead.
> See changes in these tests for examples:
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml>
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml>
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml>
> Parameters with template value using itself were substituted as is. For
> example: "timer: '{timer}'" was expanded to "{timer}". Now it leads to
> recursive parameter error.
> See changes in this test for example:
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/error_fixtures/parameter_name_reuse_default.yaml <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/error_fixtures/parameter_name_reuse_default.yaml>
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/parameter_name_reuse_default.xml <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/parameter_name_reuse_default.xml>
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/error_fixtures/parameter_name_reuse_default.error <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/error_fixtures/parameter_name_reuse_default.error>
> When job group includes a job which was never declared, it was just
> ignored. Now it fails: job is missing.
> See changes in this test for example:
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/error_fixtures/job_group_includes_missing_job.yaml <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/error_fixtures/job_group_includes_missing_job.yaml>
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/job_group_includes_missing_job.xml <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/job_fixtures/job_group_includes_missing_job.xml>
> https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/error_fixtures/job_group_includes_missing_job.error <https://review.opendev.org/c/jjb/jenkins-job-builder/+/871965/5/tests/yamlparser/error_fixtures/job_group_includes_missing_job.error>
> Any comments and suggestions are welcomed.
>
>
> Vsevolod.
>
> --
>
> ======================================================
> [jjb] Jenkins Job Builder 4.2.0 is now available
> ======================================================
> Inbox
>
> Vsevolod Fedorov <vfed...@networkoptix.com
> <mailto:vfed...@networkoptix.com>> Unsubscribe
> <http://pypi.org>, and know that
> it's 6.0.0 that's done it.  Constructs like "${BRANCH}" in a builders
> section are throwing
>
> While formatting string '${BRANCH}': Missing parameter: 'BRANCH'
>                    label: '${BRANCH}'
>
> On 2/9/2024 12:09 PM, Dan Mick wrote:
> > Hi; we've been using jjb for years, and recently our CI started
> > breaking; it appears as though some version has started interpreting
> > things in our yml files as variables to expand.  All I know so
> far is
> > that 4.3.0 didn't have this behavior and 6.0 does.
> >
> > I can't find a place in the docs that talks about when tokens are
> > expanded or how (other than inside templates, which these config
> files
> > are not using), and I also can't find a changelog.  Can anyone
> point me at:
> >
> > 1) a definition of variable syntax and replacement rules, and/or
> > 2) a changelog describing at least major breaking changes?
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "jenkins-job-builder" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to jenkins-job-bui...@googlegroups.com
> <mailto:jenkins-job-builder%2Bunsu...@googlegroups.com>
> > <mailto:jenkins-job-bui...@googlegroups.com
> <mailto:jenkins-job-builder%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/jenkins-job-builder/f027126e-0856-4889-b716-a333d715d4d6n%40googlegroups.com <https://groups.google.com/d/msgid/jenkins-job-builder/f027126e-0856-4889-b716-a333d715d4d6n%40googlegroups.com> <https://groups.google.com/d/msgid/jenkins-job-builder/f027126e-0856-4889-b716-a333d715d4d6n%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/jenkins-job-builder/f027126e-0856-4889-b716-a333d715d4d6n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "jenkins-job-builder" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to jenkins-job-bui...@googlegroups.com
> <mailto:jenkins-job-builder%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkins-job-builder/7be60c52-f9f1-483e-9ba8-d46990079c7b%40gmail.com <https://groups.google.com/d/msgid/jenkins-job-builder/7be60c52-f9f1-483e-9ba8-d46990079c7b%40gmail.com>.
>
>
>
> --
>
> *Tony Chia*
> Sr Build and Release Engineer
>  LinkedIn <https://www.linkedin.com/company/spokeo/> •  Instagram
> <https://www.instagram.com/spokeo/> •  Youtube <https://bit.ly/2oh8YPv>
>
> --
> You received this message because you are subscribed to the Google
> Groups "jenkins-job-builder" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jenkins-job-bui...@googlegroups.com
> <mailto:jenkins-job-bui...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkins-job-builder/CAJdSq2J-MiWGc5V4FbSHpWf3aTGf%3DUbKe1uZg%2B%2BkPupRdyjaXQ%40mail.gmail.com <https://groups.google.com/d/msgid/jenkins-job-builder/CAJdSq2J-MiWGc5V4FbSHpWf3aTGf%3DUbKe1uZg%2B%2BkPupRdyjaXQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
OpenPGP_0x635DF1E692C806E7.asc
OpenPGP_signature.asc
Reply all
Reply to author
Forward
0 new messages