Changes are not picked up with autoUpdate=false and a timer trigger

[Alexey Savchkov]

I have a long-running set of nightly tests which I want to trigger only on schedule during non-working hours. I have set up the following material using the Groovy configuration plugin

materials {
  git('my-repo') {
    url = 'g...@mygit.org:mygroup/myrepo.git'
    branch = 'mybranch'
    destination = 'dest'
    autoUpdate = false
  }
}

with the intention that regular daily commits won't trigger the pipeline but it will be triggered by timer in the night and pick up all the changes which have accumulated so far,

However I see that when the pipeline starts by timer it doesn't recognise that new commits are present and runs on some old commit.

The server log looks like this

2023-03-23 05:04:00,001 INFO  [DefaultQuartzScheduler_Worker-4] TimedBuild:67 - Timer scheduling pipeline 'NightlyTests-Dev' using spec '0 */2 * * * ? *'
2023-03-23 05:04:00,720 INFO  [goThreadPoolTaskScheduler-4] ScheduleService:158 - [Pipeline Schedule] Scheduling pipeline NightlyTests-Dev with build cause [ManualForcedBuildCause: Forced by timer]
2023-03-23 05:04:00,730 INFO  [goThreadPoolTaskScheduler-4] PipelineRepository:78 - Start updating pipeline timeline
2023-03-23 05:04:00,734 INFO  [goThreadPoolTaskScheduler-4] PipelineRepository:84 - Pipeline timeline updated
2023-03-23 05:04:06,678 INFO  [109@MessageListener for WorkFinder] BuildAssignmentService:190 - [Agent Assignment] Assigned job [JobIdentifier[NightlyTests-Dev, 26, 26, Build, 1, Postgres-14, 17323]] to agent [Agent [linux-agent-3, 10.5.50.74, 9254d4f7-55a2-486e-bfff-847ed8d49041]]

and so on.

Is is not assumed that a timer-triggered pipeline should update its materials before running?

GoCD server version: 23.1.0 (16079-21e78c998e1eb35d8d489c1d3e3e9813dc18233a)

GoCD Agent version: 22.3.0-15301

Thanks a lot.

[Chad Wilson]

If you turn material updates off, this is the expected behaviour because you've told the polling subsystem not to check this material for updates unless you manually "trigger with latest".

You probably need to leave autoUpdate turned on for the material and make the first stage a manual approval stage to get the behaviour you want using https://github.com/gocd-contrib/gocd-groovy-dsl-config-plugin/blob/6d754dfb7628f1d52e16c4e3a03c73b594fd3f31/dsl/src/main/java/cd/go/contrib/plugins/configrepo/groovy/dsl/Stage.java#L65-L72

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/57bf6593-14b4-47ef-a1ea-5118b1e7695en%40googlegroups.com.

[Alexey Savchkov]

Yes, you are perfectly right, leaving autoUpdate turned on and adding a manual approval property to the first stage produced the behaviour I wanted and although I find this configuration somewhat counter-intuitive I admit that this is exactly the solution to my original problem. Great thanks for your help!

четверг, 23 марта 2023 г. в 13:02:32 UTC+7, Chad Wilson:

[Chad Wilson]

It's typically confusing to folks because almost no other similar tools look at 'materials' as a first class citizen independent from the pipelines that use them, which is one of the things that distinguishes GoCD from other tools. So whether you configure something at the 'material' level vs the 'pipeline' level makes a logical difference in GoCD's model.

Although i agree that some of the way GoCD presents the configuration and tries to strike a balance between 'easy to create a pipeline with a git material' and 'this material is used by N pipelines' can be confusing, because it does 'de-duplication' of materials automatically based on the URL/branch etc.

On the way scheduling is handled you'll note when viewing via the UI there is a UI convenience to set this on the pipeline level as 'automatic pipeline scheduling' being disabled.

The same convenience could possibly be offered with some syntactic sugar in the groovy DSL if someone wanted to work on a PR with the appropriate validations to make sure the stage level approval and pipeline-level config don't conflict.

-Chad

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/ebc8d07a-4b54-4d1f-b98a-4803cb39e88dn%40googlegroups.com.