[feature question] Is it possible to trigger a Pipeline for *each* commit to a git branch?

[boito...@gmail.com]

Hi,

Our organisation is following Gitflow, and we would need some automated tasks to take place for each commit pushed to some branches (notably the 'develop' branch).

Because it is possible that a developer is pushing several commits on a branch at the same time, we cannot simply rely on automatic polling, as we observed that it only uses the latest commit.

We do not see a "git revision" parameter to the Web API "Notify git materials" either (https://api.gocd.org/current/#notify-git-materials)

Is there a way with GoCD to trigger the pipeline for each commit on the branch, even if several new commits are available at once?

Thank you for reading,

  Ad

[Ankit Srivastava]

Hi,

GoCD doesn't build on each commits. In GoCD, there is a material update process which polls the changes every minute. Pipeline build/triggers based on the material update response. In that period of time material update process can poll whatever the available commits (it could be one or more than that). Thus, when there are multiple commits happens in a short span of time (say between material update1 and material update2), pipeline scheduled after material update2 will have all the in-between commits.

This (https://api.gocd.org/current/#notify-git-materials) might not help as well, because once you fire this api it will trigger the material update process which will batch all the commits happened in close interval.

Regards,

Ankit

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Varsha Varadarajan]

Hi, 

I believe there's a request for this - https://github.com/gocd/gocd/issues/938 that hasn't been picked up.

Also, may be you could try the following out as a workaround. Turn off automatic scheduling for your pipeline (note that this is different from automatic polling, which is also something you need to turn off for using the notify materials API). The option for automatic scheduling is available on the pipeline settings page and the first stage setting's page. Using webhooks and the "Notify git materials API", notify GoCD of every commit that happens. In addition, use the pipeline schedule API to trigger the pipeline in question with the specific git revision. You can check out the API docs for scheduling pipelines with specific revision - https://api.gocd.org/18.7.0/#scheduling-pipelines.

Hope this helps.

Thank you,

Varsha

[boito...@gmail.com]

  Thank you for your detailed answer Varsha,

I finally found some time to further experiment with this, and finally wrote some Python script to:

1. extract a range of commits from a Bitbucket PUSH webhook notification

2. Use GoCD scheduling API (https://api.gocd.org/18.7.0/#scheduling-pipelines) to trigger the Pipelines on the corresponding range

Sadly, now that it is almost complete, I found out that, apparently, it is not possible to queue up "schedule requests" with the API (as per this closed issue: https://github.com/gocd/gocd/issues/2253).

Is there a way to actually queue up those requests. I can not wait for the pipeline to complete on a given version to trigger the next revision.

Thank you for reading,

  Adrien