Updating git repo on pipeline trigger if not set to polling

[Marius Ciotlos]

Hi, 

We have the following setup:

A (pipeline)--->B (pipeline) --->C

                      ^

                      |

                     D (Git repo)

We don't want B to trigger when someone commits to D (Git repo) as we just want B to pull from the Git repo when A finishes with success. 

A is a deployment pipepline with autolocking

B is a testing pipeline

D is the git repo that holds the testing sources

If we trigger B on any change in D we could end up triggering testing while A is deploying

We want to trigger B only when A is successful, but we want it to always pull the latest code from D

What we've noticed is that D only gets pulled to latest commit if we re-trigger manually pipeline B. I assume this is for optimisation, when you only "fetch" the upstream changed material, and as D is no longer set to trigger on "polling", GO doesn't see it as changed upstream dependency. 

Is there any way to work around this?

Cheers,

Marius

[srinivas upadhya]

Keep polling turned on for D and set blacklist for D to ignore everything. That way Go always fetches latest of D but B does not trigger off of it.

[Marius Ciotlos]

Great, thanks!

[Jaume Sancho]

genius!

This has solved a headache that I had with my "tools" dependency. 

Which is pretty much used by all pipelines and would trigger unnecessarily trigger them all, whenever we added a new script or binary to this folder.

Thanks very much Srinivas,

Jaume

[Marius Ciotlos]

@Jaume: I had similar issue when I used "tools" as a dependency on my pipelines, however we've figured out a better approach. 

1. You remove the dependency on all pipelines

2. You go to each agent and clone the "tools" in a predefined folder (like /home/go/tools )

3. On your tasks you use absolute paths for your "tools" scripts, /home/go/tools/example.sh

4. Add a new pipeline (or multiple if you use environments) and create a job that does a simple cd /home/go/tools and does git pull (make sure the material is the tools repo and has polling active)

5. Make the job in this pipeline "Run on all agents" to make it update all agents with the new code

6. (optional) If you have multiple agents on one VM you can tag just one of those on each VM as "primary-agent" (resource) and assign to the job in the updating pipeline to use as resource "primary-agent"

This way you don't have to keep re-adding the "tools" dependency on each pipeline

This way you update all your agents automatically after commit (or any method you wish, including cron scheduled)

This way if you want to update something regarding tooling setup you do it in 1 place (the update pipeline).

Cheers, 

Marius

[Marius Ciotlos]

Hi, 

I have tried to this for a Perforce repo but I get an error:

        <p4 port="perforce.dev.uk.betfair:1666" username="" dest="tests" autoUpdate="true" materialName="perforce">

          <filter>

            <ignore pattern="/*" />

          </filter>

          <view><![CDATA[//depot/se/development/HEAD/tests/... //${P4CLIENT}/...]]></view>

        </p4>

The following error(s) need to be resolved in order to perform this action:

• Invalid content was found starting with element 'filter'. One of '{view}' is expected.

On Thursday, August 14, 2014 9:35:34 AM UTC+3, srinivas upadhya wrote:

[Jaume Sancho]

Hi Marius, 

That actually makes a lot of sense. Originally I started with hardcoded paths and then migrated to the "tools" pipeline, which seemed like a good idea until the number of dependencies started growing significantly.

Having the tools folder as a separate pipeline that does not need to be "wired in" makes a lot of sense.

Best of both worlds as it were,

Thanks for the tip,

Jaume

--
You received this message because you are subscribed to a topic in the Google Groups "go-cd" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/go-cd/rBouLywo3a4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to go-cd+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[srinivas upadhya]

On Tuesday, August 19, 2014 at 1:57:20 PM UTC+5:30, Marius Ciotlos wrote:

Hi, 

I have tried to this for a Perforce repo but I get an error:

        <p4 port="perforce.dev.uk.betfair:1666" username="" dest="tests" autoUpdate="true" materialName="perforce">

          <filter>

            <ignore pattern="/*" />

          </filter>

          <view><![CDATA[//depot/se/development/HEAD/tests/... //${P4CLIENT}/...]]></view>

        </p4>

Try putting filter tag below view tag. 

[Bill Birch]

Hi,

I raised a GitHub issue for this, since according to the GoCD documentation, the pipeline should start with the latest code from all materials. See https://github.com/gocd/gocd/issues/948

Cheers
Bill

[Christopher Roth]

Srinivas,

If I 'blacklist' everything on the D git repository then wouldn't NOTHING get checked out?  And thus the material would be ignored completely?

We have this same issue and just did a co as the first 'task' so that only A will trigger B.

I'd love a better solution but this one doesn't make sense to me.

Thanks for your input.