Environment variables in downstream pipelines

[Edward Petersen]

All,

Before I make a feature request (or two), I thought I'd check in here to see if anyone had a good way to accomplish my current issue. I'm using the pipeline counter number heavily for dynamic versioning of our software (as the build number). I note that if you make Pipeline A a material for Pipeline B, that several environment variables are created and exported into each of Pipeline B's task shells. Specifically I have been using the one GO_DEPENDENCY_LABEL_[material name in all caps]. This has been great on "fan in" to know the most recent version of previously run dependencies, which includes the build number (x.y.z_build) in downstream pipelines. However, even though a further downstream pipeline, such as a Pipeline C that uses Pipeline B as a material, has access to artifacts posted by Pipeline A, it does not inherit the environment variables exported into Pipeline B's tasks.

To currently solve this, I echo the version of Pipeline A into a file and save it as an artifact. Later, Pipeline C has it by cat-ing the file for use. The problem is, I have to do that cat in each task.

It would be really nice to have one of the following features:

a) all downstream pipelines get the environment variables of upstream materials - not just from its own materials; or

b) Being able to create a task in an early stage that adds an environment variable that is valid and exported into all tasks for the rest of the pipeline run.

Thoughts? Do people have other good ideas? Or should I make a feature request? Or even code up a feature for contributing to GoCD itself?

thanks for listening!

-Ed

[srinivas upadhya]

Passing environment variables to downstream is a little tricky. If you consider the following situation:

git -> A -> C
      |       ^
      +>B +

and A-1 is triggered off of git-1, B-1 is triggered with git-1 and B-2 is trigerred with git-2. Now, if C is trigerred with A-1 and B-2 then what version is of git should be exposed as enviornment variable?

--

In your case, what you can do is add pipeline A as material for pipeline C to get value published as env. var. I understand its not easy but thats an option.

What you are currently doing (writing to file) and passing it as artifact is good one. The ambiguity i mentioned is resolved by the path specified by fetch artifact path.

I will get back to you if i have better suggestions.

[Edward Petersen]

I would venture that the environment variable name could be created to map the pathway the value took through the stream.

Also, by directly making A a material of C, a successful build of A would trigger C, no? And if we only want C to kick off after a successful build of B that could be a bad race condition. Does anyone know how GoCD handles such a thing?

We're passing the artifact file - it seems to be working. It's growing on me - feels less hackish as we continue using it.

-Ed

[srinivas upadhya]

On Wed, Sep 24, 2014 at 9:47 PM, Edward Petersen <edward....@gmail.com> wrote:

I would venture that the environment variable name could be created to map the pathway the value took through the stream.

​That would mean everytime you changed upstream dependency you would have to change the env var. being consumed? I will think & get back if i have something on this.​

 

Also, by directly making A a material of C, a successful build of A would trigger C, no? And if we only want C to kick off after a successful build of B that could be a bad race condition. Does anyone know how GoCD handles such a thing?

​Fan-in should handle that. C will wait for B's successful run.​ But yes i would suggest not to add dependency just to get a env var. exposed!

 

We're passing the artifact file - it seems to be working. It's growing on me - feels less hackish as we continue using it.

-Ed

On Saturday, September 20, 2014 7:40:08 AM UTC-4, srinivas upadhya wrote:

Passing environment variables to downstream is a little tricky. If you consider the following situation:

git -> A -> C
      |       ^
      +>B +

and A-1 is triggered off of git-1, B-1 is triggered with git-1 and B-2 is trigerred with git-2. Now, if C is trigerred with A-1 and B-2 then what version is of git should be exposed as enviornment variable?

--

In your case, what you can do is add pipeline A as material for pipeline C to get value published as env. var. I understand its not easy but thats an option.

What you are currently doing (writing to file) and passing it as artifact is good one. The ambiguity i mentioned is resolved by the path specified by fetch artifact path.

I will get back to you if i have better suggestions.

--

You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.