need help understanding Fan-In concept

[Syed Rakib Al Hasan]

Referring to the picture of sample VSM from the GoCD Getting Started tutorial. 

Here we have 2 fan-ins.

1. Production Pipeline has fan-in from 3 upstream (Git Material, User Acceptance Pipeline, Capacity Testing Pipeline)
2. Capacity Testing Pipeline has fan-in from 2 upstream (SVN Material, Build Pipeline)

Does a fan-in concept mean that all upstreams of a pipeline need to be successful in order for that pipeline to be triggered?

If that is the case, as i seem to understand from the fan-out and fan-in section of the GoCD tutorial, Production Pipeline will be triggered only if all 3 of the Git Material, User Acceptance Pipeline and Capacity Testing Pipeline are successful.

Now, if that's the case, how is Capacity Testing Pipeline ever supposed to get triggered by only the successful completion of Build Pipeline whereas the Capacity Testing Pipeline also has the SVN material as an upstream. Does it not mean that Capacity Testing Pipeline will be triggered if-and-only-if BOTH Build Pipeline completes and the SVN Material has a new commit?

Learning GoCD for the first time. Everything seemed great in the Getting Started tutorial. Except, this very last section of the tutorial (dealing with fan-in-fan-out concept) seems a little confusing.

[Aravind SV]

Some conversation in Gitter about this, between Syed and @tomzo: https://gitter.im/gocd/gocd?at=57d00bdb77c1b70d7ff135cd

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Syed Rakib Al Hasan]

Yes. That would be me and @Tomzo. However, a few matters remained unclear in the brief discussion. Also, it's kind of difficult to keep focus on one topic in a gitter chat since there are so many discussions happening in a single thread. Hence, wanted to ask the question separately over here with a dedicated thread.

To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.

[Syed Rakib Al Hasan]

So the main question is,

does having multiple upstreams to a single pipeline require ALL of its upstreams to complete successfully to trigger the pipeline? Or will the pipeline be triggered by ANY ONE of the upstreams being successfully completed?

[Aravind SV]

On Thu, Sep 8, 2016 at 11:37 AM, Syed Rakib Al Hasan <syed.raki...@gmail.com> wrote:

Yes. That would be me and @Tomzo. However, a few matters remained unclear in the brief discussion. Also, it's kind of difficult to keep focus on one topic in a gitter chat since there are so many discussions happening in a single thread. Hence, wanted to ask the question separately over here with a dedicated thread.

Makes sense. I replied to the other mail of yours (with the subject "VSM vs MultiStage/MultiJob design") and my response from there goes for this as well.

does having multiple upstreams to a single pipeline require ALL of its upstreams to complete successfully to trigger the pipeline? Or will the pipeline be triggered by ANY ONE of the upstreams being successfully completed?

For a downstream pipeline, it needs successful upstreams for it to trigger. All of them.

[Syed Rakib Al Hasan]

If triggering a pipeline requires ALL of its upstreams to be successful (and not just ANY ONE), does it mean that Capacity Testing Pipeline (which shows that it has 2 upstreams - the BuildPipeline and the SVN Material) will be triggered only if Build Pipeline is successful and ALSO there is a new commit in SVN material? It means, the completion of Build Pipeline will successfully trigger the User Acceptance Pipeline but it will not trigger the Capacity Testing Pipeline because the Capacity Testing Pipeline must ALSO wait for a new commit in SVN material.

Is that correct?

[Aravind SV]

No. Think of it this way: A pipeline will trigger only if all of its upstreams are "green". Here's an example:

Time 00:00 - Commit "git_1" happens.

Time 00:00 - Commit "svn_1" happens

Time 00:01 - Build pipeline starts (Label: BP_1)

Time 00:02 - Build pipeline finishes successfully (Label: BP_1)

Time 00:03 - User acceptance pipeline starts (Label: UP_1) since build pipeline finished successfully (Dependency which caused trigger: BP_1)

Time 00:03 - Capacity testing pipeline starts (Label: CP_1) - (Dependencies which caused trigger: BP_1 and svn_1)

Time 00:04 - Capacity testing finishes successfully (Label: CP_1)

Time 00:04 - Production pipeline does NOT trigger, since the correct version of User acceptance pipeline is not yet finished (the one with BP_1)

Time 00:05 - User acceptance pipeline finishes successfully (Label: UP_1)

Time 00:06 - Production pipeline starts since its upstreams passed (CP_1 and UP_1) and fan-in shows that they're compatible (lead back to BP_1 and then git_1).

Time 00:07 - Commit "svn_2" happens

Time 00:08 - Capacity pipeline starts (Label: CP_2) and dependencies are svn_2 and BP_1 (latest green, since there is no fan-in here - build pipeline does not depend on svn repo).

Time 00:09 - Capacity pipeline finishes successfully (Label: CP_2)

Time 00:10 - Production pipeline starts with UP_1 and CP_2. That's because of fan-in. UP_1 is built by BP_1 which leads to git_1. Similarly, CP_2 is build by BP_1 which leads to git_1. So, consistent.

Time 00:11 - Commit "git_2" happens.

Time 00:12 - Build pipeline starts (Label: BP_2)

Time 00:13 - Build pipeline fails (Label: BP_2)

Time 00:14 - Capacity pipeline does NOT trigger. Upstream is not successful.

Time 00:15 - Commit "svn_2" happens.

Time 00:16 - Capacity pipeline does NOT trigger. Latest run of upstream pipeline is not successful, since there is no fan-in here.

I'd suggest trying these out. These should be easy to make happen using a small script which exits with 0 (success) or 1 (failure).

Cheers,

Aravind

[Syed Rakib Al Hasan]

Hi Aravinda, thank you for the elaborate example with timestamps. Really helpful.

However, at Time 00:08 and at Time 00:16 you said "since there is no fan-in here". What do you mean by "since there is no fan-in here". What defines something as whether it is a fan-in or not? Is it not simply the converging of more than one upstreams into a single pipeline which is defined as a fan-in? Or am i failing to clearly define what a fan-in is?

[Rakib]

Yes. That would be me and @Tomzo. However, a few matters remained unclear in the brief discussion and it's kind of difficult to keep focus on one topic in a gitter chat, since there are so many concerns and/or discussions happening in a single thread. Hence, wanted to ask the question here.

regards
Rakib
--------------------------------------------------
PS: Please consider the environment before printing this email.

--
You received this message because you are subscribed to a topic in the Google Groups "go-cd" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/go-cd/QgBdEKbw7BY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to go-cd+unsubscribe@googlegroups.com.

[Syed Rakib Al Hasan]

Hi Aravinda (and others in the community). Hope you can see this query and help me out in solving this puzzle.