Checkout sources into the top level pipelie folder

[Alexey Savchkov]

I have a large (a few GB) Git repository which is used in N pipelines. This results in the repository being checked out N times for each pipeline which in turn occupies a lot of disk space on the agent. Therefore I would like to reuse the repository between pipelines. Is there a way to specify an SCM destination path starting from the top level pipelie folder (/var/lib/go-agent/pipelines) rather than /var/lib/go-agent/pipelines/<pipeline name>? Specifying destination = '..' in the material properties is intentionally prohibited and doesn;t pass the validation check.
Currently I'm thinking of using symlinks but having an option to set this in GoCD in the first place would be much cleaner and reliable.

Many thanks.

[Ashwanth Kumar]

I wouldn't recommend gocd does it automatically because think of 2 pipelines running simultaneously with a different SCM commit of the same material, the result would be undefined and not reliable from CD principles.

A better approach would be to expose the repo as an artifact from a pipeline and just download that in all downstream pipelines. To keep the server disk usage in control may be run that pipeline manually on demand and have something gocd-janitor to make sure you've only last few versions. 

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/68d12835-d845-46bd-b047-e5b7aacb9256n%40googlegroups.com.

[Alexey Savchkov]

> 2 pipelines running simultaneously with a different SCM commit

I can see no problem with this as long as the shared repository remains within one agent's hierarchy as there can be only one job running at a time. In practice I had a similar arrangement in Jenkins where one could specify the path of the agent workspace and I never had a conflict with thousands of pipeline runs.

> A better approach would be to expose the repo as an artifact from a pipeline and just download that in all downstream pipelines.

I will try this suggestion, thank you for the hint.

суббота, 25 марта 2023 г. в 20:48:46 UTC+7, ashwant...@gmail.com: