Python pipeline

[Naor Yuval]

Hi, 

I am new to go-cd and am looking to create a pipeline that builds, tests and deploys python processes and web application.

I do see a python task that runs a python script. is there way to setup a virtualenv? 

Thanks,

N

[magnus...@aitellu.com]

Den söndag 23 november 2014 kl. 16:49:39 UTC+1 skrev Naor Yuval:

I am new to go-cd and am looking to create a pipeline that builds, tests and deploys python processes and web application.

I do see a python task that runs a python script. is there way to setup a virtualenv? 

Did you get anywhere with this? I have the same question.

[Fredrik Wendt]

Hi Magnus,

The way I see it, how you setup the environments where your Go agents are operating, is not within the scope of Go.cd. The way I prefer to have control of this, is to to run the Go agents within a Docker container. These containers provides a controlled environment and so can easily be reproduced, allow for firing up identical Go agents if you want/need that. (Launching new Go agents based on workload from the Go server, ie auto-scaling of the whole continuous integration/delivery/whatever process, is on the roadmap but I'm not up to date on the actual plans or implementation details.)

Another options is to make all build steps run in docker containers (where a virtualenv, or simply just one set of Python env), which is what I've done myself for Ruby stuff. (Had some collisions with Puppet and Jekyll which was fixed by having the host OS run what was needed for Puppet, and the website (Jekyll) ran it's stuff in a container.)

/ Fredrik

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

+46 702 778511

[Axel Haustant]

Hi!

I have slightly the same approach as Magnus: Docker + Docker Compose + Virtualenv

The difference is that the agent is not contained in the docker but it launch the commands using docker-compose allowing to have middleware started in dockers too.

My main use case is:

• mongo official docker
• elasticsearch official docker
• redis official docker
• a specific docker for the build/test environment with linkage on the previous ones and the workspace as volume

It allows me to write my tasks like that:

docker-compose run -T myapp mytask.sh

This will start all required services as dockers. The -T option is important because without you won't have any output.

This permit to have all agents configured the same way: just install docker, docker-compose and add the go user to the docker group.

It can handle any configuration.

The main caveat is permissions handling: docker runs as root so written files have the root:root permissions unless you create a go user with the same uid in your build/test docker.

This is why I have a git material dedicated to pipelines scripts and tooling. with a Dockerfile written like that:

# Extend the generic build dockerfile to add the go user

FROM myapp/build

# Use the same go user and group as the agent.

RUN groupadd -r -g 5000 go && useradd -r -g go -u 5000 -m go

USER go

--

Axel H.