Running daemon processes

[andrea crotti]

Hello everyone,

I'm trying to move some integration tests for our django app to Jenkins.

I already succesfully managed to run integration tests for our business logic doing with coverage and violations as well, using something like:

COVER_OPTIONS="--with-coverage --cover-package=company_core --cover-inclusive --cover-erase"

ve/bin/pip install -r test-requirements.txt

source ve/bin/activate

nosetests --nologcapture --with-xunit $COVER_OPTIONS company_core/tests/unit company_core/tests/integration

python -m coverage xml -i

pylint -f parseable -d I0011,R0801 company_core | tee pylint.out

However for the app itself, we need something more, because I need to be able to run locally one API server on port 9797 and the django app itself on port 8000.

So ideally the flow should be the following on Jenkins:

- checkout everything

- run api and app daemons

- run integration tests

- collect results and kill daemons

I still haven't found a proper way to run the daemon processes though, any suggestions?

Thanks

[JonathanRRogers]

AFAIK, managing daemons is outside of the scope of Jenkins unless there's a plugin for managing a specific kind of daemon.  Just use ordinary commands to start and stop daemons in a shell script within the job. It may be necessary to set environment variable "BUILD_ID=dontKillMe" to tell Jenkins not to kill a daemon process:
<URL:https://wiki.jenkins-ci.org/display/JENKINS/ProcessTreeKiller>

[andrea crotti]

So I was trying to do something like this, and I created this bash script which locally works perfectly fine.

However it doesn't seem to work at all with the error pasted below.

Is Jenkins able to run any bash script?

It seems like it should be I'm not sure why it gives this error though..

#!/bin/bash

SPOTLIGHT_PID="not"

API_PID="not"

function create_ve {

    if [ ! -d ve ]

    then virtualenv ve

    fi

}

function start_spotlight {

    cd wazoku-spotlight/spotlight

    create_ve

    ve/bin/pip install -r test-requirements.txt

    ve/bin/python manage.py runserver &

    SPOTLIGHT_PID=$!

    cd -

}

start_spotlight

cd wazoku-spotlight/spotlight

ve/bin/python manage.py test tests/integration --with-xunit

kill $SPOTLIGHT_PID

ERROR:

Pip.....................................................................................................................................................................................................................................................................................................................................done.

./run-tests-jenkins.sh: ve/bin/pip: "/var/lib/jenkins/jobs/spotlight: bad interpreter: No such file or directory

/var/lib/jenkins/jobs/spotlight integration and selenium/workspace

Traceback (most recent call last):

  File "run.py", line 1, in <module>

    from bottle import run

ImportError: No module named bottle

New python executable in ve/bin/python

Installing Setuptools..............................................................................................................................................................................................................................done.

Installing 

[Jonathan Rogers]

andrea crotti wrote:
> So I was trying to do something like this, and I created this bash
> script which locally works perfectly fine.
>
> However it doesn't seem to work at all with the error pasted below.
> Is Jenkins able to run any bash script?
> It seems like it should be I'm not sure why it gives this error though..

In "Execute shell" blocks, Jenkins just tries to use the system shell by
default AFAIK. On Unix-like systems, that should be /bin/sh, which could
be Bash or something else, depending on the system. You should try to
make the scripts you put in "Execute shell" blocks standard Bourne shell
and/or test them with /bin/sh. Alternatively, you can set a specific
shell executable in the global Jenkins config.

>
> #!/bin/bash
> SPOTLIGHT_PID="not"
> API_PID="not"
>
> function create_ve {
> if [ ! -d ve ]
> then virtualenv ve
> fi
> }
>
> function start_spotlight {
> cd wazoku-spotlight/spotlight
> create_ve
> ve/bin/pip install -r test-requirements.txt
> ve/bin/python manage.py runserver &
> SPOTLIGHT_PID=$!
> cd -
> }
>
> start_spotlight
>
> cd wazoku-spotlight/spotlight
> ve/bin/python manage.py test tests/integration --with-xunit
>
> kill $SPOTLIGHT_PID

Is the above script what you put in an "Execute shell" block? Don't put
a shebang in those, though it shouldn't matter much.

>
> ERROR:
>
> Pip.....................................................................................................................................................................................................................................................................................................................................done.
> ./run-tests-jenkins.sh: ve/bin/pip: "/var/lib/jenkins/jobs/spotlight:
> bad interpreter: No such file or directory
> /var/lib/jenkins/jobs/spotlight integration and selenium/workspace
> Traceback (most recent call last):
> File "run.py", line 1, in <module>
> from bottle import run
> ImportError: No module named bottle
> New python executable in ve/bin/python
> Installing
> Setuptools..............................................................................................................................................................................................................................done.
> Installing

It's pretty clear that something couldn't find
"/var/lib/jenkins/jobs/spotlight". Figure out what was looking for that
file and why it couldn't find it.

--
Jonathan Rogers

[Geoff Cummings]

Not sure what that script is doing, but make sure to consider spaces in the jenkins job name.

/var/lib/jenkins/jobs/spotlight integration and selenium/workspace

Personally I don't allow spaces in job names..

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[andrea crotti]

Argh true I missed that, I expected that jenkins would have quoted the spaces since it lets me enter them, is that a known issue than?
I'll try without tomorrow, thanks

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/zheMR_OVYI8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.

[Jonathan Rogers]

andrea crotti wrote:
>
> Argh true I missed that, I expected that jenkins would have quoted the
> spaces since it lets me enter them, is that a known issue than?
> I'll try without tomorrow, thanks
>

First, is the script in question in an "Execute shell" block? Second, is
your script doing anything with the Jenkins project (job) name? Third,
in what context would you expect the name of the job to be quoted?
Without knowing the context, it's impossible to answer the question
usefully since many factors can affect how spaces are handled, escaped
or quoted, especially when a shell is involved.

--
Jonathan Rogers

[Daniel Beck]

Good tools and scripts can handle spaces and other weirdness in file/folder names. This is not a Jenkins issue (unless it happened in a Jenkins component -- but it doesn't look like it!).

If you write your own shell scripts, remember to double-quote variable references. For example, `some_command "$FOO"` is often better than `some_command $FOO` (except when it's not :-) ).

[andrea crotti]

So well I tried renaming the job removing the spaces (after I noticed that the other jobs didn't have names in) and now it works fine.

The script is running in the "Execute shell" block yes.

But the thing is that the error comes before my script is executed, when Jenkins is still trying to set up the directories to run the job, so I'm a bit surprised because it looks like at this point it should handle the spaces.

I'm not using any Jenkins variable in my script if that makes a difference..

Anyway it's fine now so problem solved, thanks