[win32] celeryd keeps spawning processes

[throwness]

Hi, I hope I didn't miss anything obvious but when I startup celeryd
(with no daemon managers of any kind), it keeps spawning new celeryd
processes. Originally I tried it with Python 2.5 and the
multiprocessing backport, but then I tried it on a fresh machine with
2.6 installed, but same thing. This will just keep repeating unless I
ctrl-c. I tried a number of options (including concurrency=1 but no
go) Here's what it looks like:

C:\Celery>celeryd --loglevel=DEBUG
celery@Blahhost v1.0.0 is starting.
Configuration ->
. broker -> amqp://m...@127.0.0.1:5672/mainhost
. queues ->
. celery -> exchange:celery (direct) binding:celery
. concurrency -> 1
. loader -> celery.loaders.default
. logfile -> [stderr]@DEBUG
. events -> OFF
. beat -> OFF
. tasks ->
. celery.delete_expired_task_meta
. celery.execute_remote
. celery.map_async
. celery.ping
. celery.task.http.HttpDispatchTask
. tasks.add
Celery has started.
[2010-03-03 02:51:03,703: DEBUG/MainProcess] Instantiating thread
components...
[2010-03-03 02:51:03,717: DEBUG/MainProcess] Starting thread
TaskPool...
[2010-03-03 02:51:03,858: DEBUG/MainProcess] created semlock with
handle 1632
[2010-03-03 02:51:03,858: DEBUG/MainProcess] created semlock with
handle 1620
[2010-03-03 02:51:03,890: DEBUG/MainProcess] Starting thread
Mediator...
[2010-03-03 02:51:03,890: DEBUG/MainProcess] Starting thread
ScheduleController.
..
[2010-03-03 02:51:03,890: DEBUG/MainProcess] Starting thread
CarrotListener...
[2010-03-03 02:51:03,890: DEBUG/MainProcess] CarrotListener: Re-
establishing con
nection to the broker...
[2010-03-03 02:51:03,967: DEBUG/MainProcess] CarrotListener:
Connection Establis
hed.
[2010-03-03 02:51:03,983: DEBUG/MainProcess] CarrotListener: Starting
message co
nsumer...
[2010-03-03 02:51:03,983: DEBUG/MainProcess] CarrotListener: Ready to
accept tas
ks!
celery@Blahhost v1.0.0 is starting.
Configuration ->
. broker -> amqp://m...@127.0.0.1:5672/mainhost
. queues ->
. celery -> exchange:celery (direct) binding:celery
. concurrency -> 1
. loader -> celery.loaders.default
. logfile -> [stderr]@DEBUG
. events -> OFF
. beat -> OFF
. tasks ->
. celery.delete_expired_task_meta
. celery.execute_remote
. celery.map_async
. celery.ping
. celery.task.http.HttpDispatchTask
. tasks.add
Celery has started.

I'm using a bare celeryconfig.py with just the broker configs,
celery_backend = "amqp"

Any hints would be much appreciated!

[Ask Solem]

On Mar 3, 8:52 am, throwness <thrown...@gmail.com> wrote:
> Hi, I hope I didn't miss anything obvious but when I startup celeryd
> (with no daemon managers of any kind), it keeps spawning new celeryd
> processes. Originally I tried it with Python 2.5 and the
> multiprocessing backport, but then I tried it on a fresh machine with
> 2.6 installed, but same thing. This will just keep repeating unless I
> ctrl-c. I tried a number of options (including concurrency=1 but no
> go) Here's what it looks like:
>
>

Apparently celery 1.0 doesn't work well with windows,
and I haven't had the time to look at it yet :(

[FreeSoft]

Is there some news about this problem ? Actually I have the same
problem with celery 1.2, the worker keeps spawning processes until I
do the ctl+C if not I can see (in the task manager) a process
(python.exe) added every ~seconde, after a moment the system broke
down... and every thing stop...

I'm using the latest version of celery (1.2) and python2.6...

Is that related to the fork() function which is not available in
windows ? maybe an os.spawnv could resolve this ?

Thanks for help :)

--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To post to this group, send email to celery...@googlegroups.com.
To unsubscribe from this group, send email to celery-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

[Ask Solem]

On Apr 30, 2010, at 1:58 PM, FreeSoft wrote:

> Is there some news about this problem ? Actually I have the same
> problem with celery 1.2, the worker keeps spawning processes until I
> do the ctl+C if not I can see (in the task manager) a process
> (python.exe) added every ~seconde, after a moment the system broke
> down... and every thing stop...
>
> I'm using the latest version of celery (1.2) and python2.6...
>
> Is that related to the fork() function which is not available in
> windows ? maybe an os.spawnv could resolve this ?
>

How do you run celeryd?

I remember this happend if I used "celeryd", but "python -m celery.bin.celeryd" worked fine.
Also I had trouble using celeryd with django on windows when running "python manage.py celeryd", I had to set the settings file explicitly with "python manage.py celeryd --settings=settings"


--
{Ask Solem,
Developer ~ Webteam,
Opera Software,
+47 98435213 | twitter.com/asksol }.

[FreeSoft]

First of all thank you for this fast answer !

Exact, I was running the process like I do with unix (celeryd --
lolevel=INFO), but now it works fine.

btw, I need the windows worker to only do some stuff that I can't do
in Unix (running some windows software), so at the moment I don't need
django in that side...

Thanks :)