about worker multithreading model.

[Terry Cho]

Hello guys.

followed by document

http://purplefox.github.io/vert.x/manual.html#worker-verticles

Worker verticles are never executed concurrently by more than one thread. Worker verticles are also not allowed to use TCP or HTTP clients or servers. Worker verticles normally communicate with other verticles using the vert.x event bus, e.g. receiving work to process.

As i know worker is also running on single thread only. Even if i set up multiple instances in worker.

So i made very simple worker example https://github.com/bwcho75/vertx_study/tree/master/worker_sample

Sample is

1. accept HTTP request

2. send event message to worker by using eventbus.send

3. in worker it consumes message and wait 5 sec by using time.sleep(5)//python.

and i gave a load by using ab with 10 concurrent client.

My expectation was,

worker will wait about 5 sec. so the logs from worker should be displayed sequencially every 5 sec.

but in my test. after i ran the test. log message from worker is displayed in same time.

It means, workers are runned in multiple thread concurrently.

Is there any change in worker thread model?

Could u please kindly advice me the current sympthom?

Thanx in advance

-Terry

[Tim Yates]

Doesn't that code deploy 20 instances of the worker?

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Terry Cho]

runner.py

import vertx

vertx.deploy_verticle('http_server.py')
vertx.deploy_worker_verticle('worker.py','{"dummy":"dummy"}',20)

2014년 1월 28일 화요일 오후 11시 49분 28초 UTC+9, Tim Yates 님의 말:

[Tim Yates]

Yeah, so it's my understanding that you will have 20 workers sat waiting, and then you send 10 requests at a time.

So half your workers will be sat doing nothing, while the other 10 process the sleep simultaneously

I may have read the code wrong though...

[Terry Cho]

Thanx tim.

My concern is, as i know worker cannot be run on multiple thread at one time.

As a mentioned in original question

"Worker verticles are never executed concurrently by more than one thread. Worker verticles are also not allowed to use TCP or HTTP clients or servers. Worker verticles normally communicate with other verticles using the vert.x event bus, e.g. receiving work to process."

Worker can be run only one thread.

Did i miss something?

2014년 1월 29일 수요일 오전 12시 16분 7초 UTC+9, Tim Yates 님의 말:

[Tim Yates]

Yeah, you have 20 instances of your worker, each with a thread.

This is my understanding of the matter anyway, hopefully if I'm wrong someone can jump in to correct me :-/

[Tim Fox]

Vert.x maintains a pool of worker threads and these are shared out across the workers. A particular worker is never executed by more than one thread concurrently (unless it's marked as multi-threaded, but that's an advanced feature).

The question I would be asking Terry is why he is using workers in the first place...

[Tim Yates]

Is the default pool size > 10?

[Tim Fox]

Yes, it's 20

[Terry Cho]

The question I would be asking Terry is why he is using workers in the first place...

--> I just want to understand worker threading model. That was my purpose. 

Vert.x maintains a pool of worker threads and these are shared out across the workers

--> Yes.. 

 A particular worker is never executed by more than one thread concurrently (unless it's marked as multi-threaded, but that's an advanced feature).

--> I didn't turn on "muti-thread" feature. i just used default configuration

--> But in my test, it works like multi threading.

as i mentioned, if the worker is working as single threaded model.

for example i have 10 request to worker.

Because there is a sleep code in the worker.

the log message comes from worker should be.

log from worker..

(after 5 sec)

log from worker..

(after 5 sec)

log from worker..

(after 5 sec)

:

but, current result is

log from worker..

log from worker..

log from worker..

log from worker..

:

(in same time)

2014년 1월 29일 수요일 오전 12시 26분 21초 UTC+9, Tim Fox 님의 말:

[javadevmtl]

You have to send 40 requests to get sequential logging as the expected behaviour. Sp  the first 20 will wait and the next 20 will execute after 5 seconds.

Like Tim said workers each get their own thread. So with your setup you are creating 20 workers and 20 threads.

Or change to

vertx.deploy_worker_verticle('worker.py','{"dummy":"dummy"}',1)

[Jez P]

You don't have one worker, you have twenty (twenty instances of the worker verticle). Each of those instances will run on a single thread. When a message arrives, one of those workers will process it and then sleep (as per your code). However there are 19 workers, each with a thread available to it, still available during that sleep to process the next message which arrives.

Multiple instances of a worker verticle = multiple threads (up to the max size of the worker thread pool). 

[Terry Cho]

Than Jaz.

It brings me confusion.

Followed by manual "Worker verticles are never executed concurrently by more than one thread."

Worker seems like running in only on thread only in a time.

But as u mentioned "Multiple instances of a worker verticle = multiple threads (up to the max size of the worker thread pool). "

It seems like working as multithreading.

I mean one worker can be run in multiple thread in same time.

If it is correct. I think it needs to be fixed.

but As @Tim.Fox confirmed, worker is running on single thread.

what i missing now? can u help me to understand?

2014년 1월 29일 수요일 오전 1시 46분 34초 UTC+9, Jez P 님의 말:

[Terry Cho]

Thanx. i know what u mentioned.

and as i know, worker doesn't get it's thread. thread pool is shared across workers.

2014년 1월 29일 수요일 오전 1시 43분 52초 UTC+9, javadevmtl 님의 말:

[Nick Scavelli]

If I understand your confusion, you are not understanding the concept of instances. When it says "Worker verticles are never executed concurrently by more then one thread", it means your instance of that worker verticle is never accessed by more then one thread concurrently.

[John Smith]

Also remember that the any verticle who calls that "slow" worker your execution time will be as fast as the slowest worker...

You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/VPevuqJbBQE/unsubscribe.
To unsubscribe from this group and all of its topics, send an email to vertx+un...@googlegroups.com.

[Jez P]

You need to understand what an instance is. If you create ten instances of a worker verticle (I think what Tim referred to as a "worker" is a single instance of a worker verticle, I hope I'm not putting words in his mouth) that's ten potential parallel processors (although the thread pool may limit how many can be active at one time). 

Each of those instances can only be accessed by a single thread at a given time. That is not an error, and therefore does not need to be fixed :) Maybe the documentation needs clarifying slightly to say "A given worker verticle instance is never executed concurrently by more than one thread" - if it was worded like that would you be less confused? 

Tim F, please correct me if I'm completely misrepresenting the behaviour of worker verticles, but I hope I'm not :)

[Tim Fox]

On 29/01/14 08:04, Jez P wrote:

You need to understand what an instance is. If you create ten instances of a worker verticle (I think what Tim referred to as a "worker" is a single instance of a worker verticle, I hope I'm not putting words in his mouth) that's ten potential parallel processors (although the thread pool may limit how many can be active at one time). 

Each of those instances can only be accessed by a single thread at a given time. That is not an error, and therefore does not need to be fixed :) Maybe the documentation needs clarifying slightly to say "A given worker verticle instance is never executed concurrently by more than one thread" - if it was worded like that would you be less confused?

We could change it if it makes things easier to understand.