WorkerVerticle and Thread blocked VertxException

[Dan McHarness]

vertx newb here.

I have a while loop that asks postgres for notifications. So I created a WorkerVerticle and placed the loop code in the start() method of that WorkerVerticle.

A minute after the application boots vertx throws an exception in the WorkerVerticle saying the thread is blocked. 

I thought the WorkerVerticle is just the ticket for known blocking code, so I was surprised that vertx throws the exception.

After re-reading the docs, I'm still not groking why vertx throws the exception, and how I should re-think that loop code.

The top level exception trace:

[vertx-blocked-thread-checker] [user:  WARN] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-worker-thread-3,5,main] has been blocked for 60798 ms, time limit is 60000 

io.vertx.core.VertxException: Thread blocked
   at java.lang.Thread.sleep(Native Method) ~[na:1.8.0_66]
   at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source) ~[na:na]
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_66]
   at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
   at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) ~[groovy-all-2.4.4.jar:2.4.4]
   at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) ~[groovy-all-2.4.4.jar:2.4.4]
   at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite$StaticMetaMethodSiteNoUnwrap.invoke(StaticMetaMethodSite.java:133) ~[groovy-all-2.4.4.jar:2.4.4]
   at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.call(StaticMetaMethodSite.java:91) ~[groovy-all-2.4.4.jar:2.4.4]
   at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) ~[groovy-all-2.4.4.jar:2.4.4]
   at blah.ace.PersistanceMonitorVerticle.start(PersistanceMonitorVerticle.groovy:54) ~[main/:na]
   at io.vertx.lang.groovy.GroovyVerticle.start(GroovyVerticle.groovy:64) ~[vertx-lang-groovy-3.2.0.jar:na]
 

[Thomas SEGISMONT]

Hi,

The #start method should be used to setup the verticle. In your case, I'd suggest you create a periodic timer, instead of writing your own loop around Thread#sleep:
http://vertx.io/docs/vertx-core/groovy/#_executing_periodic_and_delayed_actions

Thomas

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/7f6448b6-c5da-4937-a82b-7c1bc017bc23%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dan McHarness]

Thomas,

Worked perfect. From now on it's vertx timers instead of loops in my worker verticles.

mucho thanks!

Dan

[Arnaud Estève]

Hello,

I got the exact same issue and don't understand at all neither why it's happening nor why I can get my head around it.

I got a worker verticle, with a single instance, designed for very long blocking operations, periodically (1 per 24 hours).

In the start method if this verticle, I'm completing the future right away and create a timer to delay the blocking operation :

future.complete();

vertx.setTimer(1000, this::longBlockingOperation)

And I still get the BlockedThreadChecker yelling at me after 1min.

I don't understand why... at all... I know this (worker) thread will be blocked, I designed it to do so. My non-blocking operations are deployed in another (standard) verticle, isolated

What should I do to get rid of this annoying BlockedThreadChecker within a worker verticle ??

[Julien Viet]

Hi Arnaud,

the worker pool is designed to block for a relative low amount of time in the 0-60 seconds range (typical use case is remote operation (IO / JDBC), file system access, etc…) that cannot be handled on the event loop.

Keep in mind that a pool from the worker pool is a resource and if you borrow it for a very long time, you create starvation in the system. The checker’s task is to ensure you don’t starve the worker pool, as it may be necessary for other parts of your app.

If you need to block for a very long time like you are saying, I would suggest instead you start your own thread or you fragment your very long operation into small blocking chunks and schedule them on a worker thread.

Julien

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/81dfe286-6f24-4ac5-b776-8e759ee6c33a%40googlegroups.com.

[Arnaud Estève]

OK crystal clear.

For that operations, I was only needing Vert.x for scheduling (setTimer) so I guess I should be fine with my own dedicated thread + sleeping.

There are a lot of JDBC / Redis operations happening so I have two choices :

1/ Rewrite everything using asynchronous Vert.x APIs but then I'm afraid I'll be fighting with : 

- JsonArrays "hell" with JDBC

- Callback hell (getting the connection then executing the query then scrolling through a lot of results then inserting into Redis, that'll be alot of callbacks)

2/ Start my own thread then scheduling tasks in a vanilla (non-vertx) fashion

Thanks for your answer Julien. To me it looked that the worker pool was designed for any long-blocking operation. 

[Senthil Nathan]

Hi Julien,

For ease of use, if I still wanted to use worker verticles, should I be fine using a dedicated vertx worker pool.