How to undeploy Vertx 3 application on deployment error?

[Assen Todorov]

Hi,

In Vertx 2 the following code was enough to report deployment errors to Vert.x and exit the JVM:

public class StartVerticle extends Verticle
{
   @Override
   public void start(final Future<Void> startFuture)
   {
      // deploy other verticles

      // if something does wrong undeploy
      startFuture.setFailure(new IllegalArgumentException("Cannot start"));
   }
}

The Vertx 3 equivalent for the code above does not exit the JVM correctly and hangs. How is it supposed to handle deployment errors and exit/stop the JVM correctly?

public class StartVerticle extends AbstractVerticle
{
   @Override
   public void start(final Future<Void> startFuture) throws Exception
   {
      // deploy other verticles

      // if something does wrong undeploy
      startFuture.fail(new IllegalArgumentException("Cannot start"));
   }
}

Regards,

Assen

[Assen Todorov]

Hi,

The code below, reproduces the problem. I looked into the source code and it looks like vertx.close() gets never called.

vertx.close(), which closes all resources inclusive event-loop threads, gets invoked only from JVM shutdown hook, which is defined in Starter.runVerticle(). This hook in turn will be triggered only if all non-demon threads are stopped, which is definitely not the case with the event-loop threads (they get closed only from vertx.close())

Currently you have to call vertx.close() manually in order to exit the JVM. 

public class MyVerticle extends AbstractVerticle
{
   public void start(Future<Void> startFuture)
   {
      // Now deploy some other verticle:

      vertx.deployVerticle("force-exception", res -> {
         if(res.succeeded())
         {
            startFuture.complete();
         }
         else
         {
            startFuture.fail("After this log message JVM hangs !");
         }
      });
   }

public static void main(final String[] args)
{
   final String[] arguments = {"run", "test.MyVerticle"};
   Starter.main(arguments);
}

[Tim Fox]

Assen,

Just to clarify - are you referring to running at the command line?

Could you be a bit more specific about how you're running your application and what you expect to happen?

Thanks

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Assen Todorov]

Hi Tim,

Thanks for your reply.

On Wednesday, July 1, 2015 at 10:47:40 PM UTC+2, Tim Fox wrote:

Assen,

Just to clarify - are you referring to running at the command line?

 

 We deploy our application as fat-jar and run it with java -jar on the command line. But the problem could also be reproduced, if we start our application in the IDE via main-method (see my second post for reproducer)

 

Could you be a bit more specific about how you're running your application and what you expect to happen?

Expectations

When we start our application (java -jar ...), we would like that the JVM exits, if some application initialization error occur (for instance the application did not found some configuration file). To achieve this, we implemented the async version of the start-Method in our main verticle. On initialization error, we just call startFuture.fail() and expect, that the Vertx framework stops the deployment and exits the JVM (i.e. there is no more java process in the background and we see immediately the command prompt again :-) )

Observations

When we start our application (java -jar ...) and some application initialization error occur, the JVM does not exit (i.e. it hangs and we do not see the comman prompt again). Investigation with jvisualvm showed us, that there are still event-loop threads in background. 

How to reproduce? 

1. Take the project maven-verticle from vert-x3/vertx-examples. 

2. Copy-paste the reproducer verticle from my second post into this project under src/main/java/io/vertx/example. Preferably do it in some IDE so that all necessary classes are imported.

3. In the pom.xml replace the <main.verticle>-tag with io.vertx.example.MyVerticle.

4. Build the fat-jar and start with java -jar

After the start you will see the message "After this log message JVM hangs !"  and you will notice that the JVM did not exit.

The same code in Vertx 2 worked as expected (of course implemented with the old Vertx API)

Just a hint:

I looked into the source code and it looks as if it has something to do with the logic in the method addShutdownHook() in io.vertx.core.Starter. This hook invokes vertx.close(), which in turn stops all event-loop threads. But this hook will be triggered only if all non-daemon threads are stopped. Some kind of "chicken or the egg dilemma" ;-)

Regards,

Assen

[Tim Fox]

HI Assen,

On 02/07/15 08:20, Assen Todorov wrote:

Hi Tim,

Thanks for your reply.

On Wednesday, July 1, 2015 at 10:47:40 PM UTC+2, Tim Fox wrote:

Assen,

Just to clarify - are you referring to running at the command line?

 

 We deploy our application as fat-jar

Are you using Starter as your main class, or are you using your own main class?

and run it with java -jar on the command line. But the problem could also be reproduced, if we start our application in the IDE via main-method (see my second post for reproducer)

 

Could you be a bit more specific about how you're running your application and what you expect to happen?

Expectations

When we start our application (java -jar ...), we would like that the JVM exits, if some application initialization error occur (for instance the application did not found some configuration file). To achieve this, we implemented the async version of the start-Method in our main verticle. On initialization error, we just call startFuture.fail() and expect, that the Vertx framework stops the deployment and exits the JVM

That's tricky. In general a Vert.x instance can deploy many verticles and if a particular deployment fails stopping the Vert.x instance won't usually be the right thing to do. So we can't do that in the general case, it just wouldn't make sense.

However in the case of a *single* verticle run using Starter, that might be the correct thing to do.

(i.e. there is no more java process in the background and we see immediately the command prompt again :-) )

Observations

When we start our application (java -jar ...) and some application initialization error occur, the JVM does not exit (i.e. it hangs and we do not see the comman prompt again). Investigation with jvisualvm showed us, that there are still event-loop threads in background.

This is deliberate. In Vert.x 2 Vert.x threads were daemon so they would not prevent the JVM from exiting, but this means if you embedded Vert.x in a little main, it would immediately exit! So you'd have to write some code that blocked the main thread, which is a PITA...

I.e. https://gist.github.com/purplefox/aeb4794c519e62e397d2

[Assen Todorov]

Hi Tim,

On Thursday, July 2, 2015 at 1:19:56 PM UTC+2, Tim Fox wrote:

HI Assen,

On 02/07/15 08:20, Assen Todorov wrote:

Hi Tim,

Thanks for your reply.

On Wednesday, July 1, 2015 at 10:47:40 PM UTC+2, Tim Fox wrote:

Assen,

Just to clarify - are you referring to running at the command line?

 

 We deploy our application as fat-jar

Are you using Starter as your main class, or are you using your own main class?

We are using Starter as main class. The Starter deploys a *single* verticle, which in turn deploys other 4 verticles. We do not use Vert.x in embedded mode.

 

and run it with java -jar on the command line. But the problem could also be reproduced, if we start our application in the IDE via main-method (see my second post for reproducer)

 

Could you be a bit more specific about how you're running your application and what you expect to happen?

Expectations

When we start our application (java -jar ...), we would like that the JVM exits, if some application initialization error occur (for instance the application did not found some configuration file). To achieve this, we implemented the async version of the start-Method in our main verticle. On initialization error, we just call startFuture.fail() and expect, that the Vertx framework stops the deployment and exits the JVM

That's tricky. In general a Vert.x instance can deploy many verticles and if a particular deployment fails stopping the Vert.x instance won't usually be the right thing to do. So we can't do that in the general case, it just wouldn't make sense.

However in the case of a *single* verticle run using Starter, that might be the correct thing to do.

IMHO this is our case - *single* verticle run using Starter. But, if the fact matters, that this *single* verticle deploys other verticles, then this could be our problem.

 

(i.e. there is no more java process in the background and we see immediately the command prompt again :-) )

Observations

When we start our application (java -jar ...) and some application initialization error occur, the JVM does not exit (i.e. it hangs and we do not see the comman prompt again). Investigation with jvisualvm showed us, that there are still event-loop threads in background.

This is deliberate. In Vert.x 2 Vert.x threads were daemon so they would not prevent the JVM from exiting,

 

Thanks, this explains why Vert.x 2 worked as we expected.

[Tim Fox]

On 02/07/15 14:07, Assen Todorov wrote:

Hi Tim,

On Thursday, July 2, 2015 at 1:19:56 PM UTC+2, Tim Fox wrote:

HI Assen,

On 02/07/15 08:20, Assen Todorov wrote:

Hi Tim,

Thanks for your reply.

On Wednesday, July 1, 2015 at 10:47:40 PM UTC+2, Tim Fox wrote:

Assen,

Just to clarify - are you referring to running at the command line?

 

 We deploy our application as fat-jar

Are you using Starter as your main class, or are you using your own main class?

We are using Starter as main class. The Starter deploys a *single* verticle, which in turn deploys other 4 verticles. We do not use Vert.x in embedded mode.

 

and run it with java -jar on the command line. But the problem could also be reproduced, if we start our application in the IDE via main-method (see my second post for reproducer)

 

Could you be a bit more specific about how you're running your application and what you expect to happen?

Expectations

When we start our application (java -jar ...), we would like that the JVM exits, if some application initialization error occur (for instance the application did not found some configuration file). To achieve this, we implemented the async version of the start-Method in our main verticle. On initialization error, we just call startFuture.fail() and expect, that the Vertx framework stops the deployment and exits the JVM

That's tricky. In general a Vert.x instance can deploy many verticles and if a particular deployment fails stopping the Vert.x instance won't usually be the right thing to do. So we can't do that in the general case, it just wouldn't make sense.

However in the case of a *single* verticle run using Starter, that might be the correct thing to do.

IMHO this is our case - *single* verticle run using Starter. But, if the fact matters, that this *single* verticle deploys other verticles, then this could be our problem.

Ok, I've changed the behaviour of Starter, so a failed deployment will close the Vert.x instance. (in master)

[Assen Todorov]

Hi Tim,

Awesome! Many thanks.