Multi-tenancy in vert.x
Tim Fox
of development.
One of the first things I want to do is a new piece of development to
allow vert.x to support multiple applications concurrently. In real
world installations its likely that system administrators may want a
single vert.x instance hosting multiple user applications, and for
those applications to be (at last partially) isolated from each other.
During this piece of work I'd also like to plug a current hole in the
concurrency model: Currently the NetServer connect handler and the
HTTP server request handler can actually be called by multiple event
loops concurrently. This breaks the vert.x concurrency guarantee that
any piece of user code will always be called by the event loop, and
can lead to concurrency issues, e.g. if that handler contains mutable
state which is changed when the handler is invoked.
To fix this, vert.x can simply ensure that the connect or request
handler is always called by the same event loop. This fixes the
concurrency issue, but means the application is effectively single
threaded - which isn't good if you want to scale it over all the cores
on your server.
The solution I propose to have our cake and eat it, is to indeed
ensure that each instance of an application is strictly single
threaded, _but_ to allow multiple instances of that application to be
deployed on a single vert.x instance. This gives us the advantage of
code never being executed by more than one thread, but still scaling
nicely over multiple cores.
We can do this without any changes to the current API, by allowing
multiple NetServers (or HTTPServers) to be created from user
application code with the same host/port values. When this occurs,
vert.x will detect the host/port are the same and only create one
actual server under the covers. As connections come in to the server,
handlers will be selected from those set from user code in a round
robin fashion.
To create multiple servers listening on same host/port, the user app
can either do this programatically, or more commonly would do this by
deploying and starting multiple instances of their app on a vert.x
instance.
To allow this we need to introduce a new command to start a vert.x
server (with no apps running);
vertx-start <optional management port>
The vertx server will listen on a special management port for
commands, defined using a simple text protocol:
DEPLOY JAVA <NAME> <CLASSPATH> <MAIN> (deploys a Java app)
START <NAME> <INSTANCES> (starts a number of instances of named app)
STOP <NAME> INSTANCES (stops number of instances of named app)
SHOW <NAME> - (prints instances)
Each deployed application instance will run using its own classloader
so different deployed application instances can't see each other.
We will also provide a simple CLI that speaks the text protocol so the
commands can be typed directly into the console.
Summary:
1) Each vert.x application _instance_ will be strictly single threaded
- this removes any possibility of race conditions in user code
2) We allow multiple instances of any app to be deployed on any vert.x
server - this allows us to scale across cores
3) We provide a simple text protocol and CLI to manage the instances.
Tim Fox
A few changes from my last email:
I've introduced the following commands:
vertx start -port <port>
To start a vertx server, listening for commands on the (optional)
port. Default is 22571
vertx stop -port <port>
Stops a vertx server (port is optional)
vertx deploy -java|-ruby -name name -main main -cp classpath -
instances instances -port port
Deploys a vertx app to the server on port <port> (or default 25571 if
port is not specified).
-java if it's a Java app
-ruby if it's a Ruby app
name is an optional name given to the app, otherwise one will be
generated
main is the main class (Java) or main script name (Ruby)
cp is the classpath (LOAD_PATH in Ruby)
instances - how many instances of the app should be deployed. To scale
over multiple cores you need to deploy multiple instances since a
single instance is strictly single threaded. This defaults to number
of cores on server.
port - optional port that vert.x instance is listening on to deploy
to.
In the end I didn't use a text protocol since vertx deploy currently
can only be used from the same machine the server is on.
Tim Fox
JS and Groovy apps too. Vertx can now run JS/Groovy scripts, but the
vertx app is not available in JS/Groovy yet so they can't do much yet.
For now I've used Rhino, but I'm very interested in swapping it out
for DynJS once it has progressed more in its development :)
Douglas Campos
> for DynJS once it has progressed more in its development :)
We're working to make this happen rather soonish than later(ish)
:)
Tim Fox
Sounds good. I look forward to integrating it :)