vitess and java

[Montassar Dridi]

how can I run a java application in vitess and connect it to the database tablets ?
is there any example for java to follow as the guestbook python example provided in the vitess GitHub

[Michael Berlin]

Yes, there is :-)

Please see: https://github.com/youtube/vitess/tree/master/java/example/src/main/java/com/youtube/vitess/example

VitessClientExample.java - direct usage of the Vitess Java client

VitessJDBCExample.java - using the Vitess JDBC driver

On Wed, Nov 23, 2016 at 2:37 PM, Montassar Dridi <montass...@gmail.com> wrote:

how can I run a java application in vitess and connect it to the database tablets ?
is there any example for java to follow as the guestbook python example provided in the vitess GitHub

--
You received this message because you are subscribed to the Google Groups "vitess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vitess+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Montassar Dridi]

I deployed my java web application in kubernetes using DEPLOYMENTS and was able to scale it and connect it to a database POD, but then I wanted to scale the database too but as you know is not possible or not recommended for production. So I tried vitess and was able to scale my database but don't know how or where should I create my java web application DEPLOYMENTS/REPLICAS and connect them to the database through vtgate.

On Wednesday, November 23, 2016 at 5:41:33 PM UTC-5, Michael Berlin wrote:

Yes, there is :-)

Please see: https://github.com/youtube/vitess/tree/master/java/example/src/main/java/com/youtube/vitess/example

VitessClientExample.java - direct usage of the Vitess Java client

VitessJDBCExample.java - using the Vitess JDBC driver

On Wed, Nov 23, 2016 at 2:37 PM, Montassar Dridi <montass...@gmail.com> wrote:

how can I run a java application in vitess and connect it to the database tablets ?
is there any example for java to follow as the guestbook python example provided in the vitess GitHub

--
You received this message because you are subscribed to the Google Groups "vitess" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vitess+un...@googlegroups.com.

[Anthony Yeh]

As Michael mentioned, you need to swap out the driver that your Java app uses to talk to the database, to use the Vitess driver instead of the plain MySQL driver. This will require some code changes in your app, and some queries may need to be rewritten to avoid constructs that are incompatible with Vitess.

Once your app is compatible with Vitess, deploying it in Kubernetes will be the same as you did before, except you will point the app pods to connect to the VTGate service via DNS, like this:

https://github.com/youtube/vitess/blob/master/examples/kubernetes/guestbook/main.py#L84

[Montassar Dridi]

thank you so much for your help, that's exactly what I'm trying to avoid, app code changing.
sorry to bother you but do you know any other way of scaling mysql database through kubernetes ? 

[Anthony Yeh]

For scaling replica reads horizontally (adding more slaves), there is a new Beta feature in Kubernetes 1.5 (set to be released in December) called StatefulSet that works like ReplicaSet but for databases.

You can see an early preview of documentation for using this with MySQL here:

https://deploy-preview-1722--kubernetes-io-vnext-staging.netlify.com/docs/tutorials/replicated-stateful-application/run-replicated-stateful-application/

If you need to scale master traffic (writes) horizontally as well, you will probably need to implement application-level sharding. At that point, the changes needed to support Vitess instead would almost certainly be much much smaller.

[Montassar Dridi]

Again I appreciate your help thanks

[Montassar Dridi]

Hi anthony,

I start experiencing with statefulsets, and I'm following this link on kubernetes website 

http://kubernetes.io/docs/tutorials/stateful-application/run-replicated-stateful-application/

How can I connect my java  application to the mysql database statefulset (they are all in same kubernetes cluster node and namesapce)? what service should I connect to ? which address should I use ? 

I tried mysql-read..default.svc.cluster.local:3306 didn't work !!!

BTW I online attended your presentation at VMware meetup at Portworx HQ in Mountain View, CA. great job !!

[Montassar Dridi]

[Michael Berlin]

Hi,

I'm not Anthony but I can already answer some of your questions.

On Thu, Jan 5, 2017 at 12:12 AM Montassar Dridi <montass...@gmail.com> wrote:

Hi anthony,

I start experiencing with statefulsets, and I'm following this link on kubernetes website 

http://kubernetes.io/docs/tutorials/stateful-application/run-replicated-stateful-application/

Note that that page is talking about a pure MySQL setup and not mentioning Vitess at all.

If you want to run Vitess as well, it needs to be run on top of MySQL. To achieve that you should use our Kubernetes configuration instead. See: http://vitess.io/getting-started/

btw: The special thing about the example in the link above is that it's using the Kubernetes StateFulSet (in contrast to the support for stateless applications). This is a new feature which isn't supported by our Kubernetes configuration yet. (Vitess is able to run stateless on Kubernetes by restoring data using the MySQL replication and backups.) Vitess will support StateFulSet in the future as well: Anthony is actively working on a Vitess Kubernetes configuration for it.

How can I connect my java  application to the mysql database statefulset (they are all in same kubernetes cluster node and namesapce)? what service should I connect to ? which address should I use ? 

I tried mysql-read..default.svc.cluster.local:3306 didn't work !!!

This question is specific to the pure MySQL Kubernetes tutorial you linked above and not related to Vitess.

You should ask about that on the Kubernetes support channels.

To me it looks like your problem is that you're not running your application within Kubernetes as well. Otherwise, the DNS name "mysql-read" should resolve to one of the MySQL replicas. Maybe look into exposing the Kubernetes services externally i.e. outside of the cluster? Or just run your app within Kubernetes?

Cheers,

Michael

[Montassar Dridi]

Hi Michael

Thanks for your response, I already posted my question on kubernetes groups but no answer. I was desperate sorry I didn't mean to post it on vitess group, but in his VMware presentation Anthony showed two example, one with vitess and the other with statefulset, that's why I thought of contacting him here, since we talked about it in previous posts and his familiar with the issue.

Again my bad.

My application is running within kubernetes like I said same cluster , node and namespace. When I put the IP address of the master or any of the slaves, the connection works fine. But when I try any of the service DNS name, it doesn't work.

When I look at the mysql pod logs in kubernetes, I get this note [Note] Access denied for user 'root'@'localhost' (using password: NO), even though the connection works fine. I already set up a password for my root and create new users and imported my data !!!

Best Regards,

Montassar

[Anthony Yeh]

If you've set a password for the root user, I think that explains the "Access denied" errors in the log. There are parts of the tutorial config (such as readiness probes) that assume root has no password, and these will need to be adjusted if you assign a password.

When you say the service doesn't work, can you clarify what error you get exactly? Is it a DNS resolution failure? Or connection denied/timeout? If it's the latter, my guess is that due to the "Access denied" errors above, the pods are stuck in Unready state, meaning the service has no Ready pods to send connections to. You can check whether pods are ready with 'kubectl get pods'.

[Montassar Dridi]

Hi anthony

Sorry to bother you again I know you have a busy schedule. I appreciate your help

You are right, I get this error 

Readiness probe failed: ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

kubectl get podsNAME READY STATUS RESTARTS AGEmysql-0 1/2 Running 0 16h

 

Also, when I delete the slave pods they don't get recreated again, only the master get to be recreated once it's been deleted.

[Anthony Yeh]

Yes that looks consistent with my hypothesis. If you're still just experimenting, I recommend reverting to allowing passwordless root. Then you can continue to experiment with other aspects of the system.

If your goal is to evolve the tutorial config into a production-ready one ASAP, then you'll need to come up with a way to handle authentication. When writing the tutorial, I intentionally avoided solving that problem because the goal was to teach fundamental StatefulSet concepts, not to produce a production-ready MySQL config.

If that is your goal, one place to get started is to look at the non-StatefulSet MySQL chart, which does attempt to address authentication by using Secrets:

https://github.com/kubernetes/charts/tree/master/stable/mysql

[Montassar Dridi]

Again I'm so thankful for all your advises and suggestions, and I appreciate your time. That was really spot-on helpful.