Regarding vitess for mysql client connection and vtctlclient.

[amit.a...@iacpublishinglabs.com]

Hi,

I am new to vitess and recently started working on this.

I have installed the vitess following doc "http://vitess.io/getting-started/local-instance.html” and vtctlclient using doc 

https://golang.org/doc/install?download=go1.6.2.linux-amd64.tar.gz

http://vitess.io/doc/GettingStartedKubernetes/

go get github.com/youtube/vitess/go/cmd/vtctlclient

1. As a part of execrcie in doc http://vitess.io/getting-started/local-instance.html we ran the client application ./client.sh which calls client.py, it ran perfectly fine on the container, but if we want to run the same from client machine, what all we need to make it work.
2. IF some mysql developer wants to insert and select the data from mysql instance which is the part of Vitess, how he can do that using mysqlclient (or it is just possible using vtctlclient, if yes how can developer do the same using  vtctlclient).

Thanks,

Amit

[Michael Berlin]

Hi Amit,

Thanks for trying out Vitess!

Let me try to answer your questions. See inline.

On Mon, Jun 27, 2016 at 11:21 AM, <amit.a...@iacpublishinglabs.com> wrote:

Hi,

I am new to vitess and recently started working on this.

I have installed the vitess following doc "http://vitess.io/getting-started/local-instance.html” and vtctlclient using doc 

https://golang.org/doc/install?download=go1.6.2.linux-amd64.tar.gz

http://vitess.io/doc/GettingStartedKubernetes/

go get github.com/youtube/vitess/go/cmd/vtctlclient

1. As a part of execrcie in doc http://vitess.io/getting-started/local-instance.html we ran the client application ./client.sh which calls client.py, it ran perfectly fine on the container, but if we want to run the same from client machine, what all we need to make it work.

The example should work on your local machine as well.

Note that the name "client.py" is a bit misleading. It's more an example application which demonstrates how to use the Vitess Python client.

If you like, you can run the client.py example application against a vtgate which runs on a different machine. Therefore, see the --server parameter:

https://github.com/youtube/vitess/blob/master/examples/local/client.py#L27

As you can see, by default it points to "localhost:15991".

1. IF some mysql developer wants to insert and select the data from mysql instance which is the part of Vitess, how he can do that using mysqlclient (or it is just possible using vtctlclient, if yes how can developer do the same using  vtctlclient).

Please don't use vtctlclient for this. vtctlclient is meant to control Vitess, but not to read or write data. vtctlclient is only used to interact with the adminstration service called "vtctld".

For comparison, see our architecture overview: https://raw.githubusercontent.com/youtube/vitess/master/doc/VitessOverview.png

What's missing on that picture is a box called "vtctlclient" which only points to the "vtctld" box.

With "mysqlclient" you seem to refer to the MySQL Python client library?

Our Python client library is compliant with the Python DB API (https://www.python.org/dev/peps/pep-0249/).

You can use it as a drop in replacement for any other DB client library.

I recommend to have a look at the client.py which you already found. It's an example how to use our Python client library.

https://github.com/youtube/vitess/blob/master/examples/local/client.py

Let me know if you have more questions.

Cheers,

Michael

 

Thanks,

Amit

--
You received this message because you are subscribed to the Google Groups "vitess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vitess+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[amit.a...@iacpublishinglabs.com]

Thanks Michael for the details which you have provided.

I did figure out the port and server in client.py and i have corresponding port mapping from container to localhost (docker engine server).

Problem is when we run the client.sh it sets up the path for PYTHONPATH 

for pkg in `find $VTROOT/dist -name site-packages`; do

  export PYTHONPATH=$pkg:$PYTHONPATH

done

export PYTHONPATH=$VTROOT/py-vtdb:$PYTHONPATH

Since we don't have VTROOT on client machine so it doesn't source the PYTHONPATH properly.

Initially it was failing for "import argparse" so i did install the python 7 on remote host and now it is failing for "from vtdb import grpc_vtgate_client"

So to make client.sh/client.py what exactly we need to do to get "vtdb" (Which is there inside /vt/py-vtdb on container but not on remote client machine).

Thanks,

Amit

[Michael Berlin]

Hi Amit,

To recap, you're asking which Python source files you need to bundle to run the Vitess Python client library on any machine. Preferably, you don't always want to ship the full Vitess source directory.

To answer this question, let's take a step back and look at the directory hierarchy on disk.

That directory hierarchy is set up for a Go project in mind and therefore it's a little bit confusing.

When you run Git clone, we let you copy the repository into a sub directory tree called src/github.com/youtube/vitess. See the argument which we pass to git clone:

git clone https://github.com/youtube/vitess.git \
    src/github.com/youtube/vitess

This tree is placed in a directory which our scripts refer to as $VTROOT.

The reason for that is that $VTROOT is also set up as a Go workspace (see https://golang.org/doc/code.html#Workspaces for more details). Therefore, $VTROOT has the sub directories "src", "pkg" and "bin" after you cloned our repository and ran "make build" for the first time.

Additionally, we also maintain a directory "dist" in $VTROOT where we leave external, compiled dependencies (for all kinds of programming languages, not just Go). Our most important dependency is gRPC, an RPC framework, which is needed by the Vitess Python client library to communicate with the vtgate servers.

Putting everything together, you need the following directories to run the Vitess Python client library:

$VTROOT/dist/grpc/usr/local/lib/python2.7/site-packages (gRPC library, installed by the script "$VTTOP/travis/install_grpc.sh")

$VTROOT/src/github.com/youtube/vitess/py (Vitess Python client library)

These two directories need to be in your PYTHONPATH and client.sh ensures this.

Note that the second path is the same as $VTROOT/py-vtdb because "$VTROOT/py-vtdb" links to it.

You can probably also skip the first directory and install the package via "pip" instead. See: https://github.com/grpc/grpc/tree/master/examples/python

Based on these instructions, a "pip install grpcio" should install it for you. Note that we currently use the gRPC version 0.13.

Cheers,

Michael

[amit.a...@iacpublishinglabs.com]

Thanks Michael for your help and finally i was able to run the client.sh from remote desktop.

I just wanted to connect to mysql instance locally (from container itself) getting password error

vitess@504750a93619:/vt/src/github.com/youtube/vitess/examples/local$ mysql -u vitess --socket=/vt/vtdataroot/vt_0000000100/mysql.sock

ERROR 1045 (28000): Access denied for user 'vitess'@'localhost' (using password: NO)

you have any clue whats the password for this user and why it was not needed when ran the client.py from remote host?

Thanks,

Amit

[Michael Berlin]

Hi Amit,

Please use the "vt_dba" MySQL user if you want to access the MySQL daemon from the examples directly.

See the file init_db.sql for the MySQL configuration of the local example: https://github.com/youtube/vitess/blob/master/config/init_db.sql

Note that we only allow access via the socket file. Since you're already using the socket file, you should be fine there.

Cheers,

Michael

[amit.a...@iacpublishinglabs.com]

Thanks Michael for all your support and help.

I am able to do the stuff which i wanted to do.

I might bug you in future but as of now i am good :).

Thanks a lot,

Amit

[amit.a...@iacpublishinglabs.com]

Hi Michael,

Does vitess support JDBC connection from some application or tool?

Thanks,

Amit

[amit.a...@iacpublishinglabs.com]

In the meantime i want to create the 10GB table using following and getting error

vitess@e3e413a58afd:/vt/src/github.com/youtube/vitess/examples/local$ ./lvtctl.sh ApplySchema -sql "$(cat aq_keyword.sql)" test_keyspace

bash: xrealloc: .././subst.c:5273: cannot allocate 18446744071562067968 bytes (4295237632 bytes allocated)

[root@mysqlmha001iad ~]# docker ps

you have any clue how i can resolve this error?

Thanks,

Amit

[Anthony Yeh]

It looks like your aq_keyword.sql is a full data dump file. Our ApplySchema command is only meant for DDLs, not for importing data dumps. What's happening here is that your bash shell is trying to read the whole .sql file into memory so it can pass the contents of the file as a command-line parameter.

To import existing data, you can start with an unsharded keyspace, and import your data dump directly to the underlying MySQL master as you normally would. Then you can do a Vitess resharding after that.

[Harshit Gangal]

To answer about tool.
You can use Squirrel Client to connect to Vitess

[amit.a...@iacpublishinglabs.com]

Thanks Anthony...Thats what i was planning to do but thought to confirm with experts :)

Amit

[amit.a...@iacpublishinglabs.com]

Thanks Harshit, i will try this tool...

Amit