Installing Redis cluster in anaconda environment using redis-py-cluster?

[Jasleen Kaur]

Hi,

Can someone tell me how to create a redis cluster with 6 machines(3 masters and 3 slaves) using redis-py-cluster?

I tried creating a cluster by installing redis from source on all the 6 machines, but when i try to run my python scripts on it using my anaconda environment(which has other dependencies installed needed for running my python scripts) , it doesn't seem to work and i get an error saying:

"No module name redis: import redis"

Thanks

[Itamar Haber]

Hi Jasleen,

Creating the cluster is done by installing the machines (servers) and starting up Redis on them in cluster mode. Once the cluster is up, you can connect to it from your code. The error you're getting indicates that the redis-py package (e.g. `pip install redis`) is not installed. The redis-py package is a dependency of the redis-cluster-py package, so I'm guessing redis-cluster-py wasn't installed correctly in your environment.

Cheers,

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/3b76e927-4581-4567-ad56-a4df21580419%40googlegroups.com.

--

Itamar Haber Technicalist Evangely Phone: +972.54.567.9692

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here.

[Jasleen Kaur]

Thanks for your reply, Itamar.

However, I'm still not sure how to do it. Earlier on my 6 machines, I installed redis server instances, started all of them at once, and then create redis cluster with 1 replica each using redis-cli --cluster create command.

And then when i activated my anaconda environment to be able to run my code, my code wasn't detecting redis. I didn't install redis thorugh pip in my anaconda environment cause i figured i have already installed it from source on the mcahines itself.

Please tell me what was wrong in this? And how to use my redis cluster in my anaconda environment to be able to run my code which uses redis and also other packages(which are installed in my anaconda environment)

Thanks

On Thursday, April 30, 2020 at 3:04:35 PM UTC+5:30, Itamar Haber wrote:

Hi Jasleen,

Creating the cluster is done by installing the machines (servers) and starting up Redis on them in cluster mode. Once the cluster is up, you can connect to it from your code. The error you're getting indicates that the redis-py package (e.g. `pip install redis`) is not installed. The redis-py package is a dependency of the redis-cluster-py package, so I'm guessing redis-cluster-py wasn't installed correctly in your environment.

Cheers,

On Thu, Apr 30, 2020 at 11:50 AM Jasleen Kaur <jasleen...@gmail.com> wrote:

Hi,

Can someone tell me how to create a redis cluster with 6 machines(3 masters and 3 slaves) using redis-py-cluster?

I tried creating a cluster by installing redis from source on all the 6 machines, but when i try to run my python scripts on it using my anaconda environment(which has other dependencies installed needed for running my python scripts) , it doesn't seem to work and i get an error saying:

"No module name redis: import redis"

Thanks

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.

To unsubscribe from this group and stop receiving emails from it, send an email to redi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/3b76e927-4581-4567-ad56-a4df21580419%40googlegroups.com.

[Benjamin Sergeant]

I think that your problem, at this point, has nothing to do with redis but entirely with anaconda / your python environment.

Can you pip install a simple module in anaconda, and get your python module to import cleanly ?

Is there a place in the anaconda forums where they could help you with those import problems ?

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/0da26e3f-b1c4-41d8-a4a0-9685b80ab7c3%40googlegroups.com.

[Benjamin Sergeant]

$ cd /tmp

(venv) /tmp$ python3 -mvenv venv

(venv) /tmp$ source venv/bin/activate

(venv) /tmp$ pip install redis

Collecting redis

  Downloading https://files.pythonhosted.org/packages/d2/07/20cb8df2ded4b5db176a65da48b7c8d4295d868776296580b11071218a2b/redis-3.5.0-py2.py3-none-any.whl (71kB)

    100% |████████████████████████████████| 81kB 550kB/s 

Installing collected packages: redis

Successfully installed redis-3.5.0

You are using pip version 18.1, however version 20.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

(venv) /tmp$ python

Python 3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43) 

[Clang 6.0 (clang-600.0.57)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> import redis

>>> print(redis)

<module 'redis' from '/private/tmp/venv/lib/python3.7/site-packages/redis/__init__.py'>

>>> 

Can you create a virtualenv with anaconda ?

[Itamar Haber]

Thanks Benjamin for the walkthrough.

Jasleen (btw, welcome to the community, seen your posts a couple of times ;)), you need a Redis client to connect from your Python code to Redis. The servers you've installed are managing the database, whereas your code will make requests to these via the client. You should follow the above instructions from Mr. Sergeant to get on the right path.

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/5ADEF9AD-6B63-436C-9881-6157683D6AE0%40gmail.com.

[Jasleen Kaur]

Hi Benjamin,

yes, as per your steps I can download the redis into my anaconda environment, however now if i have to create a redis cluster and then run my code on it, how to do that?

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/0da26e3f-b1c4-41d8-a4a0-9685b80ab7c3%40googlegroups.com.

[Jasleen Kaur]

Hi Itama,

Thanks.

As per Benjamin's steps,I am able to install redis in my environment but i actually need to create a redis cluster and run my python code on it? So, how to achieve that?

Thanks.

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/0da26e3f-b1c4-41d8-a4a0-9685b80ab7c3%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.

To unsubscribe from this group and stop receiving emails from it, send an email to redi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/5ADEF9AD-6B63-436C-9881-6157683D6AE0%40gmail.com.

[Itamar Haber]

To connect to a Redis cluster from Python you'll need this: https://github.com/Grokzen/redis-py-cluster

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/fbcb658a-6aa4-4b35-ac16-e504cc6e7aeb%40googlegroups.com.

[Jasleen Kaur]

Yes, but I'm not really able to understand how it works?

Can you please help me get started?

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/fbcb658a-6aa4-4b35-ac16-e504cc6e7aeb%40googlegroups.com.

[Itamar Haber]

Have you followed the README for redis-py-cluster? Specifically, look at the installation and usage: https://github.com/Grokzen/redis-py-cluster#installation

Installation

Latest stable release from pypi

$ pip install redis-py-cluster

This major version of redis-py-cluster supports redis-py >=3.0.0, <3.5.0.

Usage example

Small sample script that shows how to get started with RedisCluster. It can also be found in examples/basic.py

>>> from rediscluster import RedisCluster

>>> # Requires at least one node for cluster discovery. Multiple nodes is recommended.
>>> startup_nodes = [{"host": "127.0.0.1", "port": "7000"}]

>>> rc = RedisCluster(startup_nodes=startup_nodes, decode_responses=True)

>>> rc.set("foo", "bar")
True
>>> print(rc.get("foo"))
'bar'

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/3c1bde57-d482-4bc0-8449-d6f41cb35934%40googlegroups.com.

[Jasleen Kaur]

Yes, I did look through it but i don't understand how to use it in my case of 6 machines? DO i also have to install redis simply as: pip install redis?

And what about redis-py? Does it have to be installed too, for using redis-py-cluster?

 

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/3c1bde57-d482-4bc0-8449-d6f41cb35934%40googlegroups.com.

[Itamar Haber]

You need only to install redis-py-cluster - `pip install redis-py-cluster` - it brings redis-py with it as a dependency. You need to install redis-py-cluster in your application's environment, not on each of the nodes in your cluster. To use it, provide redis-py-cluster the details (host/ip and port) of one (or more) of the nodes in your cluster as shown in the example.

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/1c34a573-9210-424a-9e89-600dbc5161b4%40googlegroups.com.

[Jasleen Kaur]

Okay. 

So, by what you said, i need to write these lines:

from rediscluster import RedisCluster

>>> # Requires at least one node for cluster discovery. Multiple nodes is recommended.
>>> startup_nodes = [{"host": "127.0.0.1", "port": "7000"}]

>>> rc = RedisCluster(startup_nodes=startup_nodes, decode_responses=True)

>>> rc.set("foo", "bar")
True
>>> print(rc.get("foo"))
'bar'

in my python code itself? Like if my application has 6 python files which run one within each other, and I have 1 main.py file which i run it as python main.py

then, in my main.pay I need to write these lines and then when running it will automatically detect redis cluster??

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/1c34a573-9210-424a-9e89-600dbc5161b4%40googlegroups.com.

[Itamar Haber]

That's kinda of it. You will need your client (i.e. redis-py-cluster) in the scope where you need to make reads or writes to Redis. You could put it in main.py if you need it there, or in any of the other files - that's really a matter of how your project is structured and your design choices.

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/7f815888-3dcc-49b2-b9bb-9a5455b508e3%40googlegroups.com.

[Jasleen Kaur]

Also, the redis cluster has to be setup manually before hand right? Like i did earlier on those 6 machines?

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/7f815888-3dcc-49b2-b9bb-9a5455b508e3%40googlegroups.com.