Redis stores

[Daniel Stonier]

Hi,

While hacking around a multi-gateway solution, I'm finding it will need alot of xmlrpc nodes, zeroconf advertising of all of these and interaction between all of them.

Which brought me back to thinking about redis. I don't know much about redis yet, but probably pertinent points for us include:

• Server side it runs like a database stored in ram (lists, dictionaries, ...). Fast and convenient.
• It has pubsub style connections which I believe allow clients to receive callbacks on data store changes.
• Many client languages supported.

If this works, then zeroconf advertisements would be reduced to just advertising the redis server. Clients autodiscover the redis server and dump gateway and monitoring information there. 

All multimaster data and synchronisation processes could be accumulated and triggered from there. 

The advantages - much simpler, far less code, easy to introspect and has other potential uses.

The disadvantages - requires launching of a separate component (the redis server) to do multimaster synchronisations. 

Note that the alternative, with xmlrpc nodes launched at each robot, no additional infrastructure is needed for multimaster synchronisations. Is launching a separate component inconvenient enough to disregard this option?

Thoughts?

Regards,

Daniel.

[Ken Conley]

A long time ago I had spec'd Red is for the next version of the master, for many reasons, including multi master and TTL. So,

+1

[Jack O'Quin]

I just took a first look at redis. Seems like a good fit.

Availability is good, Ubuntu has packaged redis-server for quite a long time.
--
joq

[Piyush]

+1 for redis as well.

I believe the warehouse stack(http://www.ros.org/wiki/warehousewg)
using mongodb is supposed to accomplish the same things as redis. Are
there pros/cons to using one or the other?

Piyush

[Daniel Stonier]

On 4 September 2012 00:57, Piyush <piy...@gmail.com> wrote:

+1 for redis as well.

I believe the warehouse stack(http://www.ros.org/wiki/warehousewg)
using mongodb is supposed to accomplish the same things as redis. Are
there pros/cons to using one or the other?

Piyush

I didn't realise mongodb existed in RAM as well.

Keeping in mind that I have zero past experience with these....;)

It seems that their characteristics serve quite different use cases. Redis is about having temporary key stores which are just what we need for storing multimaster related runtime information - topics, node uri's, platform info etc. Redis also has some pubsub messaging, which would be quite convenient (and I think - testing tomorrow) parallels the usage of xmlrpc handlers. TTL is built in. It's faster.

On the other side, mongodb is more aligned with data storage. Easier to do permanent storage?

 

Daniel.

--
Phone : +82-10-5400-3296 (010-5400-3296)
Home: http://snorriheim.dnsdojo.com/

Yujin R&D: http://rnd.yujinrobot.com/

[Armstrong-Crews, Nicholas - 1002 - MITLL]

-1 mongodb

 

Have used it quite a bit, love it, great for storage, but redis sounds more appropriate for real-time pub/sub

 

That said, I’ve never touched redis… but I trust the positive evaluation by you guys :-)

 

-N

[Piyush]

> Keeping in mind that I have zero past experience with these....;)

About the same here :)

Looking into the warehouse stack further, it seems it explicitly adds
pub/sub on top of mongodb using normal ROS infrastructure. I think
that makes is quite unsuitable for a multi-master approach.
-1 mongodb

Piyush

[Daniel Stonier]

We're going to go ahead and give redis a try as a centralised hub for multimaster connection details (and perhaps use it for parameterisation later as well). There are a couple of issues - the server isn't well supported across platforms, but for now, just running it on a centralised linux system suits us fine. I read some people have had some endianness concerns for arm based clients, so we'll also have to test that. 

If we do have problems later, we can always switch to something else or fallback to xmlrpc everywhere without too much time lost since the redis stuff is easy.

@Piyush, let me know when you have time it'd be great to have you experimenting and working on parts of it.