Are there any plans to allow storing binary data?

[Wilton]

I need to be able to store binary values such as Doubles, not just Strings.  Are there any plans to allow this in the future?

[Salvatore Sanfilippo]

On Wed, Mar 9, 2011 at 7:21 PM, Wilton <risen...@gmail.com> wrote:
> I need to be able to store binary values such as Doubles, not just Strings.
>  Are there any plans to allow this in the future?

Redis is binary safe since its first apparence in the world :)

Salvatore

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
>

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware

http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele

[Wilton]

Hi Salvatore,

Thanks for the quick response.  I'm using Jedis (Java) as my language binding and there is no way for me to store a non-String as a value in the database.  I'm using HSET.  Here is the link to the Javadoc API.

http://www.jarvana.com/jarvana/view/redis/clients/jedis/1.3.1/jedis-1.3.1-javadoc.jar!/redis/clients/jedis/Jedis.html#hset(java.lang.String, java.lang.String, java.lang.String)

How can I store an Object as a value in the database?

[Pierre Chapuis]

Wilton <risen...@gmail.com> a écrit :

I haven't touched Java for a long time, but in other languages I use MessagePack to encode the object into String format, so I guess you could do the same. It adds headers (1 byte for simple types) that make it able to retrieve the type of the object when decoding, you can strip them to save memory if you don't need them.

--
Pierre 'catwell' Chapuis

[Jonathan Leibiusky]

Hi! I'm the author of jedis and you can store binary safe data in
redis. Just use BinaryJedis :)
There are plenty of examples in the tests but if you need more help
let me know and I will be glad to answer!

Jonathan

[Jonathan Leibiusky]

Actually you can use Jedis class as well. There are overloads for all
the commands with byte[] signature instead of String.
Unless I forgot HSET you should be able to use hset(byte[] key, byte[]
name, byte[] value) method. And if it not there let me know and I will
add it right away!

[Wilton]

Hi Jonathan,

Thanks for responding so quickly.  I'm honestly not sure what to do with a signature like hset(byte[] key, byte[] name, byte[] value) since that's not a method signature that matches any of my typical use cases.  I think if you added hset(String, key, String name, Object value), this would become much more usable for me.  Additionally, if you modified the get routines to return Objects instead of Strings, it would eliminate a lot of headache as well.  

I frankly don't know how to pack a Double into a byte[] value, also taking into account the Java Double cases that include POSITIVE_INFINITY, NEGATIVE_INFINITY, etc.  If you can provide some advice on this, it would be really helpful.

[Jonathan Leibiusky]

So basically you can do the following:

jedis.hset("foo".getBytes(), "bar".getBytes(), yourData);

where yourData is an array of bytes.

why not Object? because I didn't want to add to Jedis a level of complexity where it decides how to serialize stuff. Just serialize it as you want. You can use java serialization api or something else.

For example, Java serialization would be something like:

Double yourdata = 1234d;
ByteArrayOutputStream bytestream = new ByteArrayOutputStream();
DataOutputStream datastream = new DataOutputStream(bytestream);
datastream.writeDouble(yourdata);
datastream.flush();
byte[] bytes = bytestream.toByteArrray();

jedis.hset("foo".getBytes(), "bar".getBytes(), bytes);

So as you can see, serialization strategy was left for the application using Jedis and not included in Jedis itself.

[Wilton]

Thanks for this information.  I'll try this out and see how it works.

However, I think you are missing a very big opportunity by not providing automatic serialization in the driver.  Having to write this code isn't something that most Java developers care to do, and it's not a typical requirement of other database drivers.  The JDBC driver, for example, has methods that accept a wide variety of data types as parameters.  If you want people to leave JDBC and use your product, you should try to reduce the complexity of switching by providing method signatures that are at least somewhat familiar.  

That's just my 2 cents.  

[Jonathan Leibiusky]

Sure, I will think about that.
But in your case, if what you are trying to do is to store Double in a hash, you should do it without serializing it with java.
Just do:
jedis.hset("foo", "bar", yourDouble.toString());

Why? Because you want to be able to use HINCRBY and all those nice redis features. If you just java serialize it, redis won't be able to do anything there.

And when reading it just use the Double parse method. Much simpler :)

--

[Wilton]

I tried this all day yesterday but ran into problems with my Doubles that are NaN (such as POSITIVE_INFINITY).  I might have been doing something wrong, but when I tried to read it from Redis using parseDouble, they didn't turn back into the original values.

[Salvatore Sanfilippo]

In such a case I would convert this special NaN values into strings, like
"+inf", "-inf", and so forth.

Then handle them specially when reading back.
Storing floats as binary values is surely ok as Redis does not care
too much, but a bit strange mode of operation given that numbers are
easy to serialize in a human readable way.

Cheers,
Salvatore

>
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
>

--

[Wilton]

On Wednesday, March 9, 2011 3:16:01 PM UTC-8, Salvatore Sanfilippo wrote:

On Thu, Mar 10, 2011 at 12:12 AM, Wilton <risen...@gmail.com> wrote:
>
> I tried this all day yesterday but ran into problems with my Doubles that
> are NaN (such as POSITIVE_INFINITY).  I might have been doing something
> wrong, but when I tried to read it from Redis using parseDouble, they didn't
> turn back into the original values.

In such a case I would convert this special NaN values into strings, like
"+inf", "-inf", and so forth.

Then handle them specially when reading back.

Thanks for pointing this out.  I get it.  My point is that this, in my little world, should be the work of the database driver, not the application developer.  

Marshalling primitives into the most-easily usable form for the database should not be the purvue of the application developer, since the application developer (me) doesn't know the capabilities of the underlying system as well as the driver developer (you).  

I don't want to seem like I'm complaining -- I'm not.  It seems like you have a very good product here.  I'm just trying to point out a point of pain for me that I'm sure other Java developers will have as well, and there is a really easy, and obvious, opportunity for you and your team to wrap this driver with more type-friendly methods.  This problem probably exists for developers of other strongly-typed languages as well.  

[Salvatore Sanfilippo]

On Thu, Mar 10, 2011 at 12:28 AM, Wilton <risen...@gmail.com> wrote:
> Thanks for pointing this out.  I get it.  My point is that this, in my
> little world, should be the work of the database driver, not the application
> developer.

I understand your point, however I'm really not qualified to comment
this as I don't know Java.
Redis is often used in the domain of more dynamic languages, where the
serialization is more or less a non issue. For this languages a simple
API is a win, as serialization is really trivial or not needed at all
in most cases.
For instance the Ruby driver by default calls toe "to_s" method
against what you pass to Redis, to convert it into a string.
Everything is a string, basically, unless you use this string in some
special way like passing a serialized object, but this is not a common
pattern.

In Java this may be different, I'm not sure. What I noticed with
interest is that in Java there is usually a big amount of disagreement
about what a good API is. I guess there are many different
philosophies in the users of the language.

Cheers,
Salvatore

[Peter]

Try jOhm. It is built on top of Jedis and allows you to store objects.

https://github.com/xetorthio/johm

Hope this help.
Peter

On Mar 9, 2:30 pm, Wilton <risenhoo...@gmail.com> wrote:
> Hi Salvatore,
>
> Thanks for the quick response.  I'm using Jedis (Java) as my language
> binding and there is no way for me to store a non-String as a value in the
> database.  I'm using HSET.  Here is the link to the Javadoc API.
>

> http://www.jarvana.com/jarvana/view/redis/clients/jedis/1.3.1/jedis-1...,