How to store binary data in Kamanja containers?

[Koushik Kulkarni]

hi all,

We need to store java serialized binary data in containers - can someone please let me know if we can do this using KVInit? if yes, please direct me to an example. The examples from SampleApplications are using CSV or delimited data only. 

thanks,

Koushik

[Krishna Uppala]

Koushik,

 

Is this serialized binary data a field of some data structure? In any case, currently there is no mechanism in the message deserializer to decode some input data into array of bytes. Best bet is to encode the binary data as hex decimal or 64 base string and store as string and then decode up on retrieving into memory. We will add proper conversion function as part of transformation work that will be doing in near future.

 

Thanks

-Krishna

--
You received this message because you are subscribed to the Google Groups "kamanja" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kamanja+u...@ligadata.com.
To post to this group, send email to kam...@ligadata.com.
Visit this group at https://groups.google.com/a/ligadata.com/group/kamanja/.

[Koushik Kulkarni]

Thanks Krishna. We encoded the binary data in base 64 string, but now we are getting below error while loading it in HBase store using KVInit:

ERROR com.ligadata.keyvaluestore.HBaseAdapter - Failed to save a list of objects in table(s) :Row length 52139 is > 32767

I suppose it is to do with some setting for the storage adapter - can someone please let me know what setting has to be done to allow insertion of bigger row values in the data store?

Thanks,

Koushik

[Koushik Kulkarni]

Hi all,

Can someone please let me know about the below issue at the earliest possible?

Thanks,

Koushik

[Sreenivasulu Pokuri]

Hi Koushik,

 

Looks like you are keeping the KEY (ROW) value as 50KB. HBase allows only max Short.MAX_VALUE as the ROW value.

 

Can you please check once whether you are keeping all that stuff as KEY or not?

 

Thanks

Pokuri

[Koushik]

Yes Pokuri, that was the issue. I added a separate key and was able to insert the data in the data store. 

Thanks for the help!

Thanks,

Koushik