Basic Question - Multiple values with the same key

2,515 views
Skip to first unread message

Siddharth Jagtiani

unread,
Nov 22, 2011, 1:12:20 AM11/22/11
to memc...@googlegroups.com
Hi All,

My scenario needs me to retrieve multiple objects that have the same key. Infact my scenario needs me to identify objects using multiple keys too, but I can solve the multiple keys problem by adding one more entry to memcached. So thats not my question. Is it possible to store multiple values using the same key ?

A parallel caching technology is Velocity "http://msdn.microsoft.com/en-us/magazine/dd861287.aspx". Here we can create something called regions, within a cache instance. And I can split my data between regions. So that I dont need to store multiple values using the same key. I can just create regions, and every region can host my unique data.

I also referred to "http://stackoverflow.com/questions/1049833/multi-valued-hashtable-in-java". I think its possible in java. But wondering if there is a parallel in memcached (or should I do it the hard way/slower way, store a map as value, get, add, set).

Please advice
Siddharth

dormando

unread,
Nov 22, 2011, 1:57:04 AM11/22/11
to memc...@googlegroups.com

Collections aren't supported. Your options are to pack multiple values
into a single key (Not sure why this is an issue? If you're fetching it
all back anyway...), or to give them different keys and issue multigets to
fetch item collections back.

It's really never an issue in practice. If so I'd like to see benchmarks
or code showing otherwise :/ Unless there's some specific feature about
collections that you're looking for.

Siddharth Jagtiani

unread,
Nov 22, 2011, 3:08:24 AM11/22/11
to memc...@googlegroups.com
Well, if I need to put another object in the collection, I need to first get it the existing object from the cache. And then insert this new object within that collection. Reducing performance by that much. But I understand that perf will not drop considerably since a get is much faster, and its only 1 more get for every put. 

If memcached would provide such a feature, it would have to manage a collection instead of a value. And allow a api to "insert duplicate". And fetch a collection instead of a object. Much faster than the former approach above.

Ok, let me ask this question (sorry if Im being lame here, just point me to the doc's if I missed something). Is there a way to define a set (or region as it would be on microsoft velocity) ?

Siddharth

dormando

unread,
Nov 22, 2011, 3:35:12 AM11/22/11
to memc...@googlegroups.com
> Well, if I need to put another object in the collection, I need to first get it the existing object from the cache. And then insert this new object
> within that collection. Reducing performance by that much. But I understand that perf will not drop considerably since a get is much faster, and its
> only 1 more get for every put.�
> If memcached would provide such a feature, it would have to manage a collection instead of a value. And allow a api to "insert duplicate". And fetch
> a collection instead of a object. Much faster than the former approach above.
>
> Ok, let me ask this question (sorry if Im being lame here, just point me to the doc's if I missed something). Is there a way to define a set (or
> region as it would be on microsoft velocity) ?

You should read through the wiki: http://memcached.org/wiki - there're
some programming sections with examples. Using namespacing is probably
similar to what you're talking about with regions.

If you're managing a list, you may also be able to use the append and/or
prepend commands to stack sets of bytes without re-fetching the object.

Siddharth Jagtiani

unread,
Nov 22, 2011, 3:37:04 AM11/22/11
to memc...@googlegroups.com
Super Dormando. This helps me in the right direction. Thanks a ton.

Siddharth

Siddharth Jagtiani

unread,
Nov 22, 2011, 5:09:45 AM11/22/11
to memc...@googlegroups.com
Dormando,
Quick question.

So if I were to 
put (key, array_of_size_3)
and then
append (key, new_item)

value = get (key)
size of value will be 4 ?

Siddharth

On Tue, Nov 22, 2011 at 2:05 PM, dormando <dorm...@rydia.net> wrote:

dormando

unread,
Nov 22, 2011, 3:07:15 PM11/22/11
to memc...@googlegroups.com
> Dormando,Quick question.

>
> So if I were to�
> put (key, array_of_size_3)
> and then
> append (key, new_item)
>
> value = get (key)
> size of value will be 4 ?

if array_of_size_3 is "3 bytes", and new_item is "1 byte", then yes.
remember that if you're appending complex structures, you still need to be
able to parse what you get back.

Dustin

unread,
Nov 22, 2011, 4:40:22 PM11/22/11
to memc...@googlegroups.com

On Tuesday, November 22, 2011 12:07:15 PM UTC-8, Dormando wrote:
> Dormando,Quick question.
>
> So if I were to�

> put (key, array_of_size_3)
> and then
> append (key, new_item)
>
> value = get (key)
> size of value will be 4 ?

if array_of_size_3 is "3 bytes", and new_item is "1 byte", then yes.
remember that if you're appending complex structures, you still need to be
able to parse what you get back.


This entirely depends on the format of your data.  If whatever you are storing can be concatenated and make a larger version of it, then yeah.  If it's something like a JSON array, then no. 

柴俊堃

unread,
Nov 22, 2011, 9:37:44 PM11/22/11
to memc...@googlegroups.com
Hi:
    I'm a java programer. accroding to ur question, I made a test to verify my solution that u can use Collection Object. what I use memcached client is
java_memcached-release_2.6.2.jar which is from https://github.com/gwhalin/Memcached-Java-Client/downloads
 
    My test as following:
class Person implements Serializable{
 /**
  *
  */
 private static final long serialVersionUID = 4468130987525379646L;
 private String name;
 private int age;
 public String getName() {
  return name;
 }
 public void setName(String name) {
  this.name = name;
 }
 public int getAge() {
  return age;
 }
 public void setAge(int age) {
  this.age = age;
 }
 public Person(String name, int age) {
  super();
  this.name = name;
  this.age = age;
 }
}
Important: u must implements Serializable interface. cuz the framework send object to memcached server by socket finally.
then, to use:
 
  Person p1=new Person("Jack", 20);
  Person p2=new Person("Rose", 30);
  LinkedList<Person> ll= new LinkedList<Person>();
  ll.add(p1);
  ll.add(p2);
  if (memcacheService.add("KeyList", ll)){
   System.out.println("success to add linked list");
  }else{
   System.out.println("fail to add linked list");
  }
  @SuppressWarnings("unchecked")
  LinkedList<Person> tt= (LinkedList<Person>)memcacheService.get("KeyList");
  System.out.println("object count in linked list:"+tt.size());
  Person p3=tt.get(0);
  Person p4=tt.get(1);
  System.out.println(p3.getName()+p3.getAge());
  System.out.println(p4.getName()+p4.getAge());
 
may my code helps u!
 
 
2011/11/23 Dustin <dsal...@gmail.com>



--
柴俊堃 敬上

ktechie

unread,
Nov 23, 2011, 12:06:19 AM11/23/11
to memcached
We have implemented a java solution which uses memcached, the client
being xmemcached.

We are using a multiget feature, which has helped in improving
performance.

Is there any recommendations around how many objects should I try to
fetch using multiget at a time.

The default timeout is one sec, which could be changed by passing a
parameter

On Nov 23, 7:37 am, 柴俊堃 <chaijun...@gmail.com> wrote:
> Hi:
>     I'm a java programer. accroding to ur question, I made a test to verify
> my solution that u can use Collection Object. what I use memcached client
> is

> java_memcached-release_2.6.2.jar which is fromhttps://github.com/gwhalin/Memcached-Java-Client/downloads


>
>     My test as following:
> class Person implements Serializable{
>  /**
>   *
>   */
>  private static final long serialVersionUID = 4468130987525379646L;
>  private String name;
>  private int age;
>  public String getName() {
>   return name;
>  }
>  public void setName(String name) {
>   this.name = name;
>  }
>  public int getAge() {
>   return age;
>  }
>  public void setAge(int age) {
>   this.age = age;
>  }
>  public Person(String name, int age) {
>   super();
>   this.name = name;
>   this.age = age;
>  }}
>

> *Important: u must implements Serializable interface. cuz the framework
> send object to memcached server by socket finally.*


> then, to use:
>
>   Person p1=new Person("Jack", 20);
>   Person p2=new Person("Rose", 30);
>   LinkedList<Person> ll= new LinkedList<Person>();
>   ll.add(p1);
>   ll.add(p2);
>   if (memcacheService.add("KeyList", ll)){
>    System.out.println("success to add linked list");
>   }else{
>    System.out.println("fail to add linked list");
>   }
>   @SuppressWarnings("unchecked")
>   LinkedList<Person> tt= (LinkedList<Person>)memcacheService.get("KeyList");
>   System.out.println("object count in linked list:"+tt.size());
>   Person p3=tt.get(0);
>   Person p4=tt.get(1);
>   System.out.println(p3.getName()+p3.getAge());
>   System.out.println(p4.getName()+p4.getAge());
>
> may my code helps u!
>

> 2011/11/23 Dustin <dsalli...@gmail.com>

Siddharth Jagtiani

unread,
Nov 23, 2011, 3:06:42 AM11/23/11
to memc...@googlegroups.com
Thanks guys.. I think I'll go with the get collection, add record and add(replace) to memcached. Its a perf hit, but considering that I am pre-populating the cache during server load, I can afford the load time perf hit. And during runtime, its not all that bad either. Considering that I expect 90% reads and 10% writes on normal days.

柴俊堃

unread,
Nov 23, 2011, 8:50:30 PM11/23/11
to memc...@googlegroups.com
Well, I'm sorry but what I used memcached client is gwhalin's client. I searched the client that you said, wow, It's designed for high perfomance applications. Maybe I should pay more attention to this, Thx.
 
by the other side, I can tell you that you can do like this by gwhalin's client:
 
MemCachedClient mcc = new MemCachedClient();
SockIOPool pool = SockIOPool.getInstance();
pool.setSocketTO(3000);
pool.initialize();
 
then you use mcc to "put" or "get" objects. it will work by the param of pool.
when you "put" or "get" object, memcached client will try to connect to server. if the time beyonds 3000 milliseconds,
the command will be considered failure, then return false or null.
 
may my answer can help you.


 
2011/11/23 ktechie <kirandos...@gmail.com>



--
柴俊堃 敬上
Reply all
Reply to author
Forward
0 new messages