ABD with comparator

[Nagarajan R]

Hi,

We are using Tokyo Cabinet for building a search application and facing the following concerns to proceed further. 

1. Since Abstract DB supports in memory data bases, we would like to use it for improving the performance. But we are not sure on how to set the comparator for ADB to sort the key records in specified format. Is it possible to set a comparator to ADB like how it is supported by BDB?

2. When we use a BDBCUR to iterate the records,  and when the cursor points to a particular record and when we fetch the key using cur.key() method, will it also load the current records' value into memory? Or this will happen only when we explicitly call cur.val() method? In our case, each key will have million bytes of value and we want to make sure we load them into memory only when its required.

3. There is mention about "getPart" to retrieve only a portion of the value from specified offset and length.

But in file tcadb.c

<code snip>

1652           const char *kbuf;

1653           int ksiz;

1654           TCLISTVAL(kbuf, args, 0, ksiz);

1655           int off = argc > 1 ? tcatoi(TCLISTVALPTR(args, 1)) : 0;

1656           if(off < 0) off = 0;

1657           if(off > INT_MAX / 2 - 1) off = INT_MAX - 1;

1658           int len = argc > 2 ? tcatoi(TCLISTVALPTR(args, 2)) : -1;

1659           if(len < 0 || len > INT_MAX / 2) len = INT_MAX / 2;

1660           int vsiz;

1661           char *vbuf = tcmdbget(adb->mdb, kbuf, ksiz, &vsiz);

1662           if(vbuf){

1663             if(off < vsiz){

1664               rv = tclistnew2(1);

1665               vsiz -= off;

1666               if(vsiz > len) vsiz = len;

1667               if(off > 0) memmove(vbuf, vbuf + off, vsiz);

1668               tclistpushmalloc(rv, vbuf, vsiz);

</code snip>

as per my understanding, line 1661 is where we fetch the value associated with that key into char * buffer.  But it looks like we use offset later in line 1667 to limit the value to the requested level from the application. So, irrespective of the specified offset, do we end up fetching the whole value first and then limit it by the requested offset? We are more concerned about this again because every individual value will be more than million bytes in length in our case and we are interested only in fetching a few bytes every time.

Any clarification in above aspects will be highly helpful for us.

Thanks and Regards,

Nagarajan