performance :- 100 writes per second and 100 reads per 3 seconds

[Dileep Mandapam]

Hi All,

 I have a collection of 15000 domain Objects which is defined below . 

Class Domain{

 .........

 List<String> p1;

 List<String> l1;

 List<String> t1;

 long timeStamp;

}

Domain Object is immutable one .Any write to the domain Object , I am creating a new Object. Readers are supposed to eliminate duplicates  . 

My Query is in the form 

select * from Domain where p1='partner1' and l1='label1' and t1=1000 and timeStamp >1000 .

My Use-case :-

 Writes are more than reads . I am getting 100 writes for every second and 100 reads for every 3 seconds .Each read result may contain 4000 to 5000 results .

From load testing :-

 sometimes readers are taking more than 10 sec . Almost 50% reads are taking more than 1 sec .I have created HashIndexes for all multi value attributes and One Navigable Index for timeStamp .I ran the test using CompoundIndex also . But it is giving worst results .

My question is how to improve read performance  ? .Do i need to make domain Object as mutable one ?  .Please suggest .

Thanks

Dileep.

[Niall]

Hi Dileep,

HashIndex is backed by a ConcurrentHashMap, and NavigableIndex is backed by a ConcurrentSkipListMap. The concurrency characteristics of those indexes is based on the concurrency of the backing maps.

If you are seeing contention on high write concurrency, you can tune the indexes to configure the concurrencyLevel on those backing maps. Especially HashIndex. By default this is 16, but you might need a higher value.

Concurrency parameters can be tuned by supplying factories to the static factory method in HashIndex (and other indexes). Then the index will ask your factories to create the backing map and Sets hold objects, whenever it needs to.

If you don't supply any custom factories, then these factories are used by default (from HashIndex):

    /**
     * Creates an index map using default settings.
     */
    public static class DefaultIndexMapFactory<A, O> implements Factory<ConcurrentMap<A, StoredResultSet<O>>> {
        @Override
        public ConcurrentMap<A, StoredResultSet<O>> create() {
            return new ConcurrentHashMap<A, StoredResultSet<O>>();
        }
    }

    /**
     * Creates a value set using default settings.
     */
    public static class DefaultValueSetFactory<O> implements Factory<StoredResultSet<O>> {
        @Override
        public StoredResultSet<O> create() {
            return new StoredSetBasedResultSet<O>(Collections.<O>newSetFromMap(new ConcurrentHashMap<O, Boolean>()));
        }
    }

So the solution is to define your own factories which use this constructor of ConcurrentHashMap to set concurrencyLevel greater than 16.

This should improve performance with high write concurrency.

HTH,
Niall