MapStore Load with TTL

[Shelley Baker]

Is there any way to load a map entry with a custom TTL using a MapStore?

Use Case: My map entries each have a custom expiration, at which point the entry is no longer valid (the TTL is not just for limiting the size of the in-memory map, and the TTL is applied to each entry, not to the map config). I set this TTL when I initially put my entry in the map, the expiration is persisted in my underlying persistent map datastore, but I cannot reset this TTL when loading my entries from the database.

    public static class MyMapStore implements MapStore<MayKey, MapValue> {

        @Override
        public MapValue load(MayKey key) {
            MapValue value = datstore.lookup(key);
            Date ttl = value.getExpiration();
            // the value has it's own entry-specific TTL, but it seems this can't be used from the MapStore
            return value;
        }

        // . . .

    }

The MapStore documentation seems to indicate this may not be possible:

Loaded entries will be placed into the distributed map and they will stay in-memory until they are explicitly removed or implicitly evicted (if eviction is configured).

If there is no way to accomplish this with MapStore, some remaining options seem to be:

• Do not use MapStore and instead explicitly lookup entries from my persistent datastore, loading these values with the desired TTL
• This may be feasible, but with this approach, I'm no longer able to take advantage of the benefits of MapStore
• Run a background thread which periodically explicitly evicts these expired entries, rather than relying on Hazelcast's automatic eviction
• This is just a lot of overhead (additional thread, querying and entry processing) and requires that my application code checks the validity of the entry prior to use (since expired entries may exist in the map)

Is there any other way to accomplish this cleanly/easily with Hazelcast that I'm overlooking?

Cross-posting from: StackOverflow

[Ahmet Mircik]

Hi, 

Is this what you want https://gist.github.com/ahmetmircik/c4325be254c94cb2505c (assumed you already know the `key` to access `value`)

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/1b56995b-2cc4-4348-b3d0-108f3db87bff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Shelley Baker]

Thanks, that's an interesting idea, but it seems to suffer all the same limitations as this option:

• Run a background thread which periodically explicitly evicts these expired entries, rather than relying on Hazelcast's automatic eviction
• This is just a lot of overhead (additional thread, querying and entry processing) and requires that my application code checks the validity of the entry prior to use (since expired entries may exist in the map)

Specifically, with your proposal, I'd still need to invoke the EntryProcessor from a separate background thread that runs periodically, which adds overhead and may require additional indexing/querying (since I likely wouldn't want to run the processor on entries that already have TTLs). My application code would still need to check the validity of the entry prior to use (since there may be a delay between the loading of the entry and the time the processor is invoked to set the TTL). In addition, with your proposed approach, I'd also need to maintain an additional attribute on my entry's value to represent it's state in the IMap (i.e. existInMemory).


(I have multiple Hazelcast servers and clients running, and not all entries will be guaranteed to be loaded in memory at once. If my application were a simple single Hazelcast instance which loaded all entries from my MapLoader/MapStore, your solution might be fine.)

[Shelley Baker]

I just noticed your code ran "executeOnKey" rather than "executeOnEntries." The former would not work because I don't know the explicit entries that would need to be updated, and the latter approach still suffers the limitations I described above.

[Nick Pratt]

https://github.com/hazelcast/hazelcast/issues/7728

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/1bea93ce-55f0-499c-a4ae-85e2a5105b97%40googlegroups.com.