OBJECT in-memory format + EntryProcessor: how to rollback to original value?

[Frederico Ferro Schuh]

Hi all,

OBJECT in-memory format is very appealing to use with EntryProcessors, as it bypasses serialization costs.

However, there's a very important issue with it: if your EntryProcessor modifies the object and an Exception happens before entry.setValue() is called, the local object will still remain modified even though it was not committed to the cache. Worst of all, it will be only partially modified, and potentially be in an invalid state. Since there were no defensive copies made, we can't restore the original value.
Because of this reason, I feel OBJECT in-memory format is not very safe and not as useful as it could be.

However, even in the case of an Exception, a fresh, unchanged copy of the data still exists on the backup nodes (the backup processors haven't ran yet).
My question is: how could I restore the value of the original object by copying it from a backup node? Is there any way to do this while the EntryProcessor is still running?

I have tried cloning the object at the beginning of my EntryProcessor so I can restore it in case of an Exception, but this incurred in low performance as the objects are not very small and the cloning would be running in a partition thread (it actually ran slower than BINARY in-memory format).

If we could restore the original object from a backup node when Exceptions are thrown, OBJECT in-memory format would become much more useful.
Any ideas?

Thanks,
Fred

[Enes Akar]

Hi Fred;

There is no easy way to replicate from backup to owner node.

But following may be a workaround:

- Catch exception inside process()

- Return a specific error object that will include old value.

- When entry processor returns this error object, put old value back to the map.

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at http://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/ed80461b-c77e-423a-b7ea-31b564535818%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Frederico Ferro Schuh]

Thanks for the reply Enes.

The problem with doing what you're suggesting is that there doesn't seem to be a way to do it in a generic way. Basically all my code would have to be prepared with a copy of the old value so it could be restored in case of an error, and I can't think of any other way but to write a specific rollback code on a case-by-case basis. And the old value returned along with the error could become complex if multiple fields were to be modified before the Exception was thrown.

A perfect old value would be a full clone of it - by that's precisely what we're trying to avoid while running the EntryProcessor in OBJECT mode.

I suggested copying the value from the backup because that could eliminate any needs to write code to deal with such a value rollback. And also, the backup is just sitting there, so why not just access it?

Of course one could keep in mind that OBJECT in-memory format has this extra risk associated with it, and be extra careful before applying changes to the objects. However modifying multiple fields will never be an atomic operation and there's always a risk involved.

But you mentioned there's no easy way to replicate from the backup - does it mean there's a hard way to do it? I wouldn't mind, as long as it's possible to get that value :)

If there's a hard way, could point me where to look?

Thanks again.

[Nils Kilden-Pedersen]

Any reason you don't use immutable objects?

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/efab6574-1ffa-4044-a6b5-9d3065543f62%40googlegroups.com.

[Enes Akar]

These are user defined objects, they may prefer to use immutable or mutable objects.

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/CABDULvX%2BF%2BrYrtaXSwjP8SZ4rj%2BuiEDw_mS0g4MuitDB233umQ%40mail.gmail.com.

[Frederico Ferro Schuh]

It does make sense to use mutable objects. One of the big benefits of EntryProcessors is to do lock-free, in-place updates without having to copy the object over the wire first. 

With OBJECT in-memory format you can bypass any serialization. It's a big win for mutable objects, especially the bigger objects (though it's very useful for immutable objects on read-only operations as well).

Any thoughts on the original question Enes? How to access that backup copy? I wouldn't mind if I'd have to go through the SPI or use some obscure API, as long as it's possible to reach that copy.

[Enes Akar]

Why do not you pass the old value with exception as I tried to explain in my first post:

- Catch exception inside process()

- Return a specific error object that will include old value.

- When entry processor returns this error object, put old value back to the map.

Using SPI to read from backup may be possible, but it sounds me like a over-engineering. 

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/455df9b3-b7d6-46b0-80e3-9a02070258c5%40googlegroups.com.

[Frederico Ferro Schuh]

Passing the old value may work for simple mutations on the object, like a couple of fields.

But if the object was heavily modified, it gets progressively more complex to return its old value.

I was looking for a solution that can cover recovering from any kind of object mutation.

[Nils Kilden-Pedersen]

On Tue, Dec 8, 2015 at 4:58 AM, Frederico Ferro Schuh <fred....@gmail.com> wrote:

It does make sense to use mutable objects. One of the big benefits of EntryProcessors is to do lock-free, in-place updates without having to copy the object over the wire first. 

Correct, but this is unrelated to mutable/immutable.

 

With OBJECT in-memory format you can bypass any serialization.

In EntryProcessor, correct, and again unrelated to mutable/immutable.

 

It's a big win for mutable objects, especially the bigger objects

If you use structural sharing, the size ought not matter.

Perhaps you can describe your object type and the needed changes? I'd like to understand this better.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/455df9b3-b7d6-46b0-80e3-9a02070258c5%40googlegroups.com.

[Frederico Ferro Schuh]

I can't see how I'd be able to do in-place updates on an immutable object.

How come in-place updates are unrelated to being mutable/immutable?

Unless you're talking about structural immutability?

My object does not use immutable collections. Which means I do have to add/remove items to underlying lists and maps.

I also perform field modifications on the object.

Sometimes more than one collection will be modified by a single EntryProcessor run, causing a rather complex change to the object.

Returning the old value from an Exception would be a bit difficult in such cases.