serialising/deserialising CAS (Nodejs API)

[Steven Barlow]

Copied from http://www.couchbase.com/communities/q-and-a/serialisingdeserialising-cas, because I'm noticing that there's very little activity there unfortunately!

Using the Nodejs Couchbase API, I'm noticing that I have to use the exact cas object returned from a "get" operation as the cas option in a subsequent "set" operation. If I make a copy of the cas object e.g. cas = JSON.parse(JSON.stringify(cas)) then the "set" call fails with keyAlreadyExists.

So the question is, how do I manage to implement cas update conflict prevention when the object retrieved by "get" required serialisation before it is modified, and subsequently "set"?

(In my case this is because the document modification occurs on a remote client communicating with the server over http REST interface, which surely must be a very common use case?)

Cheers,

Steven

[Steven Barlow]

So I got round to making an example illustrating my issue. The two set methods (one commented out) I would expect to behave identically, but using "copiedCas" fails with [Error: Bad CAS] code: 4098 (or with keyAlreadyExists if you use the Mock server).

var should = require("should");

var couchbase = require('couchbase');

var connection = new couchbase.Connection({ host: 'localhost:8091', bucket: 'unit_tests' });

//var connection = new couchbase.Mock.Connection();

describe('issue with copying cas', function() {

it('should succeed', function(done){

var testObj = { foo: "bar" };

connection.set('test', testObj, function(err, result) { // Insert with no cas option i.e. overwrite any existing data

should.not.exist(err);

connection.get('test', function(err, result) { // Retrieve the just added document

should.not.exist(err);

should.exist(result.cas);

should.exist(result.value);

result.value.foo = "bar2";

var copiedCas = JSON.parse(JSON.stringify(result.cas));

copiedCas.should.be.eql(result.cas); // The two cas values are equal

connection.set('test', result.value, { cas : copiedCas }, function(err, result) { // Doing it this way fails

// connection.set('test', result.value, { cas : result.cas }, function(err, result) { // Doing it this way succedes

should.not.exist(err);

should.exist(result);

done();

});

});

});

});

});

[Mark Getz]

Steven,

I have not tested this with the new driver but with the old one, (from over a year ago) the same problem existed. One of our guys modified the C code to convert that CAS Object to a serializable number or string and back again when it returned. We mentioned the issue to the Couchbase guys at the time but unfortunately we never submitted the change back to them. I looked for the old code or any notes on what we had done and did not find it. 

So besides confirming having the same issue before, I cannot help much. However, I will ask my team member who did the change to the C code if he still has that lying around somewhere.

Thanks.

Mark

[Steven Barlow]

So it's good to know that I'm not alone, but worrying to hear that this has been an issue for so long. As far as I'm concerned it's a fairly major show stopper bug in what I assumed would be a pretty heavily consumed and field tested component.

Any further light you can shed will be appreciated Mark, and hopefully the Couchbase guys can turn around a fix rapidly, as this issue is basically a blocker for me adopting Couchbase.

Cheers,
Steven

[M. Nunberg]

Keep in mind that since a CAS is a 64 bit integer you would need to pass
it around as a string. The 64 bit width is not merely theoretical but
rather quite common. This is the primary reason why the API maintains
this as an opaque object.

That being said, it should be simple to modify the bits here to also
accept properly formatted strings (but _not_ numbers) for CAS values.

https://github.com/couchbase/couchnode/blob/master/src/cas.cc#L66

[Steven Barlow]

However it is represented in serialised form, there should be a simple way of reconstructing a valid deserialised cas. I'm assuming that there is some hidden deeper structure on the cas object prototype that's getting lost. Hopefully if I go digging, I may be able to isolate it, but it's a bit frustrating: this should be pretty elementary behaviour I'd have thought.

[Mark Getz]

Thank you Mark!
Seeing the code and your comments bring it all back. We had converted
it to a string for the change we made to the code. That would be
perfect for our use case. We hope to have time to switch to the
Couchnode driver in the next month or two.

By the way, thanks for all the work you are doing on Couchnode and on
Libcouchbase. I see a lot of activity and great things going on there,
especially with libcouchbase.

Thanks.
Mark

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Couchbase" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/couchbase/2pxCMwV6Nz0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> couchbase+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Mark Nunberg]

I guess there would be several ways to serialize a CAS. The naive way would be to transport it as a string (i.e. "1234566"); however I'm personally not a fan of putting things as strings unless absolutely necessary; an alternative would be to serialize it as a two element array of integers (i.e. uint32_t) whose sum will form the uint64_t for the CAS; thus a cas of 7 would be [3,4]. This is a more failsafe way of ensuring the CAS stays opaque while also remaining serializable. We could also enforce hexadecimal representation in the string format, etc. etc (i.e. "0xf00f1e"). Just thoughts.

Or we could end up supporting all of the above. I'll wait for Brett to respond to this and see what he has to say :)

Mark

On Apr 30, 2014, at 7:47 AM, Steven Barlow <stem...@gmail.com> wrote:

> However it is represented in serialised form, there should be a simple way of reconstructing a valid deserialised cas. I'm assuming that there is some hidden deeper structure on the cas object prototype that's getting lost. Hopefully if I go digging, I may be able to isolate it, but it's a bit frustrating: this should be pretty elementary behaviour I'd have thought.
>

> --
> You received this message because you are subscribed to the Google Groups "Couchbase" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to couchbase+...@googlegroups.com.