cursor seems to be bound to end of database

[Michael Möllney]

I have a time series scenario:

One thread is pushing in key,record pairs at the end. Another thread is reading key,record pairs from the end or inside the list of pairs.
The key is a 64bit unsigned int coding of the current time.

The write task is done by a dedicated transaction where a (write) cursor is created within an transaction, the cursor is used to insert the new pair, then the cursor is closed and finally the transaction is committed. The cursor is used to put in a key,record pair, that is at the end of the contained data. The key is the highest value that is inserted into the db. (As time moves on ...)

The read task is done with a local (read) cursor inside a method, but still using the automatic transactions.
At first the cursor is created on stack for each read.
The cursor normally is put at the end of the tile series by asking for the last entry: cursor.move_last();
This request is done repeatedly to have a polling for the newest entry....

For the read cursor there is an option to walk "backward" in time to see old entries. This is realized by requesting the key/records by giving a time key and the option HAM_FIND_LT_MATCH in the cursor find call.
This works well.
But if the cursor is used to step forward in time by the same technique: Give a key for a time and ask for the HAM_FIND_GT_MATCH in the cursor find call.
Then the cursor gives the same key as result, but the record of the last entry of the time line.
But it should have provided the next younger entry instead....


Example: The time series is created sequentially by inserting (1,1) (2,2) (3,3) ...

Key, Value
1,1
2,2
3,3
4,4
5,5
6,6

The read cursor is put at the end via cursor.move_last() to be at the end of the series shortly after a new entry is inserted. So the read cursor is at the end while a new entry is put "behind" the end. This seems to be a critical action, because if this is not this constellation the following does not happen!

Read cursor is at position 5.
Write cursor puts in (6,6).
Read cursor is positioned via cursor.move_last().
Now the read cursor is put again at position 6 (ok)
Now the read cursor is asked to find(key,record,HAM_FIND_LT_MATCH) with key = 6. The result is key=5 and record=5 (ok)
Now repeat the step backward in time: find(key,record,HAM_FIND_LT_MATCH) with key=5. The result is key=4 and record = 4 (ok)
Now ask for the step forward in time: find(key,record,HAM_FIND_GT_MATCH)  with key=4. The result is key=4 and record=6 (?????)

All this is NOT happening, if the last entry is put at the end and the read cursor is not at the last position.

Is this a problem with the automatic transaction?
Next I will try to have a manually instanced transaction for the read cursor, too... Maybe this helps?

[Christoph Rupp]

Hi Michael,

thanks for the explanation. I'll try to reproduce it and fix it. That's clearly a bug.

best regards
Christoph

--
You received this message because you are subscribed to the Google Groups "hamsterdb User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hamsterdb-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Christoph Rupp]

Hi Michael,

i tried to reproduce the issue and in my test the problem was a different one: when placing the read cursor at 5/5 (with find(key,record,HAM_FIND_LT_MATCH)) then it actually returned 6/5 instead of 5/5. The problem was that although the cursor was moved correctly, the key structure was not properly updated.

Can you please compare my test case with your code? it looks like mine is a bit different, since i did not run in exactly the same issue as you have:

(the test case starts in line 3123)

https://github.com/cruppstahl/hamsterdb/blob/v2/unittests/cursor2.cpp#L3123

If you have some code that you can share with me then please send it and i'll make sure that the issue is fixed.

Thanks
Christoph

[Michael Möllney]

Hi Christoph,

I tried to boil down my code to the essential to show the problem.
However I already modified the code to work around the problems I had.
When re-implementing the "problem" in a simple example I came to the same result that you had. The key was not updated, but the record was ......

I will need some time to look through my repository to find a version, that shows my described behaviour...

However, the bahaviour that you recognized is not how it should be, should it?
I would expect that after a find the key shows the cursor position, right?

Thanks again for your time a nice feedback

[Christoph Rupp]

Hi Michael,

yes, that behaviour is a bug and i already fixed it.

I'll try to reproduce your issue once more, so don't worry if you don't have code to reproduce it.

best regards
Christoph