org.fusesource.hawtdb.api.PagingException: That page was updated with the 'put' method.

[hbf]

Hi everybody,

I am using the latest 1.2 release of HawtDb and am very happy with it,
in particular, its performance.

However, I have encounter the following exception when running a
transaction with around 8500 putIfAbsent() calls.

org.fusesource.hawtdb.api.PagingException: That page was updated with
the 'put' method.
at
org.fusesource.hawtdb.internal.page.HawtTransaction.get(HawtTransaction.java:
144)
at
org.fusesource.hawtdb.internal.index.BTreeIndex.loadNode(BTreeIndex.java:
231)
at
org.fusesource.hawtdb.internal.index.BTreeNode.getChild(BTreeNode.java:
261)
at
org.fusesource.hawtdb.internal.index.BTreeNode.getLeafNode(BTreeNode.java:
772)
at
org.fusesource.hawtdb.internal.index.BTreeNode.putIfAbsent(BTreeNode.java:
444)
at
org.fusesource.hawtdb.internal.index.BTreeIndex.putIfAbsent(BTreeIndex.java:
93)
at org.myorg.is.geocoding.HawtDbLocationRepository
$3.run(HawtDbLocationRepository.java:221)
at
org.myorg.is.geocoding.HawtDbLocationRepository.write(HawtDbLocationRepository.java:
306)
at
org.myorg.is.geocoding.HawtDbLocationRepository.find(HawtDbLocationRepository.java:
214)
at org.myorg.is.engine.XYZ.getAllLocations(XYZ.java:181)
at org.myorg.is.engine.XYZ.getLocations(XYZ.java:195)
at
org.myorg.is.server.impl.AnalysisServiceImpl.getLocations(AnalysisServiceImpl.java:
169)
at
org.myorg.is.web.server.GwtAnalysisServiceImpl.getLocations(GwtAnalysisServiceImpl.java:
52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:
562)
at
com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:
544)
at
org.myorg.is.web.server.infrastructure.GuiceRemoteServiceServlet.processCall(GuiceRemoteServiceServlet.java:
45)
at
com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:
224)
at
com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:
62)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:
727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:
820)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:
216)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:
141)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:
93)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:
63)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:
122)
at
com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:110)
at org.eclipse.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1187)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
421)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:
119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:
445)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:
225)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:
930)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:
358)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:
183)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:
866)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:
117)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:
245)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:
126)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:
113)
at org.eclipse.jetty.server.Server.handle(Server.java:351)
at
org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:
594)

My DB is configured like this:

final BTreeIndexFactory<String, LocationResponse>
indexFactory = new BTreeIndexFactory<String, LocationResponse>();
indexFactory.setKeyCodec(StringCodec.INSTANCE);

indexFactory.setValueCodec(LocationResponseCodec.INSTANCE);

final TxPageFileFactory factory = new
TxPageFileFactory();
final File file = new File(dataFile);
file.getParentFile().mkdirs();
factory.setFile(file);
factory.open();

TxPageFile pageFile = factory.getTxPageFile();

// Create if necessary
final Transaction tx = pageFile.tx();
if (!tx.allocator().isAllocated(0))
{
logger.info("Creating new locations DB...");
indexFactory.create(tx);
tx.commit();
}

The 8500 putIfAbsent() are performed in a transaction like this:

Transaction tx = pageFile.tx();
Index<String, LocationResponse> index = indexFactory.open(tx);

for (Pair<T, LocationResponse> response : responses)
{

index.putIfAbsent(locationProvider.apply(response.getFirst()),
response.getSecond());
}

try
{
tx.commit();
}
catch (OptimisticUpdateException e)
{
logger.info("Optimistic update exception in thread " +
Thread.currentThread() + "; trying again...");
}

Does the above exception tell anybody anything?

FYI: An "ls -lsah" on the data file shows that it has not been
modified after it has been created initially (64mb).

Many thanks,
Kaspar

[hbf]

On Aug 26, 11:31 pm, hbf <kaspar.fisc...@dreizak.com> wrote:
> FYI: An "ls -lsah" on the data file shows that it has not been
> modified after it has been created initially (64mb).

If that helps: The transaction that fails is not the first one.
Several other write transactions have succeeded before (but nothing
has been written to disk).

Kaspar

[hbf]

I wrote a test that saves 10000 entries in one transaction but could
not reproduce the error with it.

For now, I have switched to saving at most 1000 entries in a
transaction and that seems to work.

I am still interested in learning what the cause might be. Does the
error message tell you anything?

Kaspar

[Hiram Chirino]

Hi, I've not been able to reproduce that error yet. Any chance you
could boil it down to a Junit test case and post that?

--
Regards,
Hiram

Blog: http://hiramchirino.com

Open Source SOA
http://fusesource.com/

[hbf]

I am trying so but apparently, my test is not reproducing what happens
in reality.

Do any of the following things seem strange when you hear them?

i) The database file on disk does not contain any key text (the keys
are Strings in my case) even though the app has been running for an
hour and the last commit (with 5000 entries) took place half an hour
ago. Is this normal? I'd expect some sort of flush() to happen on a
commit.

ii) I only use putIfAbsent() and get(), while the error message says
something about 'put'. (The latter is called internally, I guess...)

I am asking because maybe my configuration could be wrong (while it is
fine in the test).

Thanks,
Kaspar

[Hiram Chirino]

In hawtdb.. commit() is only used to group unit of work. durability
is achieved by a subsequent flush() call. So if you want to guarantee
durability of a commit, follow it up with a flush. The default
behavior if flush is not manually called by the application is to
buffer in memory multiple commits in memory until it builds a big
batch and then flushes to disk async.

On Fri, Aug 27, 2010 at 12:04 PM, hbf <kaspar....@dreizak.com> wrote:
> I am trying so but apparently, my test is not reproducing what happens
> in reality.
>
> Do any of the following things seem strange when you hear them?
>
> i) The database file on disk does not contain any key text (the keys
> are Strings in my case) even though the app has been running for an
> hour and the last commit (with 5000 entries) took place half an hour
> ago. Is this normal? I'd expect some sort of flush() to happen on a
> commit.
>
> ii) I only use putIfAbsent() and get(), while the error message says
> something about 'put'. (The latter is called internally, I guess...)
>

It's an odd error. Right now I'm guessing it's a concurrency bug in
hawtdb. But it's going to hard to chase it down without a
reproducible test case.

> I am asking because maybe my configuration could be wrong (while it is
> fine in the test).
>

Your usage looked fine to me.

[hbf]

Hi Hiram,

I am pretty sure now it's a concurrency problem. I haven't encountered
it anymore until I was starting to play with serveral users.

Let me know if you have any ideas where the problem might lie or how
to create a test case producing it. I will try to look into it
further.

Regards,
Kaspar

On Aug 27, 6:13 pm, Hiram Chirino <hi...@hiramchirino.com> wrote:
> In hawtdb.. commit() is only used to group unit of work.  durability
> is achieved by a subsequent flush() call.  So if you want to guarantee
> durability of a commit, follow it up with a flush.  The default
> behavior if flush is not manually called by the application is to
> buffer in memory multiple commits in memory until it builds a big
> batch and then flushes to disk async.
>