org.mapdb.StoreDirectAbstract.fileHeaderCheck(StoreDirectAbstract.kt:99) [3.0.0-M5]

[Senhui Lee]

Hi, guys. 

Also today I had upgrade the MapDB version to 3.0.0-M5, But also the problem was appear. Now I need explain the scene, I used the MapDB store byte array data in service. Also my project is WEB and running in Jetty container server. Then the problem explain when restart the Jetty Server, the exception see below:

org.mapdb.DBException$DataCorruption: Header checksum broken. Store was not closed correctly, or is corrupted
        at org.mapdb.StoreDirectAbstract.fileHeaderCheck(StoreDirectAbstract.kt:99)
        at org.mapdb.StoreDirect.<init>(StoreDirect.kt:93)
        at org.mapdb.StoreDirect$Companion.make(StoreDirect.kt:48)
        at org.mapdb.StoreDirect$Companion.make$default(StoreDirect.kt:47)
        at org.mapdb.DBMaker$Maker.make(DBMaker.kt:392)
        at so.sao.analytics.sdk.mapdb.MessageStoreDB.<init>(MessageStoreDB.java:82)
        at so.sao.analytics.sdk.mapdb.MessageStoreDB.createInstance(MessageStoreDB.java:89)
        at io.github.elkan1788.mapdb.ProduerMessageServlet.<clinit>(ProduerMessageServlet.java:49)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
        at java.lang.Class.newInstance(Class.java:442)
        at org.eclipse.jetty.server.handler.ContextHandler$Context.createInstance(ContextHandler.java:2372)
        at org.eclipse.jetty.servlet.ServletContextHandler$Context.createServlet(ServletContextHandler.java:1166)
        at org.eclipse.jetty.servlet.ServletHolder.newInstance(ServletHolder.java:1207)
        at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:588)
        at org.eclipse.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:472)
        at org.eclipse.jetty.servlet.ServletHolder.ensureInstance(ServletHolder.java:767)
        at org.eclipse.jetty.servlet.ServletHolder.prepare(ServletHolder.java:752)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)

I also had add the closedOnJvmShutdown setting, why the file can't not open it again.

This is my code build the mapdb file.

this.db = DBMaker.fileDB(dbFile)
                .fileMmapEnable()
                .closeOnJvmShutdown()
                .make(); // also this one throw the exception.


        this.messages = this.db.hashSet(topicName, Serializer.BYTE_ARRAY).createOrOpen();
        this.topic = topicName;

So how can I do? Help me. Thanks.

[Jan Kotek]

Hi,

 

you need to close file before JVM exits. Use db.close() or DBMaker.closeOnJVMShutdown()

 

And transactions are disabled by default in 3.0. You can enable it with transactionsEnable(). That will protect your store from corruption on unclean shutdown (JVM crash etc)

 

Jan

--
You received this message because you are subscribed to the Google Groups "MapDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Senhui Lee]

Yeah, by now I also turn on the transaction operation. But my scene is :  there if the Linux OS will kill the jetty container will heap is to higher,  in this way I not sure the closeOnJVMShutdown switch can work on this action. So what can I do ?

在 2016年4月24日星期日 UTC+8下午4:56:09，Jan Kotek写道：

...

[Jan Kotek]

If JVM is killed, there is no clean shutdown. So you need to use transactions.

 

Jan

--

[Senhui Lee]

eh, I also turn on the transaction operation. There my code see below:

 this.db = DBMaker.fileDB(dbFile)
                .fileMmapEnable()
                .transactionEnable()
                .closeOnJvmShutdown()
                .make();
        this.incKey = this.db.atomicLong(topicName+"IncKey").createOrOpen();
        this.messages = this.db.hashMap(topicName, Serializer.LONG, Serializer.BYTE_ARRAY).createOrOpen();

So is there need db.comit when very message.put some new data in? 

Is there need some other switch need turn on. 

It's emergency status can you help online, Thanks.

在 2016年4月25日星期一 UTC+8下午12:17:46，Jan Kotek写道：

To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+unsubscribe@googlegroups.com.

[Jan Kotek]

Yes, you need to commit after every change (or commit every minute etc..). If crash happens you will lose changes from last commit.

 

There is a chat room on gitter

https://gitter.im/jankotek/mapdb

 

Jan

To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "MapDB" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+un...@googlegroups.com.

[Jan Kotek]

There is new option `DBMaker.checksumHeaderBypass()` in last M7 snapshot. I also updated snapshot maven repo