Issue with th MVStore used standalone

[Igor castang]

Hello,

first thank you all for coding and making H2 what it is : an awesome tool.

I have been playing with the new MVStore a bit and I think I found an issue.

I am trying to insert 15 000 000 records (string,string) into a map and after around 13 000 000 values the MVStore throws an exception (see attach test case and log)
The store file get to around 2.2gb

Am I doing something wrong ?

I am using
H2 1.3.71 / Xmx3000M / java 1.7 / linux


thanks 

[Igor castang]

It seems that it's not possible to attach file so here is the log and after the test case

storing 1000000
count stored : 1000000
stored 1000000
storing 2000000
count stored : 2000000
stored 2000000
storing 3000000
count stored : 3000000
stored 3000000
storing 4000000
count stored : 4000000
stored 4000000
storing 5000000
count stored : 5000000
stored 5000000
storing 6000000
count stored : 6000000
stored 6000000
storing 7000000
count stored : 7000000
stored 7000000
storing 8000000
count stored : 7999999
stored 8000000
storing 9000000
count stored : 9000000
stored 9000000
storing 10000000
count stored : 10000000
stored 10000000
storing 11000000
count stored : 11000000
stored 11000000
storing 12000000
count stored : 12000000
stored 12000000
storing 13000000
Exception in thread "main" java.lang.IllegalArgumentException: Negative position
        at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:688)
        at org.h2.store.fs.FileNio.write(FilePathNio.java:71)
        at org.h2.mvstore.cache.FilePathCache$FileCache.write(FilePathCache.java:112)
        at org.h2.mvstore.DataUtils.writeFully(DataUtils.java:346)
        at org.h2.mvstore.MVStore.store(MVStore.java:930)
        at org.h2.mvstore.MVStore.store(MVStore.java:757)
        at com.orecockpit.SimpleMainForTest.main(SimpleMainForTest.java:47)

Test case :

import java.io.File;
import org.h2.mvstore.MVMap;
import org.h2.mvstore.MVStore;

public class SimpleMainForTest {
    public static void main(String [] args) {
        String storageFile = "/tmp/storeFile";

        File killIt = new File(storageFile);
        killIt.delete();

        MVStore store = new MVStore.Builder().
        cacheSize(800).
        fileName(storageFile).
        writeBufferSize(0).
        writeDelay(-1).
        open();


        MVMap<String,String> map = store.openMap("test");
        int totalWrite = 15000000;
        int lineStored = 0;
        while(lineStored<=totalWrite) {
            lineStored++;
            String actualKey = lineStored+" just for length hhhhhhhhhhhhhhhhh"+lineStored;
            String value = "Just a a string that is kinda longgggggggggggg but not too much AAAAAAAAAAAAAAAAAAAAA hhhhhhhhhhhhhhhhhh"+lineStored;

            if (map.containsKey(actualKey))
                System.out.println("key exists");
           
            map.put(actualKey,value);

            if (lineStored%1000000== 0) {
                System.out.println("storing "+lineStored);
                store.store();
                System.out.println("count stored : "+map.size());
                System.out.println("stored "+lineStored);
            }
        }
        store.store();
        System.out.println("count stored : "+map.size());
        store.close();

[Noel Grandin]

Hi

Thomas, this is a preliminary patch which attempts to solve the problem.
I've also added Igor's test case to the TestMVStore class.

It still has bugs, and I'm not sure if this strategy is OK with you.

Posting it now in case you already have another solution, and because I
won't get a chance to debug it further until tomorrow.

Points to note
(*) you need to run it under a 64-bit VM or Igor's test case will fail
with OutOfMemoryError.
(*) The pageCount field of Chunk confused me for a while, because it's
not the count of pages, it's the number of times the page has been written
(*) MVStore#store(boolean) is confusing me with all of it's different
length values floating around. I can't work out which ones do what.

Regards, Noel Grandin.

[Noel Grandin]

I fixed a couple of issues with my previous patch, but it appears I
still don't understand the rules around allocating/de-allocating chunks,
especially in the presence of rollback, which means that the freelist
still gets out of sync.

[Noel Grandin]

Igor, this issue is fixed in SVN. 

--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.
To post to this group, send email to h2-da...@googlegroups.com.
Visit this group at http://groups.google.com/group/h2-database?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Igor castang]

Thanks a lot.

On Tuesday, March 26, 2013 9:02:31 AM UTC-7, Noel Grandin wrote:

Igor, this issue is fixed in SVN. 

On Sunday, 24 March 2013, Igor castang wrote:

Hello,

first thank you all for coding and making H2 what it is : an awesome tool.

I have been playing with the new MVStore a bit and I think I found an issue.

I am trying to insert 15 000 000 records (string,string) into a map and after around 13 000 000 values the MVStore throws an exception (see attach test case and log)
The store file get to around 2.2gb

Am I doing something wrong ?

I am using
H2 1.3.71 / Xmx3000M / java 1.7 / linux


thanks 

--
You received this message because you are subscribed to the Google Groups "H2 Database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscribe@googlegroups.com.

[Thomas Mueller]

Hi,

I found some more (performance) issues with large files, I'm working on it. Those problems should be resolved soon.

Regards,

Thomas

To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.