LOB errors
Rami
I keep getting quite randomly the following kind of error when reading
BLOBs from database
org.h2.message.DbException: IO Exception: "java.io.EOFException";
"/root/.lobs.db/172.temp.db" [90031-143]
at
org.h2.message.DbException.get(DbException.java:156)
at
org.h2.message.DbException.convertIOException(DbException.java:313)
at
org.h2.store.FileStore.readFully(FileStore.java:287)
at
org.h2.store.FileStoreInputStream.fillBuffer(FileStoreInputStream.java:93)
at
org.h2.store.FileStoreInputStream.readBlock(FileStoreInputStream.java:73)
at
org.h2.store.FileStoreInputStream.read(FileStoreInputStream.java:61)
Behind it comes
Caused by: org.h2.jdbc.JdbcSQLException: IO Exception:
"java.io.EOFException"; "/root/.lobs.db/172.temp.db" [90031-143]
at
org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
... 31
more
Caused by:
java.io.EOFException
at
java.io.RandomAccessFile.readFully(RandomAccessFile.java:383)
at
org.h2.store.FileStore.readFully(FileStore.java:285)
... 28 more
My webserver shows images that are in the database and it seems that the
/root folder mentioned
in the exception point's to the webserver's (that is client's) /root folder.
This happens quite randomly so that when reloading the page, that
contains a list of images,
the image (BLOB) that causes the exception changes in a seemingly random
fashion.
Sometimes the exception is different like this
java.lang.NegativeArraySizeException
at
org.h2.util.Utils.newBytes(Utils.java:362)
at
org.h2.store.Data.expand(Data.java:1119)
at
org.h2.store.Data.checkCapacity(Data.java:1114)
at
org.h2.store.FileStoreInputStream.fillBuffer(FileStoreInputStream.java:100)
at
org.h2.store.FileStoreInputStream.readBlock(FileStoreInputStream.java:73)
at
org.h2.store.FileStoreInputStream.read(FileStoreInputStream.java:61)
Any ideas what this might be about or how to debug it further?
- Rami
Thomas Mueller
I think the problem is that you try to read from the LOB object after
going to the next row in the result set, or after closing the result
set or connection. If this is not the problem, could you try with H2
version 1.2.145? Could you create a reproducible test case?
Regards,
Thomas
Rami
I created a test case where I create 10 threads that are started
approximately the same time and read 10 separate Blobs from a table
(altogether 100 blobs).
The test case is written in Beanshell and uses some of my utility
classes so you are not capable of running it but you can check it out as
pseudocode if you want (attached).
All threads report the same error:
Exception in thread "Thread-1" java.lang.OutOfMemoryError: Requested
memory: 1952885510
at org.h2.util.Utils.newBytes(Utils.java:364)
at org.h2.store.Data.expand(Data.java:1119)
at org.h2.store.Data.checkCapacity(Data.java:1114)
at
org.h2.store.FileStoreInputStream.fillBuffer(FileStoreInputStream.java:100)
at
org.h2.store.FileStoreInputStream.readBlock(FileStoreInputStream.java:73)
at org.h2.store.FileStoreInputStream.read(FileStoreInputStream.java:61)
I am quite convinced that these exceptions happen only when the reads
are concurrent.
When I limit the speed by writing to a slow output then I do not
encounter errors.
I will now try with version 1.2.145 as you suggested but it is a bit
more complicated in my case
because I have to merge separately my own changes to source (which you
did not accept as part of H2...)
- Rami
Rami
Sorry for the cheap tactics ;-P
A simple svn update was all that was needed.
So I ran the test using the head of the trunk (or 147 as it is called at
the moment).
The results were better but not optimal.
Instead of all threads throwing exceptions I got 4 exceptions with the
100 reads.
They were now of the form:
Exception in thread "Thread-4" org.h2.message.DbException: IO Exception:
"java.io.EOFException";
"/opt/dev/+apps/exodus/test/.lobs.db/216.temp.db" [90031-143]
at org.h2.message.DbException.get(DbException.java:156)
at org.h2.message.DbException.convertIOException(DbException.java:313)
at org.h2.store.FileStore.readFully(FileStore.java:287)
at
org.h2.store.FileStoreInputStream.fillBuffer(FileStoreInputStream.java:93)
at
org.h2.store.FileStoreInputStream.readBlock(FileStoreInputStream.java:73)
at org.h2.store.FileStoreInputStream.read(FileStoreInputStream.java:61)
So clearly the local copies of the blobs keep encountering EOFException
when there is a higher read concurrency.
- Rami
Thomas Mueller
For multiple reasons, your test case is not reproducible:
- the create table statements are missing
- the data is missing (insert statements)
- the test is not self-contained
- I can't reproduce the problem
My test case is: http://h2database.com/p.html#7051c1db4cfd824bb7c45e29c4d15ac6
Regards,
Thomas
Rami
CREATE TABLE PUBLIC.IMAGE (
IMAGE_ID IDENTITY PRIMARY KEY,
BYTES BLOB
);
Example URL
jdbc:h2:tcp://localhost:10000/TEST;user=test;password=user
You can download the database from Rapidshare
http://rapidshare.com/files/433495007/TestDb.tar.gz
(The size is 274MB so hopefully you are a RapidPro user :)
It contains 3157 images and the size of them all is 286 274 562 bytes = 273MB
I ran your test case against this table (obviously the server startup and
table creation / population has to be removed)
And I get exceptions like
java.lang.NegativeArraySizeException
at org.h2.util.Utils.newBytes(Utils.java:355)
at org.h2.store.Data.expand(Data.java:1079) &n bsp;
at org.h2.store.Data.checkCapacity(Data.java:1074) & nbsp;
at org.h2.store.FileStoreInputStream.readBlock(FileStoreInputStream.java:73)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
Exception in thread "Thread-6" java.lang.OutOfMemoryError: Requested memory: 987040432
at org.h2.util.Utils.newBytes(Utils.java:357)
at org.h2.store.Data.expand(Data.java:1079)
at org.h2.store.Data.checkCapacity(Data.java:1074)
at org.h2.store.FileStoreInputStream.fillBuffer(FileStoreInputStream.java:100)
at org.h2.store.FileStoreInputStream.readBlock(FileStoreInputStream.java:73)
at org.h2.store.FileStoreInputStream.read(FileStoreInputStream.java:61)
Here one can only wonder why would H2 request 941 megs of memory.
Hope this helps.
- Rami
Thomas Mueller wrote:
Thomas Mueller
I'm sorry, I can't reproduce the problem using your database. Could
you try again using the newest version, and then post the complete
stack trace? I think you didn't post the complete one (but I'm not
sure).
Regards,
Thomas
> --
> You received this message because you are subscribed to the Google Groups
> "H2 Database" group.
> To post to this group, send email to h2-da...@googlegroups.com.
> To unsubscribe from this group, send email to
> h2-database...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/h2-database?hl=en.
>
Rami Ojares
When I built it, it turned out that the jar was 1.3.147 instead of
1.2.147 as before.
So I suppose you changed the head of trunk from 1.2 to 1.3?
And indeed the errors I reported have disappeared with the newest version.
I get them everytime I use the older 1.2.147 but never with 1.3.147.
So even though you could not reproduce the problem you somehow managed
to fix it! ;-)
Thank you.
- Rami
Thomas Mueller
> I updated h2 and there were A LOT of changes.
Well, there always are.
> When I built it, it turned out that the jar was 1.3.147 instead of 1.2.147
> as before.
> So I suppose you changed the head of trunk from 1.2 to 1.3?
This is a small change: Constants.java, change VERSION_MINOR to 2.
Other settings depend on it, so this small change will have a big
effect.
Regards,
Thomas
Rami
But a new one appeared.
I started getting the following error:
java.io.FileNotFoundException: /opt/apache-tomcat-6.0.24/temp/h2.temp.76099c36b7051284.134011.temp.db (Too many open files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:98)
at org.h2.store.fs.FileObjectDisk.<init>(FileObjectDisk.java:25)
at org.h2.store.fs.FileSystemDisk.openFileObject(FileSystemDisk.java:444)
at org.h2.store.FileStore.<init>(FileStore.java:88)
at org.h2.store.FileStore.open(FileStore.java:141)
at org.h2.store.FileStore.open(FileStore.java:109)
at org.h2.engine.SessionRemote.openFile(SessionRemote.java:617)
at org.h2.value.ValueLobDb.initTemp(ValueLobDb.java:514)
at org.h2.value.ValueLobDb.createTempFromStream(ValueLobDb.java:476)
at org.h2.value.ValueLobDb.createTempBlob(ValueLobDb.java:441)
at org.h2.store.LobStorage.createBlob(LobStorage.java:565)
at org.h2.value.Transfer.readValue(Transfer.java:513)
at org.h2.result.ResultRemote.fetchRows(ResultRemote.java:216)
According to my understanding the client is reading a Lob and trying to cache it on the client side.
It can not open the file because OS tells that the limit of open files has been exceeded.
So my quess is that somehow these temp files are not closed properly.
And they stay open.
- rami
Rami
I have been following now the open file situation with command
lsof -c java | grep temp\.db | wc -l
that gives me the amount of open temporary lob files that the client java process keeps open
And it seems that they both increase and decrease so I quess there is no problem.
The problem was too low open file limits on the machine (has happened to me before.. :(
- rami