Is it possible to support online backup function (backup/resotre) for in-memory database?

126 views
Skip to first unread message

Nan Shen

unread,
Mar 11, 2010, 10:29:26 PM3/11/10
to Xerial
Hi I am trying to load my database into memory to achieve faster
accessing speed.
Thus I wonder is it possible for you to support online backup
functions like "backup" and "restore"?

Thank you for the great work!

Taro L. Saito

unread,
Mar 12, 2010, 2:34:57 AM3/12/10
to xer...@googlegroups.com
Hi,

I tried to implement backup/restore feature today, and have succeeded in it!
I will release the new version with this feature soon.

--
Taro L. Saito
<l...@xerial.org>
University of Tokyo
http://www.xerial.org/leo
Tel. +81-47-136-3985 (63985)

> --
> You received this message because you are subscribed to the Google Groups "Xerial" group.
> To post to this group, send email to xer...@googlegroups.com.
> To unsubscribe from this group, send email to xerial+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/xerial?hl=en.
>
>

Nan Shen

unread,
Mar 15, 2010, 4:50:52 AM3/15/10
to Xerial
Hi Taro,

Thank you for your super fast response!

I tried to use the new released SNAPSHOT, but restoring a .db file to
an in-memory database seems very slow, compared to doing so using the
sqlite3 command line tool.
The .db file is 40mb. restoring using jdbc takes more than half an
hour. but using the command line tool takes 5 seconds.
Am I doing the right thing?
Connection conn = DriverManager.getConnection("jdbc:sqlite::memory:");
Statement stmt = conn.createStatement();
stmt.executeUpdate("restore from 'C:\testDB.db'");

I have also tried
conn.setAutoCommit(false);
stmt.executeUpdate("restore from 'C:\testDB.db'");
conn.setAutoCommit(true);
The speed is not improving.

Also, can I restore/backup an attached database other than 'main'? I
have tried "restore testDB from ..." but it seems not working.


Thank you!
Best regards,
Nan


On Mar 12, 12:34 pm, "Taro L. Saito" <l...@xerial.org> wrote:
> Hi,
>
> I tried to implement backup/restore feature today, and have succeeded in it!
> I will release the new version with this feature soon.
>
> --
> Taro L. Saito
> <l...@xerial.org>

> University of Tokyohttp://www.xerial.org/leo
> Tel. +81-47-136-3985 (63985)

Taro L. Saito

unread,
Mar 15, 2010, 8:54:55 AM3/15/10
to xerial
Thank you for testing the new version.

To implement this feature, I simply used the sample code described in
the SQLite's web page, which uses only 5 pages for the backup/restore
iterations!
I guess this was the cause of the slow performance.

I will check how the original SQLite implement the backup/restore commands.

--
Taro L. Saito
<l...@xerial.org>
University of Tokyo
http://www.xerial.org/leo
Tel. +81-47-136-3985 (63985)

Taro L. Saito

unread,
Mar 15, 2010, 8:58:30 AM3/15/10
to xerial
Specifying database names to backup/restore is possible by using the
following syntax:

backup (database name)? to (file name)
restore (database name)? from (file name)


--
Taro L. Saito
<l...@xerial.org>
University of Tokyo
http://www.xerial.org/leo
Tel. +81-47-136-3985 (63985)


2010/3/15 leo <l...@xerial.org>:

Taro L. Saito

unread,
Mar 15, 2010, 8:02:59 PM3/15/10
to Xerial
Hi,

I changed the number of pages to be dumped from 5 to 100, which is the
same setting with the command line shell of sqlite.
This change increased the backup/restore performance significantly.

And also, I found the reason why backup/restore (db name)? syntax was
not working.

I will release a new version using sqlite-3.6.23, which was released
recently, soon.

On Mar 15, 9:58 pm, "Taro L. Saito" <l...@xerial.org> wrote:
> Specifying database names to backup/restore is possible by using the
> following syntax:
>
> backup (database name)? to (file name)
> restore (database name)? from (file name)
>
> --
> Taro L. Saito
> <l...@xerial.org>

> University of Tokyohttp://www.xerial.org/leo
> Tel. +81-47-136-3985 (63985)
>

> 2010/3/15 leo <l...@xerial.org>:
>
> > Thank you for testing the new version.
>
> > To implement this feature, I simply used the sample code described in the SQLite's web page, which uses only 5 pages for the backup/restore iterations!
> > I guess this was the cause of the slow performance.
>
> > I will check how the original SQLite implement the backup/restore commands.
>
> > --
> > Taro L. Saito
> > <l...@xerial.org>
> > University of Tokyo
> >http://www.xerial.org/leo
> > Tel. +81-47-136-3985 (63985)
>

Taro L. Saito

unread,
Mar 16, 2010, 1:53:00 AM3/16/10
to Xerial
I built a new version, sqlite-jdbc-3.6.23:
http://www.xerial.org/maven/repository/snapshot/org/xerial/sqlite-jdbc/3.6.23-SNAPSHOT/

This release improves backup/restore performance.

--
Taro L. Saito
<l...@xerial.org>
University of Tokyo
http://www.xerial.org/leo
Tel. +81-47-136-3985 (63985)

Nan Shen

unread,
Mar 28, 2010, 10:32:45 PM3/28/10
to Xerial
Hi Taro,

Thank you for the great work! Now the backup/restore works fast!

Only another little problem..
restore does not work if there is a space in the file path.
e.g. stmt.executeUpdate("restore bse from 'C:/a test/testdb.db'");
will not work.
but stmt.executeUpdate("restore bse from 'C:/atest/testdb.db'"); works
fine.

Could you please help to fix this whenever you have time?

Thanks a lot!
Best regards,
Nan

On Mar 16, 10:53 am, "Taro L. Saito" <l...@xerial.org> wrote:
> I built a new version, sqlite-jdbc-3.6.23:http://www.xerial.org/maven/repository/snapshot/org/xerial/sqlite-jdb...


>
> This release improves backup/restore performance.
>
> --
> Taro L. Saito
> <l...@xerial.org>

Taro L. Saito

unread,
Mar 29, 2010, 12:55:49 AM3/29/10
to xer...@googlegroups.com
Hi,

A quick fix is to use double quotation, instead of single quotation for
the file name.

Another way is download sqlite-jdbc-3.6.23-SNAPSHOT again; I uploaded
a new version, which allows both single and double quotations.

Regards,


--
Taro L. Saito
<l...@xerial.org>
University of Tokyo
http://www.xerial.org/leo
Tel. +81-47-136-3985 (63985)

Austin Chamberlin

unread,
Oct 19, 2011, 9:45:14 PM10/19/11
to xer...@googlegroups.com
I'm unable to get backup to work. I have an in memory db and the line:

stmt.executeUpdate("backup to 'defaultset.db'");

throws:
java.sql.SQLException: near "backup": syntax error
at org.sqlite.NativeDB.throwex(NativeDB.java:153)
at org.sqlite.NativeDB._exec(Native Method)
at org.sqlite.Stmt.executeUpdate(Stmt.java:132)

Taro L. Saito

unread,
Oct 20, 2011, 3:30:49 AM10/20/11
to xer...@googlegroups.com
Hi,

From the line number of the error, it looks like you are using an
older version of sqlite-jdbc,
which has no support for online backup/restore.

Use sqlite-jdbc-3.7.2 or later.

--
Taro L. Saito
<l...@xerial.org>
University of Tokyo
http://www.xerial.org/leo

Tel. +81-47-136-4065 (64065)

> --
> You received this message because you are subscribed to the Google Groups
> "Xerial" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/xerial/-/bWK-sf8epw8J.

Reply all
Reply to author
Forward
0 new messages