Multiple Threads with SQLite Backend

[Michael Smith]

Is there some additional configuration I need to do to allow multiple threads in the seeder to work with a sqlite backend? I get messages like this

u4rt9mds@rsgisvmw10:/$ geocache_seed -c /etc/geocache/geocache.xml -t osm -g gg -z 12,14 -n 2

seeding level 12 [14811/168100]: 0.284413 metatiles/sec (avg since start: 0.284413)sqlite backend failed on set: database is locked

seeded 3 metatiles at 0.121273 tiles/sec

And so I just have to end up seeding with just 1 thread.

-- 

Mike

Michael Smith

Remote Sensing/GIS Center

US Army Corps of Engineers

[thomas bonfort]

Hi,
what kind of storage is the sqlite db stored on (ssd/normal,
filesystem, nfs/local, etc... )?
can you try setting the value of the call to sqlite3_busy_timeout() in
src/cache_sqlite.c to something higher, e.g. 30000 and see if that
changes something?
if so, I'll add an option so that that value can be configured in the
config file.

--
thomas

> --
> You received this message because you are subscribed to the Google Groups
> "mod-geocache" group.
> To post to this group, send email to mod-ge...@googlegroups.com.
> To unsubscribe from this group, send email to
> mod-geocache...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mod-geocache?hl=en.
>

[Michael Smith]

Its an ext4 file system on a VM. The VM storage is NFS.

I upped the sqlite3_busy_timeout() to 30000 but when running the seeder it
still exits.

u4rt9mds@rsgisvmw10:/$ geocache_seed -c /etc/geocache/geocache.xml -t osm
-g gg -z 12,14 -n 2

seeding level 12 [14946/168100]: 0.433788 metatiles/sec (avg since start:
0.433788)sqlite backend failed on set: database is locked
seeded 3 metatiles at 0.127504 tiles/sec
u4rt9mds@rsgisvmw10:/$

--
Mike

Michael Smith
Remote Sensing/GIS Center
US Army Corps of Engineers

[thomas bonfort]

File locking over NFS has never been a successful combination I think,
although I would have thought that the vm layer over that would
mitigate the issue. Do you still have the issue if you remove NFS from
the setup?

--
thomas

[Michael Smith]

Hmm, its just the storage space being made available to the VM box that
uses NFS.

I'll try moving the VM to non-NFS storage and see if that makes any
difference.

Mike

[thomas bonfort]

FYI, from some limited testing, I did not run into parallel seeding
issues with the sqlite backend on a local filesystem, even when using
a high number of seeding threads.

--
thomas