xShm* methods

[Nikolaus Rath]

Hello,

The APSW documentation at
https://rogerbinns.github.io/apsw/vfs.html#apsw.VFSFile does not mention
any of the xShm* methods listed (but sadly also otherwise undocumented)
at https://sqlite.org/c3ref/io_methods.html.

If I implement my own VFSFile class in Python, will calls to xShm
silently pass through to the base VFS, or will the corresponding
sqlite3_io_methods elements be set to null? If they are not set to null,
is there a way to indicate that my VFSFile class does *not* support them
(so they should not be passed through)?

Does anyone know the exact semantics that these methods should provide?

Thanks!
-Nikolaus

--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

»Time flies like an arrow, fruit flies like a Banana.«

[Roger Binns]

On Fri, 24 Mar 2023, at 12:52, Nikolaus Rath wrote:
> The APSW documentation at
> https://rogerbinns.github.io/apsw/vfs.html#apsw.VFSFile does not mention
> any of the xShm* methods listed (but sadly also otherwise undocumented)
> at https://sqlite.org/c3ref/io_methods.html.

The APSW doc does imply that the xShm methods are dealt with for you. That is done by direct passthrough to the VFSFile being inherited from.

> If I implement my own VFSFile class in Python, will calls to xShm
> silently pass through to the base VFS, or will the corresponding
> sqlite3_io_methods elements be set to null? If they are not set to null,
> is there a way to indicate that my VFSFile class does *not* support them
> (so they should not be passed through)?

The logic is that if you inherit from a VFSFile that does define the xShm methods then the underlying sqlite3_io_methods get passed through. There is no parameter to control that.

Note that you do not have to inherit from an existing VFS which is part of why this wasn't more formally exposed and controllable, but they are required for wal to work.

I didn't try to expose the xShm methods because they are very difficult to deal with from Python code, and would also be a lot slower. You'd almost certainly need to write C code.
Hence the "make wal work" current sweet spot. (The Python mmap module couldn't be used because you can't get a pointer.)

> Does anyone know the exact semantics that these methods should provide?

Roughly speaking they correspond to mmap and munmap system calls, but you have to deal with allocating the backing files, and implement the locking logic (no system call for that).

I hadn't spotted that xFetch/xUnfetch had been added either - no mention in the release notes https://www.sqlite.org/changes.html

I am not convinced it is worth trying to make it possible to write the xShm methods in Python, but am happy for anyone to change my mind.

But improvements could be made - https://github.com/rogerbinns/apsw/issues/418

Roger