Is there a way to store session in memory?

[Iceberg]

Hi, does anyone know, is there a way to store session in memory? Even
this means sessions will be lost if web2py restarts, but that is fine
for some non-critical apps. Besides, this way we don't need to worry
about cleaning up the old sessions anymore. And perhaps slightly
faster.

I have tried these in models/db.py but failed.

(1)
db=SQLDB('sqlite:memory:') # Actually here we get a new empty db
everytime
session.connect(request,response,db=db) # So sessions in last mem db
can not bring to next click.

(2)
db=cache.ram('sqlmem',lambda:SQLDB('sqlite:memory:'),3600)
session.connect(request, response, db=db)
# It is tricky but doesn't work. Because this way sqlite complains
multi thread accessing one db instance.

(3)
Actually I tried some more complicated code, trying to backup raw
session data in cache.ram, and read them back into a new empty sqlite
memory db. It almost works, but I don't know when I shall do the
backup. The current session data seems not exist yet when the db.py is
executing?

Thanks in advance.

[mdipierro]

I think...

from gluon.contrib.memdb import MEMDB
session.connect(request, response, db=MEMDB(cache.ram))

[Iceberg]

Thanks for the hint. I did not notice MEMDB.

But, too bad that it does not work either.

Traceback (most recent call last):
File "gluon/main.py", line 323, in wsgibase
File "gluon/globals.py", line 313, in _try_store_in_db
File "gluon/contrib/memdb.py", line 250, in insert
File "gluon/contrib/memdb.py", line 285, in _create_id
AttributeError: 'CacheInRam' object has no attribute 'incr'

It seems the memdb.py does not wrap enough interface to simulate a db?

[mdipierro]

It does but it is designed for memcache, not cache.ram. It should be
easy to fix though.

Massimo

[Markus Gritsch]

This thread had the same topic:

http://groups.google.com/group/web2py/browse_thread/thread/b587b14f451da0f8/

and resulted in the same problems using sqlite:memory: :)

Robin B suggested a 'RAMDB':

> No one has published a 'RAMDB' (a DAL driver for RAM). It could
> easily be implemented as a global dict, and it could be useful for say
> caching, but I would not put sessions into ram for production. If you
> have more than 1 application server/process, you will not be able to
> share sessions between them. RAMDB would however be very useful for
> development. :)
>
> Robin

Markus

[mdipierro]

You cannot use sqlite:memory" because that connection is not
persistant. It is closed when the the page is served.

On Apr 6, 2:36 am, Markus Gritsch <m.grit...@gmail.com> wrote:
> This thread had the same topic:
>

> http://groups.google.com/group/web2py/browse_thread/thread/b587b14f45...

>
> and resulted in the same problems using sqlite:memory: :)
>
> Robin B suggested a 'RAMDB':
>
> > No one has published a 'RAMDB' (a DAL driver for RAM).  It could
> > easily be implemented as a global dict, and it could be useful for say
> > caching, but I would not put sessions into ram for production.  If you
> > have more than 1 application server/process, you will not be able to
> > share sessions between them.  RAMDB would however be very useful for
> > development. :)
>
> > Robin
>
> Markus
>

[Iceberg]

Mmm, SQLDB('sqlite:memory:') would be a most intuitive choice, it is a
pity that it does not work. No wonder it was once introduced but soon
"forgotten". :-)

So let's wait Massimo to adjust MEMDB to work with cache.ram, or
perhaps a dedicated RAMDB for it, or even SQLDB('ram://'). I am fine
with whatever naming convension, but slightly prefer the last one. :-)

[mdipierro]

The problem with sqlite:memory is that you can have multiple apps
running under the same web2py installation. There is no way to have
multiple sqlite:memory and those apps do not probably want to share
the database. One would have to cache the sqlite:memory connection
then it may be faster to cache the sessions directly.

Massimo

[Kuba Kucharski]

On Linux you can define(at the boot time, passing kernel params) a
part of your ram as a ramdisk. It behaves exactly like a disk with a
filesystem on it. So you can just use sqlite and have it working in
RAM space. Very fast solution. I know it is not exactly what you are
talking about, but maybe someone would like to know about it. I can
supply additional info if needed.

Regards
--
Kuba

[mdipierro]

An AlterEgo page about this would be very helpful. ;-)

[Kuba Kucharski]

> An AlterEgo page about this would be very helpful. ;-)

"Is there a way" in web2py to place sqlite database somewhere else
than "databases" folder ?
I couldn't quickly find info about it on the web.

As "somewhere else" I mean also a subfolder of "databases" folder.

Than we could have both type of storages in one app.

The problem is I have to mount /dev/ramX ON A FOLDER. I've tested it
with "databases" but this way we could loose a lot of important data
on an unexpected reboot ;)

--
Kuba

[mdipierro]

Yes

SQLDB('sqlite:///absolue/path')

notice the ///

[Kuba Kucharski]

I thought about ramdisk, but tmpfs might be a better solution,
practically it is the same. Main difference between tmpfs and linux
ramdisk device is that tmpfs alocate memory dynamically and also
less-used "pages" can be moved onto swap space.

On modern linux distros it is as easy as:

mount -t tmpfs tmpfs $folder_path -o rw,size=$size

where:
$folder_path is a path to the folder you wish to make "turbocharged"

$size is the amount of memory you want to dedicate( M - megabytes )

for example:

mount -t tmpfs tmpfs /usr/local/web2py/applications/example/cache -o
rw,size=200M
mount -t tmpfs tmpfs /usr/local/web2py/applications/example/sessions
-o rw,size=200M

you can of course do something like that:

mkdir /usr/local/web2py/applications/example/databases/ramdisk/
mount -t tmpfs tmpfs
/usr/local/web2py/applications/example/databases/ramdisk/ -o
rw,size=20M

and then

SQLDB('sqlite:///usr/local/web2py/applications/example/databases/ramdisk/ramstorage.db')

--
Kuba

[Kuba Kucharski]

p.s

it seems like "on modern linux distros" you don't have to pass extra
parameters to kernel anymore (to have ram-type disk). It has been a
while since I used it.

Sory if I scared someone ;)

--
Kuba

[AchipA]

A note, often the OS's disk caching (esp with linux style cache) is
enough speedwise, it is worth going for memory only if you're 100%
sure that the disk is a bottleneck. Otherwise, by dedicating memory to
cache or tmpfs you can actually slow yourself down in a low mem
situation (by forcing apache or python to swap).

[Kuba Kucharski]

Yes, thank you, Achipa. I forgot to warn about it, I will add this to
an AlterEgo entry as soon as I can edit it.

--
Kuba

[Iceberg]

Massimo, thanks for the hint about placing sqlite database in
different folder. However I found it not fully work as expect, because
those "abcdef123456_blah.table" are still created inside app/databases
folder. Could you adjust the code so that all relevant *.table files
are in the same folder as the "sqlite:///absolute/path"?

By the way, hope you also figure out how to store session in
memory. :-)