WebUI data storage

[quentin....@gmail.com]

Hi everyone,

I would like to know where are stored the data of the WebUI ? I'm creating a docker image of Rspamd but when I remove the container, the webui is reset.

Thanks :)

[Patrick Pichon]

It depends on how you configure it, but by default on 1.6.3 it is on Redis

[quentin....@gmail.com]

Thanks. But i can't find counters values of the status tab (scanned, clean, reject...etc) in my Redis database (I use Redis Desktop Manager to search in redis database). I use AOF method to enable persistence in Redis : https://redis.io/topics/persistence

redis:

    image: redis:3.2-alpine

    container_name: redis

    restart: always

    command: redis-server --appendonly yes

    sysctls:

      - net.core.somaxconn=1024

    volumes:

      - /mnt/docker/redis/db:/data

I think they are not stored in redis but in the filesystem. Can anyone confirm that ?

[Vsevolod Stakhov]

On 21/08/17 20:35, quentin....@gmail.com wrote:
> Thanks. But i can't find counters values of the status tab (scanned,
> clean, reject...etc) in my Redis database (I use Redis Desktop Manager
> to search in redis database). I use AOF method to enable persistence in
> Redis : https://redis.io/topics/persistence
>

> <https://lh3.googleusercontent.com/-EOQz3tilEpI/WZsy8CVZF1I/AAAAAAAAARk/2nVbCRKkXo0dLB6wnYTqfzOfpi55Eh4XACLcBGAs/s1600/Screenshot_20170821_212209.png>

>
>
>
>
>
>
>
> redis:
> image: redis:3.2-alpine
> container_name: redis
> restart: always
> command: redis-server --appendonly yes
> sysctls:
> - net.core.somaxconn=1024
> volumes:
> - /mnt/docker/redis/db:/data
>
> I think they are not stored in redis but in the filesystem. Can anyone
> confirm that ?

Counters are stored in the filesystem. And many other stuff is stored
there. Please use something like overlaying with a persistent DBDIR for
Rspamd (normally /var/lib/rspamd).

[quentin....@gmail.com]

Thank you for your reply.

ls -la /var/lib/rspamd/

total 788

drwxr-x---  2 _rspamd _rspamd   4096 Aug 21 18:41 .

drwxr-xr-x 23 root    root      4096 Aug 21 18:41 ..

-rw-r--r--  1 _rspamd _rspamd 797696 Aug 21 19:49 rspamd.rrd

srw-------  1 root    root         0 Aug 21 18:41 rspamd.sock

I will add this directory in another volume to persist his content, thanks :)

Other files may be in this directory ? other than rspamd.rrd and rspamd.sock ?

Does rspamd destroy the sock file when it stops ? Or I have to do it manually in s6-supervise finish script when i stop the container ? 

https://github.com/hardware/mailserver/blob/b69538979127900650202a72e19f630de738d996/rootfs/etc/s6.d/.s6-svscan/finish

[Alexander Moisseev]

On 8/21/2017 10:57 PM, quentin....@gmail.com wrote:
> Thank you for your reply.
>
> |
> ls -la /var/lib/rspamd/
> total 788
> drwxr-x--- 2 _rspamd _rspamd 4096 Aug 21 18:41 .
> drwxr-xr-x 23 root root 4096 Aug 21 18:41 ..
> -rw-r--r-- 1 _rspamd _rspamd 797696 Aug 21 19:49 rspamd.rrd
> srw------- 1 root root 0 Aug 21 18:41 rspamd.sock
> |
>
> I will add this directory in another volume to persist his content, thanks :)
>
> Other files may be in this directory ? other than rspamd.rrd and rspamd.sock ?
>

There should be stats.ucl in this directory. It is used to keep counters.

Vsevolod, BTW, it seems stats.ucl is never updating between rspamd restarts.

# ll -T /var/db/rspamd/stats.ucl
-rw-r--r-- 1 nobody nobody 196 18 авг. 08:59:28 2017 /var/db/rspamd/stats.ucl

Does it mean we loose some statistics if rspamd terminated unexpectedly?
Probably it worth to update stats.ucl periodically.

> Does rspamd destroy the sock file when it stops ?

No, it doesn't.

[quentin....@gmail.com]

the stats.ucl file is created when ? Because i have this error each time i start the container :

"rspamd[xxx]: controller; rspamd_controller_load_saved_stats: cannot load controller stats from /var/lib/rspamd/stats.ucl: No such file or directory"

https://github.com/vstakhov/rspamd/issues/1693

[quentin....@gmail.com]

ok, stats.ucl file is created after rspamd exiting.

ls -la /var/mail/rspamd/

total 800

drwxr-x--- 2 _rspamd _rspamd   4096 Aug 21 23:15 .

drwxr-xr-x 9 vmail   vmail     4096 Aug 21 22:55 ..

-rw-r--r-- 1 _rspamd _rspamd 797696 Aug 21 23:17 rspamd.rrd

srw------- 1 root    root         0 Aug 21 23:15 rspamd.sock

-rw-r--r-- 1 _rspamd _rspamd    167 Aug 21 23:15 stats.ucl

-rw-r--r-- 1 _rspamd _rspamd   4506 Aug 21 23:15 symbols.cache

I moved DDBDIR to /var/mail volume : https://github.com/hardware/debian-mail-overlay/commit/6ada99695eb004c3d1888e4a125a23399f6f8bd2

Everything is good now, thank you for your explanations :) 

[quentin....@gmail.com]

I can confirm that, stats.ucl is never updating and I still lost counters after a container restart. 

total 796K

drwxr-x---  2 statd netdev 4.0K Aug 24 23:01 .

drwxr-xr-x 12 vmail vmail  4.0K Aug 22 07:54 ..

-rw-r--r--  1 statd netdev 779K Aug 25 00:11 rspamd.rrd     <- last restart

srw-------  1 root  root      0 Aug 24 23:01 rspamd.sock

-rw-r--r--  1 statd netdev  170 Aug 22 20:59 stats.ucl      <- first restart

-rw-r--r--  1 statd netdev  204 Aug 22 20:59 symbols.cache

Le lundi 21 août 2017 22:08:18 UTC+2, Alexander Moisseev a écrit :

Vsevolod, BTW, it seems stats.ucl is never updating between rspamd restarts.