SYSLOG rotation and size limit

[Irina Fedorova]

Hey Folks,

I work in Chrome browser and we want to add a new file for some kiosk logs under /var/log/ and redirect these kiosk SYSLOG to this new file. I found this documentation on how to do it: https://chromium.googlesource.com/chromiumos/docs/+/HEAD/logging.md.

The documentation said that it is possible to do log rotation on a 7 days basis. I'm afraid that we will consume a lot of memory if we save 7 days of logs, and we actually need only 2 days of logs. Is it possible to save only 2 days of logs? Or maybe do a log rotation based on chrome session, like ui.LATEST and ui? IIUC for syslogs chrome sessions are not relevant.

Also, is it possible to limit the file size to a particular number of messages? E.g. save first 10000 messages or even better the last 10000 messages? Or maybe limit in by MB, not by messages? 

Thank you in advance! 

[Christoph Schlosser]

Hi Irina,

What new logs are you planning to store that you expect to run out of disk space if they're kept for 7 days?

> I work in Chrome browser and we want to add a new file for some kiosk logs under /var/log/ and redirect these kiosk SYSLOG to this new file. I found this documentation on how to do it: https://chromium.googlesource.com/chromiumos/docs/+/HEAD/logging.md.

Just to clarify: Are these logs originating from Chromium?
If yes, then beware that Chromium has its own logic for rotating and storing logs.

Only logs emitted through the SYSLOG(SEV) macros are sent to syslog and managed by the page you found.

> The documentation said that it is possible to do log rotation on a 7 days basis. I'm afraid that we will consume a lot of memory if we save 7 days of logs, and we actually need only 2 days of logs. Is it possible to save only 2 days of logs?

Unfortunately currently it isn't possible.

> IIUC for syslogs chrome sessions are not relevant.

Correct.

> Also, is it possible to limit the file size to a particular number of messages? E.g. save first 10000 messages or even better the last 10000 messages? Or maybe limit in by MB, not by messages?

This isn't possible either but IIRC if storage is running low there are mechanisms to clear things up.

+Sarthak and +Daniil to keep me honest. 

--

Christoph

[Daniil Lunev]

>> if storage is running low there are mechanisms to clear things up.

Not for system wide locations, and user bound locations cleanup is indiscriminate (will just blast everything it is allowed to).

But kiosk logs probably should be kiosk user bound - i.e. stored in daemon-store-cache.

Generally speaking, I would not say no if someone wants to implement a granular log control policy mechanism that could be applied to all ChromeOS emitted logs from a centralized place which can be hooked up to low disk space notifications.

Nowadays we have a bunch of low granularity controllers spread all over the place, some in Ash and some in Bash (pun not intended :) ) and it would be nice to just have a single place for all of that.

--Daniil

--

Daniil Lunev

ChromeOS Platform Storage

[Irina Fedorova]

Thank you, Christoph and Daniil for the reply!

>What new logs are you planning to store that you expect to run out of disk space if they're kept for 7 days?

We want to add a policy for admins to collect app-level logs during the kiosk session. So if a third-party kiosk app sends too many logs, I'm afraid that it will take a lot of space on the device.

>Just to clarify: Are these logs originating from Chromium?

Yes, they are from Chromium. But as far as I understand, there is no mechanism to redirect LOG(SEV) to a specific file. Because of this, I plan to use  SYSLOG(SEV).

The documentation links to chromeos-cleanup-logs script, which has the following comment:

# Remove older files, keep deleting until the directory is less than 100MB

# or only one day old files remains. ...and also delete older files if

# there are too many (the number of files to keep can optionally be configured

# by $3).

remove_old_files()

Do we run this script for all files listed in kLogsToRotate, if so, I can probably rely on this mechanism.

Thanks,

Irina Fedorova |

Software Engineer |

irfed...@google.com |

+49(1525)9966184

[Christoph Schlosser]

>>Just to clarify: Are these logs originating from Chromium?
>Yes, they are from Chromium. But as far as I understand, there is no mechanism to redirect LOG(SEV) to a specific file

There is logging::LoggingSettings::log_file_path but it's set for the whole process, not sure how these 3p apps are run.

>Do we run this script for all files listed in kLogsToRotate, if so, I can probably rely on this mechanism.

The script is run for these patterns which don't seem to overlap with kLogsToRotate.

If you can redirect your logs into a file you could add that to the cleanup script.

-- 

Christoph

[Irina Fedorova]

Yeah, if I use logging::LoggingSettings::log_file_path I will change the file for all Chrome logs, which I do not want.

Thanks for pointing out which logs are actually cleaned up with the cleanup script. I was confused that the documentation mentioned kLogsToRotate and the script at the same time, since they are not connected.

Looks like the script can also help with 2 days of rotation instead of the default 7 days. So if our PM decided that we need to change the number of days we save the logs, it will be possible. 

Thanks a lot,

Irina Fedorova |

Software Engineer |

irfed...@google.com |

+49(1525)9966184