why lmkd and logd are created to move log and lmk from kernel to userspace in L?

[Kaisa]

Any benefits ?

[Mark Salyzyn]

ObligatoryDisclaimer: Please do not assign any gravitas to the following, some portions of the following are WAGs, and I will welcome corrections or retractions.

lmkd:

Dual supported in L. For example N9 released with lmk due to its advanced architecture giving an edge to lmk over lmkd. Low memory killing is an important feature on all Android Devices and its tuning and optimization is a requirement to provide the best and balanced user experience. The option of either improves the tuning options of the device manufacturers. Support for lmk scales because of its inclusion upstream in the kernel. Our goal is to improve lmkd as an optimized replacement with algorithms that live well in the android ecosystem.

logd:

There are several security bugs outstanding for reading and clearing of data (closed as obsolete) on the kernel logger, requiring a rework. There will be no new development time spent on the kernel logger to fix or to expand functionality. OEMs can continue, in L, to release using the kernel but with some problems/klunkiness from bitrot. The kernel logger is NOT upstream and will _never_ be taken upstream adding a burden of support that does not scale well. The user space logger can remain unmodified in all releases, easier scale for support.

The source code for liblog contains unit tests that are logger-agnostic. We have confirmed on par or better overall performance for the user space logger, save for latency which was expected given the desire to reduce cpu load.

Features unavailable to the kernel-based logger, and present in the user space logger:
- crash log id, crashes would be dropped on the floor in the kernel space logger, you can fix that by adjusting the kernel logger sources.
- statistical analysis (-S, eng and userdebug builds only, non svelte device)
- smart pruning, prune by UID, PID or by statistically chattiest UID (-p -P)
- flexible pruning, hold data longer while slow logcat interprets the data.
- batch mode, tail from a specific timestamp (-d, -D) * solves CPU utilization problem and then some.
- auditd (log selinux violations)
- Set log size (-g -G) From 64Kb/each up to 16Mb/each (more if your want) and development option to control
- svelte device support (64Kb log size set at runtime)
- deferred batch processing of log data, reduced CPU utilization (reading, logcat notwithstanding, now that processing is done and measurable in batched user space it will look suspicious, but rather than high priority kernel space)
- clear by UID (rather than clearing _all_ logs), credential controlled.
- read by UID, credential controlled.
- Tighter security handling, tighter selinux control over security, NSA defense division oversight.

Future features/enhancements (no promises, currently just thoughts):
- klogd (log kmsg into kernel log id)
- syslogd (syslog relay of all logs in test lab environment, eng and userdebug builds only)
- Discover a means during kernel crash to grab the queued log in the unix domain socket feeding the daemon.
- encryption/compression of in-memory data
- Continued active support and creeping feature-ism 

Memory:
- Kernel logger takes precious contiguous Order 6 (or more) paged memory, locked in core forever, even if unused. Driver code is locked in core. None can be reused/swapped.
- User space logger takes (swappable - zram) flexible user space memory to record the data, and grows and shrinks as needed at runtime. Text space of the logging code can be reclaimed under memory pressure, capable of being read back in from disk from the executable.

Disadvantages of the user space logger:

- specialized OEM development and triage: kernel crash core dump (currently) does not have accessible Android Log messages.

- 10ms (batched cpu priority, 100HZ jiffyclock) or longer latency delay for logs to settle and become accessible for reading. This is an advantage to cpu utilization as noted above, a disadvantage for timeliness. However the timestamps are accurate to 80ns (on kernels that support zero syscall realtime clock), whereas kernel space logger has 1us accuracy.

Sincerely -- Mark Salyzyn

[Kaisa]

Thanks Mark Salyzyn