[google/syzkaller] 5e933e: all: support coverage of kernel modules

[Joey Jiaojg]

Branch: refs/heads/master
Home: https://github.com/google/syzkaller
Commit: 5e933e8c7b82e170b667885d3b99098e2e86f29d
https://github.com/google/syzkaller/commit/5e933e8c7b82e170b667885d3b99098e2e86f29d
Author: Joey Jiao <joeyj...@gmail.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/backend.go
M pkg/cover/backend/elf.go
M pkg/cover/backend/gvisor.go
M pkg/cover/report.go
M pkg/cover/report_test.go
M pkg/host/machine_info.go
M pkg/host/machine_info_linux.go
M pkg/host/machine_info_linux_test.go
M pkg/mgrconfig/config.go
M pkg/rpctype/rpctype.go
M syz-fuzzer/fuzzer.go
M syz-manager/cover.go
M syz-manager/covfilter.go
M syz-manager/html.go
M syz-manager/manager.go
M syz-manager/rpc.go
M tools/syz-cover/syz-cover.go

Log Message:
-----------
all: support coverage of kernel modules

The PCs returned for dynamic loaded module (DLKM) is not
parsed in coverage page, these PCs are dropped.

The commit is to use DLKM modules' load address and symbol file
to restore the PC and show coverage data of DLKM.

Introduced new config module_obj to specify module directories.

Example of config:
"module_obj": [
"module_path1"
"module_path2"
]

For linux target, before Manager.Connect run, load addresses are
getting from /proc/modules in order to group PCs into modules.
And so, if modules are under kernel_obj or module_obj dir,
their addresses and paths can be generated automatically.
kernel_obj is searched before module_obj dir and the first found
ko object is always used.

Also note that kaslr needs to be disabled.


Commit: a32dae5902fec8911336f5fc7d1e8b2292589fde
https://github.com/google/syzkaller/commit/a32dae5902fec8911336f5fc7d1e8b2292589fde
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/mgrconfig/config.go

Log Message:
-----------
pkg/mgrconfig: extend module_obj comment


Commit: 2649114619f2c83ca9beb0e122445b1820cc1646
https://github.com/google/syzkaller/commit/2649114619f2c83ca9beb0e122445b1820cc1646
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/cover.go
M pkg/cover/html.go
M pkg/cover/report.go
M pkg/cover/report_test.go
M pkg/mgrconfig/config.go

Log Message:
-----------
pkg/mgrconfig: move Subsystem from pkg/cover

mgrconfig used to be "leaf" package that can be imported by anything.
Recently it started importing pkg/cover for Subsystem definition.
It leads to the following import cycle if I try to import
pkg/host from pkg/cover:

package github.com/google/syzkaller/pkg/bisect
imports github.com/google/syzkaller/pkg/build
imports github.com/google/syzkaller/pkg/mgrconfig
imports github.com/google/syzkaller/pkg/cover
imports github.com/google/syzkaller/pkg/host
imports github.com/google/syzkaller/pkg/csource
imports github.com/google/syzkaller/pkg/mgrconfig:
import cycle not allowed

Move Subsystem into pkg/mgrconfig itself.


Commit: 8833b464504c55a0726372a352528f0a6b3d8d24
https://github.com/google/syzkaller/commit/8833b464504c55a0726372a352528f0a6b3d8d24
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/backend.go
M pkg/cover/backend/elf.go
M pkg/cover/backend/gvisor.go
M pkg/cover/report.go
M pkg/cover/report_test.go
M syz-manager/cover.go
M syz-manager/manager.go
M syz-manager/rpc.go
M tools/syz-cover/syz-cover.go

Log Message:
-----------
pkg/cover: pass modules as []host.KernelModule

Pass modules as []host.KernelModule to cover.MakeReportGenerator.
This avoids make(map) in callers that don't pass modules.

Store modules as []*KernelModule.
This avoids clumsy assignments to the map to update Path
and allows to store modules as *KernelModule rather than by name
(we are not scripting, pointer is more flexible and handy representation).


Commit: b2e2064c210dded5b1b226df7806224c7d84d413
https://github.com/google/syzkaller/commit/b2e2064c210dded5b1b226df7806224c7d84d413
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M syz-manager/manager.go
M syz-manager/rpc.go

Log Message:
-----------
syz-manager: refactor fuzzerConnect signature

We try to keep manager.go free of rpc details as much as possible.
fuzzerConnect specifically returned data rather than filled reply.
Pass in only modules and return cover filter bitmap as return value.


Commit: 5c9b22d8987802fb1fc6db94815db77b1d821c73
https://github.com/google/syzkaller/commit/5c9b22d8987802fb1fc6db94815db77b1d821c73
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/backend.go
M pkg/cover/backend/elf.go
M pkg/cover/report.go

Log Message:
-----------
pkg/cover: fix GroupPCsByModule

Both elf.go and gvisor.go are compiled into the same package,
so GroupPCsByModule callback gets installed for gvisor as well.

Move all modules-related logic into backend.
Splitting by modules is the only modules-related part
left in the common code. Move it into backend.
This make Modules field unnecessary in the backend.Impl.

Also move assignment to frame.Module to symbolize,
reduces overall code size.


Commit: 6831d51038f80994e7c47a0550abcab3dab96249
https://github.com/google/syzkaller/commit/6831d51038f80994e7c47a0550abcab3dab96249
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/elf.go

Log Message:
-----------
pkg/cover/backend: remove code duplication in symbolize


Commit: 9c04290738141669bf76cc2d861ffd633c90c73c
https://github.com/google/syzkaller/commit/9c04290738141669bf76cc2d861ffd633c90c73c
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/backend.go
M pkg/cover/backend/elf.go
M pkg/cover/backend/gvisor.go
M pkg/cover/report.go

Log Message:
-----------
pkg/cover/backend: rename KernelModule to Module

We don't have any other modules.
Move definition below Impl, KernelModule is not the most important thing.


Commit: c86aca3f01de03aaadc6ef5006846bbba1e5a154
https://github.com/google/syzkaller/commit/c86aca3f01de03aaadc6ef5006846bbba1e5a154
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/backend.go
M pkg/cover/backend/elf.go
M pkg/cover/report.go

Log Message:
-----------
pkg/cover: use symbols to split PCs by module

We already find symbol for every PC in lazySymbolize.
We can just use that to map PCs to modules instead
of the additional modules sorting/searching.


Commit: 8d4d5838fe0a51500959c0c10f4040a1369043c2
https://github.com/google/syzkaller/commit/8d4d5838fe0a51500959c0c10f4040a1369043c2
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M syz-manager/html.go

Log Message:
-----------
syz-manager: don't hold manager mutex while generting coverage report

Don't hold the mutex while creating report generator and generating the report,
these operations take lots of time.


Commit: a02fc45dd31c0eab033e18387da9245ac5339c5c
https://github.com/google/syzkaller/commit/a02fc45dd31c0eab033e18387da9245ac5339c5c
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/host/machine_info_linux.go

Log Message:
-----------
pkg/host: refactor getModulesInfo

Use strconv.ParseUint instead of Sscanf, it's simpler
for parsing of a single number.


Commit: fdc7af26377d1b46713b8503b73303de29652a97
https://github.com/google/syzkaller/commit/fdc7af26377d1b46713b8503b73303de29652a97
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/elf.go

Log Message:
-----------
pkg/cover/backend: refactor makeELF

1. Reduce scope of variables.
2. Remove unnecessary proxy vars.
3. Don't sort symbols for each module seprately
as we sort the combined array anyway.


Commit: d3c31814bca5812360f650cb04b6e1f426dc52e5
https://github.com/google/syzkaller/commit/d3c31814bca5812360f650cb04b6e1f426dc52e5
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/elf.go

Log Message:
-----------
pkg/cover/backend: stop treating modules[0].Addr specially

As we use symbols to split PCs to modules now,
we can leave modules[0].Addr == 0 and don't treat it specially.
This removes all checks for module.Name == "".


Commit: d791e31f76f8479015ac2a482c9784c2db1caee4
https://github.com/google/syzkaller/commit/d791e31f76f8479015ac2a482c9784c2db1caee4
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/elf.go

Log Message:
-----------
pkg/cover: avoid "else if" after return

Avoid double check for err != nil and else if after return statement.


Commit: 4fd7e7cf6192443813368e9bb984e2b517c5ac17
https://github.com/google/syzkaller/commit/4fd7e7cf6192443813368e9bb984e2b517c5ac17
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/elf.go
A pkg/cover/backend/modules.go

Log Message:
-----------
pkg/cover/backend: move module discovery into separate file

elf.go is already quite large and messy.
Module discovery is a well separatable chunk of logic,
move it to a dedicated file.
No code changes.


Commit: 3d67c6a0759e6360bdcc0d7a8716b428354ca6bd
https://github.com/google/syzkaller/commit/3d67c6a0759e6360bdcc0d7a8716b428354ca6bd
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/backend.go
M pkg/cover/backend/elf.go
M pkg/cover/backend/gvisor.go
M pkg/cover/backend/modules.go
M pkg/cover/report.go

Log Message:
-----------
pkg/cover: refactor module discovery interface

Make module discovery convert host.KernelModule to backend.Module.

Also error if we have modules on non-Linux
and make it possible to return errors from module discovery.


Commit: 2b7d56011214e1f3ac27f4837a113838ec8a78f4
https://github.com/google/syzkaller/commit/2b7d56011214e1f3ac27f4837a113838ec8a78f4
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/modules.go
A pkg/cover/backend/modules_test.go

Log Message:
-----------
pkg/cover/backend: refactor module discovery

Restructure discovery code that it can be tested and add a manual test.
Also remove some code duplication for module creation.


Commit: 2732397851f598a678a441cb641f630b94e18761
https://github.com/google/syzkaller/commit/2732397851f598a678a441cb641f630b94e18761
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/elf.go

Log Message:
-----------
pkg/cover/backend: remove unnecessary sort

We sort all ranges in makeELF, don't sort per-module.


Commit: 4ca2935d663453b098b895ae56a309f3efb6a4b9
https://github.com/google/syzkaller/commit/4ca2935d663453b098b895ae56a309f3efb6a4b9
Author: Dmitry Vyukov <dvy...@google.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/elf.go

Log Message:
-----------
pkg/cover: refactor extraction of coverage points

1. Add module.Addr right in objdump, there is not point in the separate loop.
It only adds unnecessary lines of code.

2. Eliminate excessive module.Name == "" checks.
We already check module.Name == "" in makeELF, there is no point
in the separate check in readCoverPoints. It also does 2 completely
different things, it's a bad way to structure code. Split it into
readCoverPoints and readModuleCoverPoints.

3. Refactor readModuleCoverPoints.
It can be much more compact, one reasonably short function.
Also avoid reading ELF symbols for every relocation,
it's completely unnecessary. We already read symbols in readSymbols,
so we don't need to re-read them again at all.


Commit: c753ca8ef9b9cae42f9f8ebed5a73b47511b2698
https://github.com/google/syzkaller/commit/c753ca8ef9b9cae42f9f8ebed5a73b47511b2698
Author: Joey Jiao <joeyj...@gmail.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/elf.go

Log Message:
-----------
pkg/cover: objDir is not necessary for cleanPath


Commit: 7216542e366bb6d7f750ca7cc7bbd11d9a386c09
https://github.com/google/syzkaller/commit/7216542e366bb6d7f750ca7cc7bbd11d9a386c09
Author: Joey Jiao <joeyj...@gmail.com>
Date: 2021-03-18 (Thu, 18 Mar 2021)

Changed paths:
M pkg/cover/backend/elf.go

Log Message:
-----------
pkg/cover: add back objDir to avoid potential risk


Compare: https://github.com/google/syzkaller/compare/fdb2bb2c23ee...7216542e366b