[website] _content/security: add vulncheck page
Zvonimir Pavlinovic (Gerrit)
Zvonimir Pavlinovic has uploaded this change for review.
_content/security: add vulncheck page
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
---
A _content/security/vulncheck.html
1 file changed, 334 insertions(+), 0 deletions(-)
diff --git a/_content/security/vulncheck.html b/_content/security/vulncheck.html
new file mode 100644
index 0000000..3b4876f
--- /dev/null
+++ b/_content/security/vulncheck.html
@@ -0,0 +1,325 @@
+<!--{
+ "Title": "Vulnerability Checking For Go",
+ "layout": "article"
+}-->
+
+<h2 id="overview">Overview</h2>
+
+<p>
+ One of Go's missions is to help programmers write more secure code. Scanning transitive
+ dependencies of Go programs for known vulnerabilities is a critical aspect of this
+ mission and Go's efforts to enhance software supply-chain security practices. To this end,
+ we introduce a vulnerability detection module for Go called <i>vulncheck</i>.
+</p>
+
+<h2 id="vulncheck">vulncheck</h2>
+
+<p>
+ vulncheck delivers support for detection and understanding of how user programs exercise
+ known vulnerabilities. It can detect packages and modules with known vulnerabilities that
+ are transitively imported by the user program. What makes vulncheck unique is that is also
+ tries to find runtime call stacks witnessing how user code reaches a vulnerability without
+ actually executing the code. This feature brings several benefits.
+</p>
+
+<p>
+ vulncheck is more accurate than standard package-level vulnerability detection: just because
+ a vulnerable <i>symbol</i> is imported, this does does not mean the symbol is actually used.
+ By symbol, we mean a Go function or method. Consider the following illustrative code.
+</p>
+
+<pre>
+package main
+
+import "some/third/party/pkg/P"
+
+func main() {
+ P.G()
+}
+</pre>
+<pre>
+// package some/third/party/pkg/P
+package P
+
+// F has a known vulnerability
+func F() { … }
+
+// G is safe and does not transitively invoke F
+func G() { … }
+</pre>
+
+<p>
+ Package-level vulnerability detection would issue the warning for the above code saying that the
+ package <code>P</code> has some vulnerabilities and that it has been transitively imported by the
+ user package <code>main</code>. This warning is useful, but it does not tell the whole story.
+ Since <code>F</code> never gets (transitively) called, the above user code is not affected by <code>P</code>'s
+ vulnerabilities. Programmers might choose to not address the import of <code>P</code> at all, or
+ postpone the fix until the next release, if they knew that no vulnerabilities of <code>P</code>
+ are in fact exercised. The package-level detection is inherently limited in providing the programmers
+ with such knowledge of vulnerabilities in their code. To drive this point further, let us assume
+ we also have an accompanying test code.
+</p>
+
+<pre>
+package main
+
+import (
+ "testing"
+ "some/third/party/pkg/P"
+)
+
+func TestFoo(t *testing.T) {
+ P.F()
+ …
+}
+
+func TestBar(t *testing.T) {
+ P.F()
+ …
+}
+</pre>
+
+<p>
+ Here, the vulnerable symbol <code>F</code> is indeed used by the code, but only in tests.
+ Programmers might be fine with vulnerabilities being potentially triggered in tests or, say,
+ sandboxed environments. Package and module level detection do not provide the programmers
+ with the level of detail necessary to make such decisions in an informed manner. vulncheck,
+ on the other hand, is designed precisely for that. vulncheck reports call stacks witnessing
+ how the vulnerable symbols are reachable by user code. For the above example, vulncheck
+ communicates <i>[TestFoo, P.F]</i> and <i>[TestBar, P.F]</i> call stacks to programmers.
+ If we exclude above tests, vulncheck does not report any call stacks, which the programmers
+ can interpret as no vulnerabilities are in fact reachable.
+</p>
+
+<p>
+ vulncheck's motivation for reporting call stacks to programmers goes beyond just improving
+ the precision of vulnerability detection. vulncheck's overarching goal is to help programmers
+ understand vulnerabilities, their impact, and their potential remedies. Although the
+ simplest fix often is just to update the corresponding vulnerable package to its healthy
+ version, if any, there are still some very important open questions:
+ <br>
+ <br>
+ <i>
+ Could my systems have been breached and, if so, where does the breach occur?
+ Do I need to escalate the issue?
+ Do I need to alarm my customers?
+ </i>
+ <br>
+ <br>
+ Call stacks reported by vulncheck can help programmers answer those questions as vulnerabilities
+ can be burried deep in the dependency chains at places unfamiliar to the programmers.
+ Package and module level detection on its own is very often not helpful when addressing these
+ questions.
+</p>
+
+
+<h2 id="slices">Vulnerability Slices</h2>
+
+<p>
+ vulncheck operates on both Go source programs and binaries. The source is represented as a slice
+ of <code>*vulncheck.Package</code>, which is a trimmed version of <code>*packages.Package</code>
+ containing only information necessary for vulnerability detection. Trimming is employed as to
+ reduce memory consumption when analyzing large programs.
+</p>
+
+<p>
+ Another input to vulncheck are actual vulnerabilities. These are represented using the shared
+ <a href="https://golang.org/x/vuln/osv">golang.org/x/vuln/osv</a> format. vulncheck accepts
+ a vulnerability database <a href="https://golang.org/x/vuln/client#Client">golang.org/x/vuln/client</a>
+ that is responsible for fetching, remotely or locally, the <code>osv</code> vulnerabilities.
+ Go provides an existing database available at <a href="https://vuln.go.dev">https://vuln.go.dev</a>
+ that can be used with vulncheck.
+</p>
+
+<p>
+ The output of vulncheck are subgraphs of the program call graph, package import graph, and module
+ require graph that lead to vulnerabilities. We refer to such subgraphs as <i>slices</i>. A call graph
+ slice contains only nodes and edges of the original call graph that show how vulnerable symbols are
+ reachable from the program entry points. At the call graph level, entry points are <code>main</code>s,
+ <code>init</code>s, as well as exported functions and methods of user packages. Consider the following
+ illustrative call graph.
+</p>
+
+<pre>
+ A _ B C
+ | \ | |
+ | \ | |
+ D <-- E |
+ | \ /
+ V F
+</pre>
+
+<p>
+ In the above graph, functions <code>A</code>, <code>B</code>, and <code>C</code> are entry points.
+ Function <code>V</code> is vulnerable. We omit package information of each function for brevity.
+ vulncheck will produce the following slice for this call graph.
+</p>
+
+<pre>
+ A _ B
+ | \ |
+ | \ |
+ D <-- E
+ |
+ V
+</pre>
+
+<p>
+ Functions <code>C</code> and <code>F</code> are not in the slice as they do not transitively lead
+ to <code>V</code>. All edges leading to <code>C</code> and <code>F</code> are removed as well. In
+ general, all edges not leading to vulnerable symbols are omitted, as well as nodes appearing
+ exclusively along those edges. The same principles are used to create slices of package imports
+ and module require graphs. The slicing APIs of vulncheck are as follows.
+</p>
+
+<pre>
+func Source(ctx context.Context, pkgs []*Package, cfg *Config) (*Result, err error)
+
+func Binary(ctx context.Context, exe io.ReaderAt, cfg *Config) (*Result, err error)
+</pre>
+
+<p>
+ <code>Config</code> data structure is used to specify the vulnerability database client
+ and the level at which vulncheck should detect vulnerabilities. By default, vulnerabilities
+ are detected at all levels (call graph, package import graph, and module require graph).
+ Context <code>ctx</code> carries deadlines and cancellation signals for downloading vulnerabilities.
+ The computed slices are stored in <code>Result</code>. These can be analyzed for concrete
+ witness traces.
+</p>
+
+<pre>
+func CallStacks(res *Result) map[*Vuln][]CallStack
+
+func ImportChains(res *Result) map[*Vuln][]ImportChain
+</pre>
+
+<p>
+ <code>CallStacks</code> searches <code>Result</code> and finds call stacks
+ for each vulnerability. <code>CallStack</code> is a sequence of stack frames modeled
+ as a call site and the corresponding callee. Vulnerabilities <code>Vuln</code> are
+ vulnerable symbols accompanied with their <code>osv</code> entry. More details on vulncheck data
+ structures can be found <a href="https://pkg.go.dev/golang.org/x/vuln/vulncheck">here</a>.
+ For each vulnerability, only represenative call stacks are reported. To avoid exponential
+ explosion, <code>CallStacks</code> never visits a call slice node more than once. The
+ returned stacks are heuristically ordered by how easy is to understand them:
+ shorter call stacks with less dynamic call sites appear earlier in the returned list.
+ For the call graph slice showed earlier, there are two stacks reported for <code>V</code>.
+</p>
+
+<pre>
+ A B
+ | |
+ D E
+ | |
+ V D
+ |
+ V
+</pre>
+
+<p>
+ Note that the call stack <code>[A, E, D, V]</code> is not reported since a shorter
+ reported stack <code>[A, D, V]</code> already goes through <code>D</code>. The same
+ principles are used to report representative package import chains.
+</p>
+
+
+<h2 id="algorithm">Under the Hood</h2>
+
+<p>
+ One of the main technical challenges in vulncheck is to statically compute call graph
+ information of a Go program. As this is an undecidable problem, we can only hope for an
+ approximate solution. More precise call graph algorithm will require more execution time.
+ On the other hand, a really fast algorithm could easily be very imprecise, either missing
+ call stacks or often reporting infeasible ones. vulncheck strikes this balance between
+ precision and volume of computational resources with
+ <a href="https://dl.acm.org/doi/pdf/10.1145/354222.353189">Variable Type Analysis</a> (VTA) algorithm.
+</p>
+
+<p>
+ VTA is an over-approximate call graph algorithm. VTA does not miss a call stack feasible
+ in practice (see Limitations section for exceptions to this), but it might sometimes
+ report a call stack leading to a vulnerability that cannot be exercised in practice. Our
+ experiments suggests this does not happen too often. Consider the following illustrative program.
+</p>
+
+<pre>
+package main
+
+import os
+
+type I interface {
+ Foo()
+}
+
+// A implements I
+type A struct { ... }
+
+// B implements I
+type B struct { ... }
+
+func main() {
+ b := B{...}
+ if os.Args[1] == "vulncheck" {
+ b.Foo()
+ a := A{...}
+ Bar(a)
+ }
+}
+
+func Bar(i I) {
+ i.Foo()
+}
+</pre>
+
+<p>
+ Existing algorithms, such as <a href="https://pkg.go.dev/golang.org/x/tools/go/callgraph/cha">CHA</a>
+ or <a href="https://pkg.go.dev/golang.org/x/tools/go/callgraph/rta">RTA</a>, would say that
+ <code>i.Foo()</code> call in <code>Bar</code> resolves to <code>A.Foo</code> and <code>B.Foo</code>
+ because types <code>A</code> and <code>B</code> implement interface <code>I</code> and are used in
+ the program. VTA, on the other hand, resolves <code>i.Foo</code> to only <code>A.Foo</code>.
+</p>
+
+<p>
+ VTA works on an abstract representation of a program where variables are represented
+ by their types. The types are then propagated around the program based on variable usage. In the
+ above example, variable <code>a</code> is abstracted via type <code>A</code> which is then propagated
+ to parameter <code>i</code> of <code>Bar</code>. The values actually stored to the variable are not
+ taken into account, only their types. This can lead to imprecision when types reaching an interface
+ variable depend on valuation of, say, involved conditional statements or complicated aliasing. However,
+ types of concrete variables are always the same, regardles of the complexity of the surrounding logic.
+ For instance, values reaching a variable of declared type <code>A</code> always have precisely the
+ type <code>A</code>. This rather unique property of Go's type system enables VTA to produce precise
+ call graph information.
+</p>
+
+<p>
+ In the above example, VTA propagates type <code>A</code> from <code>main</code> to <code>Bar</code>
+ because the call <code>Bar(a)</code> is static. VTA knows what function the identifier <code>Bar</code>
+ resolves to. But what if that call was dynamic? After all, VTA is supposed to construct the call graph
+ so how can it then propagate types across function boundaries? One solution is to rely on a fix-point
+ where the results of type propagation are also used to establish function call edges on which type
+ propagation then needs to be repeated, and so on. This could be very expensive, so VTA relies on an
+ initial approximation of call graph to scale. Note that the initial call graph is only used to propagate
+ types over function calls. We choose CHA as the initial call graph. As CHA can be rather imprecise, as
+ shown on the above example, it could cause VTA to be overly imprecise as well. To counter that, vulncheck
+ bootstraps VTA by VTA. After computing VTA on top of CHA, we feed the more precise resulting call graph
+ to VTA again, toning down excesive imprecision initially introduced by CHA.
+</p>
+
+<p>
+ VTA for Go is also available as an open source <a href="https://pkg.go.dev/golang.org/x/tools/go/callgraph/vta">package</a>.
+</p>
+
+<h2 id="limitations">Limitations</h2>
+
+<p>
+ vulncheck can produce call stacks that are not realizible in practice as call graphs produced
+ by VTA are over-approximate. We also note that VTA might miss some call stacks that flow through
+ <i>unsafe</i> and <i>reflect</i> packages.
+</p>
+
+<p>
+ <code>Result</code> of <code>Binary</code> does not contain call graph slice and consequently
+ will yield no call stack witnesses. This limitation exists due to the difficulty of reverse
+ engineering call information from binaries.
+</p>
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Zvonimir Pavlinovic uploaded patch set #2 to this change.
_content/security: add vulncheck page
Images showing the initial state of the page:
https://screenshot.googleplex.com/Boo3JDxkd7dtT68.png
https://screenshot.googleplex.com/4HKEYCmHFDLDjYJ.png
https://screenshot.googleplex.com/9g44QZqX2k5758F.png
https://screenshot.googleplex.com/7bWWVCGDHejkW67.png
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
---
A _content/security/vulncheck.html
1 file changed, 341 insertions(+), 0 deletions(-)
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Patch set 2:Run-TryBot +1
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker.
1 comment:
Patchset:
Note to the reviewers. This might take several rounds of reviewing as this CL is essentially a document. Let me know if there is a different way you'd like to review this.
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Jonathan Amsterdam (Gerrit)
Attention is currently required from: Julie Qiu, Roland Shoemaker, Zvonimir Pavlinovic.
33 comments:
Patchset:
This is awesome.
File _content/security/vulncheck.html:
Patch Set #2, Line 12:
modulepackage?
It depends on what exactly you want to say here.
The package is named vulncheck and its import path is golang.org/x/vuln/vulncheck.
The module is golang.org/x/vuln.
If you just want to refer loosely to the collection of packages and tools we have, then that's fine, but avoid the words "package" and "module" here.it
Patch Set #2, Line 21:
witnessingdemonstrating
add:
(function or method)Patch Set #2, Line 28:
By symbol, we mean a Go function or method.remove
a
remove
new paragraph here
Patch Set #2, Line 59:
drivedrive home
remove
Patch Set #2, Line 87:
witnessingditto
Patch Set #2, Line 96:
vulncheck'sIts
<i>
Could my systems have been breached and, if so, where does the breach occur?
Do I need to escalate the issue?
Do I need to alarm my customers?
</i>
maybe <ul>?
, because
Patch Set #2, Line 110:
the dependency chains at places unfamiliar to the programmers.unfamilar places in the code
Patch Set #2, Line 110:
can be burried deep in the dependency chains at places unfamiliar to the programmers.buried
The source is represented as a slice
of <code>*vulncheck.Package</code>, which is a trimmed version of <code>*packages.Package</code>
containing only information necessary for vulnerability detection. Trimming is employed as to
reduce memory consumption when analyzing large programs.
We're starting to get very low level here. This implementation detail probably doesn't belong here at all; I would put it in the package doc for the vulncheck package, and even at the bottom of that.
Perhaps the next topic in this article is how to use vulncheck, and for that maybe you want to talk about the govulncheck tool. But if you want to talk about the package, start with a short sample program.
Patch Set #2, Line 136:
slicesThat is confusing, since that term already means something else in Go.
Patch Set #2, Line 172:
The slicing APIs of vulncheck are as follows.You don't really get call graphs with binaries so it's a bit misleading to put Binary here. I would frame this whole discussion in terms of source code, show the Source function, then say there's also a Binary function that is similar, but different in that ...
But frankly, a lot of this reads like package documentation and not a blog post or general article. Not the general idea of pruning the call graph, which should stay here, but anything that mentions specific vulncheck types or functions.
See for example the doc for encoding/gob, which is similar in that it talks about the API but also goes into detail on the implementation.
I won't repeat myself, but the same goes for the content below. I'll flip the bit from here on and only comment on stuff I think should stay in this doc.
Patch Set #2, Line 202:
represenativerepresentative
Patch Set #2, Line 226:
<h2 id="algorithm">Under the Hood</h2>This section could stay here.
Patch Set #2, Line 231:
algorithmalgorithms
the
with the
Patch Set #2, Line 239:
feasibleI don't know what this means here.
Patch Set #2, Line 262:
if os.Args[1] == "vulncheck" {Does this affect either VTA or CHA or RTA? If not, omit.
Patch Set #2, Line 289:
regardlesregardless
the
Patch Set #2, Line 306:
excesiveexcessive
Patch Set #2, Line 316:
realiziblerealizable
Patch Set #2, Line 317:
over-approximateTry to avoid this technical term.
<code>Result</code> of <code>Binary</code> does not contain call graph slice and consequently
will yield no call stack witnesses. This limitation exists due to the difficulty of reverse
engineering call information from binaries.
I would re-order this: "Because binaries do not contain detailed call information, vulncheck cannot ..."
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Julie Qiu, Roland Shoemaker, Zvonimir Pavlinovic.
Zvonimir Pavlinovic uploaded patch set #3 to this change.
The following approvals got outdated and were removed: Run-TryBot+1 by Zvonimir Pavlinovic, TryBot-Result+1 by Gopher Robot
_content/security: add vulncheck page
Images showing the initial state of the page:
https://screenshot.googleplex.com/Boo3JDxkd7dtT68.png
https://screenshot.googleplex.com/4HKEYCmHFDLDjYJ.png
https://screenshot.googleplex.com/9g44QZqX2k5758F.png
https://screenshot.googleplex.com/7bWWVCGDHejkW67.png
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
---
A _content/security/vulncheck.html
1 file changed, 370 insertions(+), 0 deletions(-)
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Julie Qiu, Roland Shoemaker, Zvonimir Pavlinovic.
Zvonimir Pavlinovic uploaded patch set #4 to this change.
_content/security: add vulncheck page
Images showing the initial state of the page:
https://screenshot.googleplex.com/Boo3JDxkd7dtT68.png
https://screenshot.googleplex.com/4HKEYCmHFDLDjYJ.png
https://screenshot.googleplex.com/9g44QZqX2k5758F.png
https://screenshot.googleplex.com/7bWWVCGDHejkW67.png
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
---
A _content/security/vulncheck.html
1 file changed, 369 insertions(+), 0 deletions(-)
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Julie Qiu, Roland Shoemaker, Zvonimir Pavlinovic.
Zvonimir Pavlinovic uploaded patch set #5 to this change.
_content/security: add vulncheck page
Images showing the initial state of the page:
https://screenshot.googleplex.com/Boo3JDxkd7dtT68.png
https://screenshot.googleplex.com/4HKEYCmHFDLDjYJ.png
https://screenshot.googleplex.com/9g44QZqX2k5758F.png
https://screenshot.googleplex.com/7bWWVCGDHejkW67.png
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
---
A _content/security/vulncheck.html
1 file changed, 368 insertions(+), 0 deletions(-)
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker.
32 comments:
File _content/security/vulncheck.html:
Patch Set #2, Line 12:
module
package? […]
PTAL
I am using package here as vulncheck is technically that.
Done
Patch Set #2, Line 21:
witnessingdemonstrating
Done
add: […]
Done
Patch Set #2, Line 28:
By symbol, we mean a Go function or method.remove
Done
Done
remove
Done
new paragraph here
Done
Patch Set #2, Line 59:
drivedrive home
Done
remove
Done
Patch Set #2, Line 87:
witnessingditto
Done
Patch Set #2, Line 96:
vulncheck'sIts
Done
<i>
Could my systems have been breached and, if so, where does the breach occur?
Do I need to escalate the issue?
Do I need to alarm my customers?
</i>maybe <ul>?
Done
, because
Done
Patch Set #2, Line 110:
the dependency chains at places unfamiliar to the programmers.unfamilar places in the code
Done
Patch Set #2, Line 110:
can be burried deep in the dependency chains at places unfamiliar to the programmers.buried
Done
The source is represented as a slice
of <code>*vulncheck.Package</code>, which is a trimmed version of <code>*packages.Package</code>
containing only information necessary for vulnerability detection. Trimming is employed as to
reduce memory consumption when analyzing large programs.
We're starting to get very low level here. […]
PTAL
We could add a section for govulncheck (I could leave a TODO for now).
Patch Set #2, Line 136:
slicesThat is confusing, since that term already means something else in Go.
PTAL
Changed to vulnerability graphs.
Patch Set #2, Line 172:
The slicing APIs of vulncheck are as follows.
You don't really get call graphs with binaries so it's a bit misleading to put Binary here. […]
PTAL
Patch Set #2, Line 202:
represenativerepresentative
Done
Patch Set #2, Line 226:
<h2 id="algorithm">Under the Hood</h2>This section could stay here.
Ack
Patch Set #2, Line 231:
algorithmalgorithms
Done
the
Done
with the
Done
Patch Set #2, Line 239:
feasibleI don't know what this means here.
PTAL
I also changed the use of word infeasible in the paragraph above.
Patch Set #2, Line 262:
if os.Args[1] == "vulncheck" {Does this affect either VTA or CHA or RTA? If not, omit.
Good point, no need to complicate this.
Patch Set #2, Line 289:
regardlesregardless
Done
Done
Patch Set #2, Line 306:
excesiveexcessive
Done
Patch Set #2, Line 316:
realiziblerealizable
Done
Patch Set #2, Line 317:
over-approximateTry to avoid this technical term.
PTAL
There is a use of this term earlier, but there I (kind of) gave its definition.
<code>Result</code> of <code>Binary</code> does not contain call graph slice and consequently
will yield no call stack witnesses. This limitation exists due to the difficulty of reverse
engineering call information from binaries.
I would re-order this: "Because binaries do not contain detailed call information, vulncheck cannot […]
Done
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Julie Qiu (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Roland Shoemaker, Zvonimir Pavlinovic.
16 comments:
File _content/security/vulncheck.html:
Patch Set #5, Line 2:
CheckingDetection
One of Go's missions is to help programmers write more secure code. Scanning transitive
dependencies of Go programs for known vulnerabilities is a critical aspect of this
mission and Go's efforts to enhance software supply-chain security practices. To this end,
we introduce a vulnerability detection package
<a href="https://pkg.go.dev/golang.org/x/vuln/vulncheck">golang.org/x/vuln/vulncheck</a> for Go.Instead of framing this around Go's missions, I would frame it around what we are enabling the user to do. Maybe something like:
> Writing secure and reliable software requires knowing about vulnerabilities in your dependencies. This page provides an overview of the Go vulnerability detection package, <a href="https://pkg.go.dev/golang.org/x/vuln/vulncheck">golang.org/x/vuln/vulncheck</a>, which enables Go developers to scan dependencies in their Go projects for public vulnerabilities.This API is also available as a CLI tool, <a href="https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck">govulncheck</a>.
Patch Set #5, Line 16:
vulncheckmaybe "Package vulncheck" since this is discussing the API as opposed to the CLI
Patch Set #5, Line 19:
vulncheckditto re: "Package vulncheck"
Patch Set #5, Line 22:
runtime call stacksdefine call stack somewhere
maybe worth adding an h3 like "Vulnerabilities at the package and function levels" to break it up a bit?
Patch Set #5, Line 56:
(transitively)I don't think that you need the word transitively here
maybe worth adding an h3 heading like "Vulnerabilities in test code" for this section?
use <code> tags
use <code> tags
Could my systems have been breached and, if so, where does the breach occur?
Do I need to escalate the issue?
Do I need to alarm my customers?
put these in "<li>" tags
Patch Set #5, Line 115:
slicesnit: "vulnerability-graphs" (typically the id and the heading match across go.dev)
nit: I think this should be a colon instead of a period
Patch Set #5, Line 123:
illustrativenit: I don't think you need the word illustrative here
<p>
Clients of vulncheck can present the vulnerability graphs, such as the one above, to the programmers
as a way of showing how vulnerabilities are reachable in their code. However, vulnerability graphs can
get large for big projects, which would make it hard for programmers to manually inspect vulnerabilities.
In response, vulncheck also provides <code>CallStacks</code> functionality for extracting call stacks
from vulnerability call graphs.
</p>
<p>
For each pair of a vulnerable symbol and an entry point, <code>CallStacks</code> traverses the vulnerability
graph searching for call stacks starting at the entry point and ending with a call to the vulnerable symbol.
To avoid exponential explosion, each node is visited at most once. The extracted stacks for a particular
vulnerability are heuristically ordered by how easy is to understand them: shorter call stacks with less
dynamic call sites appear earlier in the extracted results. For the vulnerability call graph shown earlier,
there are two stacks reported for <code>V</code>.
</p>
nit: wrap these lines
Patch Set #5, Line 338:
VTA for Go is also available as an open source <a href="https://pkg.go.dev/golang.org/x/tools/go/callgraph/vta">package</a>.maybe:
Package VTA can be found at <a href="https://pkg.go.dev/golang.org/x/tools/go/callgraph/vta">golang.org/x/tools/go/callgraph/vta</a>
so that it is clear what package the word "package" is linking to?
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Jonathan Amsterdam (Gerrit)
Attention is currently required from: Roland Shoemaker, Zvonimir Pavlinovic.
Patch set 5:Code-Review +1
3 comments:
Patchset:
My comments addressed, will let Julie finish.
File _content/security/vulncheck.html:
Patch Set #2, Line 12:
module
PTAL […]
Ack
File _content/security/vulncheck.html:
remove apostrophe
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Roland Shoemaker, Zvonimir Pavlinovic.
Zvonimir Pavlinovic uploaded patch set #6 to this change.
_content/security: add vulncheck page
Images showing the initial state of the page:
https://screenshot.googleplex.com/Boo3JDxkd7dtT68.png
https://screenshot.googleplex.com/4HKEYCmHFDLDjYJ.png
https://screenshot.googleplex.com/9g44QZqX2k5758F.png
https://screenshot.googleplex.com/7bWWVCGDHejkW67.png
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
---
A _content/security/vulncheck.html
1 file changed, 382 insertions(+), 0 deletions(-)
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker.
Patch set 6:Run-TryBot +1
23 comments:
Patchset:
PTAL
Made few simple fixes and changes, as well as added few more headings to VTA section.
File _content/security/vulncheck.html:
The source is represented as a slice
of <code>*vulncheck.Package</code>, which is a trimmed version of <code>*packages.Package</code>
containing only information necessary for vulnerability detection. Trimming is employed as to
reduce memory consumption when analyzing large programs.
PTAL […]
Done
Patch Set #2, Line 136:
slicesPTAL […]
Done
Patch Set #2, Line 172:
The slicing APIs of vulncheck are as follows.PTAL
Done
Patch Set #2, Line 239:
feasible
PTAL […]
Done
Patch Set #2, Line 317:
over-approximate
PTAL […]
Done
File _content/security/vulncheck.html:
Patch Set #5, Line 2:
CheckingDetection
Done
One of Go's missions is to help programmers write more secure code. Scanning transitive
dependencies of Go programs for known vulnerabilities is a critical aspect of this
mission and Go's efforts to enhance software supply-chain security practices. To this end,
we introduce a vulnerability detection package
<a href="https://pkg.go.dev/golang.org/x/vuln/vulncheck">golang.org/x/vuln/vulncheck</a> for Go.
Instead of framing this around Go's missions, I would frame it around what we are enabling the user […]
Done
Patch Set #5, Line 16:
vulncheckmaybe "Package vulncheck" since this is discussing the API as opposed to the CLI
Done
Patch Set #5, Line 19:
vulncheckditto re: "Package vulncheck"
Done
Patch Set #5, Line 22:
runtime call stacksdefine call stack somewhere
PTAL
maybe worth adding an h3 like "Vulnerabilities at the package and function levels" to break it up a […]
Done
Patch Set #5, Line 56:
(transitively)I don't think that you need the word transitively here
Done
maybe worth adding an h3 heading like "Vulnerabilities in test code" for this section?
I used the name for the heading that reflects the main point of the subsection. I don't want to make it about tests, I believe there is larger message to be conveyed here.
use <code> tags
Done
use <code> tags
Done
remove apostrophe
Done
Could my systems have been breached and, if so, where does the breach occur?
Do I need to escalate the issue?
Do I need to alarm my customers?put these in "<li>" tags
Done
Patch Set #5, Line 115:
slices
nit: "vulnerability-graphs" (typically the id and the heading match across go. […]
Done
Patch Set #5, Line 123:
illustrativenit: I don't think you need the word illustrative here
Done
nit: I think this should be a colon instead of a period
Done
<p>
Clients of vulncheck can present the vulnerability graphs, such as the one above, to the programmers
as a way of showing how vulnerabilities are reachable in their code. However, vulnerability graphs can
get large for big projects, which would make it hard for programmers to manually inspect vulnerabilities.
In response, vulncheck also provides <code>CallStacks</code> functionality for extracting call stacks
from vulnerability call graphs.
</p>
<p>
For each pair of a vulnerable symbol and an entry point, <code>CallStacks</code> traverses the vulnerability
graph searching for call stacks starting at the entry point and ending with a call to the vulnerable symbol.
To avoid exponential explosion, each node is visited at most once. The extracted stacks for a particular
vulnerability are heuristically ordered by how easy is to understand them: shorter call stacks with less
dynamic call sites appear earlier in the extracted results. For the vulnerability call graph shown earlier,
there are two stacks reported for <code>V</code>.
</p>nit: wrap these lines
They are currently wrapped with <p>. Perhaps you meant adding a heading?
PTAL
Patch Set #5, Line 338:
VTA for Go is also available as an open source <a href="https://pkg.go.dev/golang.org/x/tools/go/callgraph/vta">package</a>.
maybe: […]
Done
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker, Zvonimir Pavlinovic.
Zvonimir Pavlinovic uploaded patch set #7 to this change.
The following approvals got outdated and were removed: Run-TryBot+1 by Zvonimir Pavlinovic, TryBot-Result+1 by Gopher Robot
_content/security: add vulncheck page
Images showing the initial state of the page:
https://screenshot.googleplex.com/Boo3JDxkd7dtT68.png
https://screenshot.googleplex.com/4HKEYCmHFDLDjYJ.png
https://screenshot.googleplex.com/9g44QZqX2k5758F.png
https://screenshot.googleplex.com/7bWWVCGDHejkW67.png
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
---
A _content/security/vulncheck.html
1 file changed, 382 insertions(+), 0 deletions(-)
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker.
Patch set 7:Run-TryBot +1
Julie Qiu (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker, Zvonimir Pavlinovic.
Patch set 7:Code-Review +2
3 comments:
File _content/security/vulncheck.html:
Patch Set #5, Line 22:
runtime call stacksPTAL
Done
I used the name for the heading that reflects the main point of the subsection. […]
Done
<p>
Clients of vulncheck can present the vulnerability graphs, such as the one above, to the programmers
as a way of showing how vulnerabilities are reachable in their code. However, vulnerability graphs can
get large for big projects, which would make it hard for programmers to manually inspect vulnerabilities.
In response, vulncheck also provides <code>CallStacks</code> functionality for extracting call stacks
from vulnerability call graphs.
</p>
<p>
For each pair of a vulnerable symbol and an entry point, <code>CallStacks</code> traverses the vulnerability
graph searching for call stacks starting at the entry point and ending with a call to the vulnerable symbol.
To avoid exponential explosion, each node is visited at most once. The extracted stacks for a particular
vulnerability are heuristically ordered by how easy is to understand them: shorter call stacks with less
dynamic call sites appear earlier in the extracted results. For the vulnerability call graph shown earlier,
there are two stacks reported for <code>V</code>.
</p>
They are currently wrapped with <p>. Perhaps you meant adding a heading? […]
sorry I meant so that the lines don't run over (usually there is an editor plugin that will do this for you), but you can also use something like https://github.com/yosssi/gohtml
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker, Zvonimir Pavlinovic.
Zvonimir Pavlinovic uploaded patch set #8 to this change.
The following approvals got outdated and were removed: Run-TryBot+1 by Zvonimir Pavlinovic, TryBot-Result+1 by Gopher Robot
_content/security: add vulncheck page
Images showing the initial state of the page:
https://screenshot.googleplex.com/Boo3JDxkd7dtT68.png
https://screenshot.googleplex.com/4HKEYCmHFDLDjYJ.png
https://screenshot.googleplex.com/9g44QZqX2k5758F.png
https://screenshot.googleplex.com/7bWWVCGDHejkW67.png
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
---
A _content/security/vulncheck.html
1 file changed, 383 insertions(+), 0 deletions(-)
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker, Zvonimir Pavlinovic.
Zvonimir Pavlinovic uploaded patch set #9 to this change.
_content/security: add vulncheck page
Images showing the initial state of the page:
https://screenshot.googleplex.com/Boo3JDxkd7dtT68.png
https://screenshot.googleplex.com/4HKEYCmHFDLDjYJ.png
https://screenshot.googleplex.com/9g44QZqX2k5758F.png
https://screenshot.googleplex.com/7bWWVCGDHejkW67.png
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
---
A _content/security/vulncheck.html
1 file changed, 383 insertions(+), 0 deletions(-)
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker.
Patch set 9:Run-TryBot +1
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker, Zvonimir Pavlinovic.
Zvonimir Pavlinovic uploaded patch set #10 to this change.
The following approvals got outdated and were removed: Run-TryBot+1 by Zvonimir Pavlinovic
_content/security: add vulncheck page
Images showing the initial state of the page:
https://screenshot.googleplex.com/Boo3JDxkd7dtT68.png
https://screenshot.googleplex.com/4HKEYCmHFDLDjYJ.png
https://screenshot.googleplex.com/9g44QZqX2k5758F.png
https://screenshot.googleplex.com/7bWWVCGDHejkW67.png
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
---
A _content/security/vulncheck.html
1 file changed, 383 insertions(+), 0 deletions(-)
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
Attention is currently required from: Jonathan Amsterdam, Julie Qiu, Roland Shoemaker.
Patch set 10:Run-TryBot +1
Zvonimir Pavlinovic (Gerrit)
Zvonimir Pavlinovic submitted this change.
7 is the latest approved patch-set.
The change was submitted with unreviewed changes in the following files:
```
The name of the file: _content/security/vulncheck.html
Insertions: 24, Deletions: 23.
@@ -122,12 +122,12 @@
<h2 id="vulnerability-graphs">Vulnerability Graphs</h2>
<p>
- The main output of vulncheck are subgraphs of the program call graph, package import graph, and module
- require graph that lead to vulnerabilities. We refer to such subgraphs as <i>vulnerability graphs</i>.
- A vulnerability call graph contains only nodes and edges of the original call graph that show how
- vulnerable symbols are reachable from the program entry points. At the call graph level, entry points
- are <code>main</code>s, <code>init</code>s, as well as exported functions and methods of user packages.
- Consider the following example:
+ The main output of vulncheck are subgraphs of the program call graph, package import graph,
+ and module require graph that lead to vulnerabilities. We refer to such subgraphs as
+ <i>vulnerability graphs</i>. A vulnerability call graph contains only nodes and edges of the
+ original call graph that show how vulnerable symbols are reachable from the program entry points.
+ At the call graph level, entry points are <code>main</code>s, <code>init</code>s, as well as
+ exported functions and methods of user packages. Consider the following example:
</p>
<pre>
@@ -242,20 +242,20 @@
<h3 id="vulnerability-evidence">Evidence of vulnerability uses</h3>
<p>
- Clients of vulncheck can present the vulnerability graphs, such as the one above, to the programmers
- as a way of showing how vulnerabilities are reachable in their code. However, vulnerability graphs can
- get large for big projects, which would make it hard for programmers to manually inspect vulnerabilities.
- In response, vulncheck also provides <code>CallStacks</code> functionality for extracting call stacks
- from vulnerability call graphs.
+ Clients of vulncheck can present the vulnerability graphs, such as the one above, to the
+ programmers as a way of showing how vulnerabilities are reachable in their code. However,
+ vulnerability graphs can get large for big projects, which would make it hard for programmers
+ to manually inspect vulnerabilities. In response, vulncheck also provides <code>CallStacks</code>
+ functionality for extracting call stacks from vulnerability call graphs.
</p>
<p>
- For each pair of a vulnerable symbol and an entry point, <code>CallStacks</code> traverses the vulnerability
- graph searching for call stacks starting at the entry point and ending with a call to the vulnerable symbol.
- To avoid exponential explosion, each node is visited at most once. The extracted stacks for a particular
- vulnerability are heuristically ordered by how easy is to understand them: shorter call stacks with less
- dynamic call sites appear earlier in the extracted results. For the vulnerability call graph shown earlier,
- there are two stacks reported for <code>V</code>.
+ For each pair of a vulnerable symbol and an entry point, <code>CallStacks</code> traverses the
+ vulnerability graph searching for call stacks starting at the entry point and ending with a call
+ to the vulnerable symbol. To avoid exponential explosion, each node is visited at most once. The
+ extracted stacks for a particular vulnerability are heuristically ordered by how easy is to understand
+ them: shorter call stacks with less dynamic call sites appear earlier in the extracted results.
+ For the vulnerability call graph shown earlier, there are two stacks reported for <code>V</code>.
</p>
<pre>
@@ -271,11 +271,12 @@
</pre>
<p>
- Note that the call stack <code>[A, E, D, Y.Foo, V]</code> is not reported since a shorter extracted stack
- <code>[A, D, Y.Foo, V]</code> starting at <code>A</code> already goes through <code>D</code>. The clients of
- vulncheck can present (a subset) of representative calls stacks to programmers as a more succinct evidence
- of vulnerability uses. For instance, <a href="https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck">govulncheck</a>
- by default shows only the first call stack extracted by <code>CallStacks</code>.
+ Note that the call stack <code>[A, E, D, Y.Foo, V]</code> is not reported since a shorter extracted
+ stack <code>[A, D, Y.Foo, V]</code> starting at <code>A</code> already goes through <code>D</code>.
+ The clients of vulncheck can present (a subset) of representative calls stacks to programmers as a
+ more succinct evidence of vulnerability uses. For instance,
+ <a href="https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck">govulncheck</a> by default shows only
+ the first call stack extracted by <code>CallStacks</code>.
</p>
<p>
@@ -283,7 +284,7 @@
section). Vulnerabilities are modeled using the shared
<a href="https://golang.org/x/vuln/osv">golang.org/x/vuln/osv</a> format and an existing
vulnerability database is available at <a href="https://vuln.go.dev">https://vuln.go.dev</a>.
- For more details details on vulncheck data structures and APIs, please see
+ For more details on vulncheck data structures and APIs, please see
<a href="https://pkg.go.dev/golang.org/x/vuln/vulncheck">here</a>.
</p>
```
_content/security: add vulncheck page
Images showing the initial state of the page:
https://screenshot.googleplex.com/Boo3JDxkd7dtT68.png
https://screenshot.googleplex.com/4HKEYCmHFDLDjYJ.png
https://screenshot.googleplex.com/9g44QZqX2k5758F.png
https://screenshot.googleplex.com/7bWWVCGDHejkW67.png
Change-Id: Icdbc5dda10f3571a2fbe399ddfe6d4da4069b57c
Reviewed-on: https://go-review.googlesource.com/c/website/+/407334
Reviewed-by: Julie Qiu <juli...@google.com>
TryBot-Result: Gopher Robot <go...@golang.org>
Run-TryBot: Zvonimir Pavlinovic <zpavl...@google.com>
Reviewed-by: Jonathan Amsterdam <j...@google.com>
---
A _content/security/vulncheck.html
1 file changed, 388 insertions(+), 0 deletions(-)
diff --git a/_content/security/vulncheck.html b/_content/security/vulncheck.html
new file mode 100644
index 0000000..57ea38a
--- /dev/null
+++ b/_content/security/vulncheck.html
@@ -0,0 +1,367 @@
+<!--{
+ "Title": "Vulnerability Detection For Go",
+ "layout": "article"
+}-->
+
+<h2 id="overview">Overview</h2>
+
+<p>
+ Writing secure and reliable software requires knowing about vulnerabilities in your
+ dependencies. This page provides an overview of the Go vulnerability detection package,
+ <a href="https://pkg.go.dev/golang.org/x/vuln/vulncheck">golang.org/x/vuln/vulncheck</a>,
+ which enables Go developers to scan dependencies in their Go projects for public vulnerabilities.
+
+ This package is also available as a CLI tool, <a href="https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck">govulncheck</a>.
+
+</p>
+
+<h2 id="vulncheck">Package vulncheck</h2>
+
+<p>
+ Package vulncheck delivers support for detection and understanding of how user programs exercise
+ known vulnerabilities. It can detect packages and modules with known vulnerabilities that
+ are transitively imported by the user program. What makes vulncheck unique is that it also
+ tries to find runtime call stacks, sequences of currently active function calls made by the
+ program, demonstrating how user code reaches a vulnerability without actually executing the
+ code. This feature brings several benefits.
+</p>
+
+<h3 id="vulnerabilities-levels">Vulnerabilities at the package and function levels</h3>
+
+<p>
+ vulncheck is more accurate than standard package-level vulnerability detection: just because
+ a vulnerable <i>symbol</i> (function or method) is imported, this does does not mean the
+ symbol is actually used. Consider the following illustrative code.
+</p>
+
+<pre>
+package main
+
+import "some/third/party/pkg/p"
+
+func main() {
+ p.G()
+}
+</pre>
+<pre>
+// package some/third/party/pkg/p
+package p
+
+// F has a known vulnerability
+func F() { … }
+
+// G is safe and does not transitively invoke F
+func G() { … }
+</pre>
+
+<p>
+ Package-level vulnerability detection would issue a warning for the above code saying that the
+ package <code>p</code> has some vulnerabilities and that it has been transitively imported by the
+ user package <code>main</code>. This warning is useful, but it does not tell the whole story.
+ Since <code>F</code> never gets called, the above user code is not affected by <code>p</code>'s
+ vulnerabilities. Programmers might choose to not address the import of <code>p</code> at all, or
+ postpone the fix until the next release, if they knew that no vulnerabilities of <code>p</code>
+ are in fact exercised. Package-level detection is inherently limited in providing the programmers
+ with such knowledge of vulnerabilities in their code.
+</p>
+
+<h3 id="understanding vulnerabilities">Understanding vulnerabilities</h3>
+
+<p>
+ To drive home this point, let us assume we also have an accompanying test code.
+</p>
+
+<pre>
+package main
+
+import (
+ "testing"
+ "some/third/party/pkg/p"
+)
+
+func TestFoo(t *testing.T) {
+ p.F()
+ …
+}
+
+func TestBar(t *testing.T) {
+ p.F()
+ …
+}
+</pre>
+
+<p>
+ Here, the vulnerable symbol <code>F</code> is indeed used by the code, but only in tests.
+ Programmers might be fine with vulnerabilities being potentially triggered in tests or, say,
+ sandboxed environments. Package and module level detection do not provide programmers
+ with the level of detail necessary to make such decisions in an informed manner. vulncheck,
+ on the other hand, is designed precisely for that. vulncheck reports call stacks demonstrating
+ how the vulnerable symbols are reachable by user code. For the above example, vulncheck
+ communicates <code>[TestFoo, p.F]</code> and <code>[TestBar, p.F]</code> call stacks to programmers.
+ If we exclude above tests, vulncheck does not report any call stacks, which the programmers
+ can interpret as no vulnerabilities are in fact reachable.
+</p>
+
+<p>
+ As shown by the above example, vulncheck's motivation for reporting call stacks to programmers
+ goes beyond just improving the precision of vulnerability detection. Its overarching goal is
+ to help programmers understand vulnerabilities, their impact, and their potential remedies.
+ Although the simplest fix often is just to update the corresponding vulnerable package to its
+ healthy version, if any, there are still some very important open questions:
+ <ul>
+ <li>Could my systems have been breached and, if so, where does the breach occur?</li>
+ <li>Do I need to escalate the issue?</li>
+ <li>Do I need to alarm my customers?</li>
+ </ul>
+ Call stacks reported by vulncheck can help programmers answer those questions, because
+ vulnerabilities can be buried deep in unfamiliar places in the code. Package and module
+ level detection on its own is very often not helpful when addressing these questions.
+</p>
+
+
+<h2 id="vulnerability-graphs">Vulnerability Graphs</h2>
+
+<p>
+ The main output of vulncheck are subgraphs of the program call graph, package import graph,
+ and module require graph that lead to vulnerabilities. We refer to such subgraphs as
+ <i>vulnerability graphs</i>. A vulnerability call graph contains only nodes and edges of the
+ original call graph that show how vulnerable symbols are reachable from the program entry points.
+ At the call graph level, entry points are <code>main</code>s, <code>init</code>s, as well as
+ exported functions and methods of user packages. Consider the following example:
+</p>
+
+<pre>
+package p
+
+import "some/package/q"
+
+type X struct { ... }
+
+func (x X) Foo() { ... } // makes no further calls
+
+func A(x X) {
+ q.D(x)
+ q.E(x)
+}
+
+func B(x X) {
+ q.E(x)
+}
+
+func C(x X) {
+ x.Foo()
+}
+</pre>
+
+<pre>
+// package some/package/q
+package q
+
+import "vulnerable/package/vuln"
+
+type I interface {
+ Foo()
+}
+
+type Y struct { ... }
+
+func (y Y) Foo() {
+ vuln.V()
+}
+
+func D(i I) {
+ i.Foo()
+ y := Y{...}
+ y.Foo()
+}
+
+func E(i I) {
+ i.Foo()
+ D(i)
+}
+</pre>
+
+<pre>
+// package vulnerable/package/vuln
+package vuln
+
+func V() {...} // known to be vulnerable, makes no further calls
+</pre>
+
+<p>
+ vulncheck's <code>Source</code> function takes this program as input and first constructs its
+ call graph, shown below. We omit package information of each function for brevity.
+</p>
+
+<pre>
+ A _ B C
+ | \ | |
+ | \ | |
+ D <-- E |
+ | \ \ |
+ | \ \ |
+ | \ \ |
+ Y.Foo -> X.Foo
+ |
+ V
+</pre>
+
+<p>
+ The entry points are functions <code>A</code>, <code>B</code>, and <code>C</code> of the input
+ package <code>p</code>. These functions mainly pass <code>X</code> values to exported functions
+ of package <code>q</code> that in turn call <code>X.Foo</code>. <code>D</code> also calls
+ <code>Y.Foo</code>.
+</p>
+
+<p>
+ The call to <code>Y.Foo</code> is problematic as it itself makes a call to the vulnerable function
+ <code>V</code> of <code>vuln</code>. We thus have a call to a vulnerable function in a dependent
+ package that is not under control of the author of the package <code>p</code>. This can be hard to
+ trace down for programmers by relying on just package-level vulnerability detection. vulncheck detects
+ this and computes the following vulnerability call graph.
+</p>
+
+<pre>
+ A _ B
+ | \ |
+ | \ |
+ D <-- E
+ |
+ Y.Foo
+ |
+ V
+</pre>
+
+<p>
+ Functions <code>C</code> and <code>X.Foo</code> are not in the vulnerability graph as they do not
+ transitively lead to <code>V</code>. In general, all edges not leading to vulnerable symbols are
+ omitted, as well as nodes appearing exclusively along those edges. The same principles are used to
+ create vulnerability graphs of package imports and module require graphs.
+</p>
+
+<h3 id="vulnerability-evidence">Evidence of vulnerability uses</h3>
+
+<p>
+ Clients of vulncheck can present the vulnerability graphs, such as the one above, to the
+ programmers as a way of showing how vulnerabilities are reachable in their code. However,
+ vulnerability graphs can get large for big projects, which would make it hard for programmers
+ to manually inspect vulnerabilities. In response, vulncheck also provides <code>CallStacks</code>
+ functionality for extracting call stacks from vulnerability call graphs.
+</p>
+
+<p>
+ For each pair of a vulnerable symbol and an entry point, <code>CallStacks</code> traverses the
+ vulnerability graph searching for call stacks starting at the entry point and ending with a call
+ to the vulnerable symbol. To avoid exponential explosion, each node is visited at most once. The
+ extracted stacks for a particular vulnerability are heuristically ordered by how easy is to understand
+ them: shorter call stacks with less dynamic call sites appear earlier in the extracted results.
+ For the vulnerability call graph shown earlier, there are two stacks reported for <code>V</code>.
+</p>
+
+<pre>
+ A B
+ | |
+ D E
+ | |
+ Y.Foo D
+ | |
+ V Y.Foo
+ |
+ V
+</pre>
+
+<p>
+ Note that the call stack <code>[A, E, D, Y.Foo, V]</code> is not reported since a shorter extracted
+ stack <code>[A, D, Y.Foo, V]</code> starting at <code>A</code> already goes through <code>D</code>.
+ The clients of vulncheck can present (a subset) of representative calls stacks to programmers as a
+ more succinct evidence of vulnerability uses. For instance,
+ <a href="https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck">govulncheck</a> by default shows only
+ the first call stack extracted by <code>CallStacks</code>.
+</p>
+
+<p>
+ Few notes. vulncheck can also analyze Go binaries with some limitations (see Limitations
+ section). Vulnerabilities are modeled using the shared
+ <a href="https://golang.org/x/vuln/osv">golang.org/x/vuln/osv</a> format and an existing
+ vulnerability database is available at <a href="https://vuln.go.dev">https://vuln.go.dev</a>.
+ For more details on vulncheck data structures and APIs, please see
+ <a href="https://pkg.go.dev/golang.org/x/vuln/vulncheck">here</a>.
+</p>
+
+<h2 id="algorithm">Call Graph Construction</h2>
+
+<p>
+ One of the main technical challenges in vulncheck is to statically compute call graph
+ information of a Go program. As this is an undecidable problem, we can only hope for an
+ approximate solution. More precise call graph algorithms will require more execution time.
+ On the other hand, a really fast algorithm could easily be very imprecise, either missing
+ call stacks or often reporting ones that do not appear at runtime. vulncheck strikes the
+ balance between precision and volume of used computational resources with the
+ <a href="https://dl.acm.org/doi/pdf/10.1145/354222.353189">Variable Type Analysis</a> (VTA) algorithm.
+</p>
+
+<h3 id="variable-type-analysis">Variable type analysis</h3>
+
+<p>
+ VTA is an over-approximate call graph algorithm. VTA does not miss a call stack realizable
+ in practice (see Limitations section for exceptions to this), but it might sometimes
+ report a call stack leading to a vulnerability that cannot be exercised in practice. Our
+ experiments suggests this does not happen too often.
+</p>
+
+<p>
+ Consider again the program from the previous section. Existing algorithms, such as
+ <a href="https://pkg.go.dev/golang.org/x/tools/go/callgraph/cha">CHA</a> or
+ <a href="https://pkg.go.dev/golang.org/x/tools/go/callgraph/rta">RTA</a>, would say that
+ <code>i.Foo()</code> call in <code>E</code> resolves to <code>X.Foo</code> and <code>Y.Foo</code>
+ because types <code>X</code> and <code>Y</code> implement interface <code>I</code> and are used in
+ the program. If vulncheck relied on these two algorithms, it would report vulnerable call stack
+ <code>[B, E, Y.Foo, V]</code> that is in fact not realizable in practice. VTA, as we hinted
+ earlier, correctly resolves that call to only <code>X.Foo</code> that does not lead to <code>V</code>.
+</p>
+
+<p>
+ VTA works on an abstract representation of a program where variables are represented
+ by their types. The types are then propagated around the program based on variable usage. For
+ the running example, parameter <code>x</code> of <code>B</code> is abstracted via type <code>X</code>
+ which is then propagated to parameter <code>i</code> of <code>E</code>. The values actually
+ stored to the variable are not taken into account, only their types. This can lead to imprecision
+ when types reaching an interface variable depend on valuation of, say, involved conditional
+ statements or complicated aliasing. However, types of concrete variables are always the same,
+ regardless of the complexity of the surrounding logic. For instance, values reaching a variable
+ of type <code>X</code> always have precisely the type <code>X</code>. This rather unique property
+ of Go's type system enables VTA to produce precise call graph information.
+</p>
+
+<h3 id="achieving-scale">Achieving scale</h3>
+
+<p>
+ In the current example, VTA propagates type <code>X</code> from <code>B</code> to <code>E</code>
+ because the call <code>E(i)</code> is static. VTA knows what function the identifier <code>E</code>
+ resolves to. But what if that call was dynamic? After all, VTA is supposed to construct the call graph
+ so how can it then propagate types across function boundaries? One solution is to rely on a fix-point
+ where the results of type propagation are also used to establish function call edges on which type
+ propagation then needs to be repeated, and so on. This could be very expensive, so VTA relies on an
+ initial approximation of the call graph to scale. Note that the initial call graph is only used to propagate
+ types over function calls. We choose CHA as the initial call graph. As CHA can be rather imprecise, as
+ shown on the earlier example, it could cause VTA to be overly imprecise as well. To counter that, vulncheck
+ bootstraps VTA by VTA. After computing VTA on top of CHA, we feed the more precise resulting call graph
+ to VTA again, toning down excessive imprecision initially introduced by CHA.
+</p>
+
+<p>
+ Package VTA can be found at <a href="https://pkg.go.dev/golang.org/x/tools/go/callgraph/vta">golang.org/x/tools/go/callgraph/vta</a>.
+</p>
+
+<h2 id="limitations">Limitations</h2>
+
+<p>
+ As VTA can produce call stacks that are not realizable in practice, vulncheck can claim that a vulnerable
+ symbol is reachable while in fact it is not. We also note that VTA might miss some call stacks that
+ go through <i>unsafe</i> and <i>reflect</i> packages.
+</p>
+
+<p>
+ Because binaries do not contain detailed call information, vulncheck cannot compute vulnerability call
+ graphs and call stack witnesses for Go binaries.
+</p>
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.
Zvonimir Pavlinovic (Gerrit)
1 comment:
File _content/security/vulncheck.html:
Clients of vulncheck can present the vulnerability graphs, such as the one above, to the programmers
as a way of showing how vulnerabilities are reachable in their code. However, vulnerability graphs can
get large for big projects, which would make it hard for programmers to manually inspect vulnerabilities.
In response, vulncheck also provides <code>CallStacks</code> functionality for extracting call stacks
from vulnerability call graphs.
</p>
<p>
For each pair of a vulnerable symbol and an entry point, <code>CallStacks</code> traverses the vulnerability
graph searching for call stacks starting at the entry point and ending with a call to the vulnerable symbol.
To avoid exponential explosion, each node is visited at most once. The extracted stacks for a particular
vulnerability are heuristically ordered by how easy is to understand them: shorter call stacks with less
dynamic call sites appear earlier in the extracted results. For the vulnerability call graph shown earlier,
there are two stacks reported for <code>V</code>.
</p>
sorry I meant so that the lines don't run over (usually there is an editor plugin that will do this […]
Done
(Forgot to send this earlier, sorry)
To view, visit change 407334. To unsubscribe, or for help writing mail filters, visit settings.