License violations for dependencies of Rust and Go programs?

[John Thorvald Wodder II]

I am a concerned citizen who, while looking into prior art for handling
dependency licenses in order to inform some of my own projects, stumbled upon
what appear to be systemic license violations in the Debian repositories
regarding dependencies of statically-linked compiled binary programs (primarily
those written in Rust or Go). I would normally expect Debian to be rather
strict about licensing, yet the packages I've looked at unambiguously violate
what I understand the licenses require, so I'm wondering what I'm missing.

It is my understanding that when an executable program that depends (directly
or indirectly) on libraries licensed under (picking one license here) the MIT
license is compiled into a binary that statically links these libraries, and
this binary is then distributed to third parties, the binary must be
accompanied by the license text & copyright notices for all of the program's
direct & indirect MIT-licensed dependencies. In particular, since applying the
MIT license to a project typically involves placing a copyright line containing
the author's name in the same file as the base license text, this copyright
line must accompany all derivatives of the project, and so compiled,
statically-linked executable programs must be accompanied by the copyright
lines of all MIT-licensed dependencies.

Unfortunately, I've come across some software in the official Debian
repositories that do not seem to properly honor these requirements.

EXAMPLE #1: The program hyperfine (https://github.com/sharkdp/hyperfine),
written in Rust. Rust is a compiled language that usually statically-links
binaries, and so the compiled binaries that Debian distributes contain code
from all of hyperfine's dependencies.

- The webpage for the hyperfine v1.17.0 package in Debian Trixie is
https://packages.debian.org/trixie/hyperfine. Clicking on "Copyright File"
on the right side of the page leads to a debian/copyright file that declares
that the source files and the Debian packaging files are both licensed under
"MIT or Apache-2.0", and the copyright line listed for the source files
credits only the author of hyperfine. (Note that most if not all of
hyperfine's dependencies are written by other authors.) The file also
contains the base text of the MIT license and a reference to where the text
of the Apache 2.0 license can be found.

- Downloading & inspecting a binary .deb file for hyperfine (I picked amd64)
shows that it contains the same "copyright" file and no other copyright
information, though the Debian "control" file does contain an
"X-Cargo-Built-Using" field listing the names & versions of the Debian Rust
packages used to build the .deb. Note that these "built using" packages are
only used at build time and are not listed as runtime dependencies, and so
they are not installed when installing hyperfine from the Debian
repositories. Furthermore, running `ldd` on the hyperfine binary only lists
fundamental C libraries as being dynamically linked, supporting the
conclusion that the Rust dependencies are statically linked into the binary.

- For a specific dependency of hyperfine's whose license I believe is being
violated, consider the direct dependency "indicatif"
(https://github.com/console-rs/indicatif). The "X-Cargo-Built-Using" field
for the binary package I inspected above states that the hyperfine binary was
built with indicatif v0.17.3, the source for which contains the LICENSE file
https://github.com/console-rs/indicatif/blob/0.17.3/LICENSE, which consists
of a copyright line and the text of the MIT license. This copyright line
does not appear in the above-mentioned "copyright" file or anywhere in the
.deb package.

EXAMPLE #2: The program gh (https://github.com/cli/cli), written in Go. Go is
a compiled language that usually statically-links binaries, and so the compiled
binaries that Debian distributes contain code from all of gh's dependencies.

- The webpage for the gh v2.27.0 package in Debian Trixie is
https://packages.debian.org/trixie/gh. Clicking on "Copyright File" on the
right side of the page leads to a debian/copyright file that declares that
the source files and the Debian packaging files are both licensed under the
Expat license (another name for the MIT license), and the copyright line
listed for the source files credits only the author of gh. License
information is also given for several third-party dependencies that are
included in the Debian source package in order to handle some dependency
versioning issues. No information is given on the dependencies not bundled
in the source package (of which there are many).

- Downloading & inspecting a binary .deb file for gh (I picked amd64) shows
that it contains the same "copyright" file and no other copyright
information, though the Debian "control" file does contain a
"Static-Built-Using" field listing the names & versions of the Debian Go
packages used to build the .deb. Note that these "built using" packages are
only used at build time and are not listed as runtime dependencies, and so
they are not installed when installing gh from the Debian repositories.
Furthermore, running `ldd` on the gh binary only lists fundamental C
libraries as being dynamically linked, supporting the conclusion that the Go
dependencies are statically linked into the binary.

- For a specific dependency of gh's whose license I believe is being violated,
consider the direct dependency "heredoc"
(https://github.com/MakeNowJust/heredoc). The "Static-Built-Using" field for
the binary package I inspected above states that the gh binary was built with
heredoc v1.0.0, the source for which contains the LICENSE file
https://github.com/makenowjust/heredoc/blob/v1.0.0/LICENSE, which consists of
a copyright line and the text of the MIT license. This copyright line does
not appear in the above-mentioned "copyright" file or anywhere in the .deb
package.

Other packages I looked at that had similar licensing issues:

- age
- bat (In addition to the type of problem discussed above, the source code for
bat has an Apache 2.0 `NOTICE` file, yet this is not included in the .deb
package.)
- dfrs
- exa
- reposurgeon
- restic
- ripgrep
- tty-share

I suspect that this problem applies to all programs written in Go or Rust that
Debian distributes. Is Debian handling dependency licenses for these packages
incorrectly, or is there something I'm missing?

-- John Wodder

[Stephan Verbücheln]

On Wed, 2023-09-27 at 08:36 +0800, Paul Wise wrote:
> This more general problem is very hard to impossible to solve,
> since it would mean patching every single build toolchain and
> source package [...]

Are the upstream developers not already legally required to include all
this information into various places including their “Help-About” menu?

Regards
Stephan

[John Thorvald Wodder II]

On 2023 Sep 26, at 20:36, Paul Wise <pa...@debian.org> wrote:
> Your analysis is correct, some extra context for this problem:
>
> The problem you have identified applies to other statically linked
> languages too, so I have updated the wiki page to link to it.
>
> https://wiki.debian.org/StaticLinking

So was this problem previously known but under-acknowledged, or was it simply
not brought up before now? I find it surprising that Debian would allow so
many license violations to get this far. Is fixing the tooling to handle this
considered a priority? If the author of an uncredited dependency were to
complain, would Debian be more likely to focus on fixing the tooling posthaste
or to just pull whatever packages use the dependency in question?

-- John Wodder

[John Thorvald Wodder II]

On 2023 Sep 26, at 22:09, Paul Wise <pa...@debian.org> wrote:

>
> On Tue, 2023-09-26 at 14:20 -0400, John Thorvald Wodder II wrote:
>
>> - bat (In addition to the type of problem discussed above, the source code for
>> bat has an Apache 2.0 `NOTICE` file, yet this is not included in the .deb
>> package.)
>

> Please file a severity serious bug report against bat about the NOTICE
> issue, I've mentioned it on the #debian-rust IRC channel though.
>
> https://www.debian.org/Bugs/Reporting
>
> I note that lintian detects the NOTICE issue, so I have requested that
> the ftp-master team turn on auto-rejections for the lintian tag.

On further inspection, it turns out that bat itself compiles the text of its NOTICE file into the binary, and the text is displayed when running `batcat --acknowledgements`, so bat's Apache 2.0 license is being followed. If it's Debian policy to include the NOTICE file in the .deb anyway, let me know so I can file a more appropriate bug report.

-- John Wodder

[Mihai Moldovan]

* On 9/27/23 16:41, John Thorvald Wodder II wrote:
> On 2023 Sep 26, at 20:36, Paul Wise <pa...@debian.org> wrote:
>> Your analysis is correct, some extra context for this problem:
>>
>> The problem you have identified applies to other statically linked
>> languages too, so I have updated the wiki page to link to it.
>>
>> https://wiki.debian.org/StaticLinking
>
> So was this problem previously known but under-acknowledged, or was it simply
> not brought up before now? I find it surprising that Debian would allow so
> many license violations to get this far.

The reason is likely a lot more mundane: it's terribly difficult to get license
information right (and to comply with licensing terms verbatim) in such cases.

When just linking in a shared fashion, the work is split up into small,
manageable packages, and every package that depends on another package needs not
care about the other dependencies (as long as one knows that they are compatible).

Static linking is a beast.

Rather, I'd wager that it's important for maintainers to provide the software in
question at all than to delve into such issues for extended periods of time.

The severity of the issue is also very debatable. If it were the case that
non-free parts are mixed with free parts and statically linked, or incompatible
licenses mixed through static linking, than that would be a much graver concern.

In this case, we're "just" talking about missing notices for dependencies that
are pulled in, which might not be nice, but also, realistically, nobody would
really care about or try to enforce it (unless somebody has malicious intent,
which indeed did happen in the past).


Richard Laager wrote a beautiful statement on this list a few months ago, albeit
regarding a different matter, which I want to quote:

> Furthermore, courts are not robots blindly executing code. Seriously,
> can you imagine standing in court trying to argue to a judge that this
> distinction matters and somehow causes you damage‽


This also extends to Sam Hartman:

> Sometimes the best approach to licensing is to take a defensible
> position and not to try and find problems.

> Is fixing the tooling to handle this
> considered a priority? If the author of an uncredited dependency were to
> complain, would Debian be more likely to focus on fixing the tooling posthaste
> or to just pull whatever packages use the dependency in question?

If someone were to complain and the complain were to be rightful, the easiest
way to handle the situation would be to remove the affected packages from the
archives. Fixing it properly would take more time.

Truth be told, in my opinion, this whole thing is a bit blown out of proportion.
The real-world implications are minimal, and while it does make sense to rectify
the blunders that probably exist, it feels more like a trade-off between a very
complicated situation that binds maintainer's time tightly (particularly because
no automated solution exists and manual intervention for packages with hundreds
of dependencies that are then statically linked is a time drain nightmare) for
minimal benefits of having it gotten it done absolutely waterproof.

Which is not to say that slips shouldn't be eventually fixed - it does make
sense to actually point them out and to start working on them.



Mihai

[Sam Hartman]

>>>>> "Mihai" == Mihai Moldovan <io...@ionic.de> writes:

Mihai> In this case, we're "just" talking about missing notices for
Mihai> dependencies that are pulled in, which might not be nice, but
Mihai> also, realistically, nobody would really care about or try to
Mihai> enforce it (unless somebody has malicious intent, which
Mihai> indeed did happen in the past).

I agree with your overall conclusion that in practice we are unlikely to
have significant legal liability or cause significant damages here.

However, I disagree on one point. You imply that you believe anyone
complaining about a violation here would be malicious.

One of my former house mates was part of a BSD-licensed free software
project related to a new technology.
That project cared a lot about having their code acknowledged in
supporting documentation, because they were trying to demonstrate wide
adoption of their technology.
It significantly impacted their ability to raise money and impacted
their satisfaction with their work to be able to demonstrate all the
wide variety of products that incorporated their technology.

I don't think they sued (or would consider doing that) when not
credited, but it did do emotional and possibly economic harm to them
when dependencies did not credit them.

If someone like that came to Debian and asked us to do a better job of
crediting them, I would not consider that malicious at all.

[Mihai Moldovan]

* On 9/27/23 21:10, Sam Hartman wrote:
>>>>>> "Mihai" == Mihai Moldovan <io...@ionic.de> writes:
>
> Mihai> In this case, we're "just" talking about missing notices for
> Mihai> dependencies that are pulled in, which might not be nice, but
> Mihai> also, realistically, nobody would really care about or try to
> Mihai> enforce it (unless somebody has malicious intent, which
> Mihai> indeed did happen in the past).
>
> I agree with your overall conclusion that in practice we are unlikely to
> have significant legal liability or cause significant damages here.
>
> However, I disagree on one point. You imply that you believe anyone
> complaining about a violation here would be malicious.

Let me apologize for this misunderstanding, this is not what I meant. What I
really wanted to convey was that malicious intent, aiming at causing disruption
or for personal gains, is a great concern for the Debian Project, while friendly
hints at violations are usually quickly dealt with to the satisfaction of both
sides via collective work.

Though very rare, and mostly related to patents and not licenses directly, there
have been instances of what I would call malicious intent strictly for personal
gain in the past, to which I was referring.



Mihai

[Florian Weimer]

* John Thorvald Wodder, II:

> It is my understanding that when an executable program that depends (directly
> or indirectly) on libraries licensed under (picking one license here) the MIT
> license is compiled into a binary that statically links these libraries, and
> this binary is then distributed to third parties, the binary must be
> accompanied by the license text & copyright notices for all of the program's
> direct & indirect MIT-licensed dependencies.

Based on my understanding of copyright law, this is correct.
Nevertheless, there seems to be an emerging consensus throughout the
industry that (for example) mentioning the “MIT” SPDX license
identifier is sufficient to meet the notification requirement inherent
to the MIT license.

> Unfortunately, I've come across some software in the official Debian
> repositories that do not seem to properly honor these requirements.

This conclusion is incorrect for Debian, I would say. In Debian's
case, notification requirements are primarily met by shipping full
source code for the entire distribution. I'm aware of heroic efforts
to maintain debian/copyright files, but as you point out, they are
incomplete when viewed in isolation because they only reflect a
source-only view.

Personally, I do not think this is an issue. Debian does not need to
enable binary-only redistribution in cases where licenses may permit
it (Of course, there are plenty of cases where binary-only
distribution is not allowed by applicable licenses anyway.) I don't
see how it furthers Debian's goals, and it only helps a tiny subset of
Debian users.