Credits generation and whitelisted third-party code not in third_party/

[Lambros Lambrou]

Hi all,

I have been looking into restricting the abouts:credits page to only contain entries for code that is actually being shipped (for a given platform). This is tracked at http://crbug.com/178215, but our background motivation for this is http://crbug.com/697128.

We have already added support to tools/licenses.py to generate a credits file based on a GN target. The (transitive) dependencies of that target are filtered for "third_party" components, and only those associated licenses are included in the Credits report.

We are using this feature to generate credits for the "remoting_apk" target (Chrome Remote Desktop client for Android), and we propose to also use it for Chrome's about:credits page. It could also be used for WebView, and anything else we build and ship separately from the Chrome browser, such as Chrome Remote Desktop host.

The problem is, not all third-party code is included in a third_party/ directory. We have tools/copyright_scanner/ which scans all files for third-party licenses, and maintains a whitelist there (third_party_files_whitelist.txt). Conceivably, Chrome (or whatever GN target we generate credits for) might depend on one of these whitelist files, but not on the third_party/ project that would provide the license for it.

Any thoughts?

A slightly-crazy idea would be: for each whitelisted file:

• Create a "fake" GN target for it, with "third_party" within its name (for example: a/b/c:third_party__foo_cc).
• Have whatever uses that file depend on the fake GN target.
  
• Have the fake GN target "depend" on the actual third_party/ project that provides the licensing info.
  

This would fix the problem with very few (if any) changes to tools/licenses.py.

Is this a reasonable approach?

Is it easy to create "fake" targets and dependencies in GN for this purpose, or is there a better way?

[Dirk Pranke]

On Wed, Apr 12, 2017 at 4:16 PM, Lambros Lambrou <lambros...@chromium.org> wrote:

Hi all,

I have been looking into restricting the abouts:credits page to only contain entries for code that is actually being shipped (for a given platform). This is tracked at http://crbug.com/178215, but our background motivation for this is http://crbug.com/697128.

We have already added support to tools/licenses.py to generate a credits file based on a GN target. The (transitive) dependencies of that target are filtered for "third_party" components, and only those associated licenses are included in the Credits report.

We are using this feature to generate credits for the "remoting_apk" target (Chrome Remote Desktop client for Android), and we propose to also use it for Chrome's about:credits page. It could also be used for WebView, and anything else we build and ship separately from the Chrome browser, such as Chrome Remote Desktop host.

The problem is, not all third-party code is included in a third_party/ directory. We have tools/copyright_scanner/ which scans all files for third-party licenses, and maintains a whitelist there (third_party_files_whitelist.txt). Conceivably, Chrome (or whatever GN target we generate credits for) might depend on one of these whitelist files, but not on the third_party/ project that would provide the license for it.

From looking at the whitelist file, it seems possible that there are files for which we don't actually have a matching third party project that we could grab the license from. Would you add a stub target to fix that, or do something else?

Any thoughts?

A slightly-crazy idea would be: for each whitelisted file:

• Create a "fake" GN target for it, with "third_party" within its name (for example: a/b/c:third_party__foo_cc).
• Have whatever uses that file depend on the fake GN target.
  
• Have the fake GN target "depend" on the actual third_party/ project that provides the licensing info.
  

This would fix the problem with very few (if any) changes to tools/licenses.py.

Is this a reasonable approach?

Is it easy to create "fake" targets and dependencies in GN for this purpose, or is there a better way?

It seems like it would be better to query GN to figure out if there are paths from the whitelisted files to the target in question. 

I will reply further off-thread ...

-- Dirk

 

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAHbUkqEa1XBFFZ_-tOX-MZNOdEYzc%2BWH_xGHLnNa-2bm-%3Dkn7A%40mail.gmail.com.

[Brett Wilson]

Can we just make third_party directories for all of the files with other licenses? We went through some effort to do this for one file in //url a while ago so I'm surprised this was not also done elsewhere.

Brett

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEoffTDKK-OM9auhj8r3Pgt-LuU06%3Da4sPXd7d0MfFKRz-DYWw%40mail.gmail.com.

[Dirk Pranke]

If I'm understanding you correctly, you're suggesting that we actually move each  files into new third_party directories that have the correct licenses?

-- Dirk 

[Brett Wilson]

Yes, this directory is such an example of this for 2 files:

https://cs.chromium.org/chromium/src/url/third_party/mozilla/

I don't see why this can't be done for other things.

Brett

[Dirk Pranke]

If you were to go down this route, presumably you could also create real GN targets (source_sets) for the relevant files as well ?

-- Dirk

[Lambros Lambrou]

Every whitelisted file should already have some third_party/ directory that provides its licensing info in about:credits. Otherwise, we would already not be displaying their licenses in about:credits, which we hope is not the case :)

So, rather than create new third_party/ directories for them (causing duplication in about:credits), we should ideally move them into whichever third_party/ project they came from? But this might not be so simple?

For example, this entry:

# Copyright The Chromium Authors, Michael Emmel, Google Inc; BSD license. This
# third-party code is taken from WebKit, the license for which we already pick
# up from webkit/.
ui/events/keycodes/keyboard_codes_posix.h
The file comes from WebKit, but we don't really want a GN dependency from ui/ to WebKit. The assumption at the time was, everything depends on WebKit anyway. But this is not true (Chrome on iOS does not use WebKit), and there are other things we build (such as remoting/ components) that ship separately without WebKit.
In this case, we could maybe move it to:ui/events/keycodes/third_party/keyboard_codes_posix.hand put a source-set inui/events/keycodes/third_party/BUILD.gnand copy the relevant LICENSE/README files, so they get picked up by tools/licenses.py).
(as an aside, can source-sets contain just .h files?)(Also, I just did a search and it appears that keyboard_codes_posix.h is not listed in any GN file. That probably should be fixed.)

Alternatively, we could create a new directory inside WebKit and move the file there:third_party/WebKit/chromium/keyboard_codes_posix.hand a source-set inthird_party/WebKit/chromium/BUILD.gn
There would be a dependency from ui/ to third_party/WebKit/ but it wouldn't be the whole of WebKit, just the chromium subdirectory.
It sounds like we all agree that all third-party code should live in a third_party/ directory somewhere. I guess the question is how to organize things to make that happen?