Re: [chromium-dev] Credits generation and whitelisted third-party code not in third_party/

[Lambros Lambrou]

The current proposal is to move all third-party-licensed code into new third_party/ directories.

The problem with creating new third_party/ directories is that it will create new entries in about:credits that simply duplicate existing licenses.

We don't want Chrome to ship with an even larger about:credits page with lots of duplicate info.

So I think we need a way for new third_party/ directories to adopt the license from a pre-existing third_party/ directory.

For example:

ui/events/keycodes/keyboard_codes_posix.h contains code that is covered by the WebKit license.

So we would move this file to ui/events/keycodes/third_party/ and create a GN source_set for it.

Then we would place some metadata in there to say "this code is covered by license of third_party/WebKit".

In effect, the licensing metadata is "symlinked" to WebKit.

tools/licenses.py would be updated to understand this metadata in order to generate the correct credits for a given GN target.

So when we generate credits for Chrome on iOS (for example), we would still include a WebKit entry if it depends on the ui/ code, even though it doesn't depend on third_party/WebKit/.

(Right now, credits on iOS is a mess because we list everything, but manually exclude components with iOS-incompatible licenses and we assert that we don't ship those components)

I'm working on a CL that implements this - hopefully it seems a reasonable proposal?

On Fri, Apr 14, 2017 at 11:03 AM, Brett Wilson <bre...@chromium.org> wrote:

Yes, this directory is such an example of this for 2 files:

https://cs.chromium.org/chromium/src/url/third_party/mozilla/

I don't see why this can't be done for other things.

Brett

On Fri, Apr 14, 2017 at 10:59 AM, Dirk Pranke <dpr...@chromium.org> wrote:

If I'm understanding you correctly, you're suggesting that we actually move each  files into new third_party directories that have the correct licenses?

-- Dirk 

On Fri, Apr 14, 2017 at 10:43 AM, Brett Wilson <bre...@chromium.org> wrote:

Can we just make third_party directories for all of the files with other licenses? We went through some effort to do this for one file in //url a while ago so I'm surprised this was not also done elsewhere.

Brett

On Thu, Apr 13, 2017 at 6:04 PM, Dirk Pranke <dpr...@chromium.org> wrote:

On Wed, Apr 12, 2017 at 4:16 PM, Lambros Lambrou <lambros...@chromium.org> wrote:

Hi all,

I have been looking into restricting the abouts:credits page to only contain entries for code that is actually being shipped (for a given platform). This is tracked at http://crbug.com/178215, but our background motivation for this is http://crbug.com/697128.

We have already added support to tools/licenses.py to generate a credits file based on a GN target. The (transitive) dependencies of that target are filtered for "third_party" components, and only those associated licenses are included in the Credits report.

We are using this feature to generate credits for the "remoting_apk" target (Chrome Remote Desktop client for Android), and we propose to also use it for Chrome's about:credits page. It could also be used for WebView, and anything else we build and ship separately from the Chrome browser, such as Chrome Remote Desktop host.

The problem is, not all third-party code is included in a third_party/ directory. We have tools/copyright_scanner/ which scans all files for third-party licenses, and maintains a whitelist there (third_party_files_whitelist.txt). Conceivably, Chrome (or whatever GN target we generate credits for) might depend on one of these whitelist files, but not on the third_party/ project that would provide the license for it.

From looking at the whitelist file, it seems possible that there are files for which we don't actually have a matching third party project that we could grab the license from. Would you add a stub target to fix that, or do something else?

Any thoughts?

A slightly-crazy idea would be: for each whitelisted file:

• Create a "fake" GN target for it, with "third_party" within its name (for example: a/b/c:third_party__foo_cc).
• Have whatever uses that file depend on the fake GN target.
  
• Have the fake GN target "depend" on the actual third_party/ project that provides the licensing info.
  

This would fix the problem with very few (if any) changes to tools/licenses.py.

Is this a reasonable approach?

Is it easy to create "fake" targets and dependencies in GN for this purpose, or is there a better way?

It seems like it would be better to query GN to figure out if there are paths from the whitelisted files to the target in question. 

I will reply further off-thread ...

-- Dirk

 

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAHbUkqEa1XBFFZ_-tOX-MZNOdEYzc%2BWH_xGHLnNa-2bm-%3Dkn7A%40mail.gmail.com.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEoffTDKK-OM9auhj8r3Pgt-LuU06%3Da4sPXd7d0MfFKRz-DYWw%40mail.gmail.com.

[Andrew Grieve]

So happy that you're working on this! I like the symlink idea!

Makes sense to not duplicate licenses. For the "covered by third_party/WebKit" case though, would it make more sense to just put the file within third_party/WebKit?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAHbUkqH8w12B8YrZx%2ByhTYAGM1bE6JX%3DTB4g4v5HXNJHaJU0QA%40mail.gmail.com.

[Nico Weber]

There are plans to move blink out of t_p/WebKit to some top-level dir (blink/ or web/ or...). How does that mesh with putting code covered by the WebKit license into third_party folders?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CABiQX1ULn6qZBdjd%2B7RVRb4sa1LJF0m_jLdQj4m0arqC5mO7jg%40mail.gmail.com.

[Lambros Lambrou]

I guess we don't want a dependency of ui/ on WebKit, which is why we can't just put the file there.

I assume that other examples are similar - code has been copied to avoid adding dependencies on large components, or to avoid dependency layering violations.

But it's not always clear what projects the files are associated with. Just to pick some random examples from the whitelist:

# Copyright The Chromium Authors, Apple Inc; BSD license. Not used on Android.

ui/base/clipboard/clipboard_util_win.cc

# Copyright The Chromium Authors, Apple Inc and Graham Dennis; BSD license. Not

# used on Android.

ui/base/cocoa/tool_tip_base_view.mm

# Copyright The Chromium Authors, Apple Inc; BSD license. Not used on Android.

ui/base/dragdrop/os_exchange_data_provider_win.cc

I don't know which existing about:credits entries would cover these files. Maybe we do have to create new entries for these?

[Lambros Lambrou]

Regarding putting third-party-licensed code in a top-level directory: the policy says

"We put all code that isn't written by Chromium developers into src/third_party (even if you end up modifying just a few functions). We do this to make it easy to track license compliance, security patches, and supply the right credit and attributions. It also makes it a lot easier for other projects that embed our code to track what is Chromium licensed and what is covered by other licenses."

(It goes on to allow third_party subdirectories elsewhere in the tree, but recommends minimizing these.)

So there may be some complications with allowing third-party code to live in a top-level directory (especially if "other projects" are relying on current policy to track "what is Chromium licensed").

If we were to change the policy, we would need to update tools/licenses.py to understand that src/blink (for example) is a third-party directory.

The script has some new GN-related parameters which allow it to generate credits for a GN target, by looking at the dependency graph. The implementation searches the output of "gn desc" for "third_party", so the regex would be more complex if we allow non-"third_party" directories to contribute to the credits list.

[Nico Weber]

On Tue, May 2, 2017 at 1:14 PM, Lambros Lambrou <lambros...@chromium.org> wrote:

Regarding putting third-party-licensed code in a top-level directory: the policy says

"We put all code that isn't written by Chromium developers into src/third_party (even if you end up modifying just a few functions). We do this to make it easy to track license compliance, security patches, and supply the right credit and attributions. It also makes it a lot easier for other projects that embed our code to track what is Chromium licensed and what is covered by other licenses."

Sure, but blink is probably a bit of a special case. It's in src.git now, all new code written in the last 4 years was written by Chromium developers, and whatnot.

 

(It goes on to allow third_party subdirectories elsewhere in the tree, but recommends minimizing these.)

So there may be some complications with allowing third-party code to live in a top-level directory (especially if "other projects" are relying on current policy to track "what is Chromium licensed").

If we were to change the policy, we would need to update tools/licenses.py to understand that src/blink (for example) is a third-party directory.

It's not really a third-party directory. Some of the code in their used to be third-party code, but we've forked it. licenses.py should probably learn about that. If you think this is not workable, you will want to head to https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/mN9XR4YAG2g and make yourself heard.

[Lambros Lambrou]

WebKit is a bit of a special case anyway, because it doesn't seem to come under a single license AFAICT. It looks like an amalgamation of all the individual licenses of each source file.

IANAL and cannot speak officially for Chrome, but I believe that "forking" a project doesn't change the license requirements, unless you completely rewrite the project.

Hopefully someone here can give official guidance on whether the WebKit fork needs to be under third_party?

[Nico Weber]

On Tue, May 2, 2017 at 2:16 PM, Lambros Lambrou <lambros...@chromium.org> wrote:

WebKit is a bit of a special case anyway, because it doesn't seem to come under a single license AFAICT. It looks like an amalgamation of all the individual licenses of each source file.

IANAL and cannot speak officially for Chrome, but I believe that "forking" a project doesn't change the license requirements, unless you completely rewrite the project.

Hopefully someone here can give official guidance on whether the WebKit fork needs to be under third_party?

If you believe that there might be a problem with moving it out of third_party, please mention that on the blink-dev thread I linked to.

[Dirk Pranke]

On Tue, May 2, 2017 at 1:24 PM, Nico Weber <tha...@chromium.org> wrote:

On Tue, May 2, 2017 at 2:16 PM, Lambros Lambrou <lambros...@chromium.org> wrote:

WebKit is a bit of a special case anyway, because it doesn't seem to come under a single license AFAICT. It looks like an amalgamation of all the individual licenses of each source file.

IANAL and cannot speak officially for Chrome, but I believe that "forking" a project doesn't change the license requirements, unless you completely rewrite the project.

Hopefully someone here can give official guidance on whether the WebKit fork needs to be under third_party?

If you believe that there might be a problem with moving it out of third_party, please mention that on the blink-dev thread I linked to.

IANAL either, but (a) as far as I know, you are correct in that forking a project does not change licensing in any way by itself, and (b)

we do not need to keep WebKit under //third_party. The third_party rule is a guideline to make third-party reviews easier, and we don't

worry about third party reviews for Blink, it has been dealt with as an exception for a long time.

-- Dirk

[Lambros Lambrou]

OK, we can update tools/licenses.py to allow for Blink not having 'third_party' in its location. If the exceptions proliferate (seems unlikely), we might need to re-think how this works.

We will also need to add the new blink directory to the copyright scanner here. Otherwise it will report a whole ton of new files :)

I have added comments to this tracking doc to suggest adding these 2 tasks.