GeneratePackageLicenseText Errors

[Keith I Myers]

Good Morning All,

I have been getting the following build error. Any ideas on the best way to resolve

INFO    : Generating license credits page. Time:

licenses: Unhandled exception:

Traceback (most recent call last):

  File "/mnt/host/source/chromite/licensing/licenses", line 169, in <module>

    DoMain()

  File "/mnt/host/source/chromite/licensing/licenses", line 165, in DoMain

    commandline.ScriptWrapperMain(FindTarget)

  File "/mnt/host/source/chromite/lib/commandline.py", line 912, in ScriptWrapperMain

    ret = target(argv[1:])

  File "/mnt/host/source/chromite/licensing/licenses.py", line 224, in main

    licensing.GenerateHTMLLicenseOutput(opts.output)

  File "/mnt/host/source/chromite/licensing/licenses_lib.py", line 1080, in GenerateHTMLLicenseOutput

    self._GeneratePackageLicenseText(pkg)

  File "/mnt/host/source/chromite/licensing/licenses_lib.py", line 1018, in _GeneratePackageLicenseText

    license_type = self.FindLicenseType(sln, board=self.board)

  File "/mnt/host/source/chromite/licensing/licenses_lib.py", line 968, in FindLicenseType

    after fixing the license.""" % (license_name, '\n'.join(set(stock + custom))))

AssertionError:

license as-is could not be found in /mnt/host/source/src/third_party/chromiumos-overlay/licenses

/mnt/host/source/src/overlays/overlay-amd64-generic/licenses

/mnt/host/source/src/third_party/portage-stable/licenses

/mnt/host/source/src/third_party/eclass-overlay/licenses

If the license in the ebuild is correct,

a) a stock license should be added to portage-stable/licenses :

running `cros_portage_upgrade` inside of the chroot should clone this repo

to /tmp/portage/:

https://chromium.googlesource.com/chromiumos/overlays/portage/+/gentoo

find the new licenses under licenses, and add them to portage-stable/licenses

b) if it's a non gentoo package with a custom license, you can copy that license

to third_party/chromiumos-overlay/licenses/

Try re-running the script with -p cat/package-ver --generate

after fixing the license.

INFO    : Unmounting image from /mnt/host/source/src/build/images/amd64-generic/R71-11117.0.2018_10_01_1008-a1/stateful and /mnt/host/source/src/build/images/amd64-generic/R71-11117.0.2018_10_01_1008-a1/rootfs

INFO    : Unmounting temporary rootfs /mnt/host/source/src/build/images/amd64-generic/R71-11117.0.2018_10_01_1008-a1/rootfs//build/rootfs.

Cleaning up /usr/local symlinks for /mnt/host/source/src/build/images/amd64-generic/R71-11117.0.2018_10_01_1008-a1/stateful/dev_image

An error occurred in your build so your latest output directory is invalid.

[Alex Klein]

Hey Keith,

We removed the [package]"as-is" license a few months ago. I don't have the amd64-generic board built right now to be able to tell you which ones it uses, but I believe that CL should be the full list of affected packages.

Simplest way to fix it should be to just rerun build_packages. That'll be less work on your end, but will take longer.

Otherwise, from your chroot, you should be able to run `portageq match /build/amd64-generic [package]` for each of the 6 packages in the CL (you can exclude the version, e.g. just "app-text/docbook-sgml-dtd" to cover the 8 different versions of that package). 

For each package that returns an atom, run `emerge-amd64-generic =[atom]` to reinstall it and the new license should take over.

-Alex

[Alex Klein]

I was wrong, there's 3 more packages in this CL.

[Daniel Erat]

I'm getting this error every time I try to build a ToT betty image using "./build_packages --board=betty && ./build_image --board=betty --noenable_rootfs_verification test".

...

INFO    : Generating license credits page. Time:

licenses: Unhandled exception:

Traceback (most recent call last):

  File "/mnt/host/source/chromite/licensing/licenses", line 169, in <module>

    DoMain()

  File "/mnt/host/source/chromite/licensing/licenses", line 165, in DoMain

    commandline.ScriptWrapperMain(FindTarget)

  File "/mnt/host/source/chromite/lib/commandline.py", line 912, in ScriptWrapperMain

    ret = target(argv[1:])

  File "/mnt/host/source/chromite/licensing/licenses.py", line 224, in main

    licensing.GenerateHTMLLicenseOutput(opts.output)

  File "/mnt/host/source/chromite/licensing/licenses_lib.py", line 1080, in GenerateHTMLLicenseOutput

    self._GeneratePackageLicenseText(pkg)

  File "/mnt/host/source/chromite/licensing/licenses_lib.py", line 1018, in _GeneratePackageLicenseText

    license_type = self.FindLicenseType(sln, board=self.board)

  File "/mnt/host/source/chromite/licensing/licenses_lib.py", line 968, in FindLicenseType

    after fixing the license.""" % (license_name, '\n'.join(set(stock + custom))))

AssertionError:

license as-is could not be found in /mnt/host/source/src/private-overlays/overlay-betty-private/licenses

/mnt/host/source/src/private-overlays/chromeos-overlay/licenses

/mnt/host/source/src/third_party/portage-stable/licenses

/mnt/host/source/src/third_party/eclass-overlay/licenses

/mnt/host/source/src/private-overlays/overlay-amd64-generic-cheets-private/licenses

/mnt/host/source/src/third_party/chromiumos-overlay/licenses

/mnt/host/source/src/overlays/overlay-amd64-generic/licenses

/mnt/host/source/src/private-overlays/project-cheets-private/licenses

/mnt/host/source/src/private-overlays/chromeos-partner-overlay/licenses

If the license in the ebuild is correct,

a) a stock license should be added to portage-stable/licenses :

running `cros_portage_upgrade` inside of the chroot should clone this repo

to /tmp/portage/:

https://chromium.googlesource.com/chromiumos/overlays/portage/+/gentoo

find the new licenses under licenses, and add them to portage-stable/licenses

b) if it's a non gentoo package with a custom license, you can copy that license

to third_party/chromiumos-overlay/licenses/

Try re-running the script with -p cat/package-ver --generate

after fixing the license.

INFO    : Unmounting image from /mnt/host/source/src/build/images/betty/R71-11121.0.2018_10_02_1753-a1/stateful and /mnt/host/source/src/build/images/betty/R71-11121.0.2018_10_02_1753-a1/rootfs

INFO    : Unmounting temporary rootfs /mnt/host/source/src/build/images/betty/R71-11121.0.2018_10_02_1753-a1/rootfs//build/rootfs.

Cleaning up /usr/local symlinks for /mnt/host/source/src/build/images/betty/R71-11121.0.2018_10_02_1753-a1/stateful/dev_image

An error occurred in your build so your latest output directory is invalid.

Would you like to delete the output directory (y/N)?

Running build_packages doesn't seem to be helping here. I also ran "emerge-betty -C docbook-dsssl-stylesheets docbook-sgml-dtd docbook-xml-dtd smdk-dltool dnspython diffstat", but that didn't help either.

If I'm not missing it, why doesn't the licenses script print the name of the package(s) that requested the as-is license? That seems important for debugging the issue.

And why is this a fatal error for my local build? Could we instead disregard failures outside of the builders?

--
--
Chromium OS Developers mailing list: chromiu...@chromium.org
View archives, change email options, or unsubscribe:
https://groups.google.com/a/chromium.org/group/chromium-os-dev

[Mike Frysinger]

removing the packages wouldn't help if you have local binpkgs still using the old license.  you'd need to force them to be rebuilt.

we can force a revbump on the packages that had their LICENSE value changed in-place so they make it out to developers.

or you can manually run `emerge xxx` on the package to rebuild it.

you can find the packages locally by running:

  find /var/db/pkg/ /build/*/var/db/pkg/ -name LICENSE -exec grep -l as-is {} +

as for improving the output, i've filed crbug.com/891812

as for making local builds non-fatal, that's a bad idea imo.  then developers will complain that CLs fail on bots but can't be reproduced locally, and people building their own OS images will have bad licensing coverage in their spins.

-mike

[Daniel Erat]

The command you gave listed the following files in /build/betty:

/build/betty/var/db/pkg/app-text/docbook-xml-dtd-4.1.2-r6/LICENSE

/build/betty/var/db/pkg/app-text/docbook-xml-dtd-4.2-r2/LICENSE

/build/betty/var/db/pkg/app-text/docbook-xml-dtd-4.5/LICENSE

/build/betty/var/db/pkg/net-wireless/crda-1.1.1-r7/LICENSE

I ran "emerge-betty docbook-xml-dtd net-wireless/crda" successfully but I still get the same license error as before. I see this now:

$ find /var/db/pkg/ /build/*/var/db/pkg/ -name LICENSE -exec grep -l as-is {} + | grep betty

/build/betty/var/db/pkg/app-text/docbook-xml-dtd-4.1.2-r6/LICENSE

/build/betty/var/db/pkg/app-text/docbook-xml-dtd-4.2-r2/LICENSE

[Mike Frysinger]

those are multiple SLOTs so you have to do them explicitly

prob this would fix it up:

emerge-betty `qlist -ICS docbook-xml-dtd` --jobs 32

-mike

[Daniel Erat]

Thanks, that fixed it.