Bug#1030009: www.debian.org: addition of non-free-firmware for bookworm and higher

[Cyril Brulebois]

Package: www.debian.org
Severity: important

Hi,

This was mentioned a few days ago on debian-project@ by Gunnar:
https://lists.debian.org/debian-project/2023/01/msg00018.html

but much work has happened since then, with debian-installer components
having been extended to leverage this new component. The next release
of the installer should happen within the next two weeks.

I'm also trying to get documentation ready, and you can read more about
it in two merge requests (freshly opened, not reviewed yet):
- https://salsa.debian.org/installer-team/installation-guide/-/merge_requests/23
- https://salsa.debian.org/ddp-team/release-notes/-/merge_requests/138

Unless you fancy reviewing what changed in the installer, I'd skip the
former and concentrate on the latter, which briefly explains why
non-free-firmware was added.

It's available for bookworm suites and above (might be for other suites,
but I don't expect packages to show there).

TL;DR:

- bullseye systems are likely to have a sources.list configured this
way:

deb http://deb.debian.org/debian bullseye main contrib non-free

- bullseye systems getting upgraded to bookworm should be modified with
the following configuration (to make sure non-free firmware packages
that might be installed have a chance to get upgraded):

deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware

- new bookworm systems are likely to be configured with either of
these (unless one picks an expert install, enabling contrib and/or
non-free explicitly):

deb http://deb.debian.org/debian bookworm main
deb http://deb.debian.org/debian bookworm main non-free-firmware


There's also some related ongoing work, about adding some hints in apt
(linking back to the release notes), and about fixing command-not-found
in bullseye:
- https://salsa.debian.org/apt-team/apt/-/merge_requests/282
- https://bugs.debian.org/1029803


Cheers,
--
Cyril Brulebois (ki...@debian.org) <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

[Laura Arjona Reina]

Thanks Cyril for creating this bug and all the work you're doing related
to implement the decisions of the general resolution about
non-free-firmware.

I have created 2 branches in the webwml repo:

1.- branch GR-non-free-firmware
Where I plan to work on reflecting the changes that we should do in the
website on this topic

2.- branch bookworm
to prepare the changes for the website related to bookworm release that
at not related with non-free-firmware (e.g. website banner, the usual
changes we do for releases [1], etc)

Both of them are empty for now, but I plan to put some work in the
following days, and encourage everybody to help too.

[1] I think it's not well documented but I usually use these 2 wiki page
as reference:
https://wiki.debian.org/Teams/Publicity/ReleasePointAnnouncements
https://wiki.debian.org/Teams/ReleaseTeam/ReleaseCheckList/BookwormCheckList

Kind regards,

--
Laura Arjona Reina
https://wiki.debian.org/LauraArjona

El 30/1/23 a las 7:21, Cyril Brulebois escribió:

[Laura Arjona Reina]

Hello again

I have made a list of website files under /english, mentioning nonfree
or non-free. Not all of them may need updates but I think we should
review them all to be sure.

I have created a pad to ease the collaborative work of reviewing files:

https://storm.debian.net/shared/Daxsx1pQoBV0v1aPODgReIShC_qnpevLdvXNwQewADa

Shortened URL: https://deb.li/1030009

I'm pasting the list of files below for reference.

I still didn't start to review any file, and some of them I think other
teams can help (CDVendors, internationalization scripts...) I'll CC them
when I get to those files and post here (in the bug report) my
proposals, with commits to the GR-non-free-firmware branch.

--------
List of files that need review/update
Relevant files mentioning nonfree or non-free

blends/index.wml
Bugs/server-request.wml

CD/faq/index.wml
CD/http-ftp/index.wml
CD/live/index.wml
CD/netinst/index.wml
CD/torrent-cd/index.wml

devel/debian-installer/images.data
devel/debian-installer/index.wml

devel/join/nm-step3.wml

distrib/archive.wml
distrib/index.wml
distrib/netinst.wml
distrib/search_packages-form.inc

download.wml
intro/about.wml
intro/philosophy.wml
legal/cryptoinmain.wml
legal/licenses/index.wml

releases/bookworm/debian-installer/index.wml
releases/bookworm/errata.wml

releases/trixie/debian-installer/index.wml
releases/trixie/errata.wml

releases/bullseye/debian-installer/index.wml
releases/bullseye/errata.wml
releases/buster/debian-installer/index.wml
releases/buster/errata.wml
(etc, similar for former releases)

releases/proposed-updates.wml

security/faq.wml
security/index.wml

template/debian/legal_tags.wml
template/debian/release_images.wml
template/debian/release_info.wml

CD/vendors/debiancd.db
CD/vendors/getvendors.pl

international/l10n/data/synonyms
international/l10n/ddtp.wml
international/l10n/po4a/pot.wml
international/l10n/po-debconf/errors-by-pkg.inc
international/l10n/po-debconf/ok.inc
international/l10n/po-debconf/pot.wml
international/l10n/po-debconf/todo.inc
international/l10n/po-debconf/untranslated.inc
international/l10n/po/ok.inc
international/l10n/po/pot.wml
international/l10n/po/todo.inc
international/l10n/po/todo.wml
international/l10n/po/untranslated.inc
international/l10n/README
international/l10n/scripts/fix-files.sh
international/l10n/scripts/gen-files.pl
international/l10n/scripts/transmonitor-check

mirror/mirror_list.pl
----

Kind regards

[Cyril Brulebois]

Cyril Brulebois <ki...@debian.org> (2023-01-30):

> I'm also trying to get documentation ready, and you can read more about
> it in two merge requests (freshly opened, not reviewed yet):
> - https://salsa.debian.org/installer-team/installation-guide/-/merge_requests/23
> - https://salsa.debian.org/ddp-team/release-notes/-/merge_requests/138

Extra modification on the cdimage.debian.org side (that was alluded to
in the installation guide merge request), firmware material is now
available from:
https://cdimage.debian.org/cdimage/firmware/

in addition to the old location:
https://cdimage.debian.org/cdimage/unofficial/non-free/firmware/

Server-side, there's a symlink, meaning both locations serve the same
files, e.g. for all distributions (i.e. not just bookworm and above in
the new location).

That means there's no absolute need to update that URL, but it would
be nicer to do so. :)

[Computer Enthusiastic]

Hello,

What about search scripts (e.g. https://packages.debian.org/search ) and
in https://www.debian.org/distrib/packages ?

[Holger Wansing]

Hi Laura, hi all,

Laura Arjona Reina <lar...@debian.org> wrote (Sun, 5 Feb 2023 07:48:45 +0100):
> https://storm.debian.net/shared/Daxsx1pQoBV0v1aPODgReIShC_qnpevLdvXNwQewADa
>
> Shortened URL: https://deb.li/1030009
>
> I'm pasting the list of files below for reference.
>
> I still didn't start to review any file, and some of them I think other
> teams can help (CDVendors, internationalization scripts...) I'll CC them
> when I get to those files and post here (in the bug report) my
> proposals, with commits to the GR-non-free-firmware branch.

I have already updated several of files you listed (those which changings
were most likely non-debatable), but now I would like to discuss, what to
do with files like

releases/bullseye/debian-installer/index.wml
releases/bullseye/errata.wml
releases/buster/debian-installer/index.wml
releases/buster/errata.wml

Those old release pages have a warning hint like
<quote>
If any of the hardware in your system requires non-free firmware to be
loaded with the device driver, you can use one of the tarballs of common
firmware packages or download an unofficial image including these non-free
firmwares. Instructions how to use the tarballs and general information
about loading firmware during an installation can be found in the
Installation Guide.

unofficial images with firmware included - daily builds

unofficial images with firmware included - weekly builds
</quote>

First I thought we can leave this completely untouched, since these
"images with firmware included" are still there and they are also still
useful (for some hardware at least), if one wants to install those old
releases.
But now I would like to propose, we change the "unofficial images with
firmware included" phrase, since they are no longer *unofficial* according
to the non-free-firmware GR.
Thus I propose to change this into "special images with firmware included"
for those old releases.

Comments?

Holger



--
Holger Wansing <hwan...@mailbox.org>
PGP-Fingerprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076

[Cyril Brulebois]

cc += debian-cd

Holger Wansing <hwan...@mailbox.org> (2023-02-11):

I don't think we should change anything for buster and bullseye. We
aren't going to produce official images with main and non-free-firmware,
and I don't think the current implementation of those unofficial builds
(using non-free) would match the spirit of the GR.

Maybe people feel differently, but the guiding principle behind all the
changes I've driven so far is: the official status is for bookworm and
later (same story as for packages in non-free-firmware, even if that
component popped up in earlier suites in the archive).

[Steve McIntyre]

On Sat, Feb 11, 2023 at 09:07:44PM +0100, Holger Wansing wrote:
>Hi Laura, hi all,
>
>Laura Arjona Reina <lar...@debian.org> wrote (Sun, 5 Feb 2023 07:48:45 +0100):
>> https://storm.debian.net/shared/Daxsx1pQoBV0v1aPODgReIShC_qnpevLdvXNwQewADa
>>
>> Shortened URL: https://deb.li/1030009
>>
>> I'm pasting the list of files below for reference.
>>
>> I still didn't start to review any file, and some of them I think other
>> teams can help (CDVendors, internationalization scripts...) I'll CC them
>> when I get to those files and post here (in the bug report) my
>> proposals, with commits to the GR-non-free-firmware branch.
>

ACK, that's cool.

Just leave the docs for older releases alone - they'll still continue
to follow the older configuration.

--
Steve McIntyre, Cambridge, UK. st...@einval.com
"You can't barbecue lettuce!" -- Ellie Crane

[Laura Arjona Reina]

Hello
I have reviewed Holger's proposals in the branch GR-non-free-firmware (thanks a
lot!) and added updates to some other files, and updated the
https://deb.li/1030009 pad

The current status, as I see, is the following:

* I agree with all the files reviewed with comment that don't need update
* I agree with all the update proposals that are currently in the branch. The
files releases/bookworm/errata.wml and devel/debian-installer/index.wml are
already updated in the master branch so when merging we should discard changes
in the GR-non-free-firmware affecting these files and keep the ones in master.
* I agree that we shouldn't change Bugs/server-request.wml without checking
first with BTS team if the BTS already knows about non-free-firmware. So leaving
this for "after the Debian 12 release".
* No idea if packages.debian.org already knows and allows to search about
non-free-firmware but we can incorporate the updates to our files as Computer
Enthusiastic (in CC) suggested and Holger committed, and then after the release
we can try to help to update packages.d.o if needed.
* The scripts that produce the data for the l10n dashboards would need update so
I'm leaving international/l10n aside for now, I just added non-free-firmware in
the synonyms list, and to the ddtp.wml file that says that non-free is not
handled due to possible license issues ( adding here non-free-firmware does not
harm, so I added it).
* It's been suggested that we may need to update intro/why_debian.wml to reflect
the changes in the GR about non-free-firmware but I have reviewed the file and I
don't find the reason why whe should update it.
* It's been suggested that we change the links
https://cdimage.debian.org/cdimage/unofficial/non-free/firmware/ for
https://cdimage.debian.org/cdimage/firmware/ (starting with bookworm) but since
currently both URLs serve the same files, I'd would do it after the branch is
merged, since many links are going away in the merge.

So, in summary, I think we can merge the branch into master when we see suitable
(1) and just need to be careful about not overwriting the files
releases/bookworm/errata.wml and devel/debian-installer/index.wml.

(1) when we see suitable: I guess right after the Debian 12 images are ready for
download as "stable" release, but in fact I'm not sure how to handle this and
the translations. I am available to merge the branch this weekend when the
website is updated to the release, and can try to update the languages that have
no active translator these days. For the rest, if nobody proposes better way, I
will send a mail to debian-i18n and debian-www explaining that this branch will
be merged during the weekend so people are prepared to do their changes in the
corresponding language folder (if they choose to advance work in the branch, it
will be merged and I'll try to update the commit-hashes later).

Kind regards,