Reliability and build time improvements with apt-cacher-ng

[MOESSBAUER, Felix]

Dear ISAR users,

I just played around with apt-cacher-ng to circumvent the throttling
issues on snapshots.d.o, and got some impressive results:

First of all, it is easy to configure ISAR to use the apt-cacher-ng, by
just using the DISTRO_APT_PREMIRRORS:

DISTRO_APT_SNAPSHOT_PREMIRROR = "deb.debian.org/(.*)
localhost:3142/snapshot.debian.org/archive/\1/${APT_SNAPSHOT_DATE}/\n"

Then, an apt-cacher-ng instance needs to be started on the host.
When running inside the kas container, just use the external host IP,
or forward port 3142 into the container. While the build is running,
you can monitor the caching statistics under
http://localhost:3142/acng-report.html.

Second, the cache directory can - in theory - be cached in a CI as
well, so subsequent jobs can use it. Then, even less load is applied
onto snapshots.d.o. Even when using standard debian mirrors, this helps
as usually all ISAR build jobs run in parallel, hence do not share
already downloaded packages via ISARs internal cache.

Currently I'm playing around with adding a built-in apt-cacher-ng into
kas and the kas-container. By that, all users can instantly profit from
the speedup. For CI systems, an ambient caching service (e.g. via
gitlabs "services") would probably make more sense, but is also more
complicated to configure.

I'm happy to hear your thoughts.

Best regards,
Felix Moessbauer

--
Siemens AG, Technology
Linux Expert Center