Bug#1015781: autopkgtests fail if user foo exists or existed

[Marc Haber]

Package: adduser
Version: 3.122
Severity: normal

The tests expect the test environment to be fresh. They relay too much
on uids being the same and do not preseed adduser to provide
reproducible uids.

Reproducer:
- create test chroot
- adduser --system foo
- deluser foo
- verify that foo is actually gone
- run autopkgtests
- see not just one test failing.

You get other failures if you don't delete the foo user prior to running
the test.

Greetings
Marc

[Matt Barry]

On Thu, 2022-07-21 at 08:27 +0200, Marc Haber wrote:
> Package: adduser
> Version: 3.122
> Severity: normal
>
> The tests expect the test environment to be fresh. They relay too
> much
> on uids being the same and do not preseed adduser to provide
> reproducible uids.
>
> Reproducer:
> - create test chroot
> - adduser --system foo
> - deluser foo
> - verify that foo is actually gone
> - run autopkgtests
> - see not just one test failing.

I have seen similar cases where running 'autopkgtest -- null' more than
inside a debspawn session fails; it ought to be idempotent.

mb

[Jason Franklin]

On Thu, Jul 21, 2022 at 08:27:15AM +0200, Marc Haber wrote:
> The tests expect the test environment to be fresh.

When I wrote the initial autopkgtests, my thought was that this was the
entire design idea of the system.

Each stanza in the debian/tests/control file results in the
instantiation of a new chroot or container before its named collection
of tests is run.

The tests in each stanza, of course, use the same environment. That was
the reasoning behind saving and restoring /etc/passwd and friends in the
Perl BEGIN and END blocks of each test script. Not perfect, but it
speeds up the tests.

> They relay too much on uids being the same and do not preseed adduser
> to provide reproducible uids.

I can understand that this is a problem.

If new Debian system users or groups are added to the base files, or if
default ID ranges change, we have an issue when specific ID numbers are
relied upon.

This was why my tests mainly picked usernames and not ID numbers.

If Debian ever puts the user "foo" in the base system, we'll need to do
a find and replace to change "foo" to something else. ID numbers will be
harder to change, so they should be avoided or selected dynamically.

> Reproducer:
> - create test chroot
> - adduser --system foo
> - deluser foo
> - verify that foo is actually gone
> - run autopkgtests
> - see not just one test failing.

This seems like it shouldn't fail. However, there are probably side
effects here that the tests handle that aren't handled by doing this
interactively first.

In principle, I think that picking a set of canonical usernames to work
with in tests should be fine. Picking fixed ID numbers is probably not
fine.

--
Jason Franklin

[Marc Haber]

On Thu, Jul 21, 2022 at 08:37:32AM -0400, Jason Franklin wrote:
> On Thu, Jul 21, 2022 at 08:27:15AM +0200, Marc Haber wrote:
> > The tests expect the test environment to be fresh.
>
> When I wrote the initial autopkgtests, my thought was that this was the
> entire design idea of the system.

I think it is, but test environments might be recycled instead of
rebuilding them, and stupid maintainers (=me) might check whether
adduser is operational at all by using adduser --system foo before
invoking autopkgtests.

> Each stanza in the debian/tests/control file results in the
> instantiation of a new chroot or container before its named collection
> of tests is run.

I don't think that is true if autopkgtest ish run with the null
virtualization method. I didn't automate that yet, so I log in to a new
container, invoke autopkgtest . -- null and then ditch the container. I
think in that case all stanzas in the control file run in the same
container, but I might be wrong.

> > They relay too much on uids being the same and do not preseed adduser
> > to provide reproducible uids.
>
> I can understand that this is a problem.

Not a problem that needs to be worked on urgently.

> If new Debian system users or groups are added to the base files, or if
> default ID ranges change, we have an issue when specific ID numbers are
> relied upon.
>
> This was why my tests mainly picked usernames and not ID numbers.
>
> If Debian ever puts the user "foo" in the base system, we'll need to do
> a find and replace to change "foo" to something else. ID numbers will be
> harder to change, so they should be avoided or selected dynamically.

I'd fist migrate away from those easy user names like "test2a" and "foo"
and use something less likely to conflict such as "auapt-test2a"
(AddUser AutoPkgTest).

Second, we could configure adduser for the tests to use 900-999 instead
of 100-999 for system users and 50000-59999 for normal users, reducing
the chance to have UID clashes.

Third, we're allowed to break the environment, so we could just remove
all users occupying "our" names and "our" UIDs before running the tests.

Gee, this is going to be fun once adduser starts growing a memory and
not reassigning UIDs that have ever been used.

Greetings
Marc

--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421