Future of /usr/bin/which in Debian?

[Boyuan Yang]

Hi all,

I just noticed that the "which" command provided by debianutils has been
declared as deprecated in Debian Sid:

% /usr/bin/which test
/usr/bin/which: this version of 'which' is deprecated and should not be used.
/usr/bin/test

Obviously it was caused by the upload of debianutils/5.0-1 onto Unstable (see
https://salsa.debian.org/debian/debianutils/-/commit/3a8dd10b4502f7bae8fc6973c13ce23fc9da7efb
):

--------------------------------------
.SH DEPRECATION
Since type and command \-v were mandated by POSIX, this utility is no longer
useful for maintainer scripts and thus will be removed from debianutils.
--------------------------------------

I did some search and found related discussion thread at
https://lists.debian.org/debian-devel/2020/08/msg00081.html , which looks
pretty reasonable.

Now I'm just wondering what would be the good alternative that provides
/usr/bin/which. Obviously we had a round of discussion already (see
https://lists.debian.org/debian-devel/2020/08/msg00340.html ), but probably we
need a solution this time following debianutils which's deprecation. The most
reasonable choice might be GNU which anyway.

Besides, will the new "which" tool be installed in Debian by default? Since
debianutils is Essential:yes, not providing "which" tool by default could
probably break some existing packages.

Thanks,
Boyuan Yang

[Clint Adams]

On Tue, Aug 17, 2021 at 09:15:24PM -0400, Boyuan Yang wrote:
> Besides, will the new "which" tool be installed in Debian by default? Since
> debianutils is Essential:yes, not providing "which" tool by default could
> probably break some existing packages.

My personal opinion is that no one should be using `which` in maintainer
scripts and that no one should be using `which` in an interactive shell
unless it is a builtin of that shell.

There are a ton of postinst scripts relying on `which` right
now, which makes sense because `type` and `command -v` used to be
optional extensions to POSIX and not guaranteed to be provided
by /bin/sh.

[Gunnar Wolf]

Clint Adams dijo [Wed, Aug 18, 2021 at 04:20:02AM +0000]:

I agree with you, maintainer scripts should not rely on 'which'
anymore. However, what about users? 'which' is a standard Unix tool
since forever, and I expect many users to experience head scratching
when told it's not cool to use it anymore.

[Andrey Rahmatullin]

On Wed, Aug 18, 2021 at 12:36:43PM -0500, Gunnar Wolf wrote:
> I agree with you, maintainer scripts should not rely on 'which'
> anymore. However, what about users? 'which' is a standard Unix tool
> since forever, and I expect many users to experience head scratching
> when told it's not cool to use it anymore.

Why is a standard Unix tool since forever shipped by a package called
debianutils, that has a description saying "Miscellaneous utilities
specific to Debian" and "used primarily by the installation scripts of
Debian packages"? Surely a standard Unix tool would, or at least could, be
shipped somewhere else?

--
WBR, wRAR

[Clint Adams]

On Wed, Aug 18, 2021 at 08:50:04PM +0200, Ben Hutchings wrote:
> 3BSD first implemented 'which':
> <https://minnie.tuhs.org/cgi-bin/utree.pl?file=3BSD/usr/man/man1/which.1>
> (1979). This was csh-specific and I think it later became built-into
> csh.
>
> FreeBSD 2.1 introduced a new non-built-in implementation:
> <https://www.freebsd.org/cgi/man.cgi?query=which&apropos=0&sektion=0&manpath=FreeBSD+2.1.0-RELEASE&arch=default&format=html>
> (1995). This was written in Perl and then rewritten in C a few years
> later.
>
> Debian's implementation started out in 1995 or 1996 as a shell script
> calling 'type', and remains a shell script.
>
> Red Hat has yet another implementation:
> <https://www.freebsd.org/cgi/man.cgi?query=which&apropos=0&sektion=0&manpath=CentOS+7.1&arch=default&format=html>,
> <https://carlowood.github.io/which/> (1999 or earlier). This is
> written in C.
>
> So 'which' has a very long history in csh, which was the default
> interactive shell in many versions of Unix. Its availability to other
> shells dates back to the 90s (at least) but is not portable due to
> there being multiple very different implementations.

I'm happy to transition /usr/bin/which to alternatives if people are
interested in packaging all of these.

[Ben Hutchings]

3BSD first implemented 'which':
<https://minnie.tuhs.org/cgi-bin/utree.pl?file=3BSD/usr/man/man1/which.1>
(1979). This was csh-specific and I think it later became built-into
csh.

FreeBSD 2.1 introduced a new non-built-in implementation:
<https://www.freebsd.org/cgi/man.cgi?query=which&apropos=0&sektion=0&manpath=FreeBSD+2.1.0-RELEASE&arch=default&format=html>
(1995). This was written in Perl and then rewritten in C a few years
later.

Debian's implementation started out in 1995 or 1996 as a shell script
calling 'type', and remains a shell script.

Red Hat has yet another implementation:
<https://www.freebsd.org/cgi/man.cgi?query=which&apropos=0&sektion=0&manpath=CentOS+7.1&arch=default&format=html>,
<https://carlowood.github.io/which/> (1999 or earlier). This is
written in C.

So 'which' has a very long history in csh, which was the default
interactive shell in many versions of Unix. Its availability to other
shells dates back to the 90s (at least) but is not portable due to
there being multiple very different implementations.

Ben.

--
Ben Hutchings
[W]e found...that it wasn't as easy to get programs right as we had
thought. I realized that a large part of my life from then on was going
to be spent in finding mistakes in my own programs.
- Maurice Wilkes, 1949

[Geert Stappers]

On Wed, Aug 18, 2021 at 06:53:45PM +0000, Clint Adams wrote:
>
> I'm happy to transition /usr/bin/which to alternatives

Which alternatives would that be?


| $ apt show alternatives
| N: Unable to locate package alternatives
| $

> if people are interested in packaging all of these.
>

Groeten
Geert Stappers
--
Silence is hard to parse

[Clint Adams]

On Wed, Aug 18, 2021 at 09:48:29PM +0200, Geert Stappers wrote:
> Which alternatives would that be?

I meant

update-alternatives --install /usr/bin/which which /usr/bin/which.debianutils 0

in postinst and so on so that FreeBSD which and GNU which and friends could
take over.

[Geert Stappers]

On Wed, Aug 18, 2021 at 07:56:05PM +0000, Clint Adams wrote:
> On Wed, Aug 18, 2021 at 09:48:29PM +0200, Geert Stappers wrote:

> } } I'm happy to transition /usr/bin/which to alternatives

> > Which alternatives would that be?
>
> I meant
>
> update-alternatives --install /usr/bin/which which /usr/bin/which.debianutils 0
>
> in postinst and so on so that FreeBSD which and GNU which and friends could
> take over.

Please do. Make such take over possible.
It is what https://salsa.debian.org/debian/debianutils/-/merge_requests/6#note_242172
is asking for.

[Clint Adams]

On Wed, Aug 18, 2021 at 10:59:08PM +0200, Geert Stappers wrote:
> Please do. Make such take over possible.
> It is what https://salsa.debian.org/debian/debianutils/-/merge_requests/6#note_242172
> is asking for.

Okay, you have it in debianutils 5.1-2.

[Colin Watson]

On Wed, Aug 18, 2021 at 08:50:04PM +0200, Ben Hutchings wrote:

> Debian's implementation started out in 1995 or 1996 as a shell script
> calling 'type', and remains a shell script.

Not very important historical note: it's true that Debian had a "which"
command from 1995/1996 or so, but the current implementation doesn't
descend from that at all.

I wrote the earliest version of the current implementation from scratch,
as part of a job I held from 2000 to 2003; in that job, I worked on lots
of different Unix flavours, and we had NFS-mounted home directories so I
wanted a reasonably cross-platform ~/.bashrc. The spectacular lack of
consistent behaviour of "which" across those platforms got in my way, so
I wrote a shell script that I could put in my ~/bin and use everywhere.
I don't remember exactly when I wrote it, but it can't have been before
2000 and I think it was probably around 2002.

I contributed that script to Debian in 2002 in response to
https://bugs.debian.org/94507, where it became clear that the previous
implementation wasn't really salvageable; Clint merged that in
debianutils 1.16.5. It's been extended since then, but still has the
same basic approach.

--
Colin Watson (he/him) [cjwa...@debian.org]

[Boyuan Yang]

在 2021-08-18星期三的 22:59 +0200，Geert Stappers写道：

So we will have https://salsa.debian.org/debian/which-gnu providing a binary
package with name "which". I will upload it to the NEW queue soon.

Still not sure about the the package's priority. Now I am using Priority:
optional, but we may raise it later if needed.

Thanks,
Boyuan Yang

[Paul Wise]

On Thu, Aug 19, 2021 at 9:17 PM Boyuan Yang wrote:

> So we will have https://salsa.debian.org/debian/which-gnu providing a binary
> package with name "which". I will upload it to the NEW queue soon.

Some folks on IRC wanted to package the FreeBSD which too, so I
suggest using which-gnu as the binary package name too.

--
bye,
pabs

https://wiki.debian.org/PaulWise

[YunQiang Su]

Paul Wise <pa...@debian.org> 于2021年8月20日周五 上午10:50写道：

>
> On Thu, Aug 19, 2021 at 9:17 PM Boyuan Yang wrote:
>
> > So we will have https://salsa.debian.org/debian/which-gnu providing a binary
> > package with name "which". I will upload it to the NEW queue soon.
>
> Some folks on IRC wanted to package the FreeBSD which too, so I
> suggest using which-gnu as the binary package name too.
>

For such a simple tool, do we really need such many versions?

> --
> bye,
> pabs
>
> https://wiki.debian.org/PaulWise
>

--
YunQiang Su

[Calum McConnell]

On Fri, 2021-08-20 at 11:03 +0800, YunQiang Su wrote:
> Paul Wise <pa...@debian.org> 于2021年8月20日周五 上午10:50写道：
> >
> > On Thu, Aug 19, 2021 at 9:17 PM Boyuan Yang wrote:
> >
> > > So we will have https://salsa.debian.org/debian/which-gnu providing
> > > a binary
> > > package with name "which". I will upload it to the NEW queue soon.
> >
> > Some folks on IRC wanted to package the FreeBSD which too, so I
> > suggest using which-gnu as the binary package name too.
> >
>
> For such a simple tool, do we really need such many versions?

Which which witches wish to which with will wildly wander. We wish
welcome to witches which which with weird whichs, which will want whiches
which witches who were wasted winds when we were whelks used. Which which
any witch uses is a decision whence the heart, which we wish to watch each
which make.

[Andrej Shadura]

Hi,

On Thu, 19 Aug 2021, at 23:17, Boyuan Yang wrote:
> 在 2021-08-18星期三的 22:59 +0200，Geert Stappers写道:

> > > /usr/bin/which.debianutils 0
> > >
> > > in postinst and so on so that FreeBSD which and GNU which and friends
> > > could
> > > take over.
> >
> > Please do.  Make such take over possible.
> > It is what
> > https://salsa.debian.org/debian/debianutils/-/merge_requests/6#note_242172
> > is asking for.
>
> So we will have https://salsa.debian.org/debian/which-gnu providing a binary
> package with name "which". I will upload it to the NEW queue soon.

I'd rather suggest the FreeBSD which, which I'm mirroring here:
https://salsa.debian.org/andrewsh/freebsd-which/

I think it's much simpler than the GNU one.

--
Cheers,
Andrej

[Boyuan Yang]

Hi,

在 2021-08-22星期日的 17:36 +0200，Andrej Shadura写道：

The GNU which package is now in NEW queue:
https://ftp-master.debian.org/new/gnu-which_2.21-1.html

Having both freebsd which and gnu which in Debian archive is definitely ok. If
you would like, please also upload freebsd-which onto unstable.

Thanks,
Boyuan Yang

[Sean Whitton]

Hello Boyuan,

It's okay, indeed, but please do consider NEW queue workload with things
like this -- upload it if you're sure it's going to get used, not just
for completeness.

--
Sean Whitton

[Wookey]

On 2021-08-27 16:48 -0400, Boyuan Yang wrote:
> Hi,

>
> The GNU which package is now in NEW queue:
> https://ftp-master.debian.org/new/gnu-which_2.21-1.html

Thanks for this. Missing which broke bazel (at least in tensorflow)
and the implementation made it fiddly to replace <command> with
<command+option> as recommended.

Have a plain which back is much easier than fixing the build mess.

I must admit that I have no idea why replacing such a longstanding
utility is deemed necessary.

Wookey
--
Principal hats: Linaro, Debian, Wookware, ARM
http://wookware.org/

[Clint Adams]

On Sun, Sep 19, 2021 at 05:32:04PM +0100, Wookey wrote:
> I must admit that I have no idea why replacing such a longstanding
> utility is deemed necessary.

Maybe this riddle will help.

Imagine that you are the product manager for Debian `which`. According
to the hatemail in my inbox, this is the most important utility in the
history of time, such that even printing a warning to stderr causes
global devastation, block hints, and calls for impeachment. So, it
makes sense for this to be a full-time job, though perhaps you manage
another, less significant utility as well.

You go on a Gemba walk, and discover you have several user personas
amongst your customers:

(a) wants GNU `which`, to have feature parity with Red Hat Enterprise Linux
(b) wants FreeBSD `which`, to have feature parity with FreeBSD
(c) wants nothing ever to change, and the xiafs removal from Linux 2.1.21
to be reverted
(d) wants there to be exactly one version of `which` (except for all the
shell builtins) so that alternatives won't confuse and complicate things

Wearing your customer-centricity hat, what is the optimal set of
personas to unperson so that you can implement a solution that works
for everyone who still matters?

[Jonathan Dowland]

On Fri, Aug 20, 2021 at 11:03:50AM +0800, YunQiang Su wrote:
>For such a simple tool, do we really need such many versions?

At least you've asked the question. I'd love to think that there was a
proper evaluation of BSD which versus GNU which prior to the latter
being uploaded to NEW, and there's a compelling reason that the GNU one
was chosen; but if so there's no evidence of that on -devel. :(


--
Please do not CC me for listmail.

👱🏻 Jonathan Dowland
✎ jm...@debian.org
🔗 https://jmtd.net

[Wookey]

On 2021-09-20 04:48 +0000, Clint Adams wrote:
> On Sun, Sep 19, 2021 at 05:32:04PM +0100, Wookey wrote:
> > I must admit that I have no idea why replacing such a longstanding
> > utility is deemed necessary.
>
> Maybe this riddle will help.
>
> Imagine that you are the product manager for Debian `which`. According
> to the hatemail in my inbox, this is the most important utility in the
> history of time,

:-)

> such that even printing a warning to stderr causes
> global devastation, block hints, and calls for impeachment.

I do agree that it's pretty shoddy of bazel to break because a utility
several layers down issues stderr text whilst still returning a
success code. And yes I/someone should probably try to actually work
out why and fix it... But there is a lot of shit code in the world,
and only so many hours.

> You go on a Gemba walk,

Never heard of a Gemba walk before, so I learned something today :-)

> Wearing your customer-centricity hat, what is the optimal set of
> personas to unperson so that you can implement a solution that works
> for everyone who still matters?

I'm not fussy. Any of a,b,c or d would suit me fine. It's just e
(issue deprecation message), which caused me trouble.

Thanks for explaining what the actual problem is. You have my sympathy
trying to navigate the mess.

[Michael Stone]

On Mon, Sep 20, 2021 at 02:38:06PM +0100, Jonathan Dowland wrote:
>On Fri, Aug 20, 2021 at 11:03:50AM +0800, YunQiang Su wrote:
>>For such a simple tool, do we really need such many versions?
>
>At least you've asked the question. I'd love to think that there was a
>proper evaluation of BSD which versus GNU which prior to the latter
>being uploaded to NEW, and there's a compelling reason that the GNU one
>was chosen; but if so there's no evidence of that on -devel. :(

It seems to install to /usr/bin/which.gnu, implying that you could
upload /usr/bin/which.bsd if you so desire; what's the issue?

[Jonathan Dowland]

On Mon, Sep 20, 2021 at 11:02:49AM -0400, Michael Stone wrote:
>It seems to install to /usr/bin/which.gnu, implying that you could
>upload /usr/bin/which.bsd if you so desire; what's the issue?

I think we should have just one which implementation in the archive. We
should (have) pick(ed) the best one for Debian. I believe (perhaps
unfairly... I'd love to be proven wrong) that the GNU implementation was
uploaded very quickly, without the BSD implementation being considered.
Perhaps the GNU one is the best fit for our needs. It would have been
nice to see that there was an evaluation.

[Michael Stone]

On Tue, Sep 21, 2021 at 09:00:52AM +0100, Jonathan Dowland wrote:
>On Mon, Sep 20, 2021 at 11:02:49AM -0400, Michael Stone wrote:
>>It seems to install to /usr/bin/which.gnu, implying that you could
>>upload /usr/bin/which.bsd if you so desire; what's the issue?
>
>I think we should have just one which implementation in the archive. We
>should (have) pick(ed) the best one for Debian. I believe (perhaps
>unfairly... I'd love to be proven wrong) that the GNU implementation was
>uploaded very quickly, without the BSD implementation being considered.
>Perhaps the GNU one is the best fit for our needs. It would have been
>nice to see that there was an evaluation.

I think it doesn't matter how many which implementations are in debian.
If you want something with specific portable semantics, just use command
-v. The remaining consumers of which are either programs that (ipso
facto) don't care about semantic corner cases or are humans who want to
use which just because, and probably have strong opinions on how it
should behave (as, apparently, you do). We can't satisfy everybody with
one implementation, and I see no technical reason that we'd even try to
do so.

[Nicholas D Steeves]

+1 for everything above. I also think it may be more reasonable to
prefer (by default, using the alternatives mechanism) the more LSBish
one (in this case GNU) rather than the potentially more simple, clean,
and full-featured one (BSD). For precedent see netcat-traditional vs
netcat-openbsd, and GNU tar vs bsdtar and pax--particularly during the
time when bsdtar was superior (iirc) to GNU tar.

I also acknowledge that this may entrench the precedent of preferring a
GNU/Linux-standard solution that may not be the technically best. For
the record, I seem to remember using bsdtar for the period of time when
GNU tar had some issue or another (or was it a missing feature? Maybe
xattr, sparse file support, or acl-related?), and I also have a strong
personal preference for netcat-openbsd.

Thankfully we have the /etc/alternatives and Provides mechanisms to
affirm user choice in such cases, and I think most of us will agree this
is a totally equitable and reasonable compromise :-)

Regards,
Nicholas

[The Wanderer]

On 2021-09-21 at 16:16, Michael Stone wrote:

> On Tue, Sep 21, 2021 at 09:00:52AM +0100, Jonathan Dowland wrote:
>
>> On Mon, Sep 20, 2021 at 11:02:49AM -0400, Michael Stone wrote:
>>
>>> It seems to install to /usr/bin/which.gnu, implying that you
>>> could upload /usr/bin/which.bsd if you so desire; what's the
>>> issue?
>>
>> I think we should have just one which implementation in the
>> archive. We should (have) pick(ed) the best one for Debian. I
>> believe (perhaps unfairly... I'd love to be proven wrong) that the
>> GNU implementation was uploaded very quickly, without the BSD
>> implementation being considered. Perhaps the GNU one is the best
>> fit for our needs. It would have been nice to see that there was an
>> evaluation.
>
> I think it doesn't matter how many which implementations are in
> debian. If you want something with specific portable semantics, just
> use command -v.

I think I've seen that suggested a lot as an alternative for 'which',
but it doesn't seem to be comparably reliable in all contexts.

The primary issue I've run across to date is with aliases.

For example, on my computer as I type this:

$ which ls
/usr/bin/ls
$ command -v ls
alias ls='ls -N --color=auto'
$ $(which ls) /
bin home lib32 media pulse srv var
boot initrd.img lib64 mnt root sys vmlinuz
dev initrd.img.old libx32 opt run tmp vmlinuz.old
etc lib lost+found proc sbin usr
$ $(command -v ls) /
bash: alias: -N: not found
bash: alias: /: not found

And then 'ls' is broken in that shell session; I haven't yet found a way
to get it working again, short of exiting and re-launching the shell.
(Though I also haven't tried *terribly* hard.)

This seems to demonstrate that you can't safely just use 'command -v'
wherever you would otherwise use 'which'.

--
The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw

[Russ Allbery]

The Wanderer <wand...@fastmail.fm> writes:
> On 2021-09-21 at 16:16, Michael Stone wrote:

>> I think it doesn't matter how many which implementations are in
>> debian. If you want something with specific portable semantics, just
>> use command -v.

> I think I've seen that suggested a lot as an alternative for 'which',
> but it doesn't seem to be comparably reliable in all contexts.

I don't think the point is that command -v is a drop-in replacement for
which (it definitely is not). I think the point is that command -v is a
standardized, portable interface. If you want portable semantics, the
standardized command is command -v, but it doesn't do quite the same thing
in quite the same way. If you want which, you have to live with the fact
that it's not portable and different which implementations behave in
different ways.

--
Russ Allbery (r...@debian.org) <https://www.eyrie.org/~eagle/>

[Fabrice Bauzac-Stehly]

The Wanderer <wand...@fastmail.fm> writes:

> On 2021-09-21 at 16:16, Michael Stone wrote:

>> On Tue, Sep 21, 2021 at 09:00:52AM +0100, Jonathan Dowland wrote:

>> If you want something with specific portable semantics, just
>> use command -v.

> I think I've seen that suggested a lot as an alternative for 'which',
> but it doesn't seem to be comparably reliable in all contexts.
>
> The primary issue I've run across to date is with aliases.

I don't know if it is always the case, especially with bizarre zsh
setups, but on my computers, aliases are not defined in shell scripts,
only in interactive shells.

If it's an issue, a shell script can do this beforehand:

\unalias -a

See https://pubs.opengroup.org/onlinepubs/009695399/utilities/command.html

--
Fabrice Bauzac-Stehly
PGP 01EEACF8244E9C14B551C5256ADA5F189BD322B6

[Jonathan Dowland]

Michael Stone wrote:
>> I think it doesn't matter how many which implementations are in debian.
>> If you want something with specific portable semantics, just use command
>> -v. The remaining consumers of which are either programs that (ipso
>> facto) don't care about semantic corner cases or are humans who want to
>> use which just because, and probably have strong opinions on how it
>> should behave (as, apparently, you do).

*I* don't; in Clint's categorization¹ I fall into (d) "wants there to be

exactly one version of `which` (except for all the shell builtins) so

that alternatives won't confuse and complicate things". The point I've
tried to make (too clumsily I guess) is the process of choosing one
should not be shoot-from-the-hip: there should be some consideration as
to which `which` would be the best fit for Debian. I hadn't seen any
evidence of that, until,

On Tue, Sep 21, 2021 at 05:41:49PM -0400, Nicholas D Steeves wrote:
>I also think it may be more reasonable to prefer (by default, using the
>alternatives mechanism) the more LSBish one (in this case GNU) rather
>than the potentially more simple, clean, and full-featured one (BSD).

^ this is an example of exactly what I would have hoped took place to
decide upon which `which` we provided.

>Thankfully we have the /etc/alternatives and Provides mechanisms to
>affirm user choice in such cases, and I think most of us will agree this
>is a totally equitable and reasonable compromise :-)

Unless there's a really compelling reason for there to be more than one
`which`, we could avoid the burden of alternatives entirely.


I should get off my soapbox now.


¹ Message-ID: <YUgSrpfv...@scru.org>