test suite

[Ian Zimmerman]

I wonder if there is a test suite to verify shell POSIX conformance;
especially wrt corner cases such as "$@".

If there is, a secondary question is of course how many shells actually
pass it :-P

--
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
Do obvious transformation on domain to reply privately _only_ on Usenet.

[Lew Pitcher]

Ian Zimmerman wrote:

> I wonder if there is a test suite to verify shell POSIX conformance;
> especially wrt corner cases such as "$@".

Yes, there is. It is administered by the IEEE in conjunction with The Open
Group, and is part of the general POSIX certification test suite.
http://get.posixcertified.ieee.org/certification_guide.html

> If there is, a secondary question is of course how many shells actually
> pass it :-P

That, I do not know.
--
Lew Pitcher
"In Skills, We Trust"
PGP public key available upon request

[Geoff Clare]

Lew Pitcher wrote:

> Ian Zimmerman wrote:
>
>> I wonder if there is a test suite to verify shell POSIX conformance;
>> especially wrt corner cases such as "$@".
>
> Yes, there is. It is administered by the IEEE in conjunction with The Open
> Group, and is part of the general POSIX certification test suite.
> http://get.posixcertified.ieee.org/certification_guide.html
>
>> If there is, a secondary question is of course how many shells actually
>> pass it :-P
>
> That, I do not know.

A partial answer could be had by looking up which systems have been
certified as passing the test suite, and then working out which shell
they use as their "POSIX sh". Of course, this wouldn't include any
shells that pass the test suite but are not used as the POSIX sh on
a certified system.

Lists of certified systems are at:

POSIX: http://get.posixcertified.ieee.org/register.html
(which is an empty list for our purposes, as there aren't any shell
and utility certifications, only system interfaces (i.e. functions
and headers)).

UNIX 03: https://www.opengroup.org/openbrand/register/cx.htm
UNIX 98: https://www.opengroup.org/openbrand/register/cw.htm
UNIX 95: https://www.opengroup.org/openbrand/register/cv.htm

My guess is that all of the listed certified UNIX systems use either
a "modified ksh88" or bash as their POSIX sh. (Solaris 11 has ksh93
as /bin/sh but its "POSIX sh" is /usr/xpg4/bin/sh which is still a
modified ksh88.)

It's also worth noting that nobody has certified to UNIX V7 yet, which
is the certification for the current standard (SUSv4/POSIX.1-2008), and
there have been quite a lot of clarifications related to shell behaviour
since SUSv3/POSIX.1-2001. So it's possible no shell passes the latest
tests.

--
Geoff Clare <net...@gclare.org.uk>

[Lew Pitcher]

Geoff Clare wrote:

> Lew Pitcher wrote:
>
>> Ian Zimmerman wrote:
>>
>>> I wonder if there is a test suite to verify shell POSIX conformance;
>>> especially wrt corner cases such as "$@".
>>
>> Yes, there is. It is administered by the IEEE in conjunction with The
>> Open Group, and is part of the general POSIX certification test suite.
>> http://get.posixcertified.ieee.org/certification_guide.html
>>
>>> If there is, a secondary question is of course how many shells actually
>>> pass it :-P
>>
>> That, I do not know.
>
> A partial answer could be had by looking up which systems have been
> certified as passing the test suite, and then working out which shell
> they use as their "POSIX sh". Of course, this wouldn't include any
> shells that pass the test suite but are not used as the POSIX sh on
> a certified system.
>
> Lists of certified systems are at:
>
> POSIX: http://get.posixcertified.ieee.org/register.html
> (which is an empty list for our purposes, as there aren't any shell
> and utility certifications,

Seemingly, there /are/ shell and utility certifications, but only as part of
a "Base System"

http://get.posixcertified.ieee.org/docs/base-2016.html
This Product Standard is for operating system environments providing
services that conform to IEEE Std 1003.1, 2016 Edition1. This supports
applications portability at the source code level and includes the
provision of a standard operating system interface and environment,
including a command interpreter (or "shell"), and common utility programs.

This Base Product Standard covers only the required utilities, functions
and facilities defined within IEEE Std 1003.1. It does not cover the
options within IEEE Std 1003.1 or subprofiling2.

CONFORMANCE REQUIREMENTS

A single configuration of the system shall meet all of the conformance
requirements defined in the following mandatory Product Standards:

1003.1™-2016 System Interfaces
1003.1™-2016 Shell and Utilities
...

And, 1003.1 leads to...

http://get.posixcertified.ieee.org/docs/su-2016.html

"This Product Standard is for operating system environments providing
services that conform to the Shell and Utilities Volume of IEEE Std 1003.1,
2016 Edition1. This supports applications portability at the source code
level and includes the provision of a command interpreter (or "shell"), and
common utility programs."

> only system interfaces (i.e. functions and headers)).
>
> UNIX 03: https://www.opengroup.org/openbrand/register/cx.htm
> UNIX 98: https://www.opengroup.org/openbrand/register/cw.htm
> UNIX 95: https://www.opengroup.org/openbrand/register/cv.htm
>
> My guess is that all of the listed certified UNIX systems use either
> a "modified ksh88" or bash as their POSIX sh. (Solaris 11 has ksh93
> as /bin/sh but its "POSIX sh" is /usr/xpg4/bin/sh which is still a
> modified ksh88.)
>
> It's also worth noting that nobody has certified to UNIX V7 yet, which
> is the certification for the current standard (SUSv4/POSIX.1-2008), and
> there have been quite a lot of clarifications related to shell behaviour
> since SUSv3/POSIX.1-2001. So it's possible no shell passes the latest
> tests.
>


--