ANN: Prolog conformance test suite

[Paulo Moura]

Hi,

Logtalk 3.00.2, released today, includes an extended Prolog conformance suite, now featuring 1513 unit tests for both official and de facto Prolog features. Special thanks to Joachim Schimpf and Jan Wielemaker for their feedback and test contributions.

The updated Prolog conformance suite notes can be found at:
https://github.com/LogtalkDotOrg/logtalk3/blob/master/tests/prolog/NOTES.md

The Prolog conformance suite changes w.r.t. the previous stable release can be found at:
https://github.com/LogtalkDotOrg/logtalk3/blob/master/RELEASE_NOTES.md

I plan to continue to evolve this test suite based on your feedback, standardization advancements, and my own work Prolog portability in the context of Logtalk.

Cheers,

Paulo

[Ulrich Neumerkel]

Paulo Moura <pjlm...@gmail.com> writes:
>The updated Prolog conformance suite notes can be found at:
>https://github.com/LogtalkDotOrg/logtalk3/blob/master/tests/prolog/NOTES.md

Which reads:

| ... several systems can also represent the character zero but the
| ISO standard specifies a domain error for predicates that peek or
| get the character zero (or its code)

Where does the standard specify a domain error for this situation?
Take peek_code/2 8.12.3.3 that features only f as a domain_error
for stream-terms:

- domain_error(stream_or_alias, S_or_a)

And behavior for the "character zero" depends on the
C and PCS which are both implementation defined.

[Jan Burse]

Hi,

ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) schrieb:

> And behavior for the "character zero" depends on the
> C and PCS which are both implementation defined.

It could be that atom restrictions are imposed on the
streams. Didn't test the peek_char/get_char itself.

Only the atoms whether they can carry a character
of code zero. No problem is seen here:

SWI-Prolog:

?- atom_codes(X,[65,0,65]).
X = 'A\000\A'.

?- atom_codes('A\000\A',X).
X = [65, 0, 65].

Jekejeke:

?- atom_codes(X,[65,0,65]).
X = 'A\0\A'

?- atom_codes('A\0\A',X).
X = [65,0,65]


On the other hand:

GNU Prolog:

?- atom_codes(X,[65,0,65]).
uncaught exception:
error(representation_error(character_code),
atom_codes/2)

?- atom_codes('A\0\A',X).
uncaught exception:
error(syntax_error('user_input:3 (char:18) invalid
character code in \\constant\\ sequence'),read_term/3)

[Paulo Moura]

On Wednesday, May 27, 2015 at 4:53:53 PM UTC+1, Ulrich Neumerkel wrote:
> Paulo Moura <pjlm...@gmail.com> writes:
> >The updated Prolog conformance suite notes can be found at:
> >https://github.com/LogtalkDotOrg/logtalk3/blob/master/tests/prolog/NOTES.md
>
> Which reads:
>
> | ... several systems can also represent the character zero but the
> | ISO standard specifies a domain error for predicates that peek or
> | get the character zero (or its code)
>
> Where does the standard specify a domain error for this situation?
> Take peek_code/2 8.12.3.3 that features only f as a domain_error
> for stream-terms:
>
> - domain_error(stream_or_alias, S_or_a)

That's a typo in the notes. It should be "representation error" instead of "domain error". Fixed in the git version. The four relevant unit tests are:

predicates/get_char_2 - sics_get_char_2_16
predicates/get_code_2 - sics_get_code_2_17
predicates/peek_char_2 - sics_peek_char_2_16
predicates/peek_code_2 - sics_peek_code_2_17

> And behavior for the "character zero" depends on the
> C and PCS which are both implementation defined.

As the test identifier indicates, these tests were ported from SICS contributed tests to the ISO Prolog conformance suite written by Péter Szabó and Péter Szeredi. I will look into it and revise the tests as necessary. Thanks for bringing this issue to my attention.

Cheers,

Paulo

[Ulrich Neumerkel]

Jan Burse <janb...@fastmail.fm> writes:
>Hi,
>
>ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) schrieb:
>> And behavior for the "character zero" depends on the
>> C and PCS which are both implementation defined.
>
>It could be that atom restrictions are imposed on the
>streams. Didn't test the peek_char/get_char itself.
>
>Only the atoms whether they can carry a character
>of code zero. No problem is seen here:
>
>SWI-Prolog:

...
>Jekejeke:
...

>
>On the other hand:
>
>GNU Prolog:

There is no problem with GNU Prolog. Please read above:
There is C and PCS which are implementation defined.
GNU Prolog does not include this character. That's all.

[Ulrich Neumerkel]

Paulo Moura <pjlm...@gmail.com> writes:
>That's a typo in the notes. It should be "representation error" instead of
>"domain error". Fixed in the git version. The four relevant unit tests are:

...

>> And behavior for the "character zero" depends on the
>> C and PCS which are both implementation defined.

It now reads:

> several systems can also represent the character zero but

> the ISO standard specifies a representation error for

> predicates that peek or get the character zero (or its code)

It is by no means clear what "... but the ISO standard specifies ..."
means. I'd say it suggests that this is required which is not
the case.

The standard does not require that error to happen for the
NULL character. It depends on how C and PCS are implementation
defined. That is, an implementation has to document in the
accompanying documentation how it defines C and PCS. PCS contains
all characters defined by char (* 6.5 *) and may include
additional members such as the NULL character.

BTW, there are (=)/2 that need to be replaced by (==)/2
as in: predicates/sort_2/tests.lgt

[Paulo Moura]

On Thursday, May 28, 2015 at 11:16:36 AM UTC+1, Ulrich Neumerkel wrote:
> Paulo Moura <pjlm...@gmail.com> writes:
> >That's a typo in the notes. It should be "representation error" instead of
> >"domain error". Fixed in the git version. The four relevant unit tests are:
> ...
>
> >> And behavior for the "character zero" depends on the
> >> C and PCS which are both implementation defined.
>
> It now reads:
>
> > several systems can also represent the character zero but
> > the ISO standard specifies a representation error for
> > predicates that peek or get the character zero (or its code)
>
> It is by no means clear what "... but the ISO standard specifies ..."
> means. I'd say it suggests that this is required which is not
> the case.

What part of my previous reply where I wrote "As the test identifier indicates, these tests were ported from SICS contributed tests to the ISO Prolog conformance suite written by Péter Szabó and Péter Szeredi. I will look into it and revise the tests as necessary. Thanks for bringing this issue to my attention." is not clear to you?

[Ulrich Neumerkel]

Paulo Moura <pjlm...@gmail.com> writes:
>On Thursday, May 28, 2015 at 11:16:36 AM UTC+1, Ulrich Neumerkel wrote:
>> Paulo Moura <pjlm...@gmail.com> writes:

>> >That's a typo in the notes. It should be "representation error" instead =
>of
>> >"domain error". Fixed in the git version. The four relevant unit tests a=
>re:
>> ...
>>=20

>> >> And behavior for the "character zero" depends on the
>> >> C and PCS which are both implementation defined.

>>=20
>> It now reads:
>>=20

>> > several systems can also represent the character zero but
>> > the ISO standard specifies a representation error for
>> > predicates that peek or get the character zero (or its code)

>>=20

>> It is by no means clear what "... but the ISO standard specifies ..."
>> means. I'd say it suggests that this is required which is not
>> the case.
>

>What part of my previous reply where I wrote ... is not clear to you?

Above quote "several systems ..." you revised, but it is still
inaccurate/misleading.

[j4n bur53]

ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) schrieb:

>> >GNU Prolog:
> There is no problem with GNU Prolog. Please read above:
> There is C and PCS which are implementation defined.
> GNU Prolog does not include this character. That's all.

I think there is. I think there is already a
problem with the ISO standard. How could somebody
define a standard, introducing two character sets
PCS and C, and saying there that C is subset PCS?

6.5
NOTES
1 Prolog text and data input from text streams consist of a sequence of
characters taken from PCS.

7.1.4.1
C, a set of characters, is an implementation defined subset of PCS, the
processor character set (6.3,

So in the standard it says that PCS is what is
read from Prolog texts and text data streams.
But what happens to a character x in PCS \ C
and the pairs of _code and _char predicates?

How can PCS be what is read, when C is what
can be represented as a single character atom?
Does this mean that get_code can read it, and
get_char cannot read it?

Indeed the ISO core standard would allow such
a behaviour, namely it contains:

8.12.1.2:
i) The entity input from the stream is not a
character.

But I am not sure whether this is really the
intention of the standard. Would this apply to
GNU Prolog in the following way:

PCS GNU: 7-bit ASCII
C GNU: 7-bit ASCII minus NUL Character

Bye

[Ulrich Neumerkel]

j4n bur53 <janb...@fastmail.fm> writes:
>But I am not sure whether this is really the
>intention of the standard. Would this apply to
>GNU Prolog in the following way:
>
> PCS GNU: 7-bit ASCII
> C GNU: 7-bit ASCII minus NUL Character

Yes, so it seems. That is: GNU processes the NUL character in a
Prolog text as a layout. This is *not* an implementation
specific extension, but just the choice of GNU for this
implementation defined feature.

If there is a "problem" with the standard, it is that this
C could - strictly speaking - be any subset of PCS, even one
that does not contain all characters from char (* 6.5 *) -
or even worse, no character at all (still a subset).

Always remember: Conformance does not guarantee that a
system is fit for any purpose. It is only a precondition.

> How can PCS be what is read, when C is what
> can be represented as a single character atom?

By implementing read/1 differently, not using get_char/1
or get_code/1.

> Does this mean that get_code can read it, and
> get_char cannot read it?

No. See in 7.1.4.1:

NOTE - There is a one-to-one mapping between members of
C (characters) and members of CC (character codes) (7.1.2.2).

But it means that read/1 might process it, whereas
get_code/1 *and* get_char/1 could not read it.


In GNU, the NUL is even accepted within a single
quoted atom (^@ denotes NUL)

g('ab^@cd').

which is read as

g('ab').

Strictly speaking, this is an implementation specific
extension of GNU.

In SICStus, any layout characters apart from a space is
correctly rejected in quoted atoms in Prolog text.

[Ulrich Neumerkel]

ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) writes:
>j4n bur53 <janb...@fastmail.fm> writes:
>>But I am not sure whether this is really the
>>intention of the standard. Would this apply to
>>GNU Prolog in the following way:
>>
>> PCS GNU: 7-bit ASCII
>> C GNU: 7-bit ASCII minus NUL Character
>
>Yes, so it seems. That is: GNU processes the NUL character in a
>Prolog text as a layout. This is *not* an implementation
>specific extension, but just the choice of GNU for this
>implementation defined feature.
>
>If there is a "problem" with the standard, it is that this
>C could - strictly speaking - be any subset of PCS, even one
>that does not contain all characters from char (* 6.5 *) -
>or even worse, no character at all (still a subset).
>
>Always remember: Conformance does not guarantee that a
>system is fit for any purpose. It is only a precondition.

In some (old) systems other layout characters, like \t might
be a problem. Not so much for reading, but for writing.

[Paulo Moura]

I wrote (replying to your observation that there's no domain error in this case):

"That's a typo in the notes. It should be "representation error" instead of "domain error". Fixed in the git version."

Let me cite the dictionary:

typo |ˈtīpō|
noun (pl. typos) informal
a typographical error.

It follows that fixing a typo is not the same as revising a paragraph in a way that completely changes its meaning.

And, again, I also wrote (replying to your observation that the character sets are implementation defined in the standard):

"I will look into it and revise the tests as necessary. Thanks for bringing this issue to my attention."

Yet, you feel the need to keep repeating yourself.

[j4n bur53]

ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) schrieb:

> In GNU, the NUL is even accepted within a single
> quoted atom (^@ denotes NUL)
>
> g('ab^@cd').
>
> which is read as
>
> g('ab').
>
> Strictly speaking, this is an implementation specific
> extension of GNU.

Ha Ha, made my day.
No time to write a long response.

[j4n bur53]

j4n bur53 schrieb:

>
> Ha Ha, made my day.
> No time to write a long response.

My speculation so far, GNU Prolog will give different
results depending on platform.

For example on Windoz, I expect including ^Z in
text causing problems.

https://latedev.wordpress.com/2012/12/04/all-about-eof/

No time for testing yet.

[j4n bur53]

Hi,

Paulo Moura schrieb:

> "That's a typo in the notes. It should be "representation error"
> instead of "domain error". Fixed in the git version."

I guess there is a second typo in the new version:

"... several systems can also represent the character zero but

the ISO standard specifies a representation error for
predicates that peek or get the character zero (or its code)"

Ulrich is friendly and calls the above "inaccurate/misleading".
I would call it simply "wrong", if I understand Ulrich correctly
now. And fix it as follows:

"... several systems can also represent the character zero but
the ISO standard specifies if the character zero is not represented

a representation error for predicates that peek or get the character
zero (or its code)"

Bye

[j4n bur53]

j4n bur53 schrieb:

> "... several systems can also represent the character zero but
> the ISO standard specifies if the character zero is not represented
> a representation error for predicates that peek or get the character
> zero (or its code)"

Which makes the comment a little bit a pleonasm. Since it
somehow redundantly explains that a non representation leads
to a representation error.

But I guess this is the nature of such comments.
Its actually a good sign:
http://en.wikipedia.org/wiki/Pleonasm#Professional_and_scholarly_use

[Paulo Moura]

Logtalk 3.00.4, released today, fixes all the know issues with the Prolog conformance suite:

Cheers,

Paulo