Re: R7RS-large backward compatibility

[John Cowan]

On Sun, Mar 8, 2020 at 5:00 AM Marc Nieper-Wißkirchen <ma...@nieper-wisskirchen.de> wrote:

If we use the syntax of SRFI 177 to write the specification of another SRFI (as opposed to merely using SRFI 177 to implement the other SRFI), that other SRFI will be tied to SRFI 177's syntax, won't it? (Whatever the final syntax of SRFI 177 will be.)

Yes, it would be, though there is the possibility of a post-finalization note, which we have used in similar circumstances.  We can also make minor adjustments to a SRFI when we vote on it for inclusion into an R7RS-large docket, and we can vote in a replacement SRFI on a later docket, as was done with SRFI 158 (admittedly, that was completely upward-compatible).

 

There is a possibility that R7RS-largw will finally allow a superior syntax for keyword arguments (e.g. by "overloading" lambda or by having SRFI 88-like keywords so that we don't need the call/kw macro, etc.). Thus, there would have to be a clear way how to map one keyword system to another one.

Not according to my interpretation of <http://scheme-reports.org/2010/working-group-2-charter.html>, which says:

Working groups 1 and 2 must work together to produce specifications that are consistent with one another. In particular, every implementation of the specifications produced by working group 2 must be an implementation of the specifications produced by working group 1. Every program that conforms to the specifications produced by working group 1 (and relies on no features beyond those guaranteed by those specifications) must also be a program that conforms to the specifications produced by working group 2.

 This admittedly needs some reinterpretation, since it presumes the WG1 and WG2 are working concurrently rather than consecutively, which turned out not to be the case (though I was certainly thinking about them concurrently).  However, I judge (as Chair) that the requirements of the second and third sentences are intact.

Of course, I am not a BDFL, and if someone requests a special ballot to override my interpretation of this provision, and it is voted in, and the Steering Committee agrees with the vote, then this can change.  For this reason, I have copied scheme-reports-wg2@ on this email.

But as things stand, I do not believe that interpreting either :foo or foo: as a keyword satisfies either of the charter requirements.  Using #:foo would do so, but I am very very reluctant to add *mandatory* lexical syntax to <https://bitbucket.org/cowan/r7rs-wg1-infra/src/default/LexicalDocket.md> that must be provided and used in order to make use of a feature.  (By comparison, array lexical syntax is not mandatory: you can construct an array using code, and #A(...) is only truly necessary if arrays are to appear in S-expression data.)

By the same token (and this is why I have also copied srfi-140@ on this email), incorporating SRFI-140 into R7RS-large violates the charter requirements, in my judgment.  The value of (string-set! (string-upcase (string #\a \#b \#c) 1 #\!) is defined by R7RS-small as "A!C".  But in a SRFI-140 world, string-upcase returns an immutable string and so the string-set! call fails.  This can be worked around by excluding string-upcase from the (srfi 140) library and importing it from (scheme char), or by importing (srfi 140 char) instead.  But by the time that has been done across all libraries (not merely the R7RS-small ones, but the R7RS-large set as well), we essentially have a completely different dialect of Scheme, one that may run old code more efficiently but can longer be called R7RS.  Overloading lambda to support keyword arguments has similarly pervasive effects.

Going back to #: syntax, it is not unimportant that it is used for keywords only in Racket and Kawa, whereas Chez uses it in the CL meaning for uninterned symbols. Someone, possibly Marc, said that whereas R7RS-small is about the past and the present, R7RS-large is about the future.  But Scheme standardization is not only about keeping old code running; it's also about keeping old implementations working with minimal change.  The Scheme community has more living implementations than any other language community, and it's clear that people prefer it this way, or they wouldn't go on writing new ones optimized for different use cases (as opposed to modifying and extending old implementations).  

In my view, R6RS failed to spread to R5RS+ implementations (other than those maintained by members of the editorial committee) precisely because it demanded too much change from them.  I don't want to happen to R7RS, and the evidence is that so far it has not.  I'd like to keep it that way.

Comments and corrections?

John Cowan          http://vrici.lojban.org/~cowan        co...@ccil.org
We want more school houses and less jails; more books and less arsenals;
more learning and less vice; more constant work and less crime; more
leisure and less greed; more justice and less revenge; in fact, more of
the opportunities to cultivate our better natures.  --Samuel Gompers

[Marc Nieper-Wißkirchen]

Am So., 8. März 2020 um 22:58 Uhr schrieb John Cowan <co...@ccil.org>:

On Sun, Mar 8, 2020 at 5:00 AM Marc Nieper-Wißkirchen <ma...@nieper-wisskirchen.de> wrote:

If we use the syntax of SRFI 177 to write the specification of another SRFI (as opposed to merely using SRFI 177 to implement the other SRFI), that other SRFI will be tied to SRFI 177's syntax, won't it? (Whatever the final syntax of SRFI 177 will be.)

Yes, it would be, though there is the possibility of a post-finalization note, which we have used in similar circumstances.  We can also make minor adjustments to a SRFI when we vote on it for inclusion into an R7RS-large docket, and we can vote in a replacement SRFI on a later docket, as was done with SRFI 158 (admittedly, that was completely upward-compatible).

 

There is a possibility that R7RS-largw will finally allow a superior syntax for keyword arguments (e.g. by "overloading" lambda or by having SRFI 88-like keywords so that we don't need the call/kw macro, etc.). Thus, there would have to be a clear way how to map one keyword system to another one.

Not according to my interpretation of <http://scheme-reports.org/2010/working-group-2-charter.html>, which says:

Working groups 1 and 2 must work together to produce specifications that are consistent with one another. In particular, every implementation of the specifications produced by working group 2 must be an implementation of the specifications produced by working group 1. Every program that conforms to the specifications produced by working group 1 (and relies on no features beyond those guaranteed by those specifications) must also be a program that conforms to the specifications produced by working group 2.

 This admittedly needs some reinterpretation, since it presumes the WG1 and WG2 are working concurrently rather than consecutively, which turned out not to be the case (though I was certainly thinking about them concurrently).  However, I judge (as Chair) that the requirements of the second and third sentences are intact.

Of course, I am not a BDFL, and if someone requests a special ballot to override my interpretation of this provision, and it is voted in, and the Steering Committee agrees with the vote, then this can change.  For this reason, I have copied scheme-reports-wg2@ on this email.

I think there is a wide agreement on your interpretation.

 

But as things stand, I do not believe that interpreting either :foo or foo: as a keyword satisfies either of the charter requirements.  Using #:foo would do so, but I am very very reluctant to add *mandatory* lexical syntax to <https://bitbucket.org/cowan/r7rs-wg1-infra/src/default/LexicalDocket.md> that must be provided and used in order to make use of a feature.  (By comparison, array lexical syntax is not mandatory: you can construct an array using code, and #A(...) is only truly necessary if arrays are to appear in S-expression data.)

Interpreting :foo or foo: as a keyword is a question of the reader and the `read' procedure.  By introducing a reader flag, say, `#!keywords' that is necessary to interpret `:foo' or `foo:' as a keyword, the charter requirements are intact.

As for adding lexical syntax like `#:foo', it will take literally less than a day to add it to any Scheme implementation, I believe.  This is not what would prevent implementations to become implementations of R7RS-large.  The hard things to implement are things like threads, ephemerons (done right), proper tail calls, continuations, ...

I am not saying that one should needlessly add lexical syntax, in particular, because it is (yet?) not namespaced.  But if it really solves the problem at hand (and solving the keyword argument problem with special keyword lexical syntax has a long tradition), it would be unwise not to use this tool.

By the same token (and this is why I have also copied srfi-140@ on this email), incorporating SRFI-140 into R7RS-large violates the charter requirements, in my judgment.  The value of (string-set! (string-upcase (string #\a \#b \#c) 1 #\!) is defined by R7RS-small as "A!C".  But in a SRFI-140 world, string-upcase returns an immutable string and so the string-set! call fails.  This can be worked around by excluding string-upcase from the (srfi 140) library and importing it from (scheme char), or by importing (srfi 140 char) instead.  But by the time that has been done across all libraries (not merely the R7RS-small ones, but the R7RS-large set as well), we essentially have a completely different dialect of Scheme, one that may run old code more efficiently but can longer be called R7RS.  Overloading lambda to support keyword arguments has similarly pervasive effects.

I agree with the problem the SRFI 140 string procedures have.  The base language (R7RS-small) would have to be evolved first.  The approach to call immutable strings something else (like `text's) seems the best way out.

 

The case with "overloading lambda" is different, though.  "Overloading lambda" is like adding the third argument to `member' and `assoc' as has been done in the evolution from R5RS to R7RS.  Every R5RS use of `member' and `assoc' is still an R7RS use.

With our interpretation of the charter, on which we all seem to agree, we cannot extend `lambda' in a way so that

`(lambda (a b &kw c d) ...)'

with auxiliary syntax `&kw' will evaluate to a procedure taking two positional arguments and two keyword arguments `c' and `d'.  The reason is that this would also be a valid procedure of R7RS-small, namely one taking five arguments irregardless to which `&kw' is bound.

On the other hand, a syntax like

`(lambda (a b (kw) c d) ...)'

would be possible.  (I do not want to suggest to use this syntax; it's just an example.)

Having said this, I wouldn't think of such R7RS-small compatible syntax extensions before the end of the R7RS-large standardization process as whatever syntax extensions come up, they will also have to be compatible with each other.

Going back to #: syntax, it is not unimportant that it is used for keywords only in Racket and Kawa, whereas Chez uses it in the CL meaning for uninterned symbols. Someone, possibly Marc, said that whereas R7RS-small is about the past and the present, R7RS-large is about the future.  But Scheme standardization is not only about keeping old code running; it's also about keeping old implementations working with minimal change.  The Scheme community has more living implementations than any other language community, and it's clear that people prefer it this way, or they wouldn't go on writing new ones optimized for different use cases (as opposed to modifying and extending old implementations).  

I can't remember whether I said this, but this doesn't mean anything. :)

We don't have to use the `#:key' syntax if we don't want to force that Chez adds a mode where `#:key' is read in our sense as long as there are letters in the alphabet.  `#&key' looks equally fine.  Apart from that, if Chez is the only system using `#:key' for something else, we can simply ask its author.  It seems that he has already once abandoned the `#:key' syntax for uninterned symbols: http://www.r6rs.org/r6rs-editors/2004-January/000043.html.

In my view, R6RS failed to spread to R5RS+ implementations (other than those maintained by members of the editorial committee) precisely because it demanded too much change from them.  I don't want to happen to R7RS, and the evidence is that so far it has not.  I'd like to keep it that way.

Making string immutable by default would be a great change.  Adding threads an even greater.  On the other hand, making a syntax like `#&key' mandatory would take less than a day to add to any implementation.  In fact, R7RS-small added some mandatory lexical syntax when compared to R5RS, which is much less straightforward to implement, namely datum labels to read and write cyclic and shared data structures.  Thus, I don't see any danger here when it comes to these trivial reader extensions.

-- Marc

 

Comments and corrections?

John Cowan          http://vrici.lojban.org/~cowan        co...@ccil.org
We want more school houses and less jails; more books and less arsenals;
more learning and less vice; more constant work and less crime; more
leisure and less greed; more justice and less revenge; in fact, more of
the opportunities to cultivate our better natures.  --Samuel Gompers

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/CAD2gp_Q1_P1LdRBq7-NqxWU-Kpc%3DYUV9iyLKRKCWt11%3DxYZtYQ%40mail.gmail.com.

[Shiro Kawai]

Regarding #:key syntax - Gauche also uses it as an uninterned symbol.

It is possible to add a reader/writer mode to switch interpretation; I hope I don't need to do so, but since we seem minority, I'd follow whatever everyone agree.

[Marc Nieper-Wißkirchen]

Am Mo., 9. März 2020 um 11:06 Uhr schrieb Shiro Kawai <shiro...@gmail.com>:

Regarding #:key syntax - Gauche also uses it as an uninterned symbol.

It is possible to add a reader/writer mode to switch interpretation; I hope I don't need to do so, but since we seem minority, I'd follow whatever everyone agree.

Is there any prior use of `#&key'?

 

To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/CALN0JNHzTHDRkh-DmWiLfsdXzaSnboSwZewAd0kgobVLSnE%2BgA%40mail.gmail.com.

--

Prof. Dr. Marc Nieper-Wißkirchen
 
Universität Augsburg
Institut für Mathematik
Universitätsstraße 14
86159 Augsburg
 
Tel: 0821/598-2146
Fax: 0821/598-2090
 
E-Mail: marc.nieper...@math.uni-augsburg.de
Web: www.math.uni-augsburg.de/alg/mitarbeiter/mnieper/

[Per Bothner]

On 3/8/20 2:58 PM, John Cowan wrote:

> By the same token (and this is why I have also copied srfi-140@ on this email), incorporating SRFI-140 into R7RS-large violates the charter requirements, in my judgment.

Charter requirements are less of a concern to me that what is better for the Scheme language
and community in the long run.

>  The value of (string-set! (string-upcase (string #\a \#b \#c) 1 #\!) is defined by R7RS-small as "A!C".  But in a SRFI-140 world, string-upcase returns an immutable string and so the string-set! call fails.

I guess it is *possible* to write a correct and meaningful program that that calls string-set!
on the result of string-upcase but I find that very hard to conceive of.
So I think the concern with breaking program is valid, but string-upcase is ulikely
to be an issue.

While breaking old programs is unfortunate, srfi-140 is the best kind of incompatibility:
Programs break cleanly (compile-time or run-time type-checking rather than wrong result or
hard-to-detect performance degradation); the fix is local and trivial (usually add a call
to string-copy); and the fixed programs work fine on older implementations.

In my limited experience after making strings by default immutable, there were very few
places I had to fix my code.

It would be helpful if someone had larger programs that use strings, and tried running
then with srfi-140 - perhaps using Kawa. I'd like to know if you had to change much,
and if it was difficult to do so.

On 3/9/20 12:42 AM, Marc Nieper-Wißkirchen wrote:
> I agree with the problem the SRFI 140 string procedures have. The base language (R7RS-small) would have to be evolved first. The approach to call immutable strings something else (like `text's) seems the best way out.

That would be an absolute disaster. It would break *every* existing string-using program,
not in the sense of no longer working, but in the sense of using a deprecated/obsolete API,
as well as performance issues for programs that are not updated.

How do you visualize the state of Scheme if the new standard recommends text-upcase etc
in place of string-upcase etc? How will tutorials and other documents be changed? Who
will change their existing programs to use the new text-xxx APIs? My guess is almost nobody.

Either leave strings R7RS-large unchanged from R7RS-small (perhaps with a deprecation
warning), or some variant of srfi-152 (also perhaps with a deprecation warning), or
go with srfi-140 (possibly with some tweaking). srfi-135 would be a disaster, IMO.

--
--Per Bothner
p...@bothner.com http://per.bothner.com/

[Marc Nieper-Wißkirchen]

Am Mo., 9. März 2020 um 17:30 Uhr schrieb Per Bothner <p...@bothner.com>:

On 3/8/20 2:58 PM, John Cowan wrote:

> By the same token (and this is why I have also copied srfi-140@ on this email), incorporating SRFI-140 into R7RS-large violates the charter requirements, in my judgment.

Charter requirements are less of a concern to me that what is better for the Scheme language
and community in the long run.

You have a point here.

There can be design errors of R7RS-small.  Do we have to live with them or can we correct them?  By the charter, the initial idea was seemingly to develop both the small and large language in parallel.  If this had happened, we wouldn't have had the keyword syntax problem and, maybe, the string problem as well.

 

>  The value of (string-set! (string-upcase (string #\a \#b \#c) 1 #\!) is defined by R7RS-small as "A!C".  But in a SRFI-140 world, string-upcase returns an immutable string and so the string-set! call fails.

I guess it is *possible* to write a correct and meaningful program that that calls string-set!
on the result of string-upcase but I find that very hard to conceive of.
So I think the concern with breaking program is valid, but string-upcase is ulikely
to be an issue.

While breaking old programs is unfortunate, srfi-140 is the best kind of incompatibility:
Programs break cleanly (compile-time or run-time type-checking rather than wrong result or
hard-to-detect performance degradation); the fix is local and trivial (usually add a call
to string-copy); and the fixed programs work fine on older implementations.

In my limited experience after making strings by default immutable, there were very few
places I had to fix my code.

It would be helpful if someone had larger programs that use strings, and tried running
then with srfi-140 - perhaps using Kawa.  I'd like to know if you had to change much,
and if it was difficult to do so.

On 3/9/20 12:42 AM, Marc Nieper-Wißkirchen wrote:
> I agree with the problem the SRFI 140 string procedures have.  The base language (R7RS-small) would have to be evolved first.  The approach to call immutable strings something else (like `text's) seems the best way out.

That would be an absolute disaster.  It would break *every* existing string-using program,
not in the sense of no longer working, but in the sense of using a deprecated/obsolete API,
as well as performance issues for programs that are not updated.

Hmmm... I didn't mean to deprecate/obsolete the old API or the old mutable strings.  In fact, I would extend their mutability.  At the moment, we do have R7RS-small strings that have two problems: (1) They are not immutable, making most code unnecessarily inefficient.  (2) The are not mutable enough.

By (2), I mean that if you ask for mutable strings, you usually want more than just the ability to change characters in a string.  For example, you want to be able to delete or insert characters as well.

If we narrow R7RS-small strings so that problem (1) is solved, we violate backward compatibility.  On the other hand, we can easily extend R7RS-small strings to something that, say, can eventually represent Emacs buffers, thus solving (2).  If we go this route, we need some new name for immutable strings.  We would end up with "texts" and "strings", which would probably be named "strings" and "buffers" if we didn't maintain backward compatibility.

Even in the case of maintaining backward compatibility, I would propose some reader flag so that strings enclosed in quotes will be read as immutable strings (whatever they are called), because most string constants wouldn't end up as general mutable buffers.

Marc

How do you visualize the state of Scheme if the new standard recommends text-upcase etc
in place of string-upcase etc?  How will tutorials and other documents be changed?  Who
will change their existing programs to use the new text-xxx APIs?  My guess is almost nobody.

Either leave strings R7RS-large unchanged from R7RS-small (perhaps with a deprecation
warning), or some variant of srfi-152 (also perhaps with a deprecation warning), or
go with srfi-140 (possibly with some tweaking).  srfi-135 would be a disaster, IMO.

--
        --Per Bothner
p...@bothner.com   http://per.bothner.com/

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/7d2ae20c-2cf0-36d7-ce58-7d5ef153c1c2%40bothner.com.

[John Cowan]

On Mon, Mar 9, 2020 at 8:05 AM Marc Feeley <fee...@iro.umontreal.ca> wrote:

> Is there any prior use of `#&key’?

That is the syntax for boxes in some Scheme systems.

In all Schemes supporting this syntax with the exception of Racket, boxes are mutable.  As SRFI 111 points out, having a lexical syntax for mutable data structures amounts to self-modifying code.  The standard makes it an error to mutate literal lists, vectors, strings, etc., but few Schemes enforce this.  I have put it on the lexical docket but I doubt it will be voted in.

Per Bothner wrote:

Charter requirements are less of a concern to me that what is better for the Scheme language
and community in the long run.

"In the long run we are all dead."  --John Maynard Keynes

"Sufficient to each day is the evil thereof". --Matt 6:34

I guess it is *possible* to write a correct and meaningful program that that calls string-set!
on the result of string-upcase but I find that very hard to conceive of.
So I think the concern with breaking program is valid, but string-upcase is ulikely
to be an issue.

It was a poor example, I agree.  Nevertheless, the idea of (say) lowercasing an input string and then converting newlines to spaces using a layer over string-set! is not absurd.  Unicode has forced us to give up on SRFI 13's string-downcase!, so the most space-efficient approach is no longer practical. In SRFI 140, these operations have to be transposed to work correctly, which is surely very arbitrary.

In my limited experience after making strings by default immutable, there were very few
places I had to fix my code.

Most code does use strings (and indeed lists) as if they were immutable.

 

On 3/9/20 12:42 AM, Marc Nieper-Wißkirchen wrote:
> I agree with the problem the SRFI 140 string procedures have.  The base language (R7RS-small) would have to be evolved first.

A job for the editors of R8RS, whoever and whenever they may be.

 

 [SRFI 135] would be an absolute disaster.  It would break *every* existing string-using program,

not in the sense of no longer working, but in the sense of using a deprecated/obsolete API,
as well as performance issues for programs that are not updated.

I do not agree that mutable strings are either deprecated or obsolete, or at all likely to become so.  A huge fraction of the SRFIs that make up R7RS-large already (and we are not even close to being done) would have to be scrutinized to determine whether they should return mutable or immutable strings.   We could make a blanket decision that all procedures return immutable strings with the exception of make-string and string-copy, but this is very like Racket's blanket decision to make all Scheme pairs immutable: it's a fine idea (and eliminates the variadic-recursion bug) but it isn't standard Scheme.  This is something about which the Racket community cares little or nothing; they can handle such code with their multilanguage facilities, and that's that.

How do you visualize the state of Scheme if the new standard recommends text-upcase etc
in place of string-upcase etc? 

I don't see such a thing happening.  Texts will be useful where the guarantee that string-ref is O(1) is important, which will probably mean when dealing with large numbers of characters.  (SRFI 135 allows sharing to a limited degree, but does not mandate it).

How will tutorials and other documents be changed?  Who
will change their existing programs to use the new text-xxx APIs?  My guess is almost nobody.

I agree, with the proviso that people who really need the additional performance will switch to texts, and that once the various advantages and disadvantages of each (all textual I/O is still done in strings, for example) are explained, people writing new code can make intelligent choices.  And since mutable strings are still in use, they of course need a library, which is what SRFI 152 is for.  Note also that unlike the situation with (srfi 140) and (srfi 140 mstrings), which cannot be used together except by renaming or prefixing, (scheme text) and (srfi 152) interoperate trivially.

John Cowan          http://vrici.lojban.org/~cowan        co...@ccil.org

Dievas dave dantis; Dievas duos duonos        --Lithuanian proverb
Deus dedit dentes; deus dabit panem           --Latin version thereof
Deity donated dentition;
  deity'll donate doughnuts                   --English version by Muke Tever
God gave gums; God'll give granary            --Version by Mat McVeagh