Striking while the iron is hot

[John Cowan]

Well, now that we've voted on the Yellow Docket, I was planning to hold a vote on the Orange Docket next, but SRFI 225 isn't quiiite complete.  So instead we're going to vote on the Hypnos Docket, which is all SRFIs with portable implementations.  Hopefully that will leave me time to catch up on some of the not-yet-SRFI-ized proposals.

So the ballot is open now at <https://tinyurl.com/wg2-hypnos-docket>. and will close on 2002-04-19T1200Z.  As usual, your choices are the SRFI (in one case two SRFIs), "No" meaning that you are voting against all the SRFIs, and "No vote" meaning that you are abstaining.  A ballot question passes if the number of votes for a particular proposal exceeds 1/2 of all votes cast, *not* counting abstentions.

Write-in votes are still possible, of course, but it winds up being me who figures out just what they mean.  Conditional votes are fine, but I just had to decide what "Only as a SRFI" means; since the proposals are already SRFIs, I took it to mean "No".  Try not to introduce still more complications if you can possibly avoid them.  As before, you can correct your vote by revoting, and all but the last vote will be removed.

Stupid Google Forms won't let me put reasonable commentaries next to the questions and answers, so as I don't wnat to write them now, ask me -- or discuss it on this list.  Or both, of course. 

As before, we can also use implementations of any pre-SRFI in <https://github.com/johnwcowan/r7rs-work/blob/master/ColorDockets.md> other than those already marked "Implementation".  Any efforts on this front will be greatly appreciated

    by your own

        Chief Cook and Bottle-Washer

[Marc Nieper-Wißkirchen]

SRFI 210 and SRFI 219 have already been voted on, I think.

SRFI 148 does not make any sense (at least in its current form) now
that we have syntax-case.

SRFI 28 is missing as an option (there is just SRFI 48).

SRFI 28/48 and SRFI 166 should not be exclusive and should appear in
different questions.

> --
> You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/CAD2gp_T-7xKO96LL4cHxNNO_-OL71d9vh7xkT-XoheV303PT%2BQ%40mail.gmail.com.

[Daphne Preston-Kendal]

Thank you, John.

Didn’t we already just vote to include SRFIs 210 and 219 on the Yellow Docket?

Here are my arguments for and against each proposal apart from those:

SRFI 223 (Binary search): Yes. I am of course the author of this SRFI, so I’m obviously biased. But do note the rationale of this SRFI: vector-binary-search in the (scheme vector) library (SRFI 133) is incompatible with every other Scheme procedure’s idea of what a comparison procedure is — it doesn’t work at all with (scheme comparator), for example. It also can’t be used, for example, to count the number of in elements within a particular range in a sorted vector, because it doesn’t guarantee which index will be returned from the binary search; and it only works on vectors, whereas SRFI 223 also provides a simple plug-in-your-own-procedures way to binary search over any random-access data structure, including one that may be on disk or similar.

SRFI 148 (Eager syntax-rules): No. If we had rejected syntax-case, it would be a yes. The benefit of SRFI 148 is that it can, in theory, be used to write macros in quasi-procedural style in Schemes which only have syntax-rules (including any small implementations which don’t adopt Large, modulo the need for them to support SRFI 147 as well). But I consider this a sufficiently niche use-case, and the SRFI itself is not going away, and you can in theory always bundle its sample implementation into your project if you need it.

SRFI 189 (Maybe/Either): No. We have established patterns for dealing with the problems which these types set out to solve (return #f, or take success/failure thunks as arguments).

(I semi-seriously suggested bundling up Haskellisms like this together into a third-party ‘hl-lib’, comparable to Elisp’s ‘cl-lib’, in #scheme the other day. I still think that would be a reasonable idea.)

SRFI 165 (Environment monad): No. Also a good candidate for hl-lib ;-)

SRFI 175 (ASCII char lib): No. We should *definitely* not be encouraging people to write programs that only work properly on ASCII input. A more minimal solution would be to add e.g. char-set:ascii, char-set:ascii-letter, char-set:ascii-lowercase etc. to (scheme charset) [sic].

[SRFIs 214 (Flexvectors), 161 (Unifiable boxes) elided: I have abstained on them.]

SRFI 166/SRFI 48 (formatting): SRFI 166 should upward-compatibly replace SRFI 158 as (scheme show). SRFI 48 was already voted out once, iirc, when SRFI 158 was adopted; I don’t see why it’s been restored to the ballot.

SRFI 180 (JSON): No, on two grounds. Firstly, JSON is certainly a popular serialization format today, but I don’t know anyone who uses it who doesn’t wish it were better. (Complaints: Not enough data types (no binary string, no set, no date-time); unclear semantics of numbers/no obvious bignum or exact rational type; verbose when records are repeated; you can’t have anything other than strings as keys in a mapping; no circular references; human-readable but frustrating to write by hand due to restrictions on comma placement and similar; no canonicalization form so can’t be hashed/signed/etc.) The number of complaints about it suggest it is ripe for replacement as the Internet’s data serialization format du jour — when considering a language with the longevity of Scheme, I think it a bad idea to jump on this bandwagon. As I said in #scheme: data formats come, data formats go — mostly they go, but Scheme is forever. (If we had designed R7RS Large twenty years ago, everyone who now wants a JSON library would presumably have wanted an XML library. Enough said, I hope.) Secondly, SRFI 180 is badly-designed as JSON APIs go. There is no need for both json-generator and json-fold; the former is a questionable application of generators; the latter takes too many positional arguments; worst, *both* of them make it to difficult to do something simple like say ‘I want objects in a SRFI 146 mapping rather than an alist, but all the other default deserialization options are okay’. There are two security parameters when at most only one is needed (and I note no other language’s JSON library that I’m aware of provides either — JSON is not vulnerable to input explosion attacks like the billion-laughs of XML, for example); one of the names of the security parameters is ungrammatical. If we must have a JSON API, I would prefer something absolutely minimal like Adam Nelson’s (schemepunk json); but in general I’m opposed to including any JSON library in R7RS.

Hooks: No, at least as long as this remains application-specific. If a future R7RS Large library proposal has a use for hooks, I would be okay with putting this back on the ballot. As it is, I think this is better solved at the level of individual applications or third-party libraries.


Daphne

[Marc Nieper-Wißkirchen]

Am Sa., 19. Feb. 2022 um 09:37 Uhr schrieb Daphne Preston-Kendal
<d...@nonceword.org>:

[...]

> Here are my arguments for and against each proposal apart from those:> SRFI 165 (Environment monad): No. Also a good candidate for hl-lib ;-)

SRFI 165 is the basis of SRFI 166 and provides primitives for writing
formatters.

SRFI 166 without SRFI 165 is possible but it's scheme-y to provide the
primitives as well.

[...]

Marc

[Ondřej Majerech]

Regarding formatting: I think C++ has shown well enough that having only manipulator-based formatting is not a good idea. Indeed, C++ has finally got std::format in C++20, which provides printf-like formatting capabilities. Scheme should therefore not be left behind and it should also provide some format-string-based formatting like every other non-toy language does.

Having said that, I can see why people would want manipulator-based/monadic formatting as well. So I support the notion that these two options should not be exclusive.

Both SRFI 48 and 166 contain some questionable parts, however. For 48 it's the ~h “formatter”. I suppose it's not really a big deal, but it's definitely goofy.

For 166, the questionable bit is the colour support. ANSI control sequences don't belong in a string formatting library, and they don't belong in a language's standard library either.

Ondra

[Marc Nieper-Wißkirchen]

Regarding SRFI 28/48 and SRFI 166, I'd like to mention the possibility
of a third option, namely something like format strings but with the
formatting procedure actually not being a procedure but a macro.

In the C world, where format strings are ubiquitous, it is generally
agreed upon that the format strings should appear as constants in the
source code so that the compiler can type-check the call to print.

An analogous approach in Scheme is to make the formatting procedure
macro so that the formatting string can be inspected at compile-time.
Now that we have procedural macros in R7RS-large, quite a lot is
possible.

In particular, formatting strings could become basically as powerful
as SRFI 166 formatters; they just use a different way to represent how
to format values. The former use strings, the latter (roughly)
s-expressions. Both have their own advantages. When it comes to
translation (e.g. through gettext), format strings have some
advantages.

The problem I see with SRFI 48 is that it is not extensible; it is
just some pile of features if you know what I mean. SRFI 28 is so
basic that that argument does not really apply.

Has WG2 ever talked about string formatting outside the question of
accepting or rejecting complete SRFIs? E.g. whether something like
SRFI 48* or SRFI 166* would be better (whatever * means in each case).
But this is once more a procedural complaint about how R7RS-large is
emerging and is not specific to formatting.

Am Sa., 19. Feb. 2022 um 23:03 Uhr schrieb Ondřej Majerech
<oxyd...@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/292d8ab0-e2e9-4880-b5e6-791356836d7en%40googlegroups.com.

[Ondřej Majerech]

I like the idea of expand-time-constant string formatting. However, SRFI 166 can do things like columnar and justified formats that, if implemented in format string library, would most likely result in some quite complicated and incomprehensible format strings. For complex formats like tables and justified text, I think the 166 approach is better.

I don't think SRFI 48 being just a pile of features is necessarily a bad thing. Imagine if it were extensible – you'd see format strings with unfamiliar directives and you'd have to go hunt for the exact extension that added this new directive and understand what it does. Worse, the same directive letter might be used for multiple different extensions used by different programs, or even different parts of the same program.

One of the advantages of format strings is their terseness, which is really only an advantage when the possible set of directives is well known to most people. Once new custom directives become a possibility, the terse format becomes a liability.

One more thing that's missing from SRFI 48 is the ability to refer to positional arguments, like printf's %n$ or Python/C++ {n} directives.

[Marc Nieper-Wißkirchen]

Am So., 20. Feb. 2022 um 00:22 Uhr schrieb Ondřej Majerech
<oxyd...@gmail.com>:
>

> I like the idea of expand-time-constant string formatting. However, SRFI 166 can do things like columnar and justified formats that, if implemented in format string library, would most likely result in some quite complicated and incomprehensible format strings. For complex formats like tables and justified text, I think the 166 approach is better.

I didn't mean SRFI 28/48 instead of SRFI 166. And that's why I think
they should be voted on independently.

> I don't think SRFI 48 being just a pile of features is necessarily a bad thing. Imagine if it were extensible – you'd see format strings with unfamiliar directives and you'd have to go hunt for the exact extension that added this new directive and understand what it does. Worse, the same directive letter might be used for multiple different extensions used by different programs, or even different parts of the same program.

SRFI 166 solves this through lexical bindings ("hygiene"). One could
think whether a similar thing is possible for format strings.

> One of the advantages of format strings is their terseness, which is really only an advantage when the possible set of directives is well known to most people. Once new custom directives become a possibility, the terse format becomes a liability.
>
> One more thing that's missing from SRFI 48 is the ability to refer to positional arguments, like printf's %n$ or Python/C++ {n} directives.

This is a good example of why a pile of features may not always be sufficient.

Yet, I understand why one would like to have some closed set of
formatting options; a way out could be to provide a library with which
general formatting strings can be realized and to provide something
like SRFI 48+ as an application of this library.

(Comparable to Scheme providing a general macro facility, but also a
number of predefined core forms.)

> To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/3b7d9427-5715-420d-86ed-c2a889699100n%40googlegroups.com.

[Martin Rodgers]

On 19/02/2022, Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
> Regarding SRFI 28/48 and SRFI 166, I'd like to mention the possibility
> of a third option, namely something like format strings but with the
> formatting procedure actually not being a procedure but a macro.
>
> In the C world, where format strings are ubiquitous, it is generally
> agreed upon that the format strings should appear as constants in the
> source code so that the compiler can type-check the call to print.

It may be how it works in modern C, but it wasn't really that long ago
when printf etc were just functions and the compiler had no special
knowledge of them, nevermind format strings.

Another option could be a formatter function, taking a control string
and returning a procedure to perform the formatting operations.

Another issue is the parameter passing overhead for the SRFI 48
reference implementation. This is avoided by SRFI 166. It might also
be avoided via a formatter procedure, performing the specialising just
once - perhaps even at compile time, if the compiler can constant-fold
the call.

I've considered define-advice syntax for general rewrite hints to a
compiler, but done no work on that yet. I want to get more experience
with an online syntax directed specialising before I do that. However,
SRFI 48 would be an ideal candidate for it. So would many procedures
in other SRFIs.

> The problem I see with SRFI 48 is that it is not extensible; it is
> just some pile of features if you know what I mean. SRFI 28 is so
> basic that that argument does not really apply.

I have no problem with the "pile of features" offered by SRFI 48, but
is SRFI 166 not extensible? I use format a lot but build procedures
and/or syntax around it, adding more specialised features. E.g.
diagnostic, which is syntax, for tracing execution but only during
debugging. If I used SRFI 166, I expect it should be easier to do this
and more. I've simply never felt a need to.

Could you give an example or two of how format could be extended?
Perhaps also an example of how a syntax version of format might be
help.

[Ondřej Majerech]

On Sunday, 20 February 2022 at 18:19:42 UTC+1 mcro...@gmail.com wrote:

On 19/02/2022, Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
> Regarding SRFI 28/48 and SRFI 166, I'd like to mention the possibility
> of a third option, namely something like format strings but with the
> formatting procedure actually not being a procedure but a macro.
>
> In the C world, where format strings are ubiquitous, it is generally
> agreed upon that the format strings should appear as constants in the
> source code so that the compiler can type-check the call to print.

It may be how it works in modern C, but it wasn't really that long ago
when printf etc were just functions and the compiler had no special
knowledge of them, nevermind format strings.

GCC 2.95 (released in 1999) already had the -Wformat warning option to give warnings about printf format string errors. Assuming that by “the compiler” you meant GCC. The option may have been introduced even earlier than that, but 2.95 is as far as the online documentation on the GCC website goes.

I don't think 1999 counts as “not really that long ago”, but I suppose that is subjective. 

[John Cowan]

On Sat, Feb 19, 2022 at 3:37 AM Daphne Preston-Kendal <d...@nonceword.org> wrote:

Thank you, John.

Didn’t we already just vote to include SRFIs 210 and 219 on the Yellow Docket?

Yes, so they will not appear in the Committee B ballot either.

SRFI 223 (Binary search): Yes. I am of course the author of this SRFI, so I’m obviously biased. But do note the rationale of this SRFI: vector-binary-search in the (scheme vector) library (SRFI 133) [is problematic...]

Removing vector-binary-search from (scheme vector) should go on your issues list tentatively.

 

SRFI 148 (Eager syntax-rules): No. If we had rejected syntax-case, it would be a yes. The benefit of SRFI 148 is that it can, in theory, be used to write macros in quasi-procedural style in Schemes which only have syntax-rules (including any small implementations which don’t adopt Large, modulo the need for them to support SRFI 147 as well). But I consider this a sufficiently niche use-case, and the SRFI itself is not going away, and you can in theory always bundle its sample implementation into your project if you need it.

Your argument is compelling.  I'll remove it from the Committee B agenda.

SRFI 189 (Maybe/Either): No. We have established patterns for dealing with the problems which these types set out to solve (return #f, or take success/failure thunks as arguments).

Thunks are equal in expressive power to Maybe/Either, but they aren't naturally reifiable: thunks are control, Maybe/Either is data.  After all, we do not represent natural numbers as lambdas, though we could.

 

SRFI 165 (Environment monad): No. Also a good candidate for hl-lib ;-)

As Marc points out, this underlies SRFI 166 and I think both should be exposed.

SRFI 180 (JSON): No, on two grounds. Firstly, JSON is certainly a popular serialization format today,

You talk as if it were some mayfly dev framework.  JSON has been stable for 20 years (modulo clarifications around the edges) and is a de facto and de jure (ECMA) standard.  What is more, it is entrenched for web development, which our charter certainly requires that we support.  As of now, only pre-1993 programming is possible in portable Scheme.

but I don’t know anyone who uses it who doesn’t wish it were better. (Complaints: Not enough data types (no binary string, no set, no date-time); unclear semantics of numbers/no obvious bignum or exact rational type; verbose when records are repeated; you can’t have anything other than strings as keys in a mapping; no circular references;

All of these are desires for "more" rather than "better".  JSON is actually rather Scheme-like, lexical syntax aside: it gives you just the minimum required for data interchange that every language can natively support (well, except C).  The moment people decide to accommodate "more" they end up with YAML, which is complicated and unstable.

human-readable but frustrating to write by hand due to restrictions on comma placement and similar;

"Don't do that, then" (what the doctor says when you complain that it hurts to stand on your head).  Trying to accommodate this desire also leads to YAML.

no canonicalization form so can’t be hashed/signed/etc.)

That is indeed an issue. but I don't know of anything except ASN.1/CER and XML that do anything about it.

The number of complaints about it suggest it is ripe for replacement as the Internet’s data serialization format du jour

You could make a much more compelling case that JavaScript is ripe for replacement as the universal front-end language, but we all know it isn't going to happen.

 — when considering a language with the longevity of Scheme, I think it a bad idea to jump on this bandwagon. As I said in #scheme: data formats come, data formats go — mostly they go, but Scheme is forever.

Remember: "practical application development".  Unicode might be replaced, and so might HTTP(S), but we still need them even though they are not timeless in the sense that red-black trees are.  If we want to write Web application servers in Scheme, they need to speak JSON because that is what front-end programmers expect.

 (If we had designed R7RS Large twenty years ago, everyone who now wants a JSON library would presumably have wanted an XML library. Enough said, I hope.)

The trouble is that while it's possible to make a clean JSON library (see below), a clean XML library (as opposed to MicroXML, say) is probably impossible.

Secondly, SRFI 180 is badly-designed as JSON APIs go.

I agree; we need a better SRFI.  Indeed, Amirouche himself isn't happy with it and is talking about replacing it.

worst, *both* of them make it to difficult to do something simple like say ‘I want objects in a SRFI 146 mapping rather than an alist, but all the other default deserialization options are okay’.

If you look at the history of SRFI 225, you'll see that abstracting over dictionary constructors is no simple matter, which is why that SRFI ended up not supporting it.   In particular, people are quite likely to want, or think they want, to store JSON objects as different kinds of record instances, which is even more difficult in the general case.  I think it's better to have a single internal representation, admittedly imperfect.

 

There are two security parameters when at most only one is needed (and I note no other language’s JSON library that I’m aware of provides either — JSON is not vulnerable to input explosion attacks like the billion-laughs of XML, for example);

No, but it's still subject to infinite-input DoS attacks.  (I agree that the depth parameter, which was my idea, is not necessary.)  This is the same provision needed when reading text lines from untrusted input.  We need a way for the caller to regain control in such situations.

one of the names of the security parameters is ungrammatical. If we must have a JSON API, I would prefer something absolutely minimal like Adam Nelson’s (schemepunk json); but in general I’m opposed to including any JSON library in R7RS.

Truly minimal would be (json-read textual-input-stream [max-chars]) and (json-write textual-output-stream object) using the same internal conventions as SRFI 180, and I'd be content with that.

Hooks:

If Committee F gives us SRFI 229, then we won't need this, so I'll put it on the back burner.

[Arthur A. Gleckler]

On Wed, Mar 16, 2022 at 2:22 PM John Cowan <co...@ccil.org> wrote:

 

Truly minimal would be (json-read textual-input-stream [max-chars]) and (json-write textual-output-stream object) using the same internal conventions as SRFI 180, and I'd be content with that.

The input length limit could be factored out.  In MIT Scheme, I've created a special type of input port that truncates its input at a specified number of bytes:

(make-length-limited-input-port input-port length)→ input-port

In some situations, it might be better to make the port's input operations throw exceptions if more than a certain number of bytes is read.  In any case, it's better to do something composable so that the same problem doesn't have to be solved both for JSON and for other input types.

[Marc Nieper-Wißkirchen]

Am Mi., 16. März 2022 um 22:22 Uhr schrieb John Cowan <co...@ccil.org>:
t.

>>
>> Hooks:
>
>
> If Committee F gives us SRFI 229, then we won't need this, so I'll put it on the back burner.

Noted.

[Marc Nieper-Wißkirchen]

Am Mi., 16. März 2022 um 22:22 Uhr schrieb John Cowan <co...@ccil.org>:

[...]

 

SRFI 148 (Eager syntax-rules): No. If we had rejected syntax-case, it would be a yes. The benefit of SRFI 148 is that it can, in theory, be used to write macros in quasi-procedural style in Schemes which only have syntax-rules (including any small implementations which don’t adopt Large, modulo the need for them to support SRFI 147 as well). But I consider this a sufficiently niche use-case, and the SRFI itself is not going away, and you can in theory always bundle its sample implementation into your project if you need it.

Your argument is compelling.  I'll remove it from the Committee B agenda.

As the author of SRFI 148, I agree.  It should be noted that SRFI 147 is also portably implementable (but would then expose own versions of define-syntax, etc.)

 

SRFI 189 (Maybe/Either): No. We have established patterns for dealing with the problems which these types set out to solve (return #f, or take success/failure thunks as arguments).

Thunks are equal in expressive power to Maybe/Either, but they aren't naturally reifiable: thunks are control, Maybe/Either is data.  After all, we do not represent natural numbers as lambdas, though we could.

While success/failure procedures are, for most Scheme applications, the best approach, sometimes we need to reify control as data and then SRFI 189 or a similar mechanism may come in handy.

 

SRFI 180 (JSON): No, on two grounds. Firstly, JSON is certainly a popular serialization format today,

You talk as if it were some mayfly dev framework.  JSON has been stable for 20 years (modulo clarifications around the edges) and is a de facto and de jure (ECMA) standard.  What is more, it is entrenched for web development, which our charter certainly requires that we support.  As of now, only pre-1993 programming is possible in portable Scheme.

I think the separation of Foundation and Batteries really helps here.  Should JSON someday no longer be popular or being used or proven to be broken, some future iteration of R7RS Large can remove that library again.  Inclusion or removal of the library won't touch the foundations.